AWS Thinkbox Discussion Forums

Bug in jobs with enabled Vray Dynamic DBR and disabled Use Local Host

Hi,

I’m currently evaluating the new DL 9 and found a little bug. When you’re submitting a DBR job (3dsCmd or 3dsmax) with Dynamic DBR Enabled and Use Local Host Disabled and there is only one render node available, then the job is finishing prematurely without any rendering being done.

Steps to reconstruct:

  1. In 3dsCmd plugin enable Dynamic DBR and disable Use Local Host options
  2. Make sure that only single render node is ready to start rendering
  3. Submit job with Dynamic DBR

Effect:
The job is starting and the master node is loading the scene due to dynamic DBR being enabled, but no rendering is happening, because there are no free render nodes, which can run VraySpawner, and Local Host rendering is disabled. After some time the job is marked as completed.

Proposed Fix:
If plugin have enabled VRay Dynamic DBR and disabled Use Local Host then master node should wait for at least one job task (other than master node) to start rendering before loading the scene.

Because this feature is something I was waiting for a long time, I decided to FIX it by myself.

The problem concerns both 3dsmax and 3dsCmd plugins in DL 9.0.0.18. The fix code can be found below. You have to replace original code with fixed one in both plugins. Original code starts In line 146 in 3dsCmd.py file and line 163 in 3dsmax.py file. I’m not sure if this works for V-Ray RT because I’m not using it. I have only tested this in 3dscmd plugin, but it should works in both 3dscmd and 3dsmax.

After applying the fix, the master node should wait for at least one job task (other than master node) to start rendering before loading the scene. Assuming the VRay Dynamic DBR is enabled and Use Local Host is disabled

Original code:

                if (self.VrayDBRJob and self.MyCmdController.VrayDBRDynamicStart) or (self.VrayRtDBRJob and self.MyCmdController.VrayRtDBRDynamicStart):
                    # Start job immediately before all tasks have been dequeued.
                    self.LogInfo( self.Prefix + "Starting distributed render immediately" )
                else:
                    # Wait until the job has no queued tasks left.
                    self.LogInfo( self.Prefix + "Waiting for all job tasks to be dequeued before starting distributed render" )
                    while True:
                        if self.IsCanceled():
                            self.FailRender( "Task canceled" )
                        
                        # When checking for queued tasks, take into account a potential queued post job task.
                        updatedJob = RepositoryUtils.GetJob( currentJob.JobId, True )
                        if updatedJob.JobQueuedTasks == 0 or (updatedJob.JobPostJobScript != "" and updatedJob.JobQueuedTasks <= 1):
                            break
                        
                        # Sleep a bit so that we're not constantly polling the job.
                        SystemUtils.Sleep( 5000 )

                    self.LogInfo( self.Prefix + "All tasks dequeued, setting up distributed config file" )

Fixed code:

                # FIX to use VRay Dynamic DBR with Use Local Host set to False:
                # if job uses V-Ray Dynamic Start and V-Ray Use Local Host is Enabled than start immediately
                # if job uses V-Ray Dynamic Start and V-Ray Use Local Host is Disabled than wait for at least one job task to start
                # if job does not use V-Ray Dynamic Start then wait for all job tasks to start
                if (self.VrayDBRJob and self.MyCmdController.VrayDBRDynamicStart and self.MyCmdController.VrayDBRUseLocalMachine) or (self.VrayRtDBRJob and self.MyCmdController.VrayRtDBRDynamicStart and self.MyCmdController.VrayRtDBRAutoStartLocalSlave):
                    # Start job immediately before all tasks have been dequeued.
                    self.LogInfo( self.Prefix + "Starting distributed render immediately" )
                elif (self.VrayDBRJob and self.MyCmdController.VrayDBRDynamicStart and not self.MyCmdController.VrayDBRUseLocalMachine) or (self.VrayRtDBRJob and self.MyCmdController.VrayRtDBRDynamicStart and not self.MyCmdController.VrayRtDBRAutoStartLocalSlave):
                    # Wait until at least one job task start
                    self.LogInfo( self.Prefix + "Waiting for at least one job tasks other than master node to start rendering before starting distributed render" )
                    while True:
                        if self.IsCanceled():
                            self.FailRender( "Task canceled" )
                        
                        updatedJob = RepositoryUtils.GetJob( currentJob.JobId, True )
                        # wait until there are more rendering tasks then just a master node
                        if updatedJob.JobRenderingTasks > 1:
                            break
                        
                        # Sleep a bit so that we're not constantly polling the job.
                        SystemUtils.Sleep( 5000 )

                    self.LogInfo( self.Prefix + "At least one job task other than master ndoe is rendering, setting up distributed config file" )
                else:
                    # Wait until the job has no queued tasks left.
                    self.LogInfo( self.Prefix + "Waiting for all job tasks to be dequeued before starting distributed render" )
                    while True:
                        if self.IsCanceled():
                            self.FailRender( "Task canceled" )
                        
                        # When checking for queued tasks, take into account a potential queued post job task.
                        updatedJob = RepositoryUtils.GetJob( currentJob.JobId, True )
                        if updatedJob.JobQueuedTasks == 0 or (updatedJob.JobPostJobScript != "" and updatedJob.JobQueuedTasks <= 1):
                            break
                        
                        # Sleep a bit so that we're not constantly polling the job.
                        SystemUtils.Sleep( 5000 )

                    self.LogInfo( self.Prefix + "All tasks dequeued, setting up distributed config file" )
                # End of FIX

– edit 03.04.2017
included VrayRtDBRAutoStartLocalSlave fix proposed by MikeOwen

Nice! Note, for V-Ray RT this property:

self.MyCmdController.VrayDBRUseLocalMachine

should be swapped for this one instead:

self.MyCmdController.VrayRtDBRAutoStartLocalSlave

(It means the same thing, but is applicable to RT and not standard V-Ray).

I’ll see about adding this to our shipping plugins as well.

Good catch. I’ve edited my answer to include your fix to RT part

Privacy | Site terms | Cookie preferences