Restart job upon Error

Deadline
1

Is there a way of restarting the entire job if Deadline errors?

I’m sending my pre-pass which renders every 10th frame, on a single machine. My trouble is that if the job errors, say 80% of the way through the job, it will start the task up again but wipe the Light Cache and Irradience Map info and start it again.

Can you tell Deadline to start the entire job again from scratch if it reports even 1 error?

Cheers,

Mark

2

Hi Mark,

Perhaps you could submit a command line job to Deadline that would build the cache. You could set up the job to be a single task, that way the full render gets restarted if an error occurs. What software are you rendering with (3dsmax, Maya, etc)?

Cheers,

  • Ryan
3

Thanks Ryan,

I’m using 3ds Max and V-Ray, but the main thing is I’m using RPManager to control and submit around 20 passes from different cameras, each with heir own IR and Light Cache pre-pass setup. Therefore I’d prefer to not go down the route of submitting a command line job.

I’ve tried making the ‘Render Task Chunk Size’ the same size as the entire job (to make it see the job as a single task), but because I’m only doing every 10th frame, it seems to ignore this and give each frame a new task.

Any other thoughts?

Cheers,
Mark

4

Hi Mark,

Unfortunately, I don’t think there is an easy workaround. The only thing I can think of is to modify the the 3dsmax plugin and the submitter. The submitter could be modified to add a setting like “ChunkStep=10” to the plugin info file in this case, and then you could modify the RenderTasks() function in 3dsmax.py to look something like this:

	# This renders the current task.
	def RenderTasks( self ):
		if self.RegionRenderingSingleJob:
			self.StartFrame = GetIntegerPluginInfoEntryWithDefault( "RegionSingleFrame", 0 )
			self.EndFrame = GetIntegerPluginInfoEntryWithDefault( "RegionSingleFrame", 0 )
		else:
			self.StartFrame = GetStartFrame()
			self.EndFrame = GetEndFrame()
		
		VerifyMonitoredManagedProcess( self.ProgramName )
		
		SetProgress( 0.0 )
		
		chunkStep = GetIntegerPluginInfoEntryWithDefault( "ChunkStep", 0 )
		if chunkStep == 0:
			denominator = self.EndFrame - self.StartFrame + 1.0
			for frame in range( self.StartFrame, self.EndFrame + 1 ):
				self.RenderFrame( frame )
				
				if denominator != 0:
					progress = ( (frame - self.StartFrame + 1.0) / denominator ) * 100.0
					SetProgress( progress )
					#SetProgress( ( (frame - self.StartFrame + 1) / denominator ) * 100.0 )
				
				VerifyMonitoredManagedProcess( self.ProgramName )
		else:
			denominator = self.EndFrame - self.StartFrame + 1.0
			for frame in range( self.StartFrame, self.EndFrame + 1, chunkStep ):
				self.RenderFrame( frame )
				
				if denominator != 0:
					progress = ( (frame - self.StartFrame + chunkStep) / denominator ) * 100.0
					SetProgress( progress )
					#SetProgress( ( (frame - self.StartFrame + 1) / denominator ) * 100.0 )
				
				VerifyMonitoredManagedProcess( self.ProgramName )
			
		SetProgress( 100.0 )

Then you could still submit the job as one task, and the plugin will know to use step frames instead of rendering the entire sequence.

Cheers,

  • Ryan