How to make a custom event/script for copying image upon completion of task

Deadline
1

Hello,

We are adding cloud rendering to our farm and I now have the problem of having to copy the completed frames from the cloud nodes back to our server.

I have made a custom event using the OnJobFinished function to achieve this upon completion of the job and it works fine. I can successfully copy all the rendered images back to our server automatically when the job finishes, but this it not what we want… we want to be able to copy the rendered image back to our server once each task has completed… we don’t want to have to wait for the entire job to complete…

Seeing as there is no OnTaskFinished function… what would be the best way to achieve this?

Thanks

2

As far as I can tell, there are no post-task event hooks.

I think you’ll be better off running this as a PostTask script instead, attached to the job itself. Implementation depends a bit on how you are submitting the jobs, but generally speaking it’s a pretty simple modification.

Optionally you could use your existing event script to insert this information to each job after catching a hook that makes sense to you, like OnJobStartedCallback or something.

3

Hey Daniel,

Thanks for the reply, though I have a few questions.

I thought the PostTask script had to be manually attended to (run on each job) and it could not function automatically like an event does. Am I wrong? (I cannot rely on anyone having to manually run anything)

Could you please explain how the OnJobStartedCallback would allow a custom script to be run after each task completes?

Cheers

4

All the scripts you can attach to a job are run automatically. I only really make extensive use of the PreJob and PostJob scripts, but from the docs:

In summary, the job scripts are executed once each, automatically. The task scripts are executed per task by the same worker that is about to – or just finished – running a task.

What I meant by the OnJobStartedCallback hook was just that your current event script could detect a new job had started, and insert the correct scripts in the slots listed above, and save the job object. However, come to think of it, you probably need to hook in earlier than when the job started.

You could perhaps use Deadline.Events.DeadlineEventListener.OnJobSubmittedCallback, which receives the job object in question. At which point you can either directly modify the job object (task scripts), or use the relevant RepositoryUtils method as described above (job scripts).

If you are modifying the job object directly, you must call Deadline.Scripting.RepositoryUtils.SaveJob for the changes to propagate.

Hope this helps, let me know if you still need something clarified.

5

Thank you for the extra info Daniel. I can see (theoretically) how this would all come together now. Creating it all however, is another thing entirely.

The syntax for the script to do my task and then the syntax for the script to add my custom script to each job as a task script is beyond me at the moment.

I might have to give up on this idea. That is unless anyone else comes forward with a working example of this running…

6

Have a look at something existing, like this: https://github.com/ThinkboxSoftware/Deadline/blob/master/Custom/events/OverrideJobName/OverrideJobName.py

That script hooks in to the same event I’m proposing that you use, and uses the same RepositoryUtils command I’m suggesting that you use to save your job object.

Just strip out all the stuff about 3dsmax and jobname which you don’t care about, and insert your own logic for adding a path to your PostTask script. As for what this script contains, presumably it would be very similar to your existing script that copies the frames for an entire job – just with some additional logic to only copy frames from a certain task.

1 Like
7

Wicked - thank you!

I have it working now. See below:

\custom\events\Add_PostTask_To_Cloud_Jobs\Add_PostTask_To_Cloud_Jobs.py

###############################################################
# Imports
###############################################################
from System import *
from Deadline.Events import *
from Deadline.Scripting import *
##################################################################################################
# This is the function called by Deadline to get an instance of the Draft event listener.
##################################################################################################
def GetDeadlineEventListener():
	return addPostTaskScript()

def CleanupDeadlineEventListener(eventListener):
	eventListener.Cleanup()

###############################################################
# The event listener class.
###############################################################
class addPostTaskScript (DeadlineEventListener):
	def __init__(self):
		self.OnJobSubmittedCallback += self.OnJobSubmitted
	
	def Cleanup(self):
		del self.OnJobSubmittedCallback

	def OnJobSubmitted(self, job):
		jobPlugin = job.JobPlugin
		jobGroup = job.JobGroup
		if "cloud" in jobGroup:
			self.LogInfo("Add Post Task script to copy images to server..")
			cloudCopyFramesScript = "\\\\MY_SERVER\\deadline10_repo\\custom\\scripts\\Tasks\\cloudCopyFramesScript.py"
			job.PostTaskScript = cloudCopyFramesScript
			self.LogInfo("postTask Script added: %s" % cloudCopyFramesScript)
			RepositoryUtils.SaveJob(job)
		else:
			self.LogInfo( "Not a cloud job. Skipping event." )

and then the script I use: cloudCopyFramesScript.py

import re
from System.IO import *
from Deadline.Scripting import *
import os
from FranticX import Environment2
from subprocess import call


def __main__( *args ):
	deadlinePlugin = args[0]
	job = deadlinePlugin.GetJob()
	outputDirectories = job.OutputDirectories
	outputFilenames = job.OutputFileNames
	paddingRegex = re.compile("[^\\?#]*([\\?#]+).*")

	deadlinePlugin.LogInfo( "Trying to run: \\\\MY_SERVER\\deadline10_repo\\custom\\scripts\\Tasks\\cloudCopyFramesScript.py" )
 
	for i in range( 0, len(outputDirectories) ):
		outputDirectory = outputDirectories[i]
		outputFilename = outputFilenames[i]

		startFrame = deadlinePlugin.GetStartFrame()
		endFrame = deadlinePlugin.GetEndFrame()
		for frameNum in range(startFrame, endFrame+1):
			outputPath = Path.Combine(outputDirectory,outputFilename)
			outputPath = outputPath.replace("//","/")

			m = re.match(paddingRegex,outputPath)
			if( m != None):
				padding = m.group(1)
				frame = StringUtils.ToZeroPaddedString(frameNum,len(padding),False)
				outputFile = outputPath.replace( padding, frame )

			deadlinePlugin.LogInfo( "Output file: " + outputFile )
			if ".exr" in outputFile:
				outputDir = str(job.GetJobInfoKeyValue("OutputDirectory0"))
				outputLocalServer = outputDir.replace("cloud", "MY_SERVER")
				if os.path.isdir('\\\\cloud\\jobs'):
					deadlinePlugin.LogInfo( "Trying to sync rendered files..." )
                    outputFile = outputFile.split("\\")[-1]
					deadlinePlugin.LogInfo( "Syncing file: %s" % outputFile )
					try:
						command_to_pass = ["robocopy", outputDir, outputLocalServer, outputFile]
						print command_to_pass
						call(command_to_pass)
						deadlinePlugin.LogInfo( "All done!" )
					except OSError:
						pass
				else:
					deadlinePlugin.LogInfo( "Not running, as this script didn't run on a cloud node.." )
			else:
				deadlinePlugin.LogInfo( "Not running, as this job didn't render .exrs.." )
2 Likes
8

Nicely done! Looks like a solid implementation :slight_smile: