Capture JobId from Command Line Submission

Deadline
1

Hey guys,

I’ve got a large automation system that has a series of processes that need to run in serial. Here’s the series of a events:

  1. DeadlineCommand7 -SubmitCommandLineJob -executable “Nuke9.0v4” -arguments “-t /scripts/client/queue_scripts/build_script.py %s %s” -frames 1 -name script_build’

  2. DeadlineCommand7 job_info.job plugin_info.job [NEED JOBID FROM FIRST JOB]

  3. DeadlineCommand7 job_info.job plugin_info.job [NEED JOBID FROM SECOND JOB]

These jobs are being launched from python scripts via os.system(). How can I capture the JobId from each job in order to set up the dependency?

Thank you for great software and documentation!

1 Like
2

I should also mention that this approach seemed to be the path of least resistance. If there is a better way to launch these jobs that helps accomplish my issue, I’m all ears.

3

I suppose another approach might be to kick off the second and third jobs using post-scripts…

4

Found a way, kinda hacky, but it works:

response = subprocess.Popen(cmd, stdout=subprocess.PIPE)
response = response.communicate()[0]

job_id = None
result = None
for l in response.split('\n'):
	if l.startswith('JobID='): 
		job_id = l.split('=')[1]
	elif l.startswith('Result='):
		result = l.split('=')[1]

print "job_id:", job_id
print "result:", result
5

Hello,

Can I ask if the second and third jobs you are submitting are dependent on each other? If so, you might want to look at the dependency options on docs.thinkboxsoftware.com/produc … ssion.html which should help you out in working this out. If not, can you advise the purpose of the JobIDs? Thanks

6

The three jobs must run in sequential order, so they each depend on the previous. I can see in the docs that I can specify JobID’s to create a dependency. That is why I need the JobID of the previous job. These three jobs are submitted programmatically, so setting a manual dependency is not an option.

I have to run this three step process hundreds of times so I am trying to set this dependency chain up via python.

7

Nothing wrong with your ‘hacky’ way. It’s flexible!

Here’s a previous forum post on this topic: viewtopic.php?f=11&t=10391&p=46459#p45046

Perhaps you would prefer to use this “-dependent” flag in deadlinecommand as it will set the dependencies for you:

SubmitMultipleJobs Submits multiple jobs at once [-dependent] This flag makes each job in the list of jobs specified dependent on the previous job [-notify] This flag displays a notification window after the jobs have been submitted [-job <Job Files>] This flag must precede each list of files for each individual job being submitted. You can also add the -dependsonprevious flag to make a single job dependent on the previous job

You can define what plugin is used for each job via creating the Job Files which you feed it above.

Alternatively, if you use our standalone Python API, which is a wrapper around our RESTful API, then you can submit jobs and get it to return the job id only. You need Pulse running with the WebService.
docs.thinkboxsoftware.com/produc … 632c9ddd36

8

Now there is some food for thought! Thanks, Mike!

I’ll do some experimenting… but nice to know there are options :smiley: