Is there any way to group multiple operations in a single job, or multiple jobs into a single entity? For instance, if I have a process that needs to run a MayaBatch process and then a render process after that, in previous tools I’ve made those into multiple layers of a single job. From what I’ve seen so far, it doesn’t seem like that’s possible to do in deadline. Is that correct, or have I missed something?
Right now, we’re grouping jobs into dependency groups just based on name - we’re naming them something like “[job_name] layer1”, which gets them to sort correctly in the monitor, but is a bit of a hack. Is there any more formal way to group a set of jobs that are all part of a larger pipeline or process?
p.s. - I’m new to deadline, so I’m sorry if this comes across as an annoying “why can’t we do it like my last studio?” question. If the answer is no, that’s totally fine. I just wanted to check before I abandon it.
The Job Batches option lets you group related jobs in the Monitor by setting a Batch Name properties in the Monitor or at submission time. Some integrated submitters generate Batches of Jobs automatically, but since the feature is relatively new, there may be some places where we could automate this that we haven’t touched yet.
For example, submitting a Multi-Region job and the associated Draft Assembly jobs naturally belong into a Batch group.
To put multiple jobs in the same batch, you can select the multiple Jobs in the Monitor, open the Job Properties dialog and type a batch name in the Batch Name field right under the Job Name, ID, Comments and Department fields:
Note that Deadline supports very complex Job and Task dependencies (controlling whether a Job should run depending on whether other Jobs and Tasks have finished, certain assets exist on disk, or dedicated Python scripts return True). These dependencies are only signaled by the Color of the dependent job (it is Orange when Pending), and can be browsed through the Job Dependencies node graph, as a tree view would be insufficient to display the actual hierarchy in many cases.
So the Batch Name is just a UI level grouping mechanism to let you show jobs with some level of relationship in a collapsible group, but it does not affect the scheduling and pending state due to actual logical dependencies.
The Batch Group’s title will show info about all its Jobs at the top level, and lets you control and delete all the jobs in the Batch by deleting the top level item. For example calling Job Properties (Ctrl+P) on the Batch entry will open the dialog for all grouped Jobs…
Currently the feature supports only one level of batch grouping. Nested Batch Jobs have been requested many times though…
Awesome, that sounds like it will work for what I’m trying to do. Is there a way to specify the batch through the job info file at launch time, or is it a post process that has to be run once everything is running? I don’t see it documented in the Manual Job Submission doc page, but if it’s a new feature maybe it’s a hidden option?
Oops. Turns out we missed adding it to our “Manual Job Submission” docs. I’ll add it now, so it’s available in the next release of Deadline. Here’s a snippet of what it will look like:
BatchName= : Specifies an optional name to logically group jobs together (default = blank).
Just to add to what Bobo said, we also provide batch naming to various parts of the in-app Maya submitter now as well, so that tile/jigsaw rendering jobs are automatically grouped together. (This was added in v7.1.0.35). The logic can all be seen in the Melscript.
There is no dedicated property in SMTDSettings, and the BatchName is not exposed to the UI. SMTD generates and adds a BatchName to jobs it knows belong together (for example multiple Jigsaw jobs, Quicktime job going out with a render job,etc.)
In SMTD of Deadline 7.2, we are adding more such things, like for example Stereo Camera submission (one job per Eye) that would share a BatchName.
In Deadline 7.1, if you are using the SMTDFunctions to submit a job, you can call SMTDFunctions.createSubmitInfoFile() with a batchName: argument to set the name yourself.
For example
SMTDFunctions.createSubmitInfoFile SMTDPaths.SubmitInfoFile batchName:"My Group Name 123"
This way, if you are submitting multiple jobs, as long as you pass the same string as the BatchName: argument, those jobs will be grouped together in the Monitor, even if they have nothing else in common…
If the answer is no, that’s totally fine. I just wanted to check before I abandon it.