Getting the job ID of a task after submission

Deadline
1

Hi everyone,

So I’m creating a plugin using Python that needs to have several tasks created with one submission button.

What the plugin is meant to do is:

1 - Create folder structure in specified path
2 - Open Maya and run some python scripts (depends on 1 being finished to start). This script then submits at least 2 other jobs, separate to the current batch (job A and job B).
3 - Open UE4 and run some more python scripts (depends on 2 being finished to start)
4 - Open Nuke and run some more python scripts (depends on 3 being finished, as well as job A and job B, before it starts)
5 - Clean unwanted files created by the previous tasks (depends on 4 to be completed before it begins)

I have the UI I want and my plugin works up to step 1, where I create a folder structure, but I cannot find how to get the job ID of that job, so I can add that ID to the dependencies list, before task 2 is created.
Then I need to do the same thing for task 3, 4 and 5 which is get the job ID from the recently submitted job and assign that to the following task.

Any thoughts/ideas on how I can do this? Thanks

2

It’s a bit unclear - are the different steps run in separate jobs? Or are they all on the submitting machine?

How are you submitting the jobs will determine how to obtain the job id.
For instance if you use RepositoryUtils.SubmitJob then the function returns the newly submitted job object.

1 Like
3

It depends on how you’re submitting your job, I do a lot of sending api requests to the web service using requests in python. the response will contain your jobID.

4

Hi, apologies for taking so long to reply.

I used the RepositoryUtils.SubmitJob in the end and that worked for what I needed at that point.

There’s one more thing I need to do that I haven’t figure out though. How can I add new dependencies to an already existing job?

For example, let’s say I submit Job 1, 2, 3, and 4 all at once. Job number 4 needs to have job 3 as a dependency. That’s all done and works fine. My issue is that Job number 2, as part of the process, creates ‘n’ amount of new jobs. These new jobs (I’ll call them job A,B,C and D) need to be a dependency to job 4 as well. How can I add these since job 4 was created before A,B,C and D were.

5

Actually we have a system like that :slight_smile:

Here is a little code, pasted verbatim, just to put you on the right track. You might have to explicitly pend the dependent jobs, to prevent them from running before new dependencies are added.

Deadline uses C# for Python or something like that, hence the odd array call.

A word of warning: since you have to RepositoryUtils.SaveJob, be careful that you do not save the job in multiple processes at once, otherwise you could have race conditions and missing data.

def update_finalize_job_with_output_deps(finalize_job, output_jobs):
    """ Add all output jobs as dependency to finalize job.
    This method is unsafe. Multiple nuke jobs may try to update the finalize job.
    """
    from Deadline.Scripting import RepositoryUtils
    from System import Array, String

    dependency_length = finalize_job.JobDependencyIDs.Length
    deps = Array.Resize[String](
        finalize_job.JobDependencyIDs,
        dependency_length + len(output_jobs))

    for i, j in enumerate(output_jobs):
        deps.SetValue(
            j.JobId,
            dependency_length + i)

    finalize_job.SetJobDependencyIDs(deps)

    return RepositoryUtils.SaveJob(finalize_job)


def pend_jobs(jobs):
    from Deadline.Scripting import RepositoryUtils
    for job in jobs:
        RepositoryUtils.PendJob(job)
1 Like
6

Thank you so much. I haven’t had a chance to look at this properly but I will later today or Monday morning and hopefully will get it working.

7

Hi there. The code makes some sense, thank you. I just have a few questions if you don’t mind.

The “finalize_job”, and “output_jobs” variables… are they job IDs of the correspoding jobs? or how are you telling it which job you apply the dependencies to?

And my other question is, no matter what I do, I keep getting this error:

from Deadline.Scripting import RepositoryUtils
ImportError: No module named Deadline.Scripting

How do you point it to be able to find the module as I’m running this from outside of Deadline.

8

The Deadline modules are compiled into the Deadline environment, and cannot be used outside of it.
In order to use them, you need deadlinecommand -ExecuteScript ../path-to-your-script.py

The jobs are Job objects. They are returned by RepositoryUtils.SubmitJob and by RepositoryUtils.GetJob.

There is also the Standalone API: Deadline Standalone Python Reference: Jobs.Jobs Class Reference (thinkboxsoftware.com)
This is a restful api for interacting with Deadline, and does not need the internal libraries . There is a pure python client library in <repo>\api\python\Deadline which can be used easily outside a Deadline context.