Set job status through python

Deadline
1

I have a script that runs in the OnJobFinished event that checks some custom parameters. If they are not valid I want to set the job status to “Failed” so it will be obvious in the monitor. Is there a way to manually change the job status with python?

2

If you’ve got a job object, you should be able to call “RepositoryUtils.FailJob(job)”.

If you only have the string id, you can get the job object by doing “RepositoryUtils.GetJob(id)”

The Deadline API is kind of a strange land. If you want to get the job’s current status, you can read it from job.JobStatus

3

I couldn’t find that in the documentation, but I tried it out and noticed something strange.

The render completes, calls the OnJobFinished event, checks some parameters, if they’re invalid it fails the job. At this point it calls the OnJobFailed event which notifies the user that the job failed. But the job status does not change in the deadline monitor. So users get notifications that it failed, but it shows up as complete in the deadline monitor.

Is this working as intended? Is there a way to force it to show as failed even if it finishes rendering?

4

I’d bet that’s a bug. Could you send your plugin here? I’ll modify it to always fail and run it through one of our farms as a test.

I’m guessing it’s some sort of race condition where the job status isn’t set until after the OnJobFinished event happens.

5

I’m probably not allowed to post the actual file, but here is the basic test case:

from Deadline.Events import *
from Deadline.Scripting import *
from Deadline.Jobs import *
from Deadline.Slaves import *


def GetDeadlineEventListener():
    return CustomEvent()


def CleanupDeadlineEventListener(eventListener):
    eventListener.Cleanup()


class CustomEvent(DeadlineEventListener):
    def __init__(self):
        self.OnJobSubmittedCallback += self.OnJobSubmitted
        self.OnJobFinishedCallback += self.OnJobFinished
        self.OnJobFailedCallback += self.OnJobFailed

    def Cleanup(self):
        del self.OnJobSubmittedCallback
        del self.OnJobFinishedCallback
        del self.OnJobFailedCallback

    def OnJobSubmitted(self, job):
        self.LogInfo('Running OnJobSubmitted event')
        # Handle some job submission parameters automatically

    def OnJobFinished(self, job):
        self.LogInfo('Running OnJobFinished event')
        # Verify that some parameters are valid, assume it failed for this test
        RepositoryUtils.FailJob(job)

    def OnJobFailed(self, job):
        self.LogInfo('Running OnJobFailed event')
        # Oops, something went wrong

The OnJobFailed event does get called, and the log will show that it both completed and failed. But it will show up as completed in the Deadline Monitor.

6

Okay, reproduced here.

I’ll see if I can figure it out from the source code. It might be that the job must be in a rendering state to be failed, but we’ll see.

Update:

From what I’ve seen in the source code, it should just work. I need to get my build environment back into shape or hand this off to someone to test and fix.

7

So, I just got schooled this morning by one of our junior guys. :stuck_out_tongue:

I forgot you can’t transition a ‘completed’ job to ‘failed’. It’s prevented in the business logic.

So, to mark as failed, you’ll need to reset the job as queued and lose all of the render statistics. This works:

def OnJobFinished(self, job): self.LogInfo('Running OnJobFinished event') # Verify that some parameters are valid, assume it failed for this test RepositoryUtils.RequeueJob(job) RepositoryUtils.FailJob(job)

8

Interesting, requeue and then fail. That does what I need and the render stats don’t matter too much as long as the images are still there (which they are).

Thank you.

9

If I try to call RepositoryUtils CompleteJob or FailJob from the PostJobScript, I get an exception in the log even though the job correctly completes or fails. See the error below…

Post job script “…”: NullReferenceException : Object reference not set to an instance of an object.
at Deadline.Controllers.DataController.CompleteJob(Job job)
at Deadline.Scripting.RepositoryUtils.CompleteJob(Job job) (FranticX.Scripting.PythonNetException)

10

Can you make a pared down version that reproduces the issue and post it here? I wonder if the job is the object which is null, or if for some reason parts inside the DataController aren’t initialized in that context.