AWS Thinkbox Discussion Forums

Can't resubmit jobs within custom deadline event

It appears that I can not call ResubmitJob from within a custom deadline event. It continues to fail when doing so with the following error in Deadline Monitor appearing…

The goal is to simply use ResubmitJob to duplicate jobs on the farm where the JobName doesn’t contain ‘PREVIEW’. But Deadline fails when calling Deadlines ResubmitJob command.

An unexpected error occurred while Resubmitting Job:
Job was cancelled by an Event Plugin. (Deadline.Submission.DeadlineSubmissionException)
   at Deadline.Controllers.DataController.SubmitJob(Job job, StringCollection2 auxiliarySubmissionFileNames, Boolean resetJobDefaults, TaskCollection taskCollectionOverride, StringWriter warningWriter, Boolean importJob)
   at Deadline.Controllers.DataController.ResubmitJob(Job job, String jobName, String frameList, Int32 chunkSize, Boolean submitSuspended, Boolean maintenanceJob, Int32 maintenanceJobStartFrame, Int32 maintenanceJobEndFrame)
   at Deadline.Monitor.WorkItems.ResubmitJobWI.InternalDoWork()
   at Deadline.Monitor.MonitorWorkItem.DoWork()

Here is the custom deadline event code… I’m not doing anything special here…

################################################################################
#  Imports
################################################################################
from System.Diagnostics import *
from System.IO import *

from Deadline.Events import *
from Deadline.Scripting import *

import os
import re


################################################################################
# Give Deadline an instance of this class so it can use it.
################################################################################
def GetDeadlineEventListener():
    return MyEvent()

################################################################################
# Cleans up deadline event when it's no longer in use
################################################################################
def CleanupDeadlineEventListener(eventListener):
    eventListener.Cleanup()


################################################################################
# The Stats event listener class.
################################################################################
class MyEvent (DeadlineEventListener):

    def __init__( self ):
        self.OnJobSubmittedCallback += self.OnJobSubmitted

    def Cleanup( self ):
        del self.OnJobSubmittedCallback
            
    def OnJobSubmitted(self, job):
        job = RepositoryUtils.GetJob(job.ID, True)

        # avoid recursive resubmits
        if re.search(job.JobName, 'PREVIEW', flags=re.IGNORECASE):
            self.LogWarning('Job does not need to generate a PREVIEW job')
            return

        # new resubmitted job we want to render first, middle, last frames only
        firstFrame = min(job.JobFramesList)
        lastFrame = max(job.JobFramesList)
        middleFrame = int((firstFrame + lastFrame) * 0.5)
        framelist = '{},{},{}'.format(firstFrame, middleFrame, lastFrame)
        
        self.LogInfo('Resubmitting PREVIEW....')

        job.JobName = job.JobName + ' | PREVIEW' # add to avoid recursion
        newJob = RepositoryUtils.ResubmitJob(
            job,
            framelist,
            job.JobFramesPerTask,
            False
        )
        self.LogInfo('New job created: {}'.format(newJob))

Anyone have any tips on why this fails. It seems like a very basic example of using the resubmit api function.

This is likely because you’re doing this in OnJobSubmitted. The job you’re handling (the one passed to the function) exists in the database but with ‘submitted=false’ so it’s not really there. Since ResubmitJob expects the job you give it to exist I imagine it’s not doing any input sanitation on the incoming job object and blows up.

Instead I’d pull all the JobInfo and PluginInfo entries out of the job object, modify the frame list and submit a new job. A couple more steps and not as tidy as a single function call but it’ll do the trick.

Or maybe this event does what you’re looking for?

How would i then take those params and resubmit them to the farm within a custom event?

Is there a way in which i could locate the jobinfo and jobplugin files and just duplicate them and modify them as needed and submit those to the farm again from within a custom event?

The sample project you sent me doesn’t really solve what im trying to do. It does something similar but doesn’t quite solve it. I wish what i was originally trying to do would work.

This system where deadline doesn’t allow access to the job during the onJobSubmitted has bit us in the ass a bunch this year with custom events we have been trying to write. Hopefully in the future you guys could implement a solution allowing for more access to jobs during this state. It would really solve a lot of problems we are having with it. I appreciate your help so far though to work around the limitation.

I like your idea of recreating the job that was submitted. But how would you recommend i get all the necessary info? It would be everything that makes up the job.

Ok just i did what you said, but it’s still appearing to error out and never submit a new job. It seems quite basic, I’m just recreating the jobInfo and pluginInfo files like you had suggested.

from System.Diagnostics import *
from System.IO import *

from Deadline.Events import *
from Deadline.Scripting import *

import subprocess
import os
import re
from pprint import pprint


######################################################################
## This is the function that Deadline calls to get an instance of the
## main DeadlineEventListener class.
######################################################################
def GetDeadlineEventListener():
    return MyEvent()

######################################################################
## This is the function that Deadline calls when the event plugin is
## no longer in use so that it can get cleaned up.
######################################################################
def CleanupDeadlineEventListener( deadlinePlugin ):
    deadlinePlugin.Cleanup()

######################################################################
## This is the main DeadlineEventListener class for MyEvent.
######################################################################
class MyEvent (DeadlineEventListener):

    def __init__( self ):
        # Set up the event callbacks here
        self.OnJobSubmittedCallback += self.OnJobSubmitted

    def Cleanup( self ):
        del self.OnJobSubmittedCallback


    def OnJobSubmitted(self, job):
        job = RepositoryUtils.GetJob(job.ID, True)

        # Quick Preview Generation...
        if re.search(job.JobName, 'QuickPreview', flags=re.IGNORECASE):
            self.LogWarning('Job does not need to generate a Quick Preview job')
            return

        firstFrame = min(job.JobFramesList)
        lastFrame = max(job.JobFramesList)
        middleFrame = int((firstFrame + lastFrame) * 0.5)
        framelist = '{},{},{}'.format(firstFrame, middleFrame, lastFrame)

        jobInfo = {}
        keys = job.GetJobInfoKeys()
        for k in keys:
            jobInfo[k] = job.GetJobInfoKeyValue(k)

        pluginInfo = {}
        keys = job.GetJobPluginInfoKeys()
        for k in keys:
            pluginInfo[k] = job.GetJobPluginInfoKeyValue(k)

        jobInfo['Name'] = jobInfo['Name'] + ' | QuickPreview'
        jobInfo['Frames'] = framelist

        # Write job files
        tempdir = PathUtils.GetSystemTempPath()

        # Write submit infoFile
        jobInfoFile = os.path.join(tempdir, 'job_submit_info.job')
        with open(jobInfoFile, 'w') as file:
            for key,value in jobInfo.items():
                file.write('{}={}\n'.format(key, value))

        # Write plugin infoFile
        pluginInfoFile = os.path.join(tempdir, 'job_plugin_info.job')
        with open(pluginInfoFile, 'w') as file:
            for key,value in pluginInfo.items():
                file.write('{}={}\n'.format(key, value))

        # resubmit job...
        auxFiles = [] # job.AuxiliaryFileNames BROKEN
        files = [jobInfoFile, pluginInfoFile]
        files.extend(auxFiles)

        res = RepositoryUtils.SubmitJob(files)
        print(res)

I don’t know what’s wrong with this line:

if re.search(job.JobName, 'QuickPreview', flags=re.IGNORECASE):

But it’s not correctly picking up if the job has QuickPreview in the name.

This works though, creating the correctly named job with the first, middle and last tasks.:

if 'QuickPreview' in job.JobName:

The issue is that since the breakout wasn’t working, each job submission would trigger another OnJobSubmitted and then another job submission. Repeat till the stack overflows :slight_smile:

The regex is probably the better way, once you figure out what’s wrong there.

The check is essentially supposed to create a quick preview job for every deadline job submitted that doesn’t contain the word quick preview in the job name.

When it creates the new QuickTime preview job it then appends quick preview to the newly created job name. In order to prevent recursion.

Only creating quick previews for jobs that dont have quickpreview in their name.

Alright i got it working. Thanks Justin. I do have one question though…how can I get a list of all the auxiliary files.

This method appears to error out no matter what i do
job.AuxiliaryFileNames() BROKEN
job.AuxiliaryFileNames BROKEN

‘Job’ object has no attribute ‘AuxiliaryFileNames’

Ok so i found that the docs are outdated or incorrectly listing that users should use this method ‘AuxiliaryFileNames’ which doesn’t exist. However this method works. But how can i get the full path to the auxillary file? Not just it’s filename?

AuxiliarySubmissionFileNames?? Convert to full filepath

os.path.join(
  RepositoryUtils.GetJobAuxiliaryPath(job),
  plugin.GetAuxiliaryFilenames()[0]
)

should do the job
Yeah things are a bit spread out over different components, but it is what it is.

1 Like
Privacy | Site terms | Cookie preferences