STD Out "Progress:" handling

im_thatoneguy · January 9, 2014, 11:25pm

Is the “Progress:” handling done inside the slave or in the job plugin?

I’m wondering if something similar couldn’t be done for printing out output paths.

E.g.

Progress: 90%
Progress: 100%
Output: \path\to\output.mov

And then accumulate that in the job output paths?

rrussell · January 10, 2014, 3:20pm

The progress handling is done by the plugin. If you look at the NukeProcess class in Nuke.py for example, we’re adding this progress handler:

self.AddStdoutHandlerCallback( ".*Frame [0-9]+ \\(([0-9]+) of ([0-9]+)\\)" ).HandleCallback += self.HandleProgress

Further down, the HandleProgress function sets the progress and status message appropriately. In theory, these functions could update the job object as well. In this particular function, we could do the following:

Get the job:

job = self.deadlinePlugin.GetJob()

Set the job’s output files (note that this function takes a string array):

job.SetJobOutputFileNames( outputFileList )

Save the job:

RepositoryUtils.SaveJob( job )

Cheers,
Ryan

im_thatoneguy · January 10, 2014, 5:58pm

Thanks, I’ll update batchDraft and this will be a nice interim solution to dynamically querying a draft script to ascertain its output list.

im_thatoneguy · January 10, 2014, 6:22pm

Error in StartJob: Initialize: Error in InitializeProcess: AttributeError : 'DraftPlugin' object has no attribute 'HandleStdoutOutput' (Python.Runtime.PythonException) [' File "none", line 93, in InitializeProcess\n'] at Deadline.Plugins.ScriptPlugin.InitializeLocalPlugin(Job job) at Deadline.Plugins.ScriptPlugin.StartJob(Job job, String& outMessage, AbortLevel& abortLevel)

So what I did was add this:

 self.AddStdoutHandlerCallback( ".*Output: (.*)" ).HandleCallback += self.HandleStdoutOutput

and this at the end…

def HandleStdoutOutput(self): job = self.deadlinePlugin.GetJob() job.SetJobOutputFileNames( [ self.GetRegexMatch( 1 ) ] ) RepositoryUtils.SaveJob( job )

What did I do wrong?

im_thatoneguy · January 10, 2014, 6:24pm

You know what you have to do to solve a problem. Post a forum question.

I was using tabs and you use spaces for indents.

im_thatoneguy · January 10, 2014, 6:42pm

Spoke too soon. So I’m passing it this output…

0: STDOUT: Output: W:\13022 L3 SPYDR Videos\Editorial\Footage\SPYDR_BTY_MP0050_COMP_B01.03_online_(5).mov

and it’s giving me this error.

[code]Scheduler Thread - Render Thread 0 threw a major error:

Exception Details
RenderPluginException – AttributeError : ‘DraftPlugin’ object has no attribute ‘deadlinePlugin’ (Python.Runtime.PythonException)
[’ File “none”, line 255, in HandleStdoutOutput\n’]
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)
RenderPluginException.Cause: JobError (2)
RenderPluginException.Level: Major (1)
RenderPluginException.HasSlaveLog: True
Exception.Data: ( )
Exception.TargetSite: Void RenderTask(System.String, Int32, Int32)
Exception.Source: deadline
Exception.HResult: -2146233088
Exception.StackTrace:
at Deadline.Plugins.Plugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.RenderCurrentTask(TaskLogWriter tlw)
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
WARNING: Exception in CleanupDeadlineEventListener: Python Error: AttributeError : ‘NotifyEventListener’ object has no attribute ‘Cleanup’ (Python.Runtime.PythonException)
Stack Trace:
[’ File “none”, line 31, in CleanupDeadlineEventListener\n’]
(System.Exception)
at FranticX.Scripting.PythonNetScriptEngine.HandlePythonError(Exception e)
at FranticX.Scripting.PythonNetScriptEngine.CallFunction(String functionName, PyObject[] args)
at Deadline.Scripting.DeadlineScriptManager.CallFunction(String scopeName, String functionName, PyObject[] args)
at Deadline.Events.DeadlineEventPlugin.Dispose()
[stack trace (maximumDepth=4)] FranticX.Diagnostics.Trace2.WriteStack line 0
Deadline.Events.DeadlineEventPlugin.Dispose line 0
Deadline.Events.DeadlineEventManager.LoadEventListeners line 0
Deadline.Events.DeadlineEventManager.OnJobError line 0
[/code]

looks like job = self.deadlinePlugin.GetJob() isn’t valid code in a deadline plugin? I tried different capitalization but that also didn’t work.

if I rewrite that line to self.GetJob() then I get this error:

[code] at Deadline.Jobs.Job.SetJobOutputFileNames(Object outputFileNames)
at FranticX.Processes.ManagedProcess.HandleStdoutLine(String line, Boolean noHandling)
at FranticX.Processes.ManagedProcess.Execute(Boolean waitForExit)
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)

[/code]

EDIT: Solved.

I had to remove the .deadlineJob.

And that was just exposing a bug in my code which was the SetJobOutputFileNames() should have been self.GetRegexMatch( 1 ) instead of just self.

im_thatoneguy · January 10, 2014, 6:59pm

Ok that worked however it didn’t actually update the job output path list.

        self.AddStdoutHandlerCallback( ".*Output: (.*)" ).HandleCallback += self.HandleStdoutOutput

def HandleStdoutOutput(self): job = self.GetJob() job.SetJobOutputFileNames( [ self.GetRegexMatch( 1 ) ] ) RepositoryUtils.SaveJob( job )

LOG:

0: STDOUT: Output: W:\13022 L3 SPYDR Videos\Editorial\Footage\SPYDR_BTY_MP0050_COMP_B01.03_online_(9).mov

rrussell · January 10, 2014, 9:20pm

Turns out calling SaveJob doesn’t update the output settings because they aren’t part of the job’s internal Properties object. We’ll have to workaround this by adding two new RepositoryUtils functions: SetJobOutputDirectories, and UpdateJobOutputFileNames, which we’ll add for RC2. In your case, the usage would look like this:

    def HandleStdoutOutput(self):
        job = self.GetJob()
        RepositoryUtils.UpdateJobOutputFileNames( job, [ self.GetRegexMatch( 1 ) ] )

Ideally, we would like to just add those settings to the internal Properties object, but we can’t do that without breaking backward capability with 6.0.

Cheers,
Ryan

im_thatoneguy · January 10, 2014, 10:11pm

Will I need to set both separately or will UpdateJobOutputFileNames automatically set the JobOutputDirectories?

rrussell · January 13, 2014, 2:20pm

Nope, UpdateJobOutputFileNames will automatically set the directories.

im_thatoneguy · January 16, 2014, 8:32pm

This works perfectly in RC2. I would request this be added as a standard feature of the Draft plugin until there is a more sophisticated way for Draft scripts to communicate with the plugin. After all a draft script might output 3 files and I suspect lots of studios don’t output to the \draft\ folder necessarily.

One more quick question, speaking of multiple file outputs. How do I query a Job’s existing output Paths list as an array? job.GetJobOutputFileNames()? Is that a function? Then I can append each STDOUT as a new output for multiple-output draft scripts. Or do I need to use the SlaveUtils.GetCurrentJobValue( “OutputFileNames” )?

rrussell · January 16, 2014, 8:42pm

I think all of our built-in Draft support automatically sends the output to the \draft\ folder. We’re a bit concerned about any potential side effects, but we can still add this to the wish list and look at it for a future release.

Yup, that should return them as a string array.

Cheers,
Ryan