Pre Job Script Not Passing Job Argument

Deadline
1

I’m trying to write a short pre-job script to change the Job’s output location if an ExtraInfo slot has been changed, but whenever it runs there’s no Job assigned to the relevant variable. It seems like it’s not passing one to the script at all for some reason. Here’s the script:

[code]import re
from System.IO import *
from Deadline.Scripting import *
from System.Diagnostics import *

def main( *args ):
deadlinePlugin = args[0]
job = deadlinePlugin.GetJob()
outputDirectories = job.OutputDirectories

target = "anim"
trigger = "Animation"

job.ExtraInfo8 = "New Location"
if job.ExtraInfo0[:len(trigger)] == trigger:
    if outputDirectories[0].find(target) == -1: #If the directory doesn't have animation in it
        jobExtraInfo9 = outputDirectories[0].replace('02_sprep_jpg','04_anim_jpg')[/code]

Setting the ExtraInfo was just a measure to first check that it was getting and setting the filepath correctly, but it seems that I’m just getting either no response at all when I do this. I did also try to use ClientUtils.LogText but that gave me no result either.

Even aside from fixing the error, I’m unclear how I’m supposed to get Deadline to give me feedback with errors and logs if ClientUtils doesn’t work (even though there is a report written about the job) so some help on getting the script to tell me anything as well as how to get it to change the output path the way I intended would be much appreciated.

EDIT: I meant to say, alternatively it’d be handy to know what the name of the setting for OutputFilename0 would be for Deadlinecommand -SetJobSetting. I was trying that method but it didn’t take either OutputFilename or OutputDirectory (with out without numbers) so it either has an obscure name or is not included as a setting that Deadlinecommand has access to.

2

I realised that I actually had made typos in the script, mostly that I forgot to refer to variables with job.JobExtraInfo9, and I also found the deadlinePlugin.LogInfo() command that allowed me to write things to the log.

Now it seems I’ve come upon an entirely different issue though. As far as I can see in the reference material, there actually isn’t a setter method to change the output directories of the files? I see that there are setters accessed easily but there’s no mention of a way to set them even with an internal Deadline script. Is this function missing or am I looking in the wrong place?

3

Hello Gary,

As far as I have been informed the only option would be that you can set ‘JobOutputFilenames’ and ‘JobOutputDirectories’ and do RepositoryUtils.SaveJob(job).

4

I did try that, but I was being told that the property is read only. That’s why I was wondering if there is a method that will write to Output that I’m meant to be using instead, thanks!

5

Hey Gary,

There’s a couple functions in RepositoryUtils for this, RepositoryUtils.SetJobOutputDirectories and RepositoryUtils.UpdateJobOutputFileNames

They’re both listed as taking in an object, but it should work with either a single string or an array of strings.

Cheers,
Jon

6

Hi Jon,

Was too busy to get back to this for a while but I took a look at this and it does exactly what I needed, thanks!

I am running into some trouble though with another pre job script I was working on. It’s basically a job script that checks if the source file referred to in the submission parameters still exists. And if it doesn’t, the script searches through the files in that folder to try and match the naming convention. This is mostly for when people want to rename files slightly, like updating a version number, and then requeue an existing job.

Here’s what the script looks like:

[code]import re, os

from Deadline.Scripting import *
from Deadline.Jobs import *

def main( *args ):
deadlinePlugin = args[0]
job = deadlinePlugin.GetJob()

#Do a check to avoid setting a new file unless the old name is invalid
path = job.GetJobPluginInfoKeyValue("SceneFile")
if os.path.exists(path):
    deadlinePlugin.LogInfo( "Scene file exists")
    return

target = job.JobExtraInfo2 #This contains a scene code from Shotgun

deadlinePlugin.LogInfo( "Searching " + os.path.dirname(path) + " for " + target)

for file in os.listdir(os.path.dirname(path)):
    if file[:len(target)] == target:
        deadlinePlugin.LogInfo( "Found new scene file: " + os.path.join(os.path.dirname(path), file))
        job.SetJobPluginInfoKeyValue("SceneFile", os.path.join(os.path.dirname(path), file))
        RepositoryUtils.SaveJob( job )
        return
deadlinePlugin.LogWarning( "No new scene file found, check that " + path + " is valid" )[/code]

The issue we’re having is that occasionally this will correctly find the new scene file and log that the new file has been found, but then not save the updated source information to the repository, so the job then attempts to render from a file that no longer exists. This has happened twice so far, and both times ran fine if the job was just requeued again, so I know the script works. But perhaps there’s some circumstance not being accounted for? Even if you can’t think what might be causing it, could you let me know if there’s a good way to get more info out of this and narrow down the source of the issue?

EDIT: I should have posted this log in before that the script generates:

[code]=======================================================
Log

2015-03-16 15:02:00: BEGIN - dogsmacserver.local\dogears
2015-03-16 15:02:00: 0: Start Job timeout is disabled.
2015-03-16 15:02:00: 0: Task timeout is being ignored because this is a Pre Job Script Task
2015-03-16 15:02:00: 0: Loaded job: WIP-ANIM_pr202_sc026 (54fd72471c773e5b02940ff4)
2015-03-16 15:02:00: 0: Plugin executing pre job script
2015-03-16 15:02:00: 0: Executing Pre Job Script: “/Volumes/Macintosh HD2/DeadlineRepository7/custom/scripts/Jobs/UpdateSourceFile.py”
2015-03-16 15:02:00: 0: INFO: Executing plugin script /Users/dogears/Library/Application Support/Thinkbox/Deadline7/slave/dogsmacserver/plugins/54fd72471c773e5b02940ff4/AnimeStudio.py
2015-03-16 15:02:00: 0: INFO: About: Anime Studio Plugin for Deadline
2015-03-16 15:02:00: 0: INFO: The job’s environment will be merged with the current environment before rendering
2015-03-16 15:02:00: 0: INFO: Executing pre job script /Volumes/Macintosh HD2/DeadlineRepository7/custom/scripts/Jobs/UpdateSourceFile.py
2015-03-16 15:02:00: 0: INFO: Searching /Volumes/Macintosh HD2/Puffin/12_animation/pr_202/pr202_sc026 for pr202_sc026_#####.anme
2015-03-16 15:02:00: 0: INFO: Found new scene file: /Volumes/Macintosh HD2/Puffin/12_animation/pr_202/pr202_sc026/pr202_sc026_bc.anme

=======================================================
Details

Date: 03/16/2015 15:02:00
Frames: 0
Job Submit Date: 03/09/2015 10:13:27
Job User: shotgun
Average RAM Usage: 0 (0%)
Peak RAM Usage: 0 (0%)
Average CPU Usage: 0%
Peak CPU Usage: 0%
Used CPU Clocks: 0
Total CPU Clocks: 0

=======================================================
Slave Information

Slave Name: dogsmacserver
Version: v7.0.2.3 R (24b5c0a7f)
Operating System: Mac OS X 10.8.3
Machine User: dogears
IP Address: 10.66.2.8
MAC Address: a8:86:dd:a0:24:91
CPU Architecture: x86_64
CPUs: 8
CPU Usage: 4%
Memory Usage: 4.0 GB / 4.0 GB (98%)
Free Disk Space: 872.764 GB
Video Card: Intel HD Graphics 4000
[/code]

As you can see in the log, it does find the new scene file (it changed xx.anme to bc.anme) and it doesn’t hit the end line of the script that should log when an update on the job failed. (“No new scene file found, check that " + path + " is valid”)
But when I check the submission parameters nothing has changed.

7

When it fails, does it still have the correct (updated) value in the properties of that Job in the Monitor? Should be able to check the value in the “Submission Params” section.

I’m also a bit confused by your log, this line:

Doesn’t seem to line up with:

deadlinePlugin.LogInfo( "Searching " + os.path.dirname(path) + " for " + target) for file in os.listdir(os.path.dirname(path)): if file[:len(target)] == target: [...]

From what I can see from your code, target (in this case) should be the prefix “pr202_sc026”, but that’s not what’s getting printed in the log (pr202_sc026_#####.anme). Are the Slaves maybe somehow pulling an older version of the script that might’ve been bugged? Because as it stands, I don’t see anything wrong with the code you posted…

8

Yeah, I check the Submission Params and it has the old filename, the one that no longer exists.
After I requeue and the script runs fine then the job renders, it has the correct one.

Oh sorry about that discrepancy, I actually had changed that part of the script after seeing the log and noticing that what I should be printing something less confusing.
The original line (that produced that log) was:

    deadlinePlugin.LogInfo( "Searching " + os.path.dirname(path) + " for " + target + "_#####.anme")

I had copy and pasted it from elsewhere and forgot to adjust it before.

9

Hey Gary,

I’m not sure what’s going on here. Like I mentioned earlier, the code seems fine, I can’t think of any reason why it would randomly not save.

Is it possible that the filename is actually getting changed (again, maybe) after the pre-job script gets run?

10

Hey Jon,

I did wonder about that, we checked with the artists and none of them did change the files after the job would have started.
Also the logs print the exact same path that’s supposed to be added to the job parameters, so nothing about that should be changing between those two lines of code but the parameters

Unless you meant that the parameters are getting switched and then switched back? I can’t think at all why that might be happening, this prescript is the only way I’m programmatically changing the source file.