Rez support in Deadline

[dani.vr-metaphysic]

Hi there!

After trying to make Rez work in Deadline Version 10.2.1.0 (b4f15d06d), I have a bunch of questions:

1. Not 100% related to Rez, how can I view the log messages of the Deadline code? I can’t see them in either the DCC console (I’m testing with Nuke), or the files in var/log/Thinkbox/Deadline10. For example, how can I see the messages from sentences like self.LogWarning in event/Rez/Rez.py? When I have to debug submit files like SubmitNukeToDeadline.py I usually use nuke.message or create log files with the logging library but it’s annoying.

2. Since I cannot see self.LogWarning messages, when an exception of some Deadline code happens I’m not aware of it. The execution is not cancelled and it seems all was good (and not). Is there any way to force the interruption of the process when an exception was raised by some Deadline code?

3. I found an error in the event/Rez/Rez.py file. I had to switch in the get_tools method from

sub = subprocess.check_output('rez-context --tools').decode("utf-8").splitlines()

to

sub = subprocess.check_output('rez-context --tools', shell=True).decode("utf-8").splitlines()

to be able to submit a job from Nuke, having Global Enabled as the State of the Rez event plugin. The Python documentation literally says:

If passing a single string, either shell must be True or else the string must simply name the program to be executed without specifying any arguments.

As the command is a string with an argument, this is a bug. Is this fixed in a newer Deadline version?

4. After fixing the bug described above, I found the exact same bug that was described in this other thread. On this thread, @Justin_B suggested that this had been fixed in version 10.3, but @Joe.Lamont-Fisher still sees the same thing after updating to that latest 10.3 version. Has there been another patch update or something fixing that? The last message on that thread was in September.

5. If I set Global Enabled in the State of the Rez event plugin, how can I use the submitters included with Deadline Monitor? Should I “Rezify” Deadline itself and run Deadline Monitor from this package to have Deadline Monitor running in a Rez environment?

6. Related to the previous question, what is the best approach to using Rez to manage Deadline customizations? For instance, if I define some custom sanity checks for Nuke submission by creating a CustomSanityChecks.py file. Should I create a Rez package with the whole content of the Deadline Repository?

Thank you very much in advance!

Dani

[Justin_B]

1. Not 100% related to Rez, how can I view the log messages of the Deadline code?

In SubmitNukeToDeadline.py you should be fine with regular Python print statements.

2. Since I cannot see self.LogWarning messages, when an exception of some Deadline code happens I’m not aware of it.

Exceptions should still be bubbling up, if it’s from the submitter the error should be in the Nuke console, and if it’s in the render job it should be in a task report, or in the Worker log.

3. I found an error in the event/Rez/Rez.py file. . .

This is fixed in Deadline 10.3!

4. After fixing the bug described above, I found the exact same bug that was described in this other thread.

That should be fixed, the thing I wasn’t sure of in that thread was if both the client and repository were running 10.3. The fix is both in the event plugin code and the code baked into our Worker that fetches executable paths. So a 10.3 Repository with a 10.2 Worker would still exhibit the issue.

5. If I set Global Enabled in the State of the Rez event plugin, how can I use the submitters included with Deadline Monitor? Should I “Rezify” Deadline itself and run Deadline Monitor from this package to have Deadline Monitor running in a Rez environment?

Yep! That way when Rez.py checks for a set environment it’ll be able to populate the job’s extrainfo settings with tool info.

6. Related to the previous question, what is the best approach to using Rez to manage Deadline customizations?

I don’t know Rez well enough to suggest a best approach, but I don’t know if Rez would be the right application. As I understand it (please correct me!) Rez is going to be best at starting applications and setting environments. For a CustomSanityCheck.py that’ll be pulled in by the submitter on load, and it’ll just pull in whatever file is in place.

For Nuke that’ll be whatever is in the [Repository]\submission\Nuke\Main folder, and there isn’t a way to override where to pull submitter scripts from an environment variable like there is on jobs for plugin files.

[dani.vr-metaphysic]

Hi @Justin_B, thank you very much for your answers! Let me add a few comments:

@Justin_B

In SubmitNukeToDeadline.py you should be fine with regular Python print statements.

One of the studio’s TDs has just confirmed to me that there is a common issue with Nuke and Python statements prints. It seems that they are only shown when the python statements are run from within Nuke, directly in the Script Editor. So, I cannot see the regular Python print statements. For example, in the SubmitNukeToDeadline.py file, I can see this code:

    # Run the sanity check script if it exists, which can be used to set some initial values.
    sanityCheckFile = os.path.join(submissionInfo[ "RepoDirs" ][ "submission/Nuke/Main" ].strip(), "CustomSanityChecks.py") # type: str
    if os.path.isfile(sanityCheckFile):
        print("Running sanity check script: " + sanityCheckFile)

and I cannot see the print in the Nuke console (nothing appears). But, if I add this line with nuke.message:

    # Run the sanity check script if it exists, which can be used to set some initial values.
    sanityCheckFile = os.path.join(submissionInfo[ "RepoDirs" ][ "submission/Nuke/Main" ].strip(), "CustomSanityChecks.py") # type: str
    if os.path.isfile(sanityCheckFile):
        print("Running sanity check script: " + sanityCheckFile)
        nuke.message("Running sanity check script using nuke.message: " + sanityCheckFile)

then I can see the Nuke popup:

@Justin_B

Exceptions should still be bubbling up, if it’s from the submitter the error should be in the Nuke console, and if it’s in the render job it should be in a task report, or in the Worker log.

The exception is not bubbled up in all cases. Continuing with the same previous example in the SubmitNukeToDeadline.py file:

    # Run the sanity check script if it exists, which can be used to set some initial values.
    sanityCheckFile = os.path.join(submissionInfo[ "RepoDirs" ][ "submission/Nuke/Main" ].strip(), "CustomSanityChecks.py") # type: str
    if os.path.isfile(sanityCheckFile):
        print("Running sanity check script: " + sanityCheckFile)
        nuke.message("Running sanity check script using nuke.message: " + sanityCheckFile)
        try:
            import CustomSanityChecks
            sanityResult = CustomSanityChecks.RunSanityCheck() # type: bool
            if not sanityResult:
                print("Sanity check returned false, exiting")
                return
        except:
            print("Could not run CustomSanityChecks.py script")
            print(traceback.format_exc())

You can see that there are two print statements when the exception is caught (into the except). As I cannot see the print statements, I don’t notice when an exception is thrown.

@Justin_B

That should be fixed, the thing I wasn’t sure of in that thread was if both the client and repository were running 10.3. The fix is both in the event plugin code and the code baked into our Worker that fetches executable paths. So a 10.3 Repository with a 10.2 Worker would still exhibit the issue.

Gotcha! Ok, I’ll try to update my Repository and Worker versions and I’ll get back to you with my results.

@Justin_B

I don’t know Rez well enough to suggest a best approach, but I don’t know if Rez would be the right application. As I understand it (please correct me!) Rez is going to be best at starting applications and setting environments

Ok, don’t worry, I can think in another approach

[dani.vr-metaphysic]

Hi there! I didn’t get back to show my results after upgrading Deadline. I can confirm that submissions and renders of jobs with Rez are working with the version 10.3.1.3.

Thank you very much!