AWS Thinkbox Discussion Forums

Deadline 10.2.1.0 - Rez PATH issues for executable

Hi,

We want to run a command using rez and for the plugin to recognize Rez, we are adding the Rez executable path to the PATH env var - but no matter how we update the PATH variable, be it using SetEnvironmentVariable, SetProcessEnvironmentVariable or event os.environ, nothing works and the job fails with:

Executable returned from RenderExecutable(), "rez", is not rooted, and does not exist in the current directory or in the PATH

Does anyone have any advice? I saw this in the latest Deadline 10.3.0.9 release notes and wonder if it might help?

Event Plugin Improvements¶
Rez

Fixed an issue with Linux/MacOS compatibility

Fixed an issue where the Rez environment was not being set when using some plugins. For the full list of compatible plugins, see the Rez event plugin documentation

input would be great, thanks!

Hello

Thanks for reaching out. Rez itself is not a plugin. The application plugins can use Rez context instead of regular executable of the plugin like to run Maya job you can use Maya tool as an executable.

What plugin are you using?
Please share how the package.py is created, we need to know what is it loading.
Have you set job extra info?
Here’s how to setup Rez: Rez — Deadline 10.3.0.9 documentation

Hi Andrew,

Did you get anywhere with this issue?

Thanks,
Marty

Hi,

I’m waiting on a pipeline developer geting back to me in the studio, I’m OOO until September but will aim to follow up then! :smiley:

Thanks,

Here is the list of the application that is compatible with Rez event plugin to work with Deadline: Rez — Deadline 10.3.0.10 documentation

We have fixed the below issue with Rez in latest Deadline 10.3.0.9 release: Release History — Deadline 10.3.0.10 documentation

  • Fixed an issue with Linux/MacOS compatibility
  • Fixed an issue where the Rez environment was not being set when using some plugins. For the full list of compatible plugins, see the Rez event plugin documentation

From our Pipe team:

We want to run a rez command from a custom plugin. Since our render nodes don’t have rez set in their system PATH environment variable, we are trying to specifically set the rez executable for the job in our custom plugin by appending the rez executable to PATH variable using the methods mentioned in the previous message. But no matter how we try to set the PATH, the modified PATH variable doesn’t appear to be understood by the job and errors with "rez", is not rooted, and does not exist in the current directory or in the PATH.
So we are just looking to understand how to make the job understand any modifications to PATH.

I will also likely deploy 10.3.0.10 but unsure if it will have any bearing on the above.

Thanks,

Unless you’re running AWS Portal 10.3.0.10 won’t do much for you.

At what point in the custom plugin’s life is the PATH variable getting updated? And how is that change getting made?

I can’t recall off the top of my head from where the environment gets pulled when fetching the executable. I’d assume we use the render sandbox’s environment which should be a copy of the machine’s at startup. It could be that the update to PATH being made isn’t taken into effect by the plugin?

I’d add an environment dump in your custom plugin code right before the GetRenderExecutable call, just to be sure.

Thanks Justin,

We have tried updating the PATH variable in pre-task, pre-job and pre-load scripts without success.

The change was made using SetEnvironmentVariable, SetProcessEnvironmentVariable and even os.environ but none of them worked.

How/where should we be updating the variable for it to be taken into effect by the plugin?

Thanks!

Additionally we printed the env before the GetRenderExecutable call and it was correctly printed but the job still errored, if that helps.

Just as a test - if you hard-code rez into the PATH on a render node for testing does this issue persist? I want to see if the issue is in reading PATH, or what we’re doing in fetching the render executable.

One other thing - are you trying to fetch Rez as the executable? Instead of something like Nuke or Redshift in comparison. If so, why? Our Rez support isn’t for Rez as an application (like Nuke, or Redshift) but as an application that gets invoked to ‘wrap’ the execution of Nuke or Redshift.

That might be the issue - instead of trying to call Rez directly, follow the setup steps for your custom plugin. Within the application plugin you shouldn’t need to worry about Rez at all. Our event plugin should step in for you.

Let me know if I’m missing context, I’m not running a studio that’s making use of Rez so my use-case assumptions might be naive!

Morning,

We try to set a pre-task script, pre-job and inside the plugin itself (before execution) all the possible variables

  • Job attribute PATH
  • plugin attribute PATH
  • python os set variable PATH

None of them are picked up by the job, even if, when inspecting each of them, they are set and accessible from the Plugin when a Job run with the correct values.

The only two ways the Job would see the correct PATH was if:

  • I hard set PATH on the machine / system level.
  • The job have an environment variable PATH passed at submission time.

Our issue is that we need a cross platform submitter render, which mean our farm can have a Windows or Linux based node/machine, and our machine submitting the Jobs can be either Windows or Linux, which make the following possibilities:

  • Windows → Windows
  • Windows → Linux
  • Linux → Windows
  • Linux → Linux

Possibly silly question - why not set Rez on the machine’s PATH? That’ll avoid cross-platform path translation issues and save Deadline having to set it.

I think the issue is that when Deadline is updating the PATH the session being run would need to re-read the PATH. Typically that’s jut done by opening a new terminal, but I’m not certain how you’d do that within an event plugin or job script.

EDIT: Does this have to be set in a script related to a job? What about if you used a OnSlaveStartedCallback? That should mean that by the time the render sandbox is fired up the correct PATH would be loaded. And should be equivalent to having the PATH set on the machine/system level.

Hi Andrew,

Are you using the ‘run as’ feature?

If so can you verify that the running user can access the files on the machine that is erroring?

We were having issues with envs being wiped via the ‘run as’ option, we could print the env fine before the invocation of the ‘run as’ side of things.

Fairly certain our developers carried out some fix but handy to know your symptoms…

Privacy | Site terms | Cookie preferences