AWS Thinkbox Discussion Forums

Houdini Environment Variables Per User

Deadline
#1

I’m using Deadline, Houdini, and Redshift
Windows 10

User A have “houdini.env” file configured for Redshift 2.6.44

User B have “houdini.env” file configured for Redshift 3.0.16

Ideally when both tasks submitted to the farm - each one should respect Environment Variables of Submitted Job and render with according versions of Redshift

Unfortunately in reality - Environment Variables overridden by Slave/Worker configuration and Job rendering with Slave/Worker User Profile configuration

How can I force Houdini/Deadline to Include Environment Variables?

I see in Repository Configuration some checkboxes for “Rendering as a user” , but Preserve Environment checkboxes available for Linux/Mac only

Help me please to solve this puzzle

#2

Our Redshift was set up to be loaded from Network. During submission we sent Redshift version number as an extra info and construct our $REDSHIFT_COREDATAPATH in JobPreLoad.py using that data.

rs_path = '/path/to/network/redshift/%s' % version_override
deadlinePlugin.SetProcessEnvironmentVariable('REDSHIFT_COREDATAPATH', rs_path)

Houdini then would only use 1 env file utilziing that variable
HOUDINI_PATH = "$REDSHIFT_COREDATAPATH/Plugins/Houdini/{houdini_version};&"

Refer to this doc for more details.

https://docs.redshift3d.com/display/RSDOCS/Custom+Install+Locations?product=houdini#CustomInstallLocations-Houdini

https://docs.redshift3d.com/display/RSDOCS/Redshift+Environment+Variables?product=houdini

1 Like
#3

Panupat

Thank you very much
Could you please elaborate a little bit more on JobPreLoad.py?

Even if I’ll modify your two lines - there might be a few more additional lines needed

Is the ant chance that you could share full (or more detailed version of the JobPreLoad.py?

Thanks a lot

Denys

#4

Refer to This doc for JobPreLoad.py starting template and the scripting reference for what you can do with it. Summary is that you first need to get access to the job object itself and then you can read any parameters from there.

Here’s a quick example, if you store the version inside Extra Info 1

def __main__(deadlinePlugin):
    job = deadlinePlugin.GetJob()
    version_override = job.JobExtraInfo1
    rs_path = '/path/to/network/redshift/%s' % version_override
    deadlinePlugin.SetProcessEnvironmentVariable('REDSHIFT_COREDATAPATH', rs_path)
3 Likes
#5

@Panupat

Can you please explain how you’re storing the version number inside “ExtraInfo1” variable?
I tried to store that as an environment variable inside houdini but deadline doesn’t pick that up in JobPreLoad.py.
I’ll also need to pass the “redshift_LICENSE” env variable.

Hoping to hear back from you.
Thanks!

#6

Hi,

I got how to store the version number now.

You can set the ExtraInfo for job through the Monitor once you submit the job. Right click on job-> Modify job properties -> Job Extra Info.

If you want to automate this on submission time, you can write this into your jobinfo file. 
- In your submitter script: C:\DeadlineRepository10\submission\Houdini\Main\SubmitHoudiniToDeadlineFunctions.py
- you could add the following lines in the section where it writes the JobInfo file:

fileHandle.write( "ExtraInfoKeyValue0=testKey=testValue\n" )
This way all your submissions will have this value. And you can use this in your JobPreLoad.py by calling

Job.GetJobExtraInfoKeyValue(string key)

I’m using this to create variables like this (in jobPreLoad.py):

	rs_path = "/path/to/network/redshift/" + rs_version
	rs_bin_path = "/path/to/network/redshift/" + rs_version + "/bin"
	deadlinePlugin.SetProcessEnvironmentVariable('HOUDINI_PATH', rs_path + "/Plugins/Houdini/18.5.532;&")
	deadlinePlugin.SetProcessEnvironmentVariable('PATH', rs_bin_path + ";$PATH")
	deadlinePlugin.SetProcessEnvironmentVariable('REDSHIFT_COREDATAPATH', rs_path)

But my jobs are still not picking up redshift. Can anyone tell me what’s going on??

Thanks!

#7

I created the JobPreLoad.py script in
\custom\plugins\Houdini
instead of
e <your_repo>/plugins/Houdini/

the script is picked up automatically and the env variables are set correctly

#8

To pass all the environment variables from the Houdini hip file to the Deadline Job you can add the “IncludeEnvironment” flag to the SubmitHoudiniToDeadlineFunctions.py script.

Script Location:
“pathtorepo”\DeadlineRepository10\submission\Houdini\Main\SubmitHoudiniToDeadlineFunctions.py

Where to add the flag:
Around line 670-685 (depending if you modified the script already)

........
                # Create submission info file
                jobInfoFile = os.path.join(homeDir, "temp", "houdini_submit_info%d.job") % ( wedgeNum * regionJobCount + regionjobNum )
                with open( jobInfoFile, "w" ) as fileHandle:
                   
                    #custom line to include ENVIRONMENT VARIABLES
                    fileHandle.write( "IncludeEnvironment=True\n" )
                   
                    fileHandle.write( "Plugin=Houdini\n" )
                    fileHandle.write( "Name=%s\n" % jobName )
                    fileHandle.write( "Comment=%s\n" % jobProperties.get( "comment", "" ) )

..........

In the Houdini File Help / About Houdini / Show Details:
You will see all the env variables of the current scene. All these variables will be passed.

In Deadline monitor, under Job Properties of the job submitted using the Deadline ROP:
under Environment tab or the Submission Params you will see all the key/value pairs added to the job.

NOTE: make sure to restart Houdini after modifing the script otherwise the changes won’t take place!

TAG: how to pass environment variables from houdini to deadline using the deadline rop submitter.

Privacy | Site terms | Cookie preferences