So I’m trying to find my way around the general python api to get an idea how to script things.
Where can I find the custom python classes/files on disk? I would like to make them part of my dev environment in Wing so I can explore the various functions and classes available (might keep from annoying the hell out of you all with silly questions all the time).

My first attempt at scripting is trying to pass the submitting machine’s environment variables to the slaves (at least a select few), including path swapping logic between osx and linux machines.

Any pointers would be greatly appreciated. I will keep reading the online resources and manual in the meantime.

Cheers,
frank

Hi Frank,

There currently aren’t any native python classes for Deadline on disk. The API is exposed through .NET functions and classes, and can only be accessed by scripts running in the context of Deadline (ie: Monitor scripts, or scripts executed with “deadlinecommand -ExecuteScript”). We will be looking at a native python API during the 6.x cycle in hopes we can release it in a later 6.x release.

We also have a feature request on our wishlist to automatically include the submitting machine’s environment with a job so that it can be applied at render time, and again we hope to include it at some point during the 6.x cycle.

For now, you could store the environment as part of the job’s arbitrary extra info properties (scroll down to the Extra Info section):
thinkboxsoftware.com/deadlin … _Info_File

You would then have to modify the rendering plugin for Deadline to apply this environment to the rendering process. The DeadlinePlugin class that the plugin subclasses has a few functions to control the environment for the rendering process:
self.SetEnvironmentVariable( string key, string value )
self.ClearEnvironmentVariables()

Note that these functions only affect the environment of the rendering process, so they don’t persist between renders. This is handy because you don’t have to worry about the environment from a previous render affecting the following render.

Cheers,

• Ryan

Thanks Russel,

I will look into that later today. I did occur to me last night to use the extra info fields to store the submitter’s environment, so am glad that you confirm this as a reasonable workaround.

Having this as a default option (applying the submitter’s environment to the slaves’ rendering environment would be grand, as it’s a very common thing for organised (non-windows) houses to want to be doing.

Cheers,
frank

so to clarify:

To store the environment variables in extra attributes I should edit the submission script for Nuke (in my case):
…/DeadlineRepository6/submission/Nuke/Main/SubmitNukeToDeadline.py

Then modify the nuke plugin to set them for the slaves:
…/DeadlineRepository6/plugins/Nuke/Nuke.py

Is that correct?

So I have modified the SubmitNukeToDeadline.py to assign a value to ExtraInfo6 like this:

	fileHandle.write( "ContinueOnError=%s\n" % dialog.continueOnError.value() )

	# TESTING 
	print 'testing extra info with: %s' % os.environ['OHU_SHOW']
	print fileHandle
	fileHandle.write( "ExtraInfo6=%s" % os.environ['OHU_SHOW'] )
	# TESTING END

	fileHandle.close()

I checked the resulting nuke_plugin_info.job file and it looks like this:

However, the ExtraInfo6 field in the monitor remains empty.

Am I doing something wrong?

frank

ok, got it to work:
I checked the job details panel and it looks like the above set an arbitrary key/value attribute instead of the ExtraInfo6 attribute that shows up in the monitor as a field.
It’s not what I expected based on the info found here, but coincidentally that’s what I was ultimately after.
So now I’m doing this in the SubmitNukeToDeadline.py to push my environment variables to Deadline as custom attributes:

	ohuEnv = [e for e in os.environ if e.startswith('OHU')]
	for ev in ohuEnv:
		fileHandle.write( "%s=%s\n" % (ev, os.environ[ev]) )

On the slave side, I am setting the environment variables by modifying the Nuke.py plugin to include this method for NukePlugin:

	def setEnv(self):
		#Environment.SetEnvironmentVariable('OHU_SHOW', self.GetPluginInfoEntry('OhuEnvKeys'))
		ohuEnvVarKeys = self.GetPluginInfoEntry('OhuEnvKeys')
		for ev in ohuEnvVarKeys.split(','):
			Environment.SetEnvironmentVariable(ev, self.GetPluginInfoEntry(ev))

then calling it in the NukePlugin.InitializeProcess method.

This seems to do what I’m after. Is this the correct way?
My main issue with this is that I will have to update the respective files again every time I re-install the repository.

Is there a better way people would recommend to get around that?

Cheers,
frank

Hi Frank,

The ExtraInfo properties I was referring to actually need to be set in the job info file, not the plugin info file. The job info file is the first file submitted with the job, and contains other generic job properties like Pool, Group, Priority, Frames, etc. Putting these extra properties in the plugin info file works too, as you’ve discovered.

For setting the environment, you should avoid Environment.SetEnvironmentVariable(). This sets the environment for the Slave process, which means the environment will persist between renders. You’ll probably want to ensure only the rendering process has these variables. What you should probably do is create a file called JobPreLoad.py and place it in the Nuke plugin folder. This script will be executed prior to the job rendering. The script should define a main function, and an instance of the DeadlinePlugin class will be passed as a parameter:

from Deadline.Plugins import *
from Deadline.Scripting import *

def __main__( deadlinePlugin ):
      ohuEnvVarKeys = deadlinePlugin.GetPluginInfoEntry('OhuEnvKeys')
      for ev in ohuEnvVarKeys.split(','):
            deadlinePlugin.SetProcessEnvironmentVariable(ev, deadlinePlugin.GetPluginInfoEntry(ev))

This should set the environment for the rendering process only, and this environment will not persist between jobs. Another benefit is that you don’t have to modify the Nuke plugin files that ship with Deadline to achieve this.

Hope this helps!

• Ryan

Perfect, that all makes sense and works nicely now, thanks!

I’d still love to see a way to do this without having to touch the files in the repository, so a re-install won’t require updating files again.
Something like a DEADLINE_SUPPORT_PATH that points to a central location which holds such customised files.

Ideally I wouldn’t have to copy and modify the submission script either, but have a support file (such as the JobPreLoad.py) live outside of the repository defining custom stuff.

If you then re-install the repository, all the customisation will still work without further work.

The nice thing about the PreJobScript is that the repository installer won’t destroy it during an update.

Things get trickier with submission scripts though. Generally if you’re making a bunch of changes to a submitter, we recommend creating a new one (even if that means copying and pasting the original and using that as your starting point). Again, those new scripts wouldn’t be touched during a repository update.

All that being said, we are looking into using environment variables or repository settings to specify an additional location to host custom scripts and plugin files. It’s on the wish list.

Cheers,

• Ryan

lovely, thank you for the info!