Hey,

I have been wondering whether its possible to package an environment up into auxiliary files?

Currently I’m packaging the environment next to the scene file to eliminate any connection issues, meaning if the slave have access to the scene file then they can render because the environment files (python files, material files etc.) are next to it.
This does however create files that aren’t easily found again, so I thought that auxiliary files might be good because you can clean up the files by cleaning up the job. The problem is how to specify the path to the auxiliary files.

Where would you want access to the aux files? If it’s in the Deadline API, you can do that with job:
docs.thinkboxsoftware.com/produc … a4d3d3ec09

If you’re submitting the scene file, I supposed you could ask Maya what the path is to the scene it has loaded.

I would like to set the job environment; docs.thinkboxsoftware.com/produc … nment.html to load for example “PYTHONPATH” from the auxiliary files.

Going further I guess I would need to have folders for the auxiliary files. Are directories even possible with auxiliary files?

The aux folder is a big pile of files with no hierarchy unfortunately. If I needed to maintain one, maybe submitting a zip file and extracting it in a PreJobScript.py script should help (info here: docs.thinkboxsoftware.com/produc … ugins.html)

Actually, reading the environment file could be done that way as well if you wanted. Just set the environment when the plugin loads.

It’s pretty unusual though that people need to read that info from a file… Maybe there’s a better way to handle it? A good example would be an OnJobSubmitted event (where you’d have access to the job object and be in the same context as DeadlineCommand), or just modifying the job info file before calling DeadlineCommand in the first place.

Not sure what the goals are here, but if you feel like laying out the pipeline a bit more maybe there’s a better solution we can come to.

Thanks Edwin for the feedback

So the current pipeline is to extract any plugins or scripts to a folder next to the scene. This is to prevent heavily customized Deadline repositories, so if the slave can access the scene file it’ll be able to process it as well since all the necessary files are located next to it.

As an example I’m using the EventScript (github.com/ThinkboxSoftware/Dea … ventScript) to do custom Ftrack updates. This is done by extracting the necessary python files to a PYTHONPATH folder that I then add to the job’s environment. Now these python files are located next to the scene file (or a subfolder relative to the scene file path), so I can set the PYTHONPATH environment variable and know the slaves will have access to it.
It is the PYTHONPATH environment variable that I was wondering whether I could set to some auxiliary path, so the files gets cleaned up when the job is deleted.

Ah! I see.

Something I used to do was use “sys” to add the paths I needed within the script itself:

>>> import sys
>>> sys.path.append('/Users/edwin.amsler/Downloads/pylu-0.1.0')
>>> import pylu

The reason I had to do that was due to fighting six different Pythons being installed on Solaris, but it worked great! It would have been more difficult modifying the system paths or wrapping things inside a shell script.

The other option is definitely creating a JobPreLoad.py or PluginPreLoad.py that just throws the job’s aux folder into PYTHONPATH. You can even put it in the root of the ‘custom/plugins’ folder now to have it apply farm-wide. Rough draft would be:

from Deadline.Scripting import *
import os
import path

def __main__( plugin ):
    import_path = os.environ['PYTHONPATH']
    import_path += ";" + os.path.dirname(plugin.GetDataFilename())
    self.SetEnvironmentVariable('PYTHONPATH', import_path)