Hi,
I am currently busy setting up a development environment which, besides other things, provides standard frameworks to all software with a python interpreter. One of these software packages is deadline.
What I like to do is use the existing python frameworks within deadline scripts, and extend it with deadline specific utilities and helpers as well.
For me it was an unfortunate surprise and is a major hurdle that ironpython (using mono 2.6 on debian 64) has problems parsing the # -- coding: iso-8859-1 -- line which is used in many standard python scripts, such inspect. It causes a long stacktrace ending with:
SystemError: An exception was thrown by the type initializer for CodecsInfo
I can remove these lines in my own code, and I will have my ways to remove them even from the python standard library, but I was wondering if there is a better way.
Another more practical issue I encountered is the usage of the PythonSearchPathString tag in the networkSettings.xml. I am adjusting it when sandboxing a deadline repository for my test suite hoping that deadlinecommand.exe would pick it up. But apparently that doesn’t happen, IRONPYTHONPATH doesn’t work either. Is the deadlineslave.exe program picking up this setting, or do I have to use other ways to centrally adjust the [IRON]PYTHONPATH for all slaves ? (I didn’t try it with the deadlineslave yet)
Thanks for your support,
Sebastian
Which version of Deadline are you using, and which version of Python are your scripts written for. The current version of IronPython that Deadline ships with is 2.6.x, which is compatible with Python 2.6 scripts. I’m sure your scripts are compatible with Python 2.6 and that this is probably a bug/oversight in IronPython, but it doesn’t hurt to ask. 
Regarding the paths, we currently don’t set IRONPYTHONPATH, and instead add search paths to the IronPython script engine directly. I wonder if it’s better to just use IRONPYTHONPATH instead though, since you’ve already confirmed it works in your case. We might just do this for the next Deadline release. Every Deadline application sets the paths in the script engine prior to running scripts, so if it’s not working with deadlinecommand.exe, then it’s not going to work with the other apps either. 
Cheers,
Hi Ryan,
Thanks for your quick response.
I use Deadline 5.0.0 and Python/IronPython2.6. IronPython caused quite a bit of trouble on my side, but I am not sure whether it may be the mono framework’s fault. It would be a great if you could check whether you can import inspect in ipy on windows. If it works, it would be interesting to know whether the first line uses the # …coding statement I mentioned, or if they just fixed it for their runtime.
Actually, I have to correct myself here to say that IRONPYTHONPATH doesn’t work, unfortunately - I neglected a line earlier in the script which parses the value manually and adjusts sys.path.
As you said that you
, I wonder why it doesn’t pick up the value I definitely add (i.e. /media/truecrypt4/mainline/py/master).
Thanks,
Sebastian
Just to confirm, is the problem that you just can’t import “inspect”, or that you can’t import anything from /media/truecrypt4/mainline/py/master? The problem with IronPython is that it doesn’t support all standard Python modules and it doesn’t support any cpython modules. So if a module you want to import (like “inspect”) has any dependencies on these unsupported modules, you won’t be able to use them.
We are looking into a solution to this problem for Deadline 5.1.
Cheers,
Inspect is a purepython module with the said # … coding line as its first line, which is incredible trouble for ironpython. Try to import it in ipy, and you might see what I mean.
Not to be able to import any c packages/modules is not entirely true though. cStringIO for instance imports just fine. I do agree though that one cannot expect all standard c modules are present yet.
Another thing I noticed is that you apparently don’t just use ironpython internally (which is probably much harder than writing this here 
), but do your very own thing. Apparently ironpython (tested through ipy) fixes some issues that come up in deadline right away. For instance, the pwd package doesn’t exist in ipy, which is why the os.path.expanduser() function on linux cannot work, as it imports pwd internally.
In ironpython it magically works, probably because they applied a monkey patch of some sort, but in deadline it doesn’t.
I wonder how other studios integrated their own frameworks, considering the inherent incompatibilities coming in through ironpython, as I cannot believe they rewrote everything using .NET.
Perhaps they do it just as I do: go through and fix all the problematic code until it works, and hope it stays like that.
Please don’t get me wrong, I do like deadline and its scriptability, but I just didn’t account for the ‘ipy’ factor .
Actually, we do. We just link against the IronPython libraries, and add a few global functions here and there when applicable. Which version of IPY are you testing with? Deadline 5.0 uses IronPython 2.6.1, so maybe you’re using version 2.6.2, and that particular bug you’re referring to was fixed for that release? We will probably look at upgrading to 2.6.2 in Deadline 5.1 regardless.
We’re definitely aware of this limitation with IronPython. It was the obvious choice when we first ported Deadline from Windows to Linux/OSX for a few reasons:
- everyone is using python these days
- it’s portable
- no local python installation required
It works great if you stick with the standard python modules or use .NET calls, but unfortunately, as you found out, it doesn’t integrate well with a studio’s native python tools. As I mentioned previously, we are working on a solution to this problem (it was necessary to solve this problem to properly integrate with Shotgun’s API), so when Deadline 5.1 is released, hopefully issues like this are a thing of the past.
Cheers,
Hi Ryan,
The ironpython version I use is namely: IronPython 2.6 Beta 2 DEBUG (2.6.0.20) on .NET 2.0.50727.1433 - this piece of information should be stored in sys.version, but in deadline you only get 2.6.1 (), so I wasn’t sure which particular build it is.
Considering that I am working on the pipeline integration now, and that the wip-version of deadline is probably much better in that respect already, it would actually be great if I could try it out to continue development on the improved deadline version right away. You can be assured I will provide plenty of feedback - please let me or Timor know if omstudios could be considered for beta testing - as we use Shotgun as well we could be a good match for your target audience.
Thank you,
Sebastian
Just one more inconvenience I noticed: If a script is executed by Deadline, the global functions, like IsRunningOnWindows(), are apparently only available in the globals() of the script being executed, as these have been set specifically.
This is problematic if you want to split your code up into multiple files to make it more maintainable, as nothing you import will have access to these global methods, which are not available in the Deadline package.
I work around this by manually updating the builtin module, such as in:
import __builtin__
__builtin__.__dict__.update(globals())
This will make the deadline global functions available to all modules in the python interpreter.
Maybe it would be possible for you to do that on your side in future, maybe in 5.1 .
Cheers,
Sebastian
Thanks for reporting the problem with the globals. We’ll see if this is something we can handle.
I’ve asked our Sales team to contact you guys about participating in the beta program.
Cheers,
Thank you very much !
We will be good testers !
Have a nice weekend,
Sebastian