We’re moving our rendering to a cloud-based provider and would like to avoid running services on windows as much as possible.
We’re using a Kubernetes based solution and have setup all services deadline needs.
However, we need to run a deadlinewebservice .exe in a docker container so we can leverage our cloud provider’s scaling features.
I’ve started by installing the Deadline Repository using a windows machine, got the Linux binaries from bin/linux and tried starting them in a mono environment, but so far we’ve encountered 3 problems:
When starting deadlinewebservice.exe, libpython2.7.so was not found. Our docker image is based on the mono docker image, which is in turn based on the debian jessie image. After installing libpython2.7, there was no /usr/lib/python2.7.so, however, there is a /usr/lib/python2.7.so.1 and /usr/lib/python2.7.so.1.0. We’ve solved this error by editing Python.Runtime.dll.config to use python2.7.so.1.
deadlinewebservice.exe(more specifically, through franticx.dll) set the python home to $BINPATH/python, which ofcourse did not contain the python standard library, but, from what I can tell, only GUI libraries, which are not needed. We solved this as follows:
rm -rf python
ln -s /usr python
The Python.Runtime.dll library appears to have been built for Windows, not Linux. Windows uses UCS2 for encoding strings, while Linux uses UCS4, pythonnet hides this, however, it seems to try to several string UCS2 functions which are not available on the linux version of python.
I’ve tried building the dll myself, however, I think there are API modifications which prevent deadline from being able to use the latest versions. I’ve tried building version 2.0.0 as well, however, some nuget dependencies are no longer available to be able to build that version.
This is the error we get when trying to start deadlinewebservice.exe:
System.EntryPointNotFoundException: PyUnicodeUCS2_FromUnicode
at (wrapper managed-to-native) Python.Runtime.Runtime:PyUnicode_FromUnicode (string,int)
at Python.Runtime.Runtime.PyUnicode_FromString (System.String s) [0x00007] in <d11f4a646d2740e6abe2a57f0492bc9b>:0
at Python.Runtime.Runtime.Initialize () [0x000d2] in <d11f4a646d2740e6abe2a57f0492bc9b>:0
at Python.Runtime.PythonEngine.Initialize () [0x00011] in <d11f4a646d2740e6abe2a57f0492bc9b>:0
at FranticX.Scripting.PythonNetScriptEngine.Initialize (System.String home, System.String programName) [0x00037] in <be5ef32d504b49f49728bce41a227812>:0
PyUnicodeUCS2_FromUnicode
We would appreciate input from the Deadline team to help solve this issue which prevents us from scaling Deadline as effectively as possible.
Deadline has various dependencies that need to be fulfilled for everything to work. A while back I posted a two-part blog article on running Deadline in containers. The article was geared towards a prior version of Deadline, but the concepts should translate to the latest versions (perhaps with slight modifications).
The articles might provide some clues. If you could share your Dockerfiles, we can try to reproduce any issues. If you prefer to share them privately, you can attach them to a Support Ticket.
Hello Coulter, thanks for replying, here are the files I used for building the container
Dockerfile:
FROM mono
RUN apt-get update && apt-get install -y libpython2.7
ADD deadline-bins-10.0.3.2-linux.tar.gz /deadline-bins
ADD deadline.ini /var/lib/Thinkbox/Deadline10/
RUN sed -i 's/so/so.1/g' /deadline-bins/Python.Runtime.dll.config
deadline.ini:
[Deadline]
NetworkRoot=/deadline-repo
deadline-bins-10.0.3.2-linux.tar.gz is an archive containing the deadline binaries, extracted from \bin\Linux\bin.zip
I only needed this to get access to deadlinewebservice, so I didn’t see any point in installing libX and libGL.
I don’t think it would have mattered anyway, since the runtime stopped at initializing python as I’ve explained in my first post.
The differences between my approach and yours is that you used the deadline installer while I did not, and you use the centos image, while I use the mono image. Perhaps there’s a different version of python that ships with centos.
I would prefer to use a debian-based image, but deadline does not appear to have been tested on debian distros.
I’ll try your approach and get back to you. If it only works on centos then i’ll stick to that until a compatible version is released.
Since deadline seems to be moving to cloud-based environments, are you planning to release updated docker images for deadline?
Edit:
I’ve just noticed the github repository with the ubuntu-based docker image example. I’ll give that a try.