AWS Thinkbox Discussion Forums

Dockerizing deadline

Hello!

We’re moving our rendering to a cloud-based provider and would like to avoid running services on windows as much as possible.
We’re using a Kubernetes based solution and have setup all services deadline needs.
However, we need to run a deadlinewebservice .exe in a docker container so we can leverage our cloud provider’s scaling features.

I’ve started by installing the Deadline Repository using a windows machine, got the Linux binaries from bin/linux and tried starting them in a mono environment, but so far we’ve encountered 3 problems:

  1. When starting deadlinewebservice.exe, libpython2.7.so was not found. Our docker image is based on the mono docker image, which is in turn based on the debian jessie image. After installing libpython2.7, there was no /usr/lib/python2.7.so, however, there is a /usr/lib/python2.7.so.1 and /usr/lib/python2.7.so.1.0. We’ve solved this error by editing Python.Runtime.dll.config to use python2.7.so.1.

  2. deadlinewebservice.exe(more specifically, through franticx.dll) set the python home to $BINPATH/python, which ofcourse did not contain the python standard library, but, from what I can tell, only GUI libraries, which are not needed. We solved this as follows:

rm -rf python
ln -s /usr python
  1. The Python.Runtime.dll library appears to have been built for Windows, not Linux. Windows uses UCS2 for encoding strings, while Linux uses UCS4, pythonnet hides this, however, it seems to try to several string UCS2 functions which are not available on the linux version of python.

I’ve tried building the dll myself, however, I think there are API modifications which prevent deadline from being able to use the latest versions. I’ve tried building version 2.0.0 as well, however, some nuget dependencies are no longer available to be able to build that version.

This is the error we get when trying to start deadlinewebservice.exe:

System.EntryPointNotFoundException: PyUnicodeUCS2_FromUnicode
  at (wrapper managed-to-native) Python.Runtime.Runtime:PyUnicode_FromUnicode (string,int)
  at Python.Runtime.Runtime.PyUnicode_FromString (System.String s) [0x00007] in <d11f4a646d2740e6abe2a57f0492bc9b>:0 
  at Python.Runtime.Runtime.Initialize () [0x000d2] in <d11f4a646d2740e6abe2a57f0492bc9b>:0 
  at Python.Runtime.PythonEngine.Initialize () [0x00011] in <d11f4a646d2740e6abe2a57f0492bc9b>:0 
  at FranticX.Scripting.PythonNetScriptEngine.Initialize (System.String home, System.String programName) [0x00037] in <be5ef32d504b49f49728bce41a227812>:0 
PyUnicodeUCS2_FromUnicode

We would appreciate input from the Deadline team to help solve this issue which prevents us from scaling Deadline as effectively as possible.

Thank you, have a great day!

Hello muxro,

Deadline has various dependencies that need to be fulfilled for everything to work. A while back I posted a two-part blog article on running Deadline in containers. The article was geared towards a prior version of Deadline, but the concepts should translate to the latest versions (perhaps with slight modifications).

Running Deadline in Containers - Part 1
Running Deadline in Containers - Part 2

The articles might provide some clues. If you could share your Dockerfiles, we can try to reproduce any issues. If you prefer to share them privately, you can attach them to a Support Ticket.

Hello Coulter, thanks for replying, here are the files I used for building the container

Dockerfile:

FROM mono

RUN apt-get update && apt-get install -y libpython2.7

ADD deadline-bins-10.0.3.2-linux.tar.gz /deadline-bins
ADD deadline.ini /var/lib/Thinkbox/Deadline10/

RUN sed -i 's/so/so.1/g' /deadline-bins/Python.Runtime.dll.config

deadline.ini:

[Deadline]
NetworkRoot=/deadline-repo

deadline-bins-10.0.3.2-linux.tar.gz is an archive containing the deadline binaries, extracted from \bin\Linux\bin.zip

I only needed this to get access to deadlinewebservice, so I didn’t see any point in installing libX and libGL.
I don’t think it would have mattered anyway, since the runtime stopped at initializing python as I’ve explained in my first post.

The differences between my approach and yours is that you used the deadline installer while I did not, and you use the centos image, while I use the mono image. Perhaps there’s a different version of python that ships with centos.

I would prefer to use a debian-based image, but deadline does not appear to have been tested on debian distros.
I’ll try your approach and get back to you. If it only works on centos then i’ll stick to that until a compatible version is released.
Since deadline seems to be moving to cloud-based environments, are you planning to release updated docker images for deadline?

Edit:
I’ve just noticed the github repository with the ubuntu-based docker image example. I’ll give that a try.

Awesome! I got it to work with the example Dockerfile from here:
https://github.com/ThinkboxSoftware/Deadline/blob/master/DevOps/Docker/BasicDockerfiles/ubuntu1404_client8/Dockerfile

I just changed the image base to debian:jessie and copied a deadline 10 installer.

Glad to hear it! Happy containerizing!

Hi,

Seem like these two links now become 404. I am new to Deadline, would you please able to guide me on how to start docker for deadline please?

I am interested in develop some custom automation process with Deadline and would like to see how can I use docker for this please.

Thank you.

@Coulter

Same question but 3 years later, any update about broken links.

I am also attempting to build my own Deadline docker container.

Cheers

Here’s a Stack I made recently. Works with Deadline 10.1.23.6, Mongo 4.2.12 and DotNet dotnet-runtime-3.1.

Probably works with newer stuff, but my Synology NAS can’t run Mongo 5. So this is far as I go until I upgrade hardware.

services:
    mongodb:
        container_name: mongodb_${MONGO_VERSION}_${MONGO_PORT}
        image: mongo:${MONGO_VERSION}
        volumes:
            - mongo_data:/data/db
        ports:
            - ${MONGO_PORT}:27017

    ubuntu:
        container_name: deadline_${DEADLINE_VERSION}_${DEADLINE_PORT}
        image: ubuntu:20.04
        volumes:
            - /volume1/Repositories/deadline/:/opt/Thinkbox
            - /volume1/Repositories/Software/Deadline/${DEADLINE_VERSION}:/repo
        ports:
            - ${DEADLINE_PORT}:8080
        command: bash -c "
            apt-get update; \
            apt-get install -y apt-transport-https libssl-dev tzdata wget; \
            wget -q https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb; \
            dpkg -i packages-microsoft-prod.deb; \
            apt-get update; \
            apt-get install -y ${DOTNET_VERSION}; \
            echo 'Installing Repository...'; \
            /repo/DeadlineR*.run --mode unattended --installmongodb false --dbhost mongodb_${MONGO_VERSION}_${MONGO_PORT} --dbport ${MONGO_PORT}; \
            echo 'Installing Client...'; \
            /repo/DeadlineC*.run --mode unattended --licensemode LicenseFree --connectiontype Repository --slavestartup false --launcherdaemon false; \
            /opt/Thinkbox/Deadline10/bin/deadlinercs.exe
            "
volumes:
    mongo_data:
1 Like

Thank you @David_Abbott

Is the dotnet-runtime a new requirement ? I don’t see it mentioned for repo nor client

Cheers

Privacy | Site terms | Cookie preferences