Subprocess error: OSError: [Errno 2] No such file or directory - for houdini submitter rop in Houdini 18

Andrew_Graham · July 4, 2021, 6:28am

I’m installing Deadline for Houdini 18.5.563 in Ubuntu for the first time. I have the deadline monitor functioning as expected. But I get this error when dropping down the deadline rop for the first time.

I’m guessing it could be a permissions error, but the traceback doesn’t say exactly what file is missing.

Traceback (most recent call last):
  File "Driver/deadline, opdef:/Driver/deadline?PreFirstCreate", line 13, in <module>
  File "/home/user/houdini18.5/python2.7libs/CallDeadlineCommand.py", line 42, in CallDeadlineCommand
    proc = subprocess.Popen(arguments, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, startupinfo=startupinfo, creationflags=creationflags)
  File "/opt/hfs18.5.563/python/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/opt/hfs18.5.563/python/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

In a houdini python shell, all the env vars seem correct.

>>> import os
>>> os.getenv('DEADLINE_SUBMITTER_DIR')
'/home/user/Thinkbox/Deadline10/submitters/HoudiniSubmitter'
>>> os.getenv('HOUDINI_PATH')
'/opt/hfs18.5.563/packages/viewer_handle_demo:/opt/hfs18.5.563/packages/kinefx:/home/user/Thinkbox/Deadline10/submitters/HoudiniSubmitter:&'
>>> os.getenv('DEADLINE_SUBMITTER_DIR')
'/home/user/Thinkbox/Deadline10/submitters/HoudiniSubmitter'
>>>

Andrew_Graham · July 4, 2021, 6:46am

ahh odd, so if you run the installer in ubuntu, it will prompt that you need to install it with sudo. but if you use sudo to install, you will end up with permissions like this:

user@z8:~/houdini18.5$ ll
total 140
drwxrwxr-x  8 user user  4096 Jul  4 15:46 ./
drwxr-xr-x 29 user user  4096 Jul  4 15:40 ../
drwxrwxr-x  7 user user  4096 Jul  4 13:13 asset_store/
drwxrwxr-x  5 user user  4096 Jun 27 19:39 config/
-rw-rw-r--  1 user user    15 Jul  4 16:09 desktops.pref
-rw-rw-r--  1 user user    70 Jul  4 16:09 dir.history
-rw-rw-r--  1 user user  5622 Jul  4 13:13 dso.cache
-rw-rw-r--  1 user user   520 Jul  4 16:09 file.history
-rw-rw-r--  1 user user 38394 Jul  4 13:13 font.cache
-rw-rw-r--  1 user user   161 Jul  4 16:09 hcommon.pref
-rw-rw-r--  1 user user  1232 Jul  4 16:09 hcomposite.pref
-rw-rw-r--  1 user user    29 Jul  4 15:46 hotkeys.pref
-rw-rw-r--  1 user user   358 Jun 27 19:37 houdini.env
-rw-rw-r--  1 user user  4858 Jul  4 16:09 houdini.pref
-rw-rw-r--  1 user user  4401 Jul  4 16:09 networkeditor.pref
-rw-rw-r--  1 user user    62 Jul  4 16:09 opencl.pref
drwxr-xr-x  2 root root  4096 Jul  4 15:46 packages/
drwxr-xr-x  2 root root  4096 Jul  4 15:46 python2.7libs/
drwx------  2 user user  4096 Jun 27 19:38 QtWebEngine/
-rw-rw-r--  1 user user   399 Jul  4 16:09 solaris.pref
drwxrwxr-x  2 user user  4096 Jun 27 19:39 toolbar/
-rw-rw-r--  1 user user     0 Jul  4 16:09 tool.pref
-rw-rw-r--  1 user user   348 Jul  4 16:09 ui.pref
-rw-rw-r--  1 user user    89 Jul  4 16:09 wingeo.pref

If the installer expects sudo permissions, should it prompt for the user name to install as?

Andrew_Graham · July 4, 2021, 9:29am

… but changing ownership did not seem to fix this. i still get this after changing the current user to own everything in /opt/Thinkbox or in $HOME/houdini18.5 recursively.

Traceback (most recent call last):
  File "Driver/deadline, opdef:/Driver/deadline?PreFirstCreate", line 13, in <module>
  File "/home/user/houdini18.5/python2.7libs/CallDeadlineCommand.py", line 42, in CallDeadlineCommand
    proc = subprocess.Popen(arguments, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, startupinfo=startupinfo, creationflags=creationflags)
  File "/opt/hfs18.5/python/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/opt/hfs18.5/python/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Any ideas?

Tanguy_BODIVIT · September 26, 2021, 10:47pm

Any luck, I have the same issues, really annoying.
And not really a pro active feedback from the dev team

mois.moshev · September 27, 2021, 6:46am

This means it is not finding deadlinecommand. For instance we have it at /opt/Thinkbox/Deadline10/bin.

martin_acht · September 28, 2021, 10:28am

we have the same issues with hfs18.5.672 on a linux worker Centos 7.7
same error
here is my thread, didnt come any further.
I think it might be a Thinkbox/houdini/linux python issue of some sorts for the 18.5 version

here is the thread:

Andrew_Graham · September 29, 2021, 12:06am

Whilst it is very annoying that installers are not configuring correct permissions at this time, I believe I have something working:

you can see where I set various permissions in my install script (you’ll need to look at the full file where I set chown and chmod)

github.com

firehawkvfx/aws-thinkbox-deadline/blob/main/install-deadline

#!/bin/bash

# This installs certificates with the DB or generate certs after DB for RCS.

set -e

readonly DEFAULT_DEADLINEUSER_NAME="deadlineuser"
readonly DEFAULT_DBPORT="27100"
readonly DEFAULT_PROXYPORT="4433"
readonly DEFAULT_HTTPPORT="8080"
readonly DEFAULT_DOWNLOAD_DIR="/var/tmp/downloads"
# readonly DEFAULT_DB_HOST_NAME="deadlinedb.service.consul"
readonly DEFAULT_DEADLINE_PROXY_CERTIFICATE="Deadline10RemoteClient.pfx"
readonly DEFAULT_DEADLINE_CLIENT_CERTIFICATE="Deadline10Client.pfx"
readonly DEFAULT_MONGO_URL="https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1604-3.6.19.tgz"
readonly DEFAULT_DOWNLOAD_INSTALLERS="true"

# Functions
function print_usage {
  echo

This file has been truncated. show original

I also set permissions after install along these lines:

chown $deadlineuser_name:$deadlineuser_name /opt/Thinkbox/
chown $deadlineuser_name:$deadlineuser_name /opt/Thinkbox/certs/
chown -R $deadlineuser_name:$deadlineuser_name /opt/Thinkbox/Deadline10/bin/pythonsync/apiclient # This might be a bug that we have to do this

Another catch was that I forgot to reinstall the deadline plugin for houdini. it is important you do this when you update deadline on your workstation.

This is with H18.5.696 and Deadline 10.1.18.5.

My installation method is probably a bit different to what people are used to, so you may not be able to use exactly what I’m doing. I have a need to configure certificates on boot and not leave secrets in the image which is bad practice - unfortunately current default behavior for installation. I hope Thinkbox consider that need for immutable infrastructure in the future.