Deadline / Fusion / Shotgrid : Draft error

spotsk · November 22, 2023, 10:44am

Hi,

I’m pretty new to deadline,
I’m evaluating setting up a renderfarm with Fusion + Nuke workstations and a Shotgrid account .
So far I can submit Fusion comps to deadline and I get rendered files,
and I can connect to SG using the pipeline tool button of the fusion submitter with no error.

But I can’t get deadline creating the shotgun_h264 and shotgun_filmstrip files.

when exploring the job report I can guess something is wrong about the file name parsing:

2023-11-22 11:19:37:  0: STDOUT: Draft 1.8.3
2023-11-22 11:19:37:  0: STDOUT: inFile=/Volumes/SnowKitty/Fusion/out/comp_????.jpg

which looks correct, my test comp files are rendered as “comp_####.jpg”
but then a few lines later:

2023-11-22 11:19:37:  0: STDOUT:   File "/Users/Shared/Thinkbox/Deadline10/workers/fab-workstation/jobsData/655dd4d25349ca19ef58ba78/DraftCreateSimpleMovie.py", line 26, in <module>
2023-11-22 11:19:37:  0: STDOUT:     frame = Draft.Image.ReadFromFile( inFile )
2023-11-22 11:19:37:  0: STDOUT: RuntimeError: Magick: UnableToOpenBlob `/Volumes/SnowKitty/Fusion/out/comp_????0000.jpg': No such file or directory @ error/blob.c/OpenBlob/2960

Obviously this looks weird: comp_???0000.jpg

Any idea ?
Thanks for your help !

I’m running deadline 10.3.0.15, Fusion Studio 17.4.6, everything local on a mac running OS X catalina 10.15.7 (intel)

spotsk · November 22, 2023, 2:29pm

replying to myself… looks like a python version mismatch

I did trash and reinstall everything (repo, client, submitters…)
rendering still works, connection to sg works
but Draft does not render sg_movies or filmstrips.
Now the error looks different: python 3.7 is missing

STDOUT: /Applications/Thinkbox/Deadline10/Resources/python3/python: line 5: /Applications/Thinkbox/Deadline10/Resources/python3/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python: No such file or directory

which is true, Python.framework/Versions/ contains only a python 3.10 directory, no more 3.7

but a few lines earlier I can see

INFO: Setting Process Environment Variable PYTHONPATH to[...]/Applications/Thinkbox/Deadline10/Resources/python3/Frameworks/Python.framework/Versions/Current/lib/python3.10

so the current python version 3.10 is detected…

also this line -which seems to launch the actual movie creation- looks correct:

INFO: Full Command: "/Applications/Thinkbox/Deadline10/Resources/python3/python" -u "/Users/Shared/Thinkbox/Deadline10/workers/fab-workstation/jobsData/655e087d9a2cc6ec36ffdfdf/DraftCreateSimpleMovie.py" frameList=1-11 startFrame=1 endFrame=11 inFile="/Volumes/SnowKitty/nuke/out.####.jpg" outFile="/Volumes/SnowKitty/nuke/Draft/shotgun_h264.mov" outFolder="/Volumes/SnowKitty/nuke/Draft" deadlineJobID=655e086da7697f7e9040753e deadlineRepository="/Volumes/SnowKitty/Thinkbox/Deadline10" taskStartFrame=1 taskEndFrame=11

… but refers to this file:
/Applications/Thinkbox/Deadline10/Resources/python3/python
which is a shell script… with python 3.7 hardcoded ?!

#!/bin/sh
cd `dirname "$0"` > /dev/null
SHELL_PATH=`pwd -P`
export DYLD_FRAMEWORK_PATH=${SHELL_PATH}/Frameworks:${DYLD_FRAMEWORK_PATH}
exec "${SHELL_PATH}/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python" "$@"
cd - > /dev/null

So I’m a bit confused… Draft is meant to run on python 3.7 or 3.10 ?

spotsk · November 22, 2023, 4:07pm

Ok some more testing…
I edited /Applications/Thinkbox/Deadline10/Resources/python3/python so it points to python 3.10

exec ${SHELL_PATH}/Frameworks/Python.framework/Versions/Current/Resources/Python.app/Contents/MacOS/Python" "$@"

and now when submitting from fusion I’m back to the previous error:

STDOUT: inFile=/Volumes/SnowKitty/Fusion/out/comp_????.jpg
...
STDOUT: RuntimeError: Magick: UnableToOpenBlob `/Volumes/SnowKitty/Fusion/out/comp_????0000.jpg': No such file or directory @ error/blob.c/OpenBlob/2960

This looks to me like a bug in the Fusion submitter ?

good news is when I try rendering from nuke with the same params it works, I get a shotgun_h264 and a shotgun_filmstrip created.

STDOUT: inFile=/Volumes/SnowKitty/nuke/out.####.jpg
...
STDOUT: Encoding Progress: 100%

But then it fails uploading to sg…

Justin_B · November 24, 2023, 7:49pm

If you save your output files with # as the stand-in for frame numbers instead of ? Draft will correctly replace it with frame numbers.

The reason it’s attempting /Volumes/SnowKitty/Fusion/out/comp_????0000.jpg is that if we don’t find any # characters we just put the frame number before the file extension.

Issue is that our Fusion submitter is using ? for some reason. I’ve attached a version that uses # instead that you can try - there may be a good reason ? is used as padding.

To install:

Make a copy of DeadlineRepository10\submission\Fusion in DeadlineRepository10\custom\submission.
Replace the existing DeadlineRepository10\custom\submission\Fusion\Main\SubmitToDeadlineMonitor.lua with the attached file.

To rollback:

Delete the DeadlineRepository10\custom\submission\Fusion directory you created in the above step 1.

SubmitToDeadlineMonitor.7z (4.2 KB)

Let me know if that sorts it out for you, or if this causes other issues.

spotsk · November 25, 2023, 3:27pm

Thanks ! it works !

In the meantime I was poking around the DraftParamParse.py code to make it works with the “???” padding generated by the Fusion submitter but your solution is much simpler and cleaner.

I have a few more questions:

I can’t get the movie uploaded to shotgun. The version is created on sg (so I guess the auth works), the movie is created locally but then the movie is not uploaded, nor the filmstrip or thumbnail. And I can’t find anything about that in the job reports or which script is involved in the process.

also:
could you confirm that the shell script Deadline10/Resources/python3/python can be tweaked to work with python 3.10 without further issues ? or is it safer to revert to an older Deadline with python 3.7 ?

And last question:
what should I modify to make the Fusion version a sticky setting in the Fusion submitter ? it always reverts to Fu 5 which is 20 years old…

thanks for your help !

Justin_B · November 27, 2023, 7:11pm

If you look in the job reports, there should be entries where the plugin is ‘Shotgun’, those should include details on why the movie is or isn’t getting uploaded. If there’s nothing from Shotgun, then make sure the event plugin is enabled in the Monitor under Tools → Configure Events → Shotgun.

I’d change 3.7 to Current instead, as we make a symlink to point Current to the 3.10 folder as a part of install on MacOS.

Is this in the integrated submitter (the one in fusion not the Monitor)? Looks like we query Fusion for its version number so that should be working. If it’s the Monitor submitter it should be retaining whatever’s set from the last submission.