Draft failing due to 'wrong' frame padding

[celluloidvfx]

Hi,

we just had some jobs that had all dependent Draft jobs fail due to them looking for frames with a 7-digit frame number instead of the usual 4 digits with our VRay renders. The strange thing is that in the Job Report it shows

inFile=Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\wc02_pac_001_010_rndr_product-pine-nl_v001_an.RGB_color.0000.exr

Which is correct. But a few lines later it says

2015-08-28 19:29:02: 0: STDOUT: Traceback (most recent call last): 2015-08-28 19:29:02: 0: STDOUT: File "C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-RS-17\jobsData\55e09a12499b5f1404fc1fd9\Draft_CreateShotgunMovie.py", line 24, in <module> 2015-08-28 19:29:02: 0: STDOUT: frame = Draft.Image.ReadFromFile( inFile ) 2015-08-28 19:29:02: 0: STDOUT: RuntimeError: exr_input_file Error: failed to open exr file Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\wc02_pac_001_010_rndr_product-pine-nl_v001_an.RGB_color.00000000.exr 2015-08-28 19:29:03: 0: INFO: Process exit code: 1
which shows that it’s suddenly looking for a file with 7 digits.

Here’s the complete log for reference

[code]=======================================================
Error

Error: Renderer returned non-zero error code, 1. Check the log for more information.
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type

RenderPluginException

=======================================================
Stack Trace

at Deadline.Plugins.Plugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.RenderCurrentTask(TaskLogWriter tlw)

=======================================================
Log

2015-08-28 19:29:01: BEGIN - CELL-RS-17\render
2015-08-28 19:29:01: 0: Plugin will be reloaded because a new job has been loaded, or one of the job files or plugin files has been modified
2015-08-28 19:29:01: 0: Loaded plugin Draft (\cell-dead-01\deadlinerepo\plugins\Draft)
2015-08-28 19:29:01: 0: Start Job timeout is disabled.
2015-08-28 19:29:01: 0: Task timeout is disabled.
2015-08-28 19:29:01: 0: Loaded job: wc02_pac_001_010_rndr_product-pine-nl_v001_an [Shotgun H264 Movie Creation] (55e09a12499b5f1404fc1fd9)
2015-08-28 19:29:01: 0: INFO: Executing plugin script C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-RS-17\plugins\55e09a12499b5f1404fc1fd9\Draft.py
2015-08-28 19:29:02: 0: INFO: Found Draft python module at: ‘C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-RS-17\Draft\Draft.pyd’
2015-08-28 19:29:02: 0: INFO: About: Draft Plugin for Deadline
2015-08-28 19:29:02: 0: INFO: The job’s environment will be merged with the current environment before rendering
2015-08-28 19:29:02: 0: Plugin rendering frame(s): 1
2015-08-28 19:29:02: 0: INFO: Draft job starting…
2015-08-28 19:29:02: 0: INFO: Stdout Redirection Enabled: True
2015-08-28 19:29:02: 0: INFO: Stdout Handling Enabled: True
2015-08-28 19:29:02: 0: INFO: Popup Handling Enabled: False
2015-08-28 19:29:02: 0: INFO: Using Process Tree: True
2015-08-28 19:29:02: 0: INFO: Hiding DOS Window: True
2015-08-28 19:29:02: 0: INFO: Creating New Console: False
2015-08-28 19:29:02: 0: INFO: Running as user: render
2015-08-28 19:29:02: 0: INFO: Looking for bundled python at: ‘C:\Program Files\Thinkbox\Deadline7\bin\dpython.exe’
2015-08-28 19:29:02: 0: INFO: Executable: “C:\Program Files\Thinkbox\Deadline7\bin\dpython.exe”
2015-08-28 19:29:02: 0: INFO: Argument: -u “C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-RS-17\jobsData\55e09a12499b5f1404fc1fd9\Draft_CreateShotgunMovie.py” frameList=0 startFrame=0 endFrame=0 inFile=“Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\wc02_pac_001_010_rndr_product-pine-nl_v001_an.RGB_color.0000.exr” outFile=“Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\Draft\shotgun_h264.mov” outFolder=“Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\Draft” deadlineJobID=55e094b874fc321b843f864d deadlineRepository="\cell-dead-01\deadlinerepo" taskStartFrame=1 taskEndFrame=1
2015-08-28 19:29:02: 0: INFO: Startup Directory: “C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-RS-17\Draft”
2015-08-28 19:29:02: 0: INFO: Process Priority: BelowNormal
2015-08-28 19:29:02: 0: INFO: Process Affinity: default
2015-08-28 19:29:02: 0: INFO: Process is now running
2015-08-28 19:29:02: 0: STDOUT: Checking for a license…
2015-08-28 19:29:02: 0: STDOUT: Draft 1.4.0.59071
2015-08-28 19:29:02: 0: STDOUT: Command line args:
2015-08-28 19:29:02: 0: STDOUT: frameList=0
2015-08-28 19:29:02: 0: STDOUT: startFrame=0
2015-08-28 19:29:02: 0: STDOUT: endFrame=0
2015-08-28 19:29:02: 0: STDOUT: inFile=Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\wc02_pac_001_010_rndr_product-pine-nl_v001_an.RGB_color.0000.exr
2015-08-28 19:29:02: 0: STDOUT: outFile=Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\Draft\shotgun_h264.mov
2015-08-28 19:29:02: 0: STDOUT: outFolder=Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\Draft
2015-08-28 19:29:02: 0: STDOUT: deadlineJobID=55e094b874fc321b843f864d
2015-08-28 19:29:02: 0: STDOUT: deadlineRepository=\cell-dead-01\deadlinerepo
2015-08-28 19:29:02: 0: STDOUT: taskStartFrame=1
2015-08-28 19:29:02: 0: STDOUT: taskEndFrame=1
2015-08-28 19:29:02: 0: STDOUT: Traceback (most recent call last):
2015-08-28 19:29:02: 0: STDOUT: File “C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-RS-17\jobsData\55e09a12499b5f1404fc1fd9\Draft_CreateShotgunMovie.py”, line 24, in
2015-08-28 19:29:02: 0: STDOUT: frame = Draft.Image.ReadFromFile( inFile )
2015-08-28 19:29:02: 0: STDOUT: RuntimeError: exr_input_file Error: failed to open exr file Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\wc02_pac_001_010_rndr_product-pine-nl_v001_an.RGB_color.00000000.exr
2015-08-28 19:29:03: 0: INFO: Process exit code: 1
2015-08-28 19:29:04: 0: An exception occurred: Error: Renderer returned non-zero error code, 1. Check the log for more information.
2015-08-28 19:29:04: at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel) (Deadline.Plugins.RenderPluginException)

=======================================================
Details

Date: 08/28/2015 19:29:04
Frames: 1
Elapsed Time: 00:00:00:06
Job Submit Date: 08/28/2015 19:27:46
Job User: alexn
Average RAM Usage: 4620102144 (7%)
Peak RAM Usage: 4621008896 (7%)
Average CPU Usage: 2%
Peak CPU Usage: 3%
Used CPU Clocks: 69146
Total CPU Clocks: 3457280

=======================================================
Slave Information

Slave Name: CELL-RS-17
Version: v7.2.0.14 R (dbc900863)
Operating System: Windows 7 Professional (SP1)
Running As Service: No
Machine User: render
IP Address: 192.168.1.117
MAC Address: AC:22:0B:0B:CE:93
CPU Architecture: x64
CPUs: 32
CPU Usage: 1%
Memory Usage: 4.3 GB / 63.9 GB (6%)
Free Disk Space: 789.130 GB (335.704 GB on C:, 453.427 GB on D:)
Video Card: Standard VGA Graphics Adapter
[/code]

Cheers,
Holger

[Conrad_Wiebe]

I am taking a look at the bug now. Can you tell me what your using for shotgun’s frame padding character? I have a feeling this might be related to our latest changes.

What is this setting in your repository:
Deadline Monitor > Tools > Configure Events > Shotgun > Frame Padding Character

[celluloidvfx]

Hi Conrad,

this is set to

%0#d

as this is the only thing that works for us in the combination Deadline/shotgun/RV.

Cheers,
Holger

[Conrad_Wiebe]

The problem appears to lie in the fact that:

inFile=Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\wc02_pac_001_010_rndr_product-pine-nl_v001_an.RGB_color.0000.exr

should actually have # symbols instead of the “0000” numbers. It should look like this:

inFile=Y:\wc02\pac_001_010\rndr\wc02_pac_001_010_rndr_product-pine-nl_v001_an\1920x1080_exr\wc02_pac_001_010_rndr_product-pine-nl_v001_an.RGB_color.####.exr

I’m not sure why it has numbers in your case, but I will investigate.

[Conrad_Wiebe]

Can you check something for me?

-In the Deadline Monitor, find the SOURCE job (After Effects, Nuke, Maya, or whatever program you’re using).
-Right-click and select “Modify Job Properties”
-Examine “Submission Params” > “Job Info Parameters” > “OutputFilename0”. What is the filename?
-It should be in the form of “myfilename####.exr”. I suspect in your case it is “myfilename0000.exr” (the zeros are what is causing this problem).

Is that the case?
Which program are you using to create the source images?

[celluloidvfx]

Hi Conrad,

Examine “Submission Params” > “Job Info Parameters” > “OutputFilename0”. What is the filename?

OutputFilename0=wc02_pac_001_010_rndr_product-pine-nl_v001_an.RGB_color.0000.exr

It’s a 3ds Max job submitted from SMTD within 3ds Max.
In this case i should probably mention that we’re still on 7.2.0.14 but using a modified/updated SMTD by Bobo from here:
https://forums.thinkboxsoftware.com/viewtopic.php?f=205&t=13681&p=61962#p61911

Cheers,
Holger

[MikeOwen]

Was it a static (still) frame that was submitted for this job instead of an animation (frame count > 1)? SMTD is coded to fix the frame number: “0000” instead of “####” if it’s a static frame for VRay VFB raw/split as it doesn’t accept any padding characters. I remember spending a lot of time dealing with the almost infinite combinations of files that can be created here, using the different SMTD workflows. I’ll have another look shortly (currently at IBC).

[celluloidvfx]

It was indeed a still (frame 0).
I attached a zip with the 3ds job and one of the Draft jobs in case this helps.
archive.zip (27 MB)

[Conrad_Wiebe]

Ok I see. I suppose it is currently a bug that still images (rather than sequences) aren’t being handled correctly in the 3dmax to Draft workflow. We will take a look at what can be done on our end.

[celluloidvfx]

Hi Conrad,

we just had another Draft error which might be related - at least partially.
I submitted a job to render a Quicktime (DNxHD 36) from Hiero. Accidentally, i also switched activated a dependent Draft job which was set to render another Quicktime. The full error log of that Draft job is below. Why i think it’s related to this issue is the part at the end of the log where it says that the input filename is supposed to be

X:\dngl\_input\2015-09-04_001\150903_fuer_Abnahme_dnxhd360000.mov

But the ‘0000’ before the ‘.mov’ is of course wrong and looks like it treats it as a still frame with frame number ‘0’.
Do you think the error message

[...] cannot have a movie-type extension, please use the Draft.VideoDecoder class instead.

is related to this? Or is this another bug?

Cheers,
Holger

[code]=======================================================
Error

Error: Renderer returned non-zero error code, 1. Check the log for more information.
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type

RenderPluginException

=======================================================
Stack Trace

at Deadline.Plugins.Plugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.RenderCurrentTask(TaskLogWriter tlw)

=======================================================
Log

2015-09-15 15:48:50: BEGIN - CELL-WS-12\render
2015-09-15 15:48:50: 0: Plugin will be reloaded because a new job has been loaded, or one of the job files or plugin files has been modified
2015-09-15 15:48:50: 0: Loaded plugin Draft (\cell-dead-01\deadlinerepo\plugins\Draft)
2015-09-15 15:48:50: 0: Start Job timeout is disabled.
2015-09-15 15:48:50: 0: Task timeout is disabled.
2015-09-15 15:48:50: 0: Loaded job: 150903_fuer_Abnahme_dnxhd36 [Quick Draft] (55f821bc935b251190c04f2d)
2015-09-15 15:48:50: 0: INFO: Executing plugin script C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-WS-12\plugins\55f821bc935b251190c04f2d\Draft.py
2015-09-15 15:48:50: 0: INFO: Found Draft python module at: ‘C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-WS-12\Draft\Draft.pyd’
2015-09-15 15:48:50: 0: INFO: About: Draft Plugin for Deadline
2015-09-15 15:48:50: 0: INFO: The job’s environment will be merged with the current environment before rendering
2015-09-15 15:48:50: 0: Plugin rendering frame(s): 1-133494
2015-09-15 15:48:50: 0: INFO: Draft job starting…
2015-09-15 15:48:50: 0: INFO: Stdout Redirection Enabled: True
2015-09-15 15:48:50: 0: INFO: Stdout Handling Enabled: True
2015-09-15 15:48:50: 0: INFO: Popup Handling Enabled: False
2015-09-15 15:48:50: 0: INFO: Using Process Tree: True
2015-09-15 15:48:50: 0: INFO: Hiding DOS Window: True
2015-09-15 15:48:50: 0: INFO: Creating New Console: False
2015-09-15 15:48:50: 0: INFO: Running as user: render
2015-09-15 15:48:50: 0: INFO: Looking for bundled python at: ‘C:\Program Files\Thinkbox\Deadline7\bin\dpython.exe’
2015-09-15 15:48:50: 0: INFO: Executable: “C:\Program Files\Thinkbox\Deadline7\bin\dpython.exe”
2015-09-15 15:48:50: 0: INFO: Argument: -u “C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-WS-12\jobsData\55f821bc935b251190c04f2d\DraftCreateMovie.py” resolution=“1” codec=“MJPEG” quality=“75” frameRate=“24” frameList=0-133493 startFrame=0 endFrame=133493 inFile=“X:\dngl_input\2015-09-04_001\150903_fuer_Abnahme_dnxhd36.mov” outFile=“X:\dngl_input\2015-09-04_001\Draft\150903_fuer_Abnahme_dnxhd36.mov” outFolder=“X:\dngl_input\2015-09-04_001\Draft” deadlineJobID=55f7fd4274fc320e0c9ee7d3 deadlineRepository="\cell-dead-01\deadlinerepo" taskStartFrame=1 taskEndFrame=133494
2015-09-15 15:48:50: 0: INFO: Startup Directory: “C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-WS-12\Draft”
2015-09-15 15:48:50: 0: INFO: Process Priority: BelowNormal
2015-09-15 15:48:50: 0: INFO: Process Affinity: default
2015-09-15 15:48:50: 0: INFO: Process is now running
2015-09-15 15:48:51: 0: STDOUT: Checking for a license…
2015-09-15 15:48:51: 0: STDOUT: Draft 1.4.0.59071
2015-09-15 15:48:51: 0: STDOUT: Command line args:
2015-09-15 15:48:51: 0: STDOUT: resolution=1
2015-09-15 15:48:51: 0: STDOUT: codec=MJPEG
2015-09-15 15:48:51: 0: STDOUT: quality=75
2015-09-15 15:48:51: 0: STDOUT: frameRate=24
2015-09-15 15:48:51: 0: STDOUT: frameList=0-133493
2015-09-15 15:48:51: 0: STDOUT: startFrame=0
2015-09-15 15:48:51: 0: STDOUT: endFrame=133493
2015-09-15 15:48:51: 0: STDOUT: inFile=X:\dngl_input\2015-09-04_001\150903_fuer_Abnahme_dnxhd36.mov
2015-09-15 15:48:51: 0: STDOUT: outFile=X:\dngl_input\2015-09-04_001\Draft\150903_fuer_Abnahme_dnxhd36.mov
2015-09-15 15:48:51: 0: STDOUT: outFolder=X:\dngl_input\2015-09-04_001\Draft
2015-09-15 15:48:51: 0: STDOUT: deadlineJobID=55f7fd4274fc320e0c9ee7d3
2015-09-15 15:48:51: 0: STDOUT: deadlineRepository=\cell-dead-01\deadlinerepo
2015-09-15 15:48:51: 0: STDOUT: taskStartFrame=1
2015-09-15 15:48:51: 0: STDOUT: taskEndFrame=133494
2015-09-15 15:48:51: 0: STDOUT: Traceback (most recent call last):
2015-09-15 15:48:51: 0: STDOUT: File “C:\Users\render.CELLULOIDVFX\AppData\Local\Thinkbox\Deadline7\slave\CELL-WS-12\jobsData\55f821bc935b251190c04f2d\DraftCreateMovie.py”, line 34, in
2015-09-15 15:48:51: 0: STDOUT: frame = Draft.Image.ReadFromFile( currInFile )
2015-09-15 15:48:51: 0: STDOUT: RuntimeError: The provided filename “X:\dngl_input\2015-09-04_001\150903_fuer_Abnahme_dnxhd360000.mov” cannot have a movie-type extension, please use the Draft.VideoDecoder class instead.
2015-09-15 15:48:52: 0: INFO: Process exit code: 1
2015-09-15 15:48:53: 0: An exception occurred: Error: Renderer returned non-zero error code, 1. Check the log for more information.
2015-09-15 15:48:53: at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel) (Deadline.Plugins.RenderPluginException)
[/code]

[Conrad_Wiebe]

These two issues are not really related. Technically, the second issue is because Quick Draft only takes image files as input, and since your input was a video file, it failed.

The addition of the “0000” is for the same reason, however it doesn’t really matter in this case. We are going to clean up the original problem and make sure it’s not an issue in the future. Thanks for pointing this out though.

[celluloidvfx]

Hi guys,

just wanted to ask whether there has been any progress on the 3ds Max still frame -> Draft workflow? We recently upgraded to 7.2.1.10 and Draft jobs of still images are still failing. This is pretty annyoing as we have quite a lot of those and it would be really great to have the proper images uploaded to shotgun. If there’s no ‘official’ solution yet, is there maybe a workaround we could use for the time being?

Cheers,
Holger

[Conrad_Wiebe]

Actually, yes. This has been fixed internally. Unfortunately the fix is for Deadline 8, but we can probably give you a 7.2 patch.

There is a workaround, and that is:
-Under “Render Setup”, you’re currently using 3dsmax’s “Single” frame mode.

Instead, switch it to “Range”, and set the “Start Frame” and “End Frame” to the same value (probably 0 to 0 in your case).

Is that workaround sufficient for you? I believe it should work.

[celluloidvfx]

Hi Conrad,

yes, if that fix could be released for 7.2 that would be awesome!
I heard there might be one more 7.2 release. Maybe it’s possible to include it with that one?

The workaround you’re proposing might do it in the meantime. Probably needs the artists to get used to it but i will definitely ask them to try this.

Thanks and cheers,
Holger

[Conrad_Wiebe]

Excellent. We can potentially add it to 7.2.2. I will discuss it internally.

For now though, using the frame range thing should work as well.

[celluloidvfx]

Is the fix just a change in the SMTD or Draft script that we can maybe just put into the ‘custom’ folder of our repo? If yes, could you maybe post it here or send it to me via PM? Your suggested workaround does the trick but it just involves the artists having to remember to send the jobs with Frame Range 0 to 0 instead of ‘Still Frame’ (which is what they’re used to) so i see some potential for human mistakes here

Cheers,
Holger

[Conrad_Wiebe]

It’s a very simple change in SMTD. You can even fix it yourself with a find-and-replace. Here’s now:

Open [deadline-repo-root]\submission\3dsmax\Main\SubmitMaxToDeadline_Functions.ms

Find:

if rendTimeType == 1 OR (findItem #(#singleFrameTiles, #singleFrameMultiRegion) SMTDSettings.RegionRenderingMode > 0) do

Replace with:

if (findItem #(#singleFrameTiles, #singleFrameMultiRegion) SMTDSettings.RegionRenderingMode > 0) do

There should be 8 occurrences to replace.