Hi,
I have an issue with frame dependent Nuke jobs and missing frames.
I have a Maya job that renders out a frame range, lets say 1-5. I then have a frame dependent Nuke job that will render out a new frame from Nuke based on the frame rendered from Maya. In my Nuke script the read node reads from where the Maya render is. The issue occurs when for example frame 2 from the Maya job is completed before frame 1, the Nuke job tries to render frame 2 (which by now exists), but still fails because frame 1 does not exist.
[10:43.23] ERROR: Read2: y:/pipeline_prototype_1/sequences/100/100_010/Comp/work/images/100_010_Comp_scene_v021/100_010_Comp_scene_v021_masterLayer.1.exr: Read error: No such file or directory
I can sort of get around it by enabling continue on error on the nuke job, but that can get confusing if there actually is an error, and it sets the task to complete.
Any ideas? Iâm on deadline 10.
Thanks a lot!
Jonas
Could you set the read nodeâs error handling to ânearest frameâ (or, in fact, anything other than âerrorâ) ?
Hi Dan, thanks for taking your time.
Well, the problem with that is that I still want the job to fail if the frame that it tries to render actually doesnât exist⌠I donât want to have tasks that are completed but are really just a rendered black/checkerboard frameâŚ
Ah, I see! But surely it only kicks off the render when the Maya task has finished, so itâll never try to render a frame that doesnât exist? Or are you concerned about Maya wigging out, not rendering an output but displaying as Complete and thus kicking off the dependent job?
How is the nuke comp made? Is it a standard one that has its input changed by an event, or is it created by the event? Either way, does it need to load in an image sequence? If each task is only ever rendering a single task, can it not load just the single frame in the read node?
(This obviously wonât work if youâre trying to do any pan-frame stuff like mblur or oflow, but then none of this will!)
Yes both Maya wigging out and there are some cases where we need to requeue the Maya tasks and then the dependent nuke tasks manually, and if we by mistake would select the wrong nuke frame or something silly like that (which could happen when done manually) we could potentially overwrite a frame that was good with a black one if the Maya frame is not there anymore. The idea is to remove the Maya frames after the Nuke job is complete since weâre really only interested in the frames from Nuke in the end.
Basically itâs a pre-made nuke script that has itâs input read and output write path replaced for each job. It would for sure not need to load any other frames then the one that that task is rendering, but iâm not sure how to accomplish that? Right now the read nodeâs first and last frame is set to first and last frame of the whole sequence that Maya will render.
The purpose of the Nuke job is just to convert Arnold rendered frames to multi-part exrs, since this is not supported by Arnold, so no mblur or oflow or things like that.
Well the start and end frames - in this case - are only relevant if youâre loading a padded sequence (âC:\renders\dog_####.exrâ or âC:\renders\dog_%04d.exrâ). How about if the read and write nodeâs input/output is replaced with simply a single, non-padded file? (ie âC:\reders\dog_0002.exrâ) Since each Nuke job has only a single frame to render, thereâs no need to involve the whole sequence in it.
Iâm working under the assumption that the nuke fileâs read and write nodes are edited manually rather than by hand.
Hmm, well all tasks are for the same job, ie using the same nuke script (though copied locally to the slave), are you suggesting to somehow dynamically change the input on render time for each task?
Correct, the Nuke file is never edited by hand.
Exactly. Or that each task has its own Nuke script thatâs generated as part of the submission process. Are you familiar with Events in Deadline? Theyâre Python scripts that get executed when X is done (and X ranges from a task being submitted to a job finishing to anything in between - there are many callbacks).
If you have a generic Nuke file thatâs set up how you want it, you could easily have a Deadline event copy and amend this nuke script to read and write from/to a single frame, and then submit this as a single-framed Nuke job. This event can be kicked off on task completion, so as soon as a Maya frame is done itâs executed, and it can get the file output path from the job too. It may take a little work to set up but once it is, itâd basically automatically convert all your outputs into .exrâs automatically without you needing to edit or copy anything anywhere - it would potentially be a bit of a mess in Monitor having so many jobs, but you could quite easily add all the Nuke jobs for a given Maya render to their own batch group so theyâre easy to ignore.
As a little proof of concept, I just wrote an incredibly simple bit of python thatâll take a nuke file, open it as text, look for âINPUT_HERE.exrâ and replace it with a filepath, ditto with âOUTPUT_HERE.exrâ and then save it out as a separate nuke file. This obviously relies on your âblankâ Nuke file (thatâs otherwise set up how you like it) having a read node with âINPUT_HERE.exrâ as the filepath and a write node having âOUTPUT_HERE.exrâ as the output - but once you have that new Nuke file, it could be automatically submitted without a problem by the same event script.
f = "C:\\tmp\\test.nk"
with open(f) as nf:
data = nf.read()
# print(data)
data = data.replace("INPUT_HERE.exr", "C:/file/here.exr")
data = data.replace("OUTPUT_HERE.exr", "C:/other/file.exr")
with open("C:\\tmp\\test2.nk", 'w+') as nf2:
nf2.write(data)
Edit: This is just one way. You could also import the nuke python module into that event script and generate the nuke script using Python rather than ASCII editing an existing file. This is likely âsaferâ but also a bit more effort to set up.
1 Like
Ah I see!
No I am not familiar with Events in Deadline, sounds like something I should definitely look in to!!
Part from the monitor becoming a bit messy as you mentioned, it would probably work the way I want. Iâm doing some similar things when setting up the Nuke script in the first place so that part is only a matter of inserting the correct frame number.
Thanks a lot for the input Dan!
1 Like