Nuke missing frame error

Deadline
1

Hi,

I have an issue with frame dependent Nuke jobs and missing frames.

I have a Maya job that renders out a frame range, lets say 1-5. I then have a frame dependent Nuke job that will render out a new frame from Nuke based on the frame rendered from Maya. In my Nuke script the read node reads from where the Maya render is. The issue occurs when for example frame 2 from the Maya job is completed before frame 1, the Nuke job tries to render frame 2 (which by now exists), but still fails because frame 1 does not exist.

[10:43.23] ERROR: Read2: y:/pipeline_prototype_1/sequences/100/100_010/Comp/work/images/100_010_Comp_scene_v021/100_010_Comp_scene_v021_masterLayer.1.exr: Read error: No such file or directory

I can sort of get around it by enabling continue on error on the nuke job, but that can get confusing if there actually is an error, and it sets the task to complete.

Any ideas? I’m on deadline 10.

Thanks a lot!
Jonas

2

Could you set the read node’s error handling to “nearest frame” (or, in fact, anything other than ‘error’) ?

3

Hi Dan, thanks for taking your time.

Well, the problem with that is that I still want the job to fail if the frame that it tries to render actually doesn’t exist… I don’t want to have tasks that are completed but are really just a rendered black/checkerboard frame…

4

Ah, I see! But surely it only kicks off the render when the Maya task has finished, so it’ll never try to render a frame that doesn’t exist? Or are you concerned about Maya wigging out, not rendering an output but displaying as Complete and thus kicking off the dependent job?

How is the nuke comp made? Is it a standard one that has its input changed by an event, or is it created by the event? Either way, does it need to load in an image sequence? If each task is only ever rendering a single task, can it not load just the single frame in the read node?

(This obviously won’t work if you’re trying to do any pan-frame stuff like mblur or oflow, but then none of this will!)

5

Yes both Maya wigging out and there are some cases where we need to requeue the Maya tasks and then the dependent nuke tasks manually, and if we by mistake would select the wrong nuke frame or something silly like that (which could happen when done manually) we could potentially overwrite a frame that was good with a black one if the Maya frame is not there anymore. The idea is to remove the Maya frames after the Nuke job is complete since we’re really only interested in the frames from Nuke in the end.

Basically it’s a pre-made nuke script that has it’s input read and output write path replaced for each job. It would for sure not need to load any other frames then the one that that task is rendering, but i’m not sure how to accomplish that? Right now the read node’s first and last frame is set to first and last frame of the whole sequence that Maya will render.

The purpose of the Nuke job is just to convert Arnold rendered frames to multi-part exrs, since this is not supported by Arnold, so no mblur or oflow or things like that.

6

Well the start and end frames - in this case - are only relevant if you’re loading a padded sequence (“C:\renders\dog_####.exr” or “C:\renders\dog_%04d.exr”). How about if the read and write node’s input/output is replaced with simply a single, non-padded file? (ie “C:\reders\dog_0002.exr”) Since each Nuke job has only a single frame to render, there’s no need to involve the whole sequence in it.

I’m working under the assumption that the nuke file’s read and write nodes are edited manually rather than by hand.

7

Hmm, well all tasks are for the same job, ie using the same nuke script (though copied locally to the slave), are you suggesting to somehow dynamically change the input on render time for each task?

Correct, the Nuke file is never edited by hand.

8

Exactly. Or that each task has its own Nuke script that’s generated as part of the submission process. Are you familiar with Events in Deadline? They’re Python scripts that get executed when X is done (and X ranges from a task being submitted to a job finishing to anything in between - there are many callbacks).

If you have a generic Nuke file that’s set up how you want it, you could easily have a Deadline event copy and amend this nuke script to read and write from/to a single frame, and then submit this as a single-framed Nuke job. This event can be kicked off on task completion, so as soon as a Maya frame is done it’s executed, and it can get the file output path from the job too. It may take a little work to set up but once it is, it’d basically automatically convert all your outputs into .exr’s automatically without you needing to edit or copy anything anywhere - it would potentially be a bit of a mess in Monitor having so many jobs, but you could quite easily add all the Nuke jobs for a given Maya render to their own batch group so they’re easy to ignore.

As a little proof of concept, I just wrote an incredibly simple bit of python that’ll take a nuke file, open it as text, look for “INPUT_HERE.exr” and replace it with a filepath, ditto with “OUTPUT_HERE.exr” and then save it out as a separate nuke file. This obviously relies on your ‘blank’ Nuke file (that’s otherwise set up how you like it) having a read node with “INPUT_HERE.exr” as the filepath and a write node having “OUTPUT_HERE.exr” as the output - but once you have that new Nuke file, it could be automatically submitted without a problem by the same event script.

f = "C:\\tmp\\test.nk"
with open(f) as nf:
    data = nf.read()
    # print(data)

data = data.replace("INPUT_HERE.exr", "C:/file/here.exr")
data = data.replace("OUTPUT_HERE.exr", "C:/other/file.exr")

with open("C:\\tmp\\test2.nk", 'w+') as nf2:
    nf2.write(data)

Edit: This is just one way. You could also import the nuke python module into that event script and generate the nuke script using Python rather than ASCII editing an existing file. This is likely “safer” but also a bit more effort to set up.

1 Like
9

Ah I see!

No I am not familiar with Events in Deadline, sounds like something I should definitely look in to!!

Part from the monitor becoming a bit messy as you mentioned, it would probably work the way I want. I’m doing some similar things when setting up the Nuke script in the first place so that part is only a matter of inserting the correct frame number.

Thanks a lot for the input Dan!

1 Like