Hi Drafters
That’s it for now 
Bill
I’m finding that for shots over, say, 100 frames, the time it takes a single farm node to create a movie is longer than optimal. What are the chances that the default solution is to break up a long shot into even intervals (say four, so 300 frames would be split into 4 jobs on 75 frames each) and then Draft would concatenate the resulting quicktime movies into a single movie automatically? ILM had this and it made a 1500 frame shot take 20 minutes for a movie instead of 2.5 hours (as one example).
That’s fairly complicated to do as I understand it. The problem being that for encoding video, there usually isn’t a ‘clean break’ for chunks to be worked on independently, because a lot of codecs have encoded frames that rely on previous frame information (and sometimes subsequent frames, too). As far as I know, FFmpeg does not support any kind of distributed encoding like this, which we would need since we’re not doing any of the encoding ourselves.
Now, you could technically have 4 different nodes encode 4 standalone videos for the chunked frame ranges, but to join them up you’d have to decode all 4 videos, and re-encode the full-length video, which would probably take longer than just doing it all on one node to begin with (unless you’re doing a lot of pre-processing on the frames).
Long story short, and Paul can correct me if I’m wrong, but I don’t think this is something that we’ll be doing.
Sorry for not commenting on these earlier, I had seen it and completely forgot to reply 
The “Proxy” checkbox in the Nuke submitter we might be able to do; I assume you’d just want this to emulate the behaviour of turning on “Proxy Mode” in Nuke itself?
As for the Node-based “Submit to Draft” option; I assume you mean a custom Nuke node that performs a Draft submission or something? I’m not too clear on how this would work, could you elaborate a bit?
“Movie pop up on screen once Draft finishes” – I’m not sure if that’s something we would do. I’m assuming this would work similar to notification, but automatically ‘open’ outputs as they complete… Which seems like it might be a bit intrusive/annoying most of the time (at least to me, anyways – but I’m just a code monkey 
). I’ll bring it up with the rest of the Deadline crew to see what they think, anyways.
Cheers,
I think the version I saw of the parallel renders of the QT movies was using Motion JPEG A, which I’m guessing is not a long GOP codec (I actually think it’s essentially got a keyframe every frame?). I do know that QT tools exist that concatenate multiple movies together very quickly but beyond that I’m not sure what they are.
Your interpretation of the “Proxy” checkbox is right on. Often times, I would run at 1K locally but want to run at 2k or 4k on the farm. If that button is there (and made sticky?) and I’ve got it set incorrectly, I could fix it with one click instead of having to close that window, turn off Proxy mode in Nuke, and then reopen the window to submit. Having it there also simply reminds me if it’s turned on or not, something that can be forgotten when it’s out of sight, out of mind.
The node-based “Submit to Draft” would work like Nuke’s “Render Selected” menu option under “Render” menu heading along the top of the screen: you pick a Write node, select “Render Selected” and it kicks that job off locally in either the fg or bg. In this case, I might have input Read nodes/frames or output Write nodes/frames that would need to be converted to movies and/or proxies; the Read example would be CG renders that never got converted after render but the comper needs to use; the Write example would be if I ran the frames of v04, then needed to just fix, say, 3 frames locally and then rerun the movie & proxies for that same take. The point is, the menu item kicks off a direct submission to Deadline of a Draft job since it knows the path to the frames and all other relevant information. Having it kick up an entire submit window seems like overkill for something that it pretty straightforward.
I know the movie popping up on the screen sounds annoying, but it’s really quite a lifesaver. When I was at Weta that was the default action of their queuer, and if you’ve got multiple shots or precomps in the queue, it allows you to check your work quickly and then drop in into whatever target (comp script, dailies review, etc) immediately, instead of having to remember it later, track it down, find the correct destination and then finally put it there. All that stuff is what the computer’s best at, so we should let it.
Similarly, to avoid costly overnight renders that aren’t correct, a great solution would be to allow the user to indicate that they want to check the first, middle and last frame (or, say, on tens – the number of and interval of frames set to user preference, really) so that they can see if their render is what they expect to see, then click a button to have it continue with the rest of the job. If the artist goes home or is away from the computer, it times out on the movie display and kicks off the remainder frames automatically. This kind of thing makes a HUGE difference between getting 1 or 3 iterations of a shot in a day, which is really the endgoal of any pipeline.
Now, on this last request, I’m not asking you to build that tool per se, but instead provide the wherewithall for an artist friendly way to make that happen. Or at least a blueprint. Dig?
Yours,
Bill
Hey Bill,
I just had a discussion with Ryan (one of my fellow Deadline devs) about your requests, and here’s what shook out of it.
With regards to the Proxy mode, Ryan mentioned that we’ve been trying to stay away from exposing knobs/settings that are part of the application itself. Once we start going down that rabbit hole, it’s hard to go back, and we run the risk of the submitter becoming bloated.
For the Render Selected stuff, we don’t currently have any plans to write standalone Draft submitters for individual apps, but you might be able to work around your scenario a little with current functionality. If you have the option enabled to submit each Write Node as a separate job, you could use the right-click Draft submission script in the Monitor to submit Draft jobs for specific output nodes, in the event the Draft stuff was missed on the initial submission. We do also have an option to only submit Jobs for the selected Nuke nodes; while this is the full Job (not just Draft) it might be useful to adapt a similar workflow to what you’re describing. None of this will work with Read nodes, however, if you’re wanting something like that you guys would have to write your own script from scratch.
For opening output on your machine after rendering, I don’t believe that’s something we’ll be implementing anytime soon, unfortunately. You might be able to develop something like this internally if you have a Programmer on board, it would require creating an Event Plugin that would push commands out to notification targets on Job completion to open up the output. This would require a bit more legwork if you only wanted to do this on a per-job basis, though.
Finally, for the last (related) request in your last post, we’ve actually had quite a few people ask us for a “preview” Job type, or something similar, which essentially consists of what you’re describing. I think the way it’d end up working is that 2 jobs would end up getting submitted to the farm; the “preview” job (consisting of a few frames, e.g. first/middle/last), and the regular job. The regular job would be dependent on the “preview” job, and could be submitted as suspended, only to be resumed if someone OKs the output of the preview job, as you describe. I don’t think this is on the roadmap for 6.0 currently (we’ve already got our hands full for that version), but might be something for 6.1
Cheers,
[/quote]
Actually this is on the roadmap, sorry to correct you !
Cb
Awesome! Sorry about that Bill – I’m a bit out of the loop on the Draft stuff nowadays, I should’ve asked Paul or Chris before making assumptions
Wow, thanks for the thorough reply! I would LOVE to have a programmer here to build tools but sadly it’s not in the cards. That’s why the approach you’ve been taking (template and example scripts that someone like me could hack their way through) has been really great.
Looking forward to all the new stuff as it comes!
Thanks
Bill
One thing I just noticed is that if one submits an AE script with two output movies that they get submitted to deadline with the same priority. This means that one has to completely finish before the other can kick in. Would it make sense to increment (or de-increment) the priority setting for each submitted job, so that they could run more in parallel?
Modifying the priority on either of the Jobs actually wouldn’t change much; whichever one ended up as higher priority would be the one slaves pick first. Either way, slaves will end up preferring one job over another.
There isn’t really a good way to do what you’re describing right now, given the way our Scheduling currently works. The best you could do is set a machine limit (of X machines, where X is whatever number is appropriate) on the Jobs when submitting it to Deadline. That way, only X slaves will work on the first job, while the next X will start on the second one. However, that would obviously put a hard cap of X slaves on each job, instead of “splitting” all available ones.
Essentially, what would be needed for our scheduling to behave the way you describe would be some sort of “round robin” mode for jobs that have the same pool & priority, instead of tie-breaking with the submitted date (which is what we do now). Some people have requested this feature already, so it’s on our wishlist somewhere; we’ve even discussed putting this in for 6.0/6.1 (no promises, though).
I should point out, though, that such a “round robin” mode wouldn’t push Jobs through any faster. If anything, individual jobs would take longer to finish. It would be good to get a few preview frames from multiple jobs of the same priority, however, so that you can figure out if something needs changing earlier.
Cheers,
It would be great if one could turn off the Draft job in the Options -> Modify Properties… drop down menu in Monitor
Alternatively (or concurrently), if the only output is a .mov file, could draft overwrite sticky settings and turn itself off?
Hey Bill,
The problem with that is that some people might have Draft templates built to process ‘.mov’ Video output, and we wouldn’t want to stop them from doing that
If all you guys are ever going to do is work with frame sequences as input, then you could edit the Draft Event Plugin yourself (events/Draft/Draft.py in your repo) to check for that before proceeding, but I think the best way to deal with this would be to instead put a check into the Draft Template itself (since the Template is the part that’s designed to only work with frame sequence input). Now, that would still create the Draft Job in the Queue (which might not be optimal), but at least it won’t waste a bunch of render time failing over and over due to incorrect input type.
Cheers,