SMTD with StereoCameraRig issues

[Bobo]

I just tried it with a recent scene that was set up with custom left/right cameras and it seems to work. We’ll continue using it today and see if it works reliably - but it’s looking good so far! I attached the scene of this test for you as reference to see if there are some things in the way it’s set up that you maybe want to consider.

Great, thanks!

Would it actually be possible to disable the ‘Stereo Camera Jobs’ menu as kind of an indicator that SMTD didn’t find any compatible setup?

No, the majority of the SMTD UI does not reflect the scene content dynamically. The menu tells SMTD “if you find a stereo camera in the active viewport, here is what to do with it”. I have some other ideas how to let you monitor what would happen if a submission would be performed, but it looks like a bigger project that will have to wait for v8.0…

Another question: right now, there is a shotgun version being created for each camera/view. Would it maybe be possible to have an option to submit all views/cameras as one single version? In most cases we don’t really need a version per view but rather have both/all cameras in one shotgun version. The “Path to Frames” in shotgun is then something like

filename_%V.%04d.exr

where the ‘%V’ is a variable that’s being interpreted by Nuke as the name of the view.

It won’t be trivial. Each stereo camera is submitted quite independently, as if you pressed the SUBMIT button multiple times. The individual jobs don’t have knowledge of each-other, except for the batch name. It would require some rethinking to make the Shotgun integration know what to do. I have some ideas, but I don’t think they will make it into 7.2.

As you also mentioned some bugfixes with QuickDraft/Integration i remember seeing another bug in the Integration tab which still seems to be there.
The info in the ‘Extra Info’ section is not updated until disabling and re-enabling the ‘Create New Version’ checkbox.
When opening SMTD, i go to the Integration tab, activate the ‘Create New Version’ and then enter the new version name, description, etc. But the ‘Extra Info’ doesn’t change - only when i disable/enable the checkbox it’s being updated. Same goes basically for the ‘Draft Template Output Options’ values. Only when pressing ‘Use Shotgun Data’ they’re being updated.

Will look into this, thanks! The Integration tab was not my code originally, so I will have to be careful not to break anything.

[celluloidvfx]

Good news: we’ve been rendering a bunch more stereo scenes today and haven’t come across any issues so far.

It won’t be trivial. Each stereo camera is submitted quite independently, as if you pressed the SUBMIT button multiple times. The individual jobs don’t have knowledge of each-other, except for the batch name. It would require some rethinking to make the Shotgun integration know what to do. I have some ideas, but I don’t think they will make it into 7.2.

Thanks for the explanation. Good to know it’s on the list.

Will look into this, thanks! The Integration tab was not my code originally, so I will have to be careful not to break anything.

Don’t you always have to be careful with that?
Thanks for taking care of it!

Cheers,
Holger

[Bobo]

Will look into this, thanks! The Integration tab was not my code originally, so I will have to be careful not to break anything.

Don’t you always have to be careful with that?

I know you were joking, but there is a huge difference between modifying my own code (even if it were 10+ years old, as is the case with some parts of SMTD), and somebody else’s code I have never seen (which is the case with the Draft & Shotgun integration functions).

Yesterday, I moved a large portion of the Quick Draft code from the UI into the Functions part because it was simply in the wrong place. However, I was probably the only person in the world to know that. It was a truly stressful exercise to move it over and make it work better than before without breaking it…
In contrast, adding the custom stereo camera support was a breeze, as I was tweaking a function I wrote myself about a month ago

[celluloidvfx]

I know you were joking, but there is a huge difference between modifying my own code (even if it were 10+ years old, as is the case with some parts of SMTD), and somebody else’s code I have never seen (which is the case with the Draft & Shotgun integration functions).

I know - especially as someone who’s just a part-time programmer and is causing this pain to myself every once in a while - and sometimes to others. But i’m working on it

Speaking of Draft: do you think there’s a chance to add an option to limit the QuickDraft jobs to the beauty pass (in the case of VRay the ‘RGB_color’ pass) of a render job? All those outputs don’t really hurt but sometimes you just don’t need/want them.

And we just stumbled upon another Draft issue related to EXR output from Max/VRay. Don’t know if this is related to the code that you’re working on but if you want to have a look, the thread is here:
https://forums.thinkboxsoftware.com/viewtopic.php?f=205&t=13720

Cheers,
Holger

[Bobo]

Speaking of Draft: do you think there’s a chance to add an option to limit the QuickDraft jobs to the beauty pass (in the case of VRay the ‘RGB_color’ pass) of a render job? All those outputs don’t really hurt but sometimes you just don’t need/want them.

It is not trivial, and is not controlled by SMTD. Basically we enable Quick Draft by writing some flags into the Job file, and then the Draft Event takes over, fishes out all paths from the file and builds jobs on the fly when the main job finishes. This will require some serious changes on the side of the Draft Event which is out of my control. But I will ask the developers to consider it…

[celluloidvfx]

Just add one more flag then!
Joking aside: completely understood. Like i wrote before, it’s not something really important, rather some nice-to-have feature.

Another question/idea:
how about adding some control next to the ‘Stereo Camera Jobs’ where the user can control the priorities of the different cameras as an offset value in relation to the first camera? I’m thinking of a scenario where you have a few scenes to render in stereo and they will take quite a while. Basic comping and editing could be started as soon as one view is rendered for all scenes. Of course, this can be done manually in the Monitor but i think it’s nice to have this in SMTD - just like with the ‘PREVIEW FRAMES’ jobs where you can set the ‘Priority +’. In the case of stereo, i think it’d be better to decrease the priorities of the other cameras in relation to the left one, though. I think it’s become kind of a de-facto standard to use the left view as the hero view and use that in case of mono versions. And i think SMTD already submits the left one first, right?

Cheers,
Holger

[Bobo]

how about adding some control next to the ‘Stereo Camera Jobs’ where the user can control the priorities of the different cameras as an offset value in relation to the first camera?

Awesome idea, I will add that.
However, it won’t make it into the next beta build (expected before the end of the week), so I will just upload the new SMTD scripts here when I have them.

[celluloidvfx]

Cool!

However, it won’t make it into the next beta build (expected before the end of the week), so I will just upload the new SMTD scripts here when I have them.

Quick question: there’s been loads of ‘inofficial’ fixes and updates of SMTD and some other scripts due to some other bugs during the current 7.2.0.14. So i was regularly backing up, overwriting scripts in the repo. I guess a better way to test those fixes would be to put them into the ‘custom’ folder as those are being used when they have the same name as the shipped ones, right?

Cheers,
Holger

[Bobo]

Quick question: there’s been loads of ‘inofficial’ fixes and updates of SMTD and some other scripts due to some other bugs during the current 7.2.0.14. So i was regularly backing up, overwriting scripts in the repo. I guess a better way to test those fixes would be to put them into the ‘custom’ folder as those are being used when they have the same name as the shipped ones, right?

Wrong, the MAXScripts always read from the \Submission\3dsmax\Main
The Custom scripts is for Deadline’s own scripts. SMTD is part of 3ds Max, shipping with Deadline, but not running IN Deadline.
Keep on doing what you were doing before - baking up the old files (in a zip or subfolder), and overwriting with the latest ones.

Here is a first prototype of the Stereo Camera Priority controls:
SMTD_StereoCameraPriority_20150827.zip (173 KB)

Instead of offering a number, it asks you for the order you want them processed in, and sets the Priority internally to 1 less than the main eye.
E.g. if Left > Right > Center was requested, and Priority was 50, it will submit Left with 50, Right with 49 and Center with 48.

If you want to tweak the actual offset value, I might expose it as a setting you can control company-wide via the Defaults.
I’d rather not add it to the UI. So you could set it to 10 and it would submit with 50,40,30…

Please let me know if this works for you.

[celluloidvfx]

Wrong, the MAXScripts always read from the \Submission\3dsmax\Main
The Custom scripts is for Deadline’s own scripts. SMTD is part of 3ds Max, shipping with Deadline, but not running IN Deadline.

I see…
Actually, it’s just a bit misleading in the docs then:

In addition, any scripts or plugins in the ‘custom’ folder will override any scripts or plugins that are shipped with Deadline if they share the same name. 

(from http://docs.thinkboxsoftware.com/products/deadline/7.1/1_User%20Manual/manual/scripting-overview.html)
So i guess it should read ‘… with the exception of STMD,…’.

I just briefly tried your new version and it seems it’s all working.

Instead of offering a number, it asks you for the order you want them processed in, and sets the Priority internally to 1 less than the main eye.

Sounds good to me. Probably not really necessary to have the prio offset exposed to the user. I was just thinking of the same behaviour as the ‘Preview Frames’ job.

E.g. if Left > Right > Center was requested, and Priority was 50, it will submit Left with 50, Right with 49 and Center with 48.

When sending Left > Right it’s going to use prio 50 and 48 (but not 50 and 49). Not at all a problem for me, just thought i let you know.

If you want to tweak the actual offset value, I might expose it as a setting you can control company-wide via the Defaults.

Yes, that’d be good. Should have at least some location to tweak it if needed.

[Bobo]

Actually, it’s just a bit misleading in the docs then:

In addition, any scripts or plugins in the ‘custom’ folder will override any scripts or plugins that are shipped with Deadline if they share the same name. 

(from http://docs.thinkboxsoftware.com/products/deadline/7.1/1_User%20Manual/manual/scripting-overview.html)
So i guess it should read ‘… with the exception of STMD,…’.

What a very Max-centric thing to say

Deadline supports 70+ applications, and a few of them ship with integrated scripts written in various scripting languages. These integrated scripts are all affected by this limitation - in fact, some of them must be copied manually (e.g. the Maya submitter) into the 3D application’s own scripts folders, and do not update automatically from the Repository like SMTD does. So they are the connection of other apps to Deadline, they ship with Deadline, but are not “Deadline Scripts” in that they don’t run inside of Deadline.

I agree we should modify the documentation to explain what kinds of scripts we mean…

[celluloidvfx]

What a very Max-centric thing to say

Yeah, pretty tough for me as well as i’m rather a Nuke than a Max user
But that’s why i put a ‘,…’ there as i still don’t quite understand which scripts are affected and which aren’t. Is there a simple rule like ‘all submission scripts’?
And yes, please make this more clear in the docs.

Cheers,
Holger

[Bobo]

What a very Max-centric thing to say

Is there a simple rule like ‘all submission scripts’?

The rule is this:

The Repository contains a \Scripts folder, which contains sub-folders called
Balancer
General
JobReports
Jobs
Limits
Pulse
SlaveReports
Slaves
Submission
Tasks
WebService

The Repository also contains a \Custom\Scripts folder, and it contains EXACTLY the same sub-folders as the \Scripts folder.

All these scripts are written in Python and run INSIDE components of Deadline- in the Monitor, on the Slaves, in the Balancer, on Pulse etc. They implement operations like job submission from the Monitor, general operations on existing jobs, connecting with the slave via VNC, defining the Balancer rules, emailing the user, opening image viewers etc.

The ones in the Custom\Scripts\ folder will never be updated by the installers, and if the same file exists in both places, the Custom one will be run instead of the Factory version. So if you take an existing shipping script, say the Deadline Monitor submitter for Nuke, and want to add functionality, you first copy it into Custom\Scripts\Submission, then modify it. This way your changes will survive multiple updates.

In the Repository there is also a \Submission folder. It contains sub-folders for every application that provides an INTEGRATED submitter that runs in the application itself (Max, Maya, Houdini, Lightwave, Modo, Nuke, Fusion, AE, etc.) These scripts are written in various scripting languages native to the respective applications. These scripts communicate with Deadline (usually by calling DeadlineCommandBG.exe), but do not run INSIDE of Deadline. Currently they do not provide a custom folder mechanism.

So the rule would be “integrated submission scripts are not supported yet”. However, we are looking into changing that in future versions.

[celluloidvfx]

Thanks for the explanation!

However, we are looking into changing that in future versions.

Cool!

Have a nice weekend!
Holger

[celluloidvfx]

Hi Bobo,

i think we found another issue with SMTD:
it’s losing the checked state of the ‘Create/Upload Movie’ and ‘Create/Upload Filmstrip’ checkboxes when you e.g. go back to the ‘Job’ tab and then again return to the ‘Integration’ tab. They’re always unchecked.
This is with your modified ‘SMTD_StereoCameraPriority_20150827.zip’ from a few posts above. We haven’t had the time to upgrade to 7.2.0.15 nor 7.2.0.16 yet.

There’s also a feature idea that came to mind during our current project. Do you think it’s possible to add a feature for the priority of dependent Draft jobs, similar to the relative priority of the stereo cams and the ‘Rest of Frames’ jobs? We just found it’d be helpful if all those Draft jobs have a higher priority than the cg/comp job they’re depending on - so they will definitely rendered before any other following cg/comp job. Sometimes those have to wait for frames of other jobs to finish before they’re being started.
Of course, this would be great to have in all submitters that are able to submit dependent Draft jobs.

Cheers,
Holger

[Bobo]

I think we found another issue with SMTD:
it’s losing the checked state of the ‘Create/Upload Movie’ and ‘Create/Upload Filmstrip’ checkboxes when you e.g. go back to the ‘Job’ tab and then again return to the ‘Integration’ tab. They’re always unchecked.
This is with your modified ‘SMTD_StereoCameraPriority_20150827.zip’ from a few posts above. We haven’t had the time to upgrade to 7.2.0.15 nor 7.2.0.16 yet.

I can see the bug and will fix it ASAP. The variables for Movie and Filmstrip were added, but not initialized correctly.

There’s also a feature idea that came to mind during our current project. Do you think it’s possible to add a feature for the priority of dependent Draft jobs, similar to the relative priority of the stereo cams and the ‘Rest of Frames’ jobs? We just found it’d be helpful if all those Draft jobs have a higher priority than the cg/comp job they’re depending on - so they will definitely rendered before any other following cg/comp job. Sometimes those have to wait for frames of other jobs to finish before they’re being started.
Of course, this would be great to have in all submitters that are able to submit dependent Draft jobs.

\

In this case, all I can do is ask the Deadline developers to add that option to the Draft event plugin. Right now the QuickDraft spawns a job when the main job finishes, and it appears that the Draft job inherits the properties of the main job. I am sure a Priority offset could be added, but it is not under my control…

[celluloidvfx]

I can see the bug and will fix it ASAP. The variables for Movie and Filmstrip were added, but not initialized correctly.

Cool!

In this case, all I can do is ask the Deadline developers to add that option to the Draft event plugin. Right now the QuickDraft spawns a job when the main job finishes, and it appears that the Draft job inherits the properties of the main job. I am sure a Priority offset could be added, but it is not under my control…

No problem. Should i post this in the Draft forum then? To have this as a parameter of the Draft plugin in general would work for me as well.

[Bobo]

No problem. Should i post this in the Draft forum then? To have this as a parameter of the Draft plugin in general would work for me as well.

Nope, it is not Draft per se, it is a Deadline feature using Draft. I will hunt down the right people and ask them about it.

[Bobo]

Could you please give this version of SMTD a go and see if it fixes the Shotgun issues you reported?
SMTD_Shotgun_Fixes_20150902.zip (173 KB)

This fixes the following issues:

• When entering a new version and description, the ExtraInfo fields will be updated immediately.
• The Movie and Filmstrip Upload checkboxes are now persistent when changing tabs.
• All Shotgun Integration options were already stored in persistent globals with the scene (if you would save the scene manually after using SMTD), but the values were only restored correctly if you first loaded the .MAX file, then relaunched SMTD. I fixed it so that opening SMTD and then loading multiple .MAX files containing different Shotgun settings will automatically update the SMTD settings and the SMTD UI without the need to restart SMTD after each loading.
• SMTD was performing some additional file operations each time a Shotgun setting was changed, resulting in awful performance hits. I removed these operations, so using SMTD with Shotgun should be much faster now.

Note that SMTD still does NOT save the current scene automatically to the original source path when you make changes to the Shotgun settings, unless the submission option “SAVE and Use Current Scene’s ORIGINAL NETWORK PATH” is selected. In order to make the Shotgun settings persistent per scene file the user MUST save the scene manually if using the other submission options.
However, the scene copy submitted with the job will have these settings stored in it, so opening it from the Repository would restore all Shotgun settings correctly.

Please let me know if you find anything else that does not work right.

[celluloidvfx]

We tried that version and the issues regarding the shotgun settings are gone.
Thanks!

Also thanks for explaining the situation regarding the handling of the shotgun settings and saving of the scene file. I think this explains some of the issues we experienced the last days when some info got mixed up in shotgun.

I’ll keep you updated should we encounter any other issues.

Cheers,
Holger