AWS Thinkbox Discussion Forums

Multi-pass EXR tile assemblies only assemble RGBA?

We’re running Redshift renders through Maya 2017 under Deadline 10.0.9.2.

To try to accelerate some very large still-frame renders we tried turning on the basic tile rendering (the one with X/Y amounts, not the jigsaw one that requires setup in the popup UI). The tiles all render fine, but if the output has AOVs, those tiles are not re-assembled, and if we set the redshift render options to put all the AOVs into a multi-pass EXR, the output of the assembly only contains the RGBA, though loading an individual tile EXR I can see all the AOVs present.

I can imagine the tile assembly not knowing about the additional AOVs as separate files, but should we expect it to work with all AOVs in a single EXR file and look for something set wrong on the job?

Thanks,
Matt

We fixed a very similar problem with Arnold back in 10.0.8 it seems:

docs.thinkboxsoftware.com/produ … notes.html

I think testing this should be easy enough. Would you be willing to create a simple scene that reproduces the problem and I can run it by the dev team?

Actually, it seems the dev team are working on this at the moment. I’ve added you to the issue there, but it likely won’t be in SP12.

How is this impacting your workflow at the moment?

Thanks for the news, glad to hear it’s already been identified and getting some attention.

Impact is annoyance, not blocking. We can reassemble the tiles in Nuke. It’s messier and adds an extra step but will work when tile rendering is worth it for faster turnaround.

Alright, the change should be in SP13 when it ships. The update for this required a significant amount of changes (apparently we needed to implement quite a bit to get it working).

If you want to give it a try now, I’m attaching the two files you’ll need to update. Make a backup of each in case something goes wrong here.

In “[repo]/plugins/MayaBatch” extract “MayaBatch.zip” after backing up “MayaBatch.py”
In “[repo]/submission/Maya/Main” extract “SubmitMayaToDeadline.zip” after backing up “SubmitMayaToDeadline.mel”

MayaBatch.zip (26.3 KB)

SubmitMayaToDeadline.zip (65.4 KB)

Ooh, thanks. I’ll get it installed and hopefully run something through today to give you some feedback.

TL;DR - Due to current limitations in Draft, if you’re making “multi-part” exrs (in this instance - with Redshift), the patch won’t solve all your problems, only some of them.

After a quick read of this, I feel the need to interject/clarify some information here.

Draft does not currently support ‘multi-part’ EXRs, which is how channels might be lost from an individual tile to the assembled image (even if you’re “tile rendering” a 1x1 region). Redshift has the ability to turn off ‘multi-part’ exr, so hopefully you can use that to work around the loss of some channels.

The changes that Edwin posted is an enhancement to maya intended to handle writing AOVs to their proper file and notify Draft of their respective tiles. Previously, if you had two AOVs pointed at the same output file, the tiles for those AOVs would be rendered to different files and we wouldn’t assemble the tiles of non-beauty passes because we didn’t know which output files to expect. With the patch, we’re now properly handling those AOV filename prefixes.

Handling multi-part EXRs is a wish-list item we currently have for Draft (among other things).

Cheers

Hi Edwin - These versions are throwing up some errors:

2018-03-01 13:52:38: 0: WARNING: Strict error checking on, ignoring the following unrecognized error or warning. If it is fatal, please email support@thinkboxsoftware.com with the error message.
2018-03-01 13:52:38: 0: STDOUT: Error: file: C:/Users/mplec/AppData/Local/Temp/tmp93BF.tmp line 11: NameError: file line 1: name ‘DeadlineMayaBatchFunctions’ is not defined
2018-03-01 13:52:38: 0: STDOUT: Error: file: C:/Users/mplec/AppData/Local/Temp/tmp93BF.tmp line 44: Render failed.
2018-03-01 13:52:38: 0: Done executing plugin command of type ‘Render Task’
2018-03-01 13:52:38: 0: An exception occurred: Error: Strict error checking on, caught the following error or warning.
2018-03-01 13:52:38: Error: file: C:/Users/mplec/AppData/Local/Temp/tmp93BF.tmp line 44: Render failed.
2018-03-01 13:52:38: If this error message is unavoidable but not fatal, please email support@thinkboxsoftware.com with the error message, and disable the Maya job setting Strict Error Checking.
2018-03-01 13:52:38: at Deadline.Plugins.PluginWrapper.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel) (Deadline.Plugins.RenderPluginException)

and on other tiles:

2018-03-01 13:27:39: 0: WARNING: Strict error checking on, ignoring the following unrecognized error or warning. If it is fatal, please email support@thinkboxsoftware.com with the error message.
2018-03-01 13:27:39: 0: STDOUT: Error: file: C:/Users/mplec/AppData/Local/Temp/tmp9E62.tmp line 43: Cannot find procedure “remapFileTextures”.
2018-03-01 13:27:39: 0: STDOUT: mel: READY FOR INPUT
2018-03-01 13:27:40: 0: Done executing plugin command of type ‘Render Task’

I can send you the complete logs offline if that helps? If so, should I email them to support@?

Maybe I’m just missing an additional file?

@Morgan - Thanks for the clarification. Understood! And as luck would have it, our supe prefers the passes as separate files, so this should solve the problem for the way we’re working.

It seems that some new files were not included in the patch, here’s the missing files that go into the same folder as the other ones. That should solve those ‘procedure not defined’ messages.

Let me know how that works for you.

Cheers
MayaBatch.zip (2.15 KB)

1 Like

Thanks for stepping in here Morgan. I didn’t remember Draft’s limitations with multipart exrs.

Thanks, we ought to be able to give it a try today. I’ll let you all know how it goes.

No luck I’m afraid. It just exited after loading the scene and marked the task complete. We needed to get a render out so we went back to the original version and the artist made a change to the scene and we re-rendered that non-tiled. I haven’t had time to go back and repro, so it could have been that particular scene file or I didn’t get all the files updated this this time, so take this with a grain of salt. I’ll try to find time to do that.

But in case it’s useful, here’s the tail of the log where it would ordinarily start rendering after loading the scene.

I’m concerned the patch work might have caused some issues, and we’re targeting the official release of patch in this thread for SP13.

Can you provide the whole render log? You can save the job report as a text file and clean out any sensitive data and send it this way. The “Errors have occurred while reading this scene that may result in data loss” is complaining that there were errors while the scene was loading, but those errors its talking about are further up.

I’m also not sure what in the patch might short-circuit the render process (if any) so my guess right now is that the renderer or the scene itself didn’t load…

Would you be willing to have a call + remote session today? My day’s pretty open, so you can just use the number in my signature there.

Thanks for sending the log! I didn’t see what I’d usually expect. I was hoping for errors with the renderer being loaded, but near as I can tell they are all " contains invalid or unused components". Though, the fractional frames warning is interesting…

For future generations, here’s a scrubbed log (removed file names) and removed Deadline ‘warnings’. Changed out the scene object names for “X” and cooked up some paths to use.

2018-03-05 18:26:58:  0: STDOUT: mel: Loading scene: C:\MainScene.mb
2018-03-05 18:27:07:  0: STDOUT: Warning: file: C:\ReferencedScene.ma line 191653: ':defaultArnoldDisplayDriver.message' is already connected to ':defaultArnoldRenderOptions.drivers'.
2018-03-05 18:27:07:  0: STDOUT: Warning: The existing frame rate 'ntscf' does not match the imported scene frame rate 'film'. Imported key frame numbers may become fractional.
2018-03-05 18:27:07:  0: STDOUT: Warning: file: C:\ReferencedScene.ma line 139186: ':defaultArnoldDisplayDriver.message' is already connected to ':defaultArnoldRenderOptions.drivers'.
2018-03-05 18:27:07:  0: STDOUT: Warning: The existing frame rate 'ntscf' does not match the imported scene frame rate 'film'. Imported key frame numbers may become fractional.
2018-03-05 18:27:07:  0: STDOUT: Logfile: "C:\Users\SomeGuy\Documents\maya\FBX\Logs\2017.1\maya2017imp.log"
2018-03-05 18:27:13:  0: STDOUT: Warning: The existing frame rate 'ntscf' does not match the imported scene frame rate 'film'. Imported key frame numbers may become fractional.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:13:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: X contains invalid or unused components.  These can be cleaned up using the Mesh Cleanup dialog.
2018-03-05 18:27:14:  0: STDOUT: Warning: file: C:/Users/SomeGuy/AppData/Local/Temp/tmp1DF0.tmp line 46: Errors have occurred while reading this scene that may result in data loss.
2018-03-05 18:27:14:  0: STDOUT: Read 5 files in  16.2 seconds.
2018-03-05 18:27:14:  0: STDOUT: mel: READY FOR INPUT
2018-03-05 18:27:15:  0: Done executing plugin command of type 'Render Task'
2018-03-05 18:27:15:  0: Render time for frame(s): 52.127 s
2018-03-05 18:27:15:  0: Total time for task: 55.986 s
2018-03-05 18:27:16:  0: Saving task log...

Hey,

We have exactly the same problem with c4d - Redshift AOV’s not rebuilding after being tile rendered through Deadline / Draft.

Has this been rectified yet?

We are running:
Deadline 10.1.3.6
Cinema 4D R21.115
RS 3.0.16

Thanks in advance,

Cheers,

C

1 Like

@eamsler I just want to upvote this, but from all your threads, I won’t hold my breath :grinning:

If nothing else, it reminds us it’s still an issue.

From a workflow perspective, is it problematic to remember to output in this mode (RGB + Alpha) or does it make some work impossible?

When using deadline region render, the tiles all get created fine in all scenarios. It is just the assembly that is bit more restrictive. We would prefer to use multi-part EXRs. However, for deadline region render and redshift renders, we have found there is no perfect solution since redshift always creates separate folders for cryptomatte, and light groups render with naming that is not passed onto Deadline. This is more of a redshift thing though.

But not using multi-part and using RGBA and combined AOVs EXRs seem to work with tile assembly.

This is a multi-part EXR with 29 parts (layers). This shows the first 6 parts. Tile Assembly will only use RGBA

This is a non multi-part EXR with combined AOVs that Tile Assembly parses OK.

Ah! That’s great detail. This isn’t our first issue with Cryptomatte in Redshift. I’m sure there are heavy technical details of this, but we also have a fun problem with our MayaBatch integration because we move the output folder out from under the renderer. I’ll add this as a talking point when I sync up with Maxon again!

1 Like
Privacy | Site terms | Cookie preferences