In both cases, the assembly is failing because the tile files it is looking for don’t exist (the error message is at the bottom of the logs):
modo: Exception: Unable to read file /mnt/CGI_server/jobsTest/003_modo_pngTileTest/Output/modo_Diff_tile_1x1_3x3_.png for tile Tile0
maya: Exception: Unable to read file /mnt/CGI_server/jobsTest/005_maya_pngTileTest/images/_tile_1x1_3x3_beauty_001.png for tile Tile0
The modo path actually looks like an invalid path, since you typically don’t see angle brackes (< and >) in file names. Could you possibly upload the modo scene file that these tiles were generated from?
The maya path looks like a valid path, so again, could you possibly upload the maya scene file that these tiles were generated from?
I will doublecheck! Modo output path and filename padding isn’t the most straight forward one. I will be more careful with it and see if it still produces problems and get back with a status and a file producing the problems if it persists!
Thanks for confirming. For modo, this is actually by design. We currently don’t know the output paths for every single pass that modo could save a file for, we just know the root folder that they will be saved to. So we tell the Assembly job to look for any tile files in that folder that matches a certain pattern, and assemble them. Since we can’t explicitly handle it on a per file basis, it’s done as a single job.
For Maya, the tile renders would be per layer, and thus the assembly jobs will be per layer.
So dividing it into more pass groups would also divide the tile assembly jobs in smaller chunks?
We have a problem where Modo (or Deadline) puts brackets in the file name and Draft doesn’t assemble them - which is understandable. Also we are having difficulties with scenes that will not submit.
Do you have any general things one should always check to be set or not set when working with Modo and Deadline?
More passes simply means more tiles to assemble because each pass has its own output with its own set of tiles. The assembly job itself doesn’t change.
We’re not familiar with this problem. Could you upload a very simple test scene that reproduces it?
Our submitters already check for basic things like local output paths, and the Cross Platform Considerations section in our modo documentation covers best practices related to cross platform rendering. Other than that, we really don’t have any other recommendations.
We have problems with scenes not submitting from the integrated script and throws errors if submitted from the monitor. I’ll have my colleague prepare a scene and i will send it to you within an hour or so.
We have a requirement for large resolution print images that need to be tile rendered across our farm.
We have success with tile rendering a large .exr image with render outputs. So for example we have a beauty channel with spec, reflection, diffuse etc all within the exr.
modo_exrTileTestMultCh_07.lxo
Issues with Pass groups and Passes;
Problems emerge when we want to use Pass Groups & Passes that each have Render Outputs assigned.
modo_exrTileTestMultCh_08.lxo
In this scene there is a single Pass Group with two passes that alternates between a red/green colour of the main sphere object. Each pass has it’s own Render Outputs. DL will only render the first pass (red) and does not pick up the second pass (green). This is unexpected behaviour from our POV, I would expect both passes within the Pass Group to be submitted separately and rendered producing two .exr’s with their respective Render Outputs in the channels.
Alternatively we have tried to render out scenes using .png’s and have the Passes and Render Outputs as seperate images. This scene contains a single Pass Group that has three passes. Again we get errors on submission so it never gets processed.
modo_passesTileTest_08.lxo
Finally in this scene I have built multiple Pass Groups that contain only a single Pass per group. For reasons unknown, I cannot submit this file via modo’s submission script or the monitor. Therefore I cannot begin to diagnose what the issue is, please could you repath the outputs and try it;
So there are a couple of things going on here. In modo_exrTileTestMultCh_08.lxo it looks like your second pass (Green) has it’s render state set to false and if you set it to true (the camera thing beside the pass) then it will render both. Unfortunately it will do everything in the same job, do to the way things are being rendered.
This brings me to the second issue which is on our end. When tile rendering, the assembly job will try to assemble every render output as if it was enabled in each render pass (I need to take a look at this to find out what we can do for it), for now you can still do the assembly you will just want to suspend(or complete/fail) the tasks that are for the assemblies that will fail.
For modo_passesTileTest_08.lxo it looks like the integrated submitter has a bug in it, attached is a new version to try. To install it you will have to put the .py file into %AppData%\Roaming\Luxology\Scripts\DeadlineModo\pyscripts\lxserv
For the monitor submitter’s error can you post what the error in the console is (I am not running into a submission error there.
Thank you for the updated script - it made things better but there still seem to be some problems.
Every time we send 2 render passes to Deadline it only renders one - even though it renders both locally in Modo. If we put them in separate render pass groups and enable “create separate job for each render pass group” it still only creates one job. Is this feature not supported with tile rendering?
The scenes you modified and got to work with Deadline - could you send that back, so we can see your approach to this kind of job/submission?
I have looked into this more and I found another issue which might have been the issue you were running into. When we created the config file for the draft tile assembler we were trying to create it in the render output directory which might not exist yet. We are now creating them in a temp directory and then copying them to the output directory if it exists. So that should hopefully solve more issues. Attached is the new commands file (same install as before) and my modified scenes.
Currently “submitting separate jobs for each render pass group” while using tile rendering is not supported, we are looking at adding it hopefully for 7.1.1 I can post the updated script here when I have it.
Modo is now rendering bot render layers in one job, so we output both the red and the green sphere. This is good!
For some weird reason Deadline creates 4 Draft tile assembly jobs. One for the red pass and one for the green and 2 “unwanted”. The red one assembles but the green one doesn’t.
So getting the passes rendering works - Draft tile assembly doesn’t!
Here is a new submitter, it should fix more of the stuff for this. To work properly each render pass needs to have channels set for the RO’s if either one does, (if one has it set and the other does not it will create extra assembly jobs), if you don’t have it set this way it will create extra assembly jobs in the wrong cases I am looking more to see if we can get around this.
For why that job failed it is because you are rendering one of the unwanted jobs (RedSphere08__greenSphereGroup.exr aka RedSphere RO in the render pass greenSphereGroup).
Thanks for the new submitter, we have it installed and have given it a run. We have experienced a repeatable bug, it seems that the submission script is not behaving properly on submission and hangs.
If I press the submission button after entering the job details, nothing happens and the job is never submitted to the farm. With the submission script open and relevant details entered I need to re-save the modo scene, then press the submission button and then the job is sent and begins being processed.
Obviously this extra manual re-save should not be required, it should do this automatically as we see with the Maya submission for example.
Regarding your last feedback on the scene we gave you, I am not sure I fully understand your feedback. So what I have done is rebuilt the scene, again with a simple colour change on the sphere in the passes. We still cannot get this to work so I have attached the scenes, here are the details;
modo_exrTilePassGrp_01.lxo - this has x1 Pass Group which contains x2 passes. The sphere has a yellow colour and then a blue colour.
modo_exrTilePassGrp_02.lxo - this has x2 Pass Groups each containing a single pass, one for the yellow sphere and one for the blue sphere.
We are seeing a similar error to the earlier scenes, the first pass (yellow) is the only one that renders, then draft errors on the stitching operations.
Is it possible for remote into our machines over here on Monday? I am at a loss to explain why this isn’t working and we don’t seem to be making much progress on understanding and solving the issues. Perhaps if you used our set-up you would pick up on an error on our side, or first hand experience what we are seeing which may help. We can arrange for you to teamview in, no problem.
