Deadline 10.0.12.1, redshift and maya 2018
When job is submitted to deadline everything renders fine but there are tasks with missing frames, task is done, time spent on it but frame is missing. I see that output size in task monitor is also missing as well.
Looking into log and frames that are fine I can see redshift bucket progress, in bad frames that part is missing.
Rendering form live maya GUI works fine.
Basically after each render I have to go to live maya and render missing to fill in holes.
Submitting only missing ones in deadline will also produce same result some frames are added some are missing.
In different projects and different slaves so probably not project and slave related.
Any ideas?
Attaching task logs for good and bad task.
For now solution is to simple render 1 frame per task and then find out those without output file size and then requeue and requeue till they are added. From what I can se so far this happens in maya 2017 and 2018.
One handy feature for missing frames is that Deadline has a feature for that:
Things to note here:
Because local rendering is on, we can see that there were no frames written to disk on the āmissingframesā task.
The tasks are a bit different in that one is the beginning of the job starting up and one is a continuation
The big issue here is that Deadline isnāt catching that the output wasnāt written. There was a feature some time ago that would throw an error if output wasnāt written, but there were some workflows where that was oddly intentional (I think it was render element output overrides).
Tested on two different projects, and both times using render local option enable and disabled final effect was the same.
But what is noticeable is that in those cases where there is no frames Redshift never actually picked up rendering, there is no buckets done, no redshift progress at all. Is it possible that something just messes up and redshift donāt do any rendering at all but it is still marked done and time is spent?
It could be. Actually, are you using concurrent tasks at all? I think we discovered recently that access to the same GPU in multiple tasks / Slaves is not support and while it usually locks up, I wonder if this may be an issue.
I suppose the easy workaround for now would be to find a way to fail if we donāt see Redshift has rendered. Iād rather see us implement something more universal though. Iāll chat with the integration team about this oneā¦
I discussed the idea of renderer short-circuit detection, but weāre not sure if that would be a good test here yet. One thing we know is that Redshift doesnāt follow the same methods when we ask it to render so this may have something to do with that implementation.
Would you be willing to run a few more test? Mainly, what weāre looking for:
Can you re-render with MayaBatch and enable ālog script contents to render logā in the Plugin configuration. Weād want a success and fail again
Can you try rendering without MayaBatch? This will disable some of the more advanced settings, but weāre trying to rule out melscript
Sorry bin a bit bussy on this side.
But just today seen this on forum as well:
[i]Hey everyone,
Weāve been using Exocortex for some time now and recently updated to Maya 2018.2.
Problem is when we send renders to our farm some nodes and workstations appear to skip-not-read the Alembic files on our servers and the renders come out with the meshes in default position.
It started happening less frequently when we set all our computers to DG evaluation, which we ended up doing cus we were also having problems with Viewport display on Maya itself.
Our workflow is moving from having the Exocortex nodes on a referenced file and then repathing, to actually creating the nodes on the file itself. In both cases, weāve got this error.
Any ideas on what could be happening?
Thanks![/i]
Missing frames along with couple more weird issues I run into like frames are rendered but objects inside scene are on and off from frame to frame and couple similar creepey stufā¦ it seems that all could be due to maya 2018.2 update!
Iāve uninstalled maya and back to pure 2018 version now and testingā¦ first couple jobs seems perfectly fineā¦ will do couple more tests to confirmā¦
Also beside these issues tthere is other one, if you submit maya 2018.2 job to deadline it wonāt render. error reading file, even tho it loads jsut fine in gui maya 2018. So there is that as wellā¦
The Maya 2018 failing to open the file is usually because itās not PU2. Maya will refuse to open a file whose version number comes from a newer version. Claims the scene is corrupt. An old hack that still works today is to save as ASCII and modify the version in the file header yourself.
For this though, mepp was discussing a discovery he made to day having to do with path mapping and file nodes which did not contain file paths. The bug seems to be that the first tasks are failing but subsequent ones for the already loaded Maya instances seem to work just fine. He has an idea of what he can do to fix that problem, so Iād like to run the script by you once we have something worth testing.
Speaking of meppā¦ Hereās an updated plugin file if youāre interested, works on top of the latest Deadline10!
Itās pretty much like Edwin described. We noticed that mayaās dirmap was failing on file nodes that didnāt actually have a filepath in them. Path mapping only happens on the first task for a Slave in mayabatch, so youād end up seeing the first task for a Slave fail with no render output and all subsequent tasks would render fine. It was pretty frustrating since there was nothing indicating what was going wrong when the dirmap failed.
If youāre curious to see if youāre affected by this bug Iāve fixed. Youāll have āDeadlineRepository10/plugins/MayaBatch/MayaBatchUtils.melā in your repository (one of the later versions of Deadline 10, (ie. 10.0.12) and the following script will tell you if youāre missing filepaths in nodes. Run this in the maya script editor with the scene in question open.
If any node is printed, then we assume it has a filepath to be path mapped, but it doesnāt actually have a path. If you have any output from this script, update the attached file in your repository.
back-up the original file at: DeadlineRepository10/plugins/MayaBatch/MayaBatchUtils.mel:
unzip and copy the attached file to the location above
Let us know if this resolves the first task failing issue. There always the possibility that more might be going on here thatāll need to be addressed.
hey. sry a bit busy here but still struggling with missing frames. render goes actually seems o be going but frames are missing.
still same thing weird most of them are there but some missed and skipped, so if you have any progress on this or something to test with Iām all up for it. Iām wasting TON of time chasing these frames
also since last time Iāve reinstalled back to pure maya 2018 no updates or anything to remove any chance of bugs there and updated repository to latest version as well. stills same thing so whatever you have to try Iām all up for it!
Also did some Houdini renders and cinema as wellā¦ all fine no missing frames so seems to be maya thingā¦not 100% sure butā¦
added another screen grab to see how it looks in monitor
The fix I posted above has not yet been released, itās in an upcoming build. Can you run that piece of code in my above post in the Maya script editor? It should hopefully reveal if my suspicions on empty filepaths being correct.
Otherwise, could you apply the fix to your repository and let us know how that works for you?