I’m saving out Position, Velocity, and Map Channel. Any guesses as to what might be going on? I’m going to try narrowing this down, but didn’t know if these messages helped.
Changing from Krak to Scanline back to Krak fixed it. This was a file from beta 15 or so, so there may have been a naughty leftover bit. Shouldn’t affect release versions.
ERR: Could not read/write file type: \anatomical\prj\RO3203\scn\05_0100_Kidney_Establish\PRT\Test_A06\MeltingFatTest\PerformanceTesting__part2of10_0000.prt
ERR: Cannot invert matrix [(0, 1.16323e-033, 0, 0),(1.88047e+024, 0, 0, 0),(0, 0, 0, 0),(0, 0, 2.10195e-044, 1)], it is only of rank 2
In Beta 18, we decided that saving particles should be identical to rendering particles with regard to color calculations. Since some methods (like blended Z depth etc.) require a camera to calculate the color and density, we now handle saving just like rendering and some bugs related to cameras might pop up in the saving process until we fix them generally.
Also the writing to Vertex Color channel mentioned in the submission is NOT the solution we promised you - next build will have a dedicated PFlow operator which will enable the copying of Scripted Vector Channels into Color and Scripted Float Channel into Density so you will just have to drag one to the flow to enable the fast shading behavior.
In beta 18, you can use the Data Operator to write colors to the Vertex Colors channel and it will shade directly much faster if you check the new “Ignore Materials” override. But this is just a temp. solution until next time.
Nope, the bug was fixed today, in the future you should be able to render from any view on Deadline. Some data was getting lost when passed to the Max Deadline plugin…
What does Deadline need for a progress update? It’s not watching buckets go 'round, it’s waiting for the particles to evaluate. If Deadline can’t see that happening, will assume that nothing is happening and that the process has hung?
Right now my tasks are running close to 2 hours per frame.
Chad
EDIT: Oh crap. Sure, it fails on the 5th frame, then moves on to frame 6. Now it has to calculate not 1, but 6 frames, at 1:40:00 per frame, that's going to be way over the update timeout, which was set at 8000, and I upped to 16000, but even that won't be sufficient. If it's not getting updates on the particles evaluating frames, then we're definately screwed.
Problem is, I really need to have Deadline force sequential, which it isn't, and I need it to either get progress updates from pflow, or barring that, have a progress update timeout for particle partitioning that's separate from the progress update timeout for regular rendering. If we removed the progress update timeout entirely, our Brazil renders would all hang machines indefinately.
I cleared the error logs and even re-queued the whole job and set the first few frames to be completed, but the slaves insist on rendering out of sequence.
When Krakatoa is loading particles or doing other things, it updates the progress and can cancel. When particle flow is updating, Krakatoa doesn’t get any feedback, so there are no updates until particle flow returns. I asked Oleg if there was a way to get the progress from particle flow, but he didn’t know a way so it’s likely not possible.
You definitely have the ‘enforce sequential rendering’ flag enabled on this job? If so, that sounds like a bug in Deadline. I’m not aware of it having in 2.7 before.
Yeah, I enabled it at submission, and I verified it was checked in the job properties.
It only has the problem if it encounters an error, but in this case the error was it not finding the FlexLM server, so it would have been best for it to just re-try the same frame.