Don’t hate me, but if I had a Pflow system where I am making 5 partitions, but I have a disk cache event, would it be possible to increment the disk cache filename, so each partition gets independent cache?
- Chad
Don’t hate me, but if I had a
Pflow system where I am making
5 partitions, but I have a
disk cache event, would it be
possible to increment the disk
cache filename, so each
partition gets independent
cache?
Since this is done on the scripted side of things, it is relatively easy to add. I would prefer not to change the GUI at this point though, but we could add an internal property to set via MAXScript to switch Krakatoa to a special Chad mode that does that. (we could expose an additional control in a point release if it shows that it works well).
What is the pattern you are using to increment your cache? SomeFile01_, SomeFile02_ or something else?
Name doesn’t matter, since I don’t need the cache once the PRT’s are written out. We just want the disk cache to be there in case a partition fails halfway through, so it doesn’t have to start at the beginning.
A post-1.0 “Chad mode” would be cool. Will there be a 1.01 beta that I could join?
- Chad
Name doesn’t matter, since I
don’t need the cache once the
PRT’s are written out. We
just want the disk cache to be
there in case a partition
fails halfway through, so it
doesn’t have to start at the
beginning.
A post-1.0 “Chad mode” would
be cool. Will there be a 1.01
beta that I could join?
I am still not sure how this would work. I assume we are talking about Disk Cache from Box #3. Do you want to SAVE the cache somehow while partitioning? Or READ from it while partitioning (that would defeat the purpose of partitioning)? How can a Disk Cache help you in a crash? Do you precache or do you expect it to save while you are partitioning and save yourself from rerunning the beginning of the calculations? I am not sure if that is possible in Slave mode at all.
I haven’t used Disk Cache myself, I tried to change the name today via MAXScript and it automatically renamed itself back, but I guess I should be adding the part1of5 pattern or something, since the cache files are frame numbered already.
I really need a solid specification of what you want, otherwise I cannot help. :o)
Cheers,
Bobo
Yeah, the idea is that as the Pflow is processing, it gets to the Disk Cache operator from Box3 or Box1 or wherever that came from, and saves out a file. Krakatoa subsequently saves out a PRT for that frame too. In a perfect world, the Pflow disk cache would never be read.
If, however, something happens, like a power outage, or someone being an ninny and hitting requeue task; when a new slave starts that task, Pflow will see the disk cache, read the previous frame, and continue on it’s merry way, preventing a lengthy run-up (which would possibly timeout).
If the name of the cache isn’t incremented along with the partition, each partition job will overwrite the other the cache and weird crap will happen.
As for the disk cache being saved by the slaves, I remember Oleg saying you could submit a render for one machine with just a really simple rasterizing, like with objects hidden, tiny output image, all the features deactivated, and just get the particles to render, then re-submit the job to the whole farm with all the rendering stuff turned back on. That way you could get the particles taken care of first, so you wouldn’t get redundant run-ups.
I tried this a while back with Krakatoa before we had the PRT Birth/Position events, and it seemed to work, and in production we’ve done it with TP, and it made a huge difference for decongesting the farm.
Giving the disk caches the same suffix as the partition PRT’s would work fine for me. I’d be cool if the prefix was based on the disk cache operator name, too.
I’d be happy if we added something like PARTITION or RENAME or something to the disk cache, like we have with NOSEED so it can be controlled which disk caches get modified when the partitions are made.
- Chad
Wow, this is a pretty cool thing actually!
I just put it in and tested it and it indeed caches as it goes and then uses the cache if available if you try to partition again, running through the frames that are already cached much faster and then continuing slower when it hits portions that have no cache files.
It will respect the same NOSEED rules as seed changes - if you put NOSEED in the Cache operator’s name, or its event or PFlow, it will not be incremented. Having a separate test for it would be really tricky.
The original file name and path will be used, with partMofN added to it when a partition is made and removed again to the original name when a partition finishes.
Obviously, the Cache has to be set to “Render” or “Viewport and Render” mode for Krakatoa partitioning to take it into account.
Will be in 1.0, (actually, in the next Beta) so no need to wait.
D’uh. Yeah, you’re right, NOSEED would be fine here. What was I thinking?
- chad