is there a way store the position, IDs and velocity of the particles into cache, so that regardless of changing shader or lights it will have this in memory?
i have tried using pCACHE but it seems it always has to retrieve particles (or i have to turn it off) in order to do any shading changes?
No, currently the material is applied to the particles after the stack evaluation and before loaded into the memory cache.
We have had a wish logged against this for as long as Krakatoa has existed, but it is very tricky and getting trickier as time passes and new features are being added.
The original idea was to have a channel that keeps the ID (nodeID) of the object the particle came from, so that changes to a material of that object would cause only the particles that are affected by the change to be updated in place in the cache without reloading.
Changing the lighting is possible though as long as the LCACHE is not checked.
ah ok,
but what about the simple act of retrieving from disk, once particles are loaded into memory, would it not still be faster to retrieve them “again” from memory and apply different colors, shader attributes, etc.
than having to physically retrieve from disk again?
eventhough all this happens befre loading into memory cache,
is there a way to keep things in memory cache and retrieve them again from there? that way we can go from memory, load, shade, etc. back to loading into memory, just work in memory space instead of falling back to disk?
does that make sense?
Here is the problem: In order to evaluate a material, you need to know what material was assigned to that particle. For that, you need to know the scene object it came from. Then, if the material has a map that requires mapping coordinates, you need that data, but IT IS NOT CACHED in memory because materials are evaluated during the loading, not during the rendering.
Right now the pipeline is (as shown in Krakatoa Schematic Flow):
*Get Particles from object (which could be PFlow, PRT Loader, PRT Volume, PRT FumeFX etc.)
*Evaluate the object space modifiers including deformations, selections, Magma/KCMs and Krakatoa Delete
*Transform the paticles into World Space
*Evaluate the world space modifiers including deformations / Krakatoa Skin Wrap
*If the base object is PRT Loader, apply the Culling
*Evaluate the Material tree
*Evaluate the Global Channel Override Magma modifiers
*Apply the Global Override Values (Color/Emission/Absorption/Density) to the channels
*Store the channels listed in the “Memory Channels” rollout - we try to keep the number of channels low to fit as many particles as possible.
If PCache is ON, once the data is written, it is kept in memory, and any operation that does not require rebuilding of the channels can be performed directly.
This includes Final Pass Density changes, Emission Strength changes and Camera-space effects like Phong Specular Highlight and Environment Reflection Maps (but not the new Raytracing which happens at Material level). Note that when a shader has the >Per-Particle option unchecked, the value to the right is used AT RENDER TIME and is not cached, so it can be manipulated freely. For example, you can change the global setting of the SpecularPower or SpecularLevel with PCache on and rerender without reloading. Same applies to the Hair shaders and the Phase of Schlick and H/G shaders.
As I explained already, we could store a channel that points back at the object that provided the particle and monitor its material or object changes, thus reloading only the particles that have changed while keeping the rest in memory, or evaluating only the material (assuming we also cache any mapping channels used). But the more features we add to the Krakatoa Renderer, the trickier it becomes to support such a workflow, so it might never happen.
It would still be nice to have an additional layer of cache between the disk and the PCache. Some ideas:
Maybe you read the UNC path in the PRT Loader so you can do easy network renders, but you optionally copy the PRT’s to a local path, like perhaps an SSD or RAM drive. That way they are accessible at what could potentially be a faster speed. The copying locally could be done on a different thread and wouldn’t hold up the initial render, but subsequent renders would read from the copy. And yes, this could also be a feature for Deadline in general, so that’s something to discuss too.
Maybe you store the “base object” particles in a separate cache. We’re finding that our modifier stacks are the things that are both fast AND change a lot, while the base objects are things that are slow AND change less often. So for PRT Loaders, or PRT Volumes, or PRT Sources, it would be nice if the output of those base objects were kept in a separate cache that was only invalidated when the base object was invalidated. This cache could be optionally in RAM or on local storage as automatically generated PRT files.
As an alternative to 2, maybe have a modifier that did PRT I/O? So it would be like PointCache2, but for PRT’s? The benefit is that you could put it anywhere in the stack, and it would be simple to toggle on and off.
None of these would affect the PCache now, meaning all would invalidate it. But it’s still better than going all the way back to the start of the pipe.