Here’s the random Krakatoa thought of the day - might be covered already, maybe not 
– Something I’ve found myself constantly wishing for is analysis tools to help me manage the particle count that I’m loading in the scene. For example some of the particle passes I’m using start off pretty low, then increase massively as particles are spawned. The questions I keep asking myself are things like:
How many particles are in a specific partition at a specific frame?
What’s the rate of increase?
What’s the peak particle count for that entire partition?
What’s the total particles count for all partitions?
What’s the peak particles count for all partitions?
What’s the memory usage estimates for these partitions? AKA How many Partitions can I get to render with the available memory?
Hmmm, when am i going to run out of memory?
What’s the max amount of particles I’ll be able to use given X GB of RAM?
Currently all these things are covered by writing vague estimates down on postit’s - whose accuracy is only determined when I find the “Bad Alloc” error staring me in the face first thing in the morning.
I’m jealously looking at some of the particle counts people are reporting round here - I’ve generally topped out at 25m on my 64-bit 4GB machines. I don’t know how I can optimize and get higher amounts when I’m so often shooting in the dark with my typically wrong estimates to guide me 
I’d be delighted if these sort of tools found their way into the particle loader roll-up!
Anyone have any thoughts?
Would be excellent if the PRT Loaders would tell me how many particles were in each partition and in all the partitions total. That would help me.
For particle counts, I’m doing all my particles in one file, and my rendering in another. It’s the only thing that can get me over 50 million.
- Chad
Some good ideas!
We intend to add particle count info to the PRT Loader and to the related rollouts. There is even a hidden label in the scripted plugin that was intended for such feedback, but we haven’t implemented the method to get the data from the loader yet. :o)
Telling you when you are going to run out of Memory might not be that easy though…
Question - are you running Max 9 32 bit on 64bit Windows? Rendering 50 million particles from 10x5M partitions shouldn’t be a problem with 64bit Max 9…
Cheers,
Borislav “Bobo” Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
Max 8 on xp64. 52 million is all I can currently get without shutting down a lot of processes.
Hmmm… Where did that max9 cd go?
- Chad
Question - are you running Max 9 32 bit on 64bit Windows? Rendering 50 million particles from 10x5M partitions shouldn’t be a problem with 64bit Max 9…
Nope - fraid not. Been sticking with Max 8 so far as we were locked to a couple plugins that haven’t been updated yet (namely Box 3 and fumeFX) They’ve been updated now, but I’m hesitant to doing any new installs when we’re in crunch mode…
/edit/ oops, and we’re stuck with only 2 64-bit machines too.
Here’s the results of an interesting experiment. I figured it would be possible to guesstimate how many particles were in a file based on the filesize and settings used to save particles to that file.
for the below figures +/- denotes:
Moving = moving vs. stationary particles
Colors = Save Particle Colors ON/OFF
Normals = Save Particle Normals ON/OFF
+Moving, +Colors, +Normals
500k particles = ~7.2 Mb
1m particles = ~14.3 Mb
2m particles = ~28.6 Mb
+Moving, +Colors, -Normals
500k particles = ~4.3 Mb
1m particles = ~8.6 Mb
2m particles = ~17.2 Mb
+Moving, -Colors, -Normals
500k particles = ~4.2 Mb
1m particles = ~8.5 Mb
2m particles = ~16.9 Mb
-Moving, -Colors, -Normals
500k particles = ~6.0 Mb
1m particles = ~11.8 Mb
2m particles = ~23.7 Mb
Thought it kinda interesting that the stationary particles had larger filesizes than the moving ones? Weird.
Scene used for generation attached. Gonna see if I can write me a quick maxscript for future guesstimation purposes…
You don’t need box3 or whatever for rendering. Make you particles in max 8, then render them in max 9.
- Chad
ya I know, just don’t want to drop 9 into the mix right in the middle of a project and crunch time at that!
Usually that’s very, very bad thing to do…
But dropping beta 10 of Krakatoa? That’s perfect! 
- Chad
heh - got no option on that! But it’s a pretty limited risk that I’m happy to take 'cos the Frantic guys are great and I have faith in them. Autodesk’s technical department on the other hand … sheesh …
If Krakatoa was Max 9 only I’d have had to consider it. But it ain’t, so I don’t.
Some technical background details:
Currently, the particle stream (and the PRT file) contains the following channels:
Particle Position
Particle Velocity
Particle Density
Particle Color (optional)
Particle Normal (optional)
The first 3 channels will always be there, even when particles are not moving.
Currently, the only 2 channels that can be disabled to save disk space are Color and Normals. So measuring moving vs. stationary does not really make much sense - the difference will be the content of the velocity channel which will be 0,0,0 in the stationary and non-zero in the moving version.
Also note that the PRT file is COMPRESSED using ZIP compression, so the actual content could affect the final file size.
It is worth mentioning that the general implementation of the particle stream allows for arbitrary particle channels (which will be likely exposed in future versions of Krakatoa), but the current Beta and the upcoming 1.0 will support only this fixed set of channels. It will remain compatible with future versions with additional channels (but would just skip data it does not know how to use).
So the disk size of a PRT file does not necessarily represent the size of the content due to the nature of ZIP compression.
The latest Beta Build 10 provides a PCache option which also reports the number of bytes actually used to store particle data. (Right-click the PCache or LCache button!)
I created a Geosphere with 10 segments and rendered with PCache on - the cache for 1002 particles had a size of 38076, which is exactly 38 bytes per particle. Also note that the LCache data is already included in this number.
Cheers,
Borislav “Bobo” Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
> ...(Right-click
>the PCache or LCache button!)
Ha! Never underestimate Bobo! Excellent.
Regarding the PRT files, what precision is the data stored as?
- Chad
:o)
The version you have does not clear the number once you release the cache. (The memory is released, it is just the method that still returns the old number). Also, for the next build we switched it to show MB instead of bytes. Long integers are really hard to read, and you don’t really care much about the size if it is below a Megabyte…
Here is the actual cache size info:
Position:12 bytes (3x4)
Velocity: 6 bytes (3x2)
Density: 2 bytes
Color: 6 bytes (3x2)
Normal: 6 bytes (3x2)
Lighting: 6 bytes (3x2)
---------------------------
TOTAL: 38 bytes.
The 6 Lighting bytes are currently always there even when you don’t have the lighting cache enabled. Right now, all 38 bytes will be used per particle even if not needed.
This will change in the future - we just finished refactoring the internal stream structure so disabling the Normals or Lighting would remove 6 bytes each per particle from the memory footprint.
When saving to PRT file with Color and Normals enabled, it saved 32 bytes (all but the lighting info of the cache). If you disable saving of Normals or Colors, these bytes are really not included, so
Position:12 bytes (3x4)
Velocity: 6 bytes (3x2)
Density: 2 bytes
-------------------------
TOTAL: 20 bytes
is the lowest data amount per particle is 20 bytes. Color adds 6 and Normals adds 6 for a total of 32. The PRT format allows for arbitrary channels, so in the future you might see options to define your own channels and store whatever you want in them.
Cheers,
Borislav “Bobo” Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
That’s IEEE real4 float and real2 half then?
I understand the importance of position, but it seems that density and velocity are only useful if you scale density or have motion blur, no?
Seems like you could save disk space, memory, and i/o by not having extraneous channels.
- Chad
Well, YOU could save memory and disk space.
We have NEVER rendered particles without velocity, Frantic is a motion pictures company 
Seriously, you are right.
I also think that Density should be optional, just like Color and Normals. I could imagine that the Velocity channel could also be one of those channels you can turn off if you don’t intend to do motion blur. When we wrote the PRT output, there were no PRT Loaders that could provide alternative color sources or manipulate the density in good ways…
We should give you more flexibility to specify what is being saved, loaded and rendered. We are working in that direction right now - our current build was refactored to use a new internal particle stream format that should make this the natural way of working in the future. But we don’t know whether we will do this for 1.0 or for a later update - we don’t want to compromise stability so late in the game. If our current build turns out to be performing well (so far it does), I would imagine that we could at least make more PRT channels optional.
The Krakatoa you are testing right now was initially written with a static set of channels with the plan to make them customizable later. The PRT file format was developed from ground up to be extensible, so the channels can be enabled and disabled, but at this point we haven’t given you all the controls needed to do so for each channel.
For out production needs, we implemented the channels we needed in the stand-alone version of Krakatoa and some of those design decisions came over in the Krakatoa for Max project, but we are working on improving this. We had to save ALL the data from Max to PRT to be able to render on a stand-alone version of Krakatoa which could not get a color or density from anywhere else - all the data had to be in the PRT. It also explains why we have an external geometry format for matte objects.
Now that we are working on a Max-specific version, we are gradually revising some of the past design decisions - compare the first Beta build with what you have now and you will notice the difference…
Keep the ideas coming!
Cheers,
Borislav “Bobo” Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
We keep coming up with ideas and suggestions so we can keep the beta going and avoid paying.
Will you be releasing a PRT spec to help us in making custom PRT editing tools?
- Chad
LOL!
We will release Krakatoa 1.0 when we feel anything you wish for is a post 1.0 feature and all known bugs are killed. Keep on asking, we push some stuff to the wishlist, but some ideas ARE valid.
As for PRT specs, I think that’s the plan.
Borislav “Bobo” Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
Some good ideas!
Here is some peek into WIP for the next build. This is the alternative output to HTML, the same data can be generated and seen inside of Max using the PRT Loader Scanner Utility.
The grid in the background of the graphs is currently set to 10 frames and is customizable, as are all colors.
Thanks for the ideas!
Just a note that this tool will most probably be expanded to scan the scene for ANY type of particles, incl. PFlow, TP, Geometry Vertices, Max particles etc., so the SCENE TOTALS would really represent what would be loaded and rendered. Also it would have to respect all the % settings in loaders, flows and also take into account custom playback settings like Playback Graph, Load Single Frame etc.
Once again, this is work in progress, but looks promising.