ID channel (and others) not loading into PFlow?

Running 1.5.1.38002 under 2010 SP1 x64…

I have a PRT Loader which the file sequence loader shows as having Position, ID, Orientation, Velocity, and Scale.

But when I pick the PRT Loader with the Krakatoa PRT Birth, or Krakatoa PRT Update, I get a warning that there is no ID channel.

Also, the PRT Update operator only shows Color as an available channel, but that channel wasn’t saved in the PRT file.

Am I missing an important step?

  • Chad

Just as a sanity check, I saved one frame of the PRT Loader out to a CSV. It has expected values for the channels described.

There was a known bug in 1.5.1 where the list of channels was not returned correctly if the current frame did not contain any particles. Is your PRT sequence by any chance empty on some frames?

This was fixed in 1.5.2 (which we posted today for Beta testing). You might want to give it a try.

The Color channel will always appear in the Krakatoa PRT Loader Update because the Stream of a PRT Loader always has a color channel, even if the input PRT sequence did not have one. That channel will contain whatever the color source of the particles is - the PRT Color channel, the Wirecolor, the Channel set by a KCM or the Material’s Diffuse color. So this is As Designed.

Yes, frame 0, the first frame is particleless.

1.5.2 installed now… I’ll check it.

Nice. 1.5.2 works. Very timely… Thank you.

Oh nuts… This isn’t working right…

I’ve got a setup in PFlow with one Source, but ~30 births in different events that get tested out to various other events. So over time particles are being born, killed, and passed between events. Nothing too odd.

In the original PFlow, when I turn on “Show Particle ID’s” the particles birthed early on have low numbers, and the ones born later have high numbers. The ID’s do not appear to be “recycled”.

But when I save that out as a PRT, and load it in to a PRT Loader and put it into a new PFlow source, the ID’s do appear to be recycled. So a particle born at frame 1 will update position correctly for 30 frames until some more particles are born, then it “disappears”, but what I think is happening is that it’s just updating the position to one of the new particles, because the new particle has the old particle’s ID. Does that make sense?

I noticed that the only option is to save “ID” but I’m wondering, which ID is that? The Birth Index? Or something else, like Event Index?

  • Chad

The ID channel saved by Krakatoa is the Birth Index (.particleID in Script Operators). That ID should be UNIQUE for a particle in a single PFlow system, other than the .particleIndex which is the internal numbering within a container without gaps.

When you save a PRT sequence with an ID channel, it is also important to remember that if you have two PF_Sources in the scene, each will have particles with potentially the same IDs since the IDs are given per system and not per scene. So it is generally a good idea to save each PF_Source to its own PRT sequence.

The Krakatoa PRT Loader Birth and Update and the Krakatoa File ID Test use their own “File ID” channel to keep track of what is in the scene vs. what is in the files. The Krakatoa PRT Loader Birth operator loads the PRT frame, looks through our own FileID channel of the scene particles and if the ID of the particle to be loaded is not found in the FileID channels of the existing particles, it gives birth to a particles and marks it with the loaded ID channel value in the FileID channel. The actual Born ID channel of PFlow could be different since those IDs are its internal affair and we don’t touch them.

On the next frame, if the next PRT file also contains that same particle with the same ID, the Birth operator finds a particle with matching FileID value and does NOT create a new particle because it is alive already. If a particle in the system has been created earlier but it is NOT found in the current PRT file, its File ID channel value is set to -1 so you can test it using the Krakatoa File ID Test operator and send it out for deletion. If you somehow delete a particle whose FileID still exists in the ID channel of the following PRT frames, it will be recreated.

The Update operator also uses the FileID in the scene and the ID channel in the file to decide what particle gets what data. So if you have given birth to a particle with ID 42, the FileID channel will contain 42 and the particle will get its Position, Orientation etc. from the PRT file’s particle with ID channel of 42. If you happen to have duplication of particle IDs in the file due to multiple PF Source saved to the sequence, there is no telling what data will end up in the particle and it could jump around…

So if you have 30 Birth operators in a PFlow, the IDs given to particles will most probably reflect the order of creation within the original system which is an internal feature of PFlow and we have no control over it. You must store the ID channel when saving to a PRT sequence to be able to recreate the birth and death of each particle when loading back into PFlow, but you don’t care what the ID values are (as long as they identify a particle uniquely). I am not sure whether PFlow does any recycling of IDs of dead particles (it might do that), and I am not sure why your setup would not work if it was created along the above outlines.

After reading this, do you see anything you are doing differently?

I only have one PF Source. There are a bunch of birthing events that test the particles into a main dynamics event.

So if a particle was killed in the Pflow that made the PRT, the PRT Loader Birth won’t kill it, since it’s just birthing, so I need to use the File ID Test to remove it? If that’s the case, then that’s the step I was missing.

  • Chad

Exactly. The test is more universal since it lets you send out particles with any range of IDs, but the default is to send out Negative File ID which are the ones marked as “Dead”. The Birth does not kill them, just brands them as “Death Row”. It is up to you to decide whether they should be deleted or just kept in another event (for example for debugging purposes).

Note that the latest RealFlow version copied exactly the same design for their PFlow operators, including the Test for deleting particles.

More questions…

I’ve got a setup with PF Source (Render) -> Event 01 (KrakPRTBirth, KrakPRTUpdate, Display, SendOut) -> Event 02 (Display)

If I turn off the Send Out, I get 86 particles and they show up in the view. If I turn on Send Out, I get 0 particles, and of course nothing in the display.

I tried replacing Send Out with a Krakatoa File ID Test set to >-1, but that didn’t help.

Seems odd that I can’t send to another event.

What I’m really trying to do is figure out a way to make Box#3’s “New In Event” work with the PRT Birth. Seems that it doesn’t filter properly when the particle count changes. I thought I’d pass the particles out to a new event and see what happened there, but I couldn’t get that working either.

  • Chad

But the operators only work on the particles in the current event, except for the Birth which always checks the IDs in the whole system.
So if you send all particles to the second event, I would expect the Birth to still work ok creating the missing particles, only the Update wouldn’t see anything but the newborn ones passing through it on the way out.

To have the Update working on the whole system, try moving it up to the global event or instancing it in both events.
The FileID Test could also be added to both events to kill particles that have a negative channel to ensure correct death.

Couldn’t get it working in my scene, but I tried that in a new scene, and it’s perfect. So there was just a bit of wacky infecting my current scene and particles just weren’t being made correctly.

My working setup ended up: PF Source (Render, KrakPRTUpdate, DataOp[Shape], Display, KrakFileIDTest[<0]) —> Event 01 (KrakPRTBirth), with the KrakFileIDTest passing to a Delete.

Saved a PRT with TextureCoord and Mapping2…

Krakatoa PRT File Update shows Mapping2, but not TextureCoord. KCM reads in TextureCoord just fine. Is that expected?

For now I’m using a KCM to copy TextureCoord to Mapping3, so I have a workaround.

Hmmm… Seems MXSInteger is not in the list either…

  • Chad

I saved a PRT with all channels (except for 99 Mappings) and loaded in the Update to see what comes over.
The channels supported appear to be:

Position, ID, LifeSpan, Age, Orientation, Velocity, Color, Acceleration, Scale, MXSVector, MXSFloat.

Not supported are:
Density, Normal, Temperature, Fuel, Fire, SignedDistance, MXSInteger, MtlIndex, TextureCoord, Tangent, SpecularLevel, SpecularPower, Eccentricity, Emission, Absorption.

Most of these make sense except for TextureCoord and MXSInteger, as you reported correctly. I assume they were simply overlooked, or there might be some deeper reason I don’t know about (I will ask next week and log as needed).

And no custom channels are brought over. Which isn’t terrible, since you do get 99x3 channels via Mapping, but would be nice if we could name them something catchy, or in cases where I want integers or unary data or whatever.

Speaking of switching channels around, is there a way, via KCM or something else to remove a channel? Like I couldn’t read in TextureCoord, so I copied it to Mapping3, but then I had duplicate data. I couldn’t figure out how to get rid of the TextureCoord, except for saving out a new PRT sequence with that channel missing. Is that the only way?

  • Chad

The new PFlow operators attempt to map the incoming data to the right native channels of PFlow. (The old ones allowed for remapping, but we are not supporting them anymore). So Position always goes to Position, Color always goes to Vertex Color channel, Orientation goes to Orientation, MXSFloat into MXSFloat and so on.
That’s why custom channels and some derivative channels like Normal and Tangent are not supported as PFlow has no corresponding channel to accommodate them.
MXSInteger and TextureCoord have been probably overlooked.

There is currently no way to kill a channel. Most channels exist only during loading and are not used by the renderer directly (TextureCoord is loaded and used by KCMs and Maps, but not by the Renderer, so it does not get cached in the PCache), so I am not sure how bad this is for memory usage.
Let’s wait to see what Darcy thinks about this.

Oooops, sorry, forget I said anything PRT Update does not read MXSInteger as you stated. Not that I said that it did, I was thinking the other way around.

Oh right, you don’t assign the channels, only load them. So of course only standard channels would work.

So unused channels aren’t used at all, other than loading off disk? Seems reasonable.

  • Chad

Looked at the docs, but didn’t see how the mapping between Pflow and BINs were done. What channels can we save to BINs and where does that data map from?

  • Chad

Ugh.
You could take a look at the RealFlow BIN loader for PFlow, it lists the channels they support.
(Or you can look in the File Sequence Manager of the PRT Loader for a list of all channels in a BIN file).
Most of these channels do not contain info we can deal with. For example, Density in a BIN file is often 0 and causes it not to render :wink:

We get the Positions and Velocities and that’s about it. You could use KCMs to remap existing channels from the BIN to something else PFlow can read (like Mapping2 and higher). When Krakatoa is saving BIN files, we leave many of these channels with default values and I heard that causes severe slowdowns when loaded for meshing back into Realflow, probably because of missing SPH-related data like Neighbors etc.

I don’t know much about BIN files. We will have to look through the code and see what was done there. Having Flood, we rarely use RealFlow anyway.