I have a 2-Cpu Xeon server with 32 Gb of DDR3 RAM. 90% (or more) of krakatoa’s rendering time is sucked into the “retreiving particles” step so the cpus are almost idle (6% total) most of the time.
Is there any known mean to speedup the particle transfer to RAM ? What slows down the process ? HDD benchmark shows 100 Mo/s performance, RAM shows over a Go/s speed, and my PRT files are 45 Mo each.
The math tells me each files should be loaded in less than second but that math is probably wrong because it takes much longer than that.
Note : i can’t activate Particle caching (Pcache) because each PRT holds particle positions for each time frames.
In light of your other post where you were trying to load 1000+ sequences in a single PRT Loader, it is quite possible that you were doing 1000x the necessary work and thus it would take 1000+ seconds to load…
If you load a single frame from the sequence in the PRT Loader and render, it should take as long as you probably expect.
The typical loading speed here on my machine is about 2.3 million per second. I just loaded 10 MP in 4.375 seconds as a test. But keep in mind that file access overhead (opening and closing streams) slows the process down, as explained here:
So loading one file with 10MP particles (as I just did) is faster than loading 100 files with 100K particles each. I just tested that, too, and got 6.531 seconds for the loading time of 100 partitions with 100K each (2 seconds slower). This would be a much heavier hit if the files were on the network.
But if I would load 100 partitions of 10MP each, then I would be loading 1 BILLION particles, and I don’t even have enough RAM to fit them in, so the paging would kill the performance (although on a 64 bit machine it would not crash). I suspect you were loading the same sequence multiple times (if I understand your other post correctly), so it would explain why your loading times were so long.
Yes, the slow render issue was linked to the fact that i was loading not 100 - but 500 times the same particles. Render time has been cut from 12 hours to 1hour …
Thanks for the link, good to know a big partition is better, with the RAM limit in mind.