Ok,
I may be missing something… but i am trying to get this render done, and it does not matter how i do it… but it just won’t cooperate.
I have tried to render it with voxels, and less particles… but that ended with half hour per frame render times… (for 300 frames…)
I then tried to Boost the # of particles… and render that… worked well up untill the frame 187 (so about 100 more again) and i start getting allocation errors, and I run out of memory. (my system has 10 gig of ram)
Now trying to use the “render particles to disk” feature. Same thing… I get to about 187 and it locks up. Files sizes jump to 2 gigs each…
My pflow source is 1666,666 particles… so not that many… this should handle it!!So i have to ask, how many particles is too many for it to handle effeciently?
Anyways… I have read the manual… and find it alittle lacking in the area of properly using “Particle Partitioning”. So i may be doing something wrong. so any step by step would be great.
Next, once i partition it properly… how do i get it back and rendered ???
thanks
craig
What sort of allocation errors? Can you post the text? You should NEVER see an allocation error in 64bit Max. Does this mean you are running 32bit Max? In that case, it is entirely possible that you are running out of memory. In 32bit apps, there is only 2-3Gb of memory space of which Max itself can eat up 1.5Gb.
So I guess it would be a good time to figure out the specs of your machine (8 core beast right?), avail memory, max version, etc.
Hey Darcy.
How are things??
Ok. here are the specs for my machine.
dual xeon e5462 @2.80 gigs
10 gigs of ram
XP 64 sp2
Nvidia GeForce 8800 gt 1 gig ram ( I think )
500 gig HD for OS and Swap
Storage on External Nas HD raids.
Running Max 2010 up to date
Hope that helps… As for the allocation error, all it says in a little pop up window is “allocation error” with a ok button.
Once i hit that, krakatoa stops working. But the memory does not clear if i watch the Task manager.I have to shut down max after that and restart it to get it to dump the memory. Just a reset will not do it.
Wish there was more i could tell you. LOL, i have to wait about 8 hours for it to have the error pop up… so it is not something that i have repeated too often
Are you making these particles with PFlow?
It is quite possible that the memory error you are seeing is related to PFlow troubles as opposed to a Krakatoa problem.
Any chance to see a screenshot of the flow? Any spawning going on in it? Any 3rd party / Box operators used?
As Darcy mentioned, it is not typical to get memory allocation errors in 64 bit Krakatoa - yesterday we were rendering hundreds of millions of particles and hit 23 GB RAM usage on a machine with 16GB and it just slowed down due to paging, but never crashed. (That being said, we were creating the particles procedurally using Krakatoa objects and not PFlow).
If you can get the particles partitioned to disk, you can simply load the PRT sequences in a PRT Loader (either placed at the origin or with Use Transform unchecked) and they should render like the PFlow ones. But if the problem is in the PFlow, then saving should crash on the same frame. Keep in mind Krakatoa does not load particles into memory when saving, so if you get any allocation errors, these are most probably related to Max/PFlow misbehaving.
Can you tell me if you are using 32 or 64 bit Max? The shortcuts that Max installs should say which one in their name. The other option is to start C:\Program Files\Autodesk\3ds Max 2010\3dsmax.exe as opposed to the C:\Program Files (x86)\ one.
Wow guys… Did not expect to see you reply today!! You need to take a break some time
Either way. glad to see it…but again not expected… thanks
Ok
Darcy.
it is max 2010 64bit.
Bobo, it is all out of pflow and there is some spawning, i will have a screen grab of the flow tomorrow… or if you wish i can send you the file. your choice.
Oh, we take Krakatoa support VERY seriously. Even on Sundays
Scene would be better, because if it crashes for us too, we would be able to debug what is causing it.
So, I launched rendering of the scene (as you sent it to me) and it is taking 18 seconds on frame 80 (first frame took a long time because you are using Position Object > Volume). I rendered 50K particles birth before that and it took less than an hour without problems. With 500K, Task Manager is showing less than 1GB in use.
I will wait to see if it will crash around frame 190, but so far it is looking good. I am using the latest build though.
So I will have to ask you to come over to the Beta forum and try out the latest Krakatoa build we are testing (in case you are not running v1.5.2.40413)
EDIT: Frame 198 took 3m49s here and Max used 5.5GB, no crash so far… This is with 500K initial particles, resulting in about 32 million particles on that frame due to spawning. Actual Krakatoa memory used for particles: slightly over 1GB.
Ok, so let’s assume your initial particle count was indeed 1,666,666 and not 500,000. Right now, on frame 212, I have 44252K particles using 1.6 GB of Krakatoa RAM, but Max is using 7+ GB because PFlow has to keep these 44MP somewhere, and it is by far not efficient as Krakatoa. Frame 212 took 5 minutes 11 seconds to render.
Assuming that 1.6MP would use a bit more than 3 times that amount, you would be already at 21GB of memory (about 6 GB Krakatoa and the rest PFlow). I can see how Pflow could easily cause a memory allocation error when called by Krakatoa, thus appearing that Krakatoa is crashing. Given that you have only 10GB RAM, it would explain the problem. (You are spawning quite excessively in that flow)
Ok Bobo,
thanks for looking at the scene.
I am not sure what is up… but it appears to lock up my system all the time. With the 500,000 even.
I was able to partition it, and that seemed to work. I am now trying to load them into krakatoa and render… but it still locks up at around the same frame. When viewing the task manager it says that i am using 11.6 gigs of memory… But this time, instead of an error it just hangs…
I am not sure how you got it only take up 1.6 gigs… did you tweak the scene at all?
the 40 min renders was using voxel shading… with the particles it was much faster…
c
As I said, it was using 7+ GB on that frame according to Task Manager, the 1.6 GB was the amount of memory used by Krakatoa Particles (you can see the value in the bottom left corner of the Extended VFB panel for each frame if you have VFB Extensions enabled in Krakatoa Preferences).
I did not tweak the scene, I just let it render as it was submitted. My machine has 16GB, so it would potentially cause problems at a different amount if the issue is memory. I am rendering in particle mode because that’s what it was set to when you sent the file. I could try voxels next.
I asked you what version of Krakatoa you are running (exact build number) and whether you downloaded the latest Beta from the Beta forum. Please let me know. Since your system has only 10GB of physical memory, using 11.6GB would result in a lot of paging. Chances are there are problems outside of Krakatoa (Max, Windows or even bad memory chips), unless you can reproduce the same issue on another machine with the same build of Krakatoa. We need EXACT info about every factor in the setup to know what to look for. If I cannot make it lock up on my system, there is no way we can debug the issue…
For the sake of debugging, please try loading half the partitioned particles and try to render the “problem” frame. If it locks up, restart, reduce to 1/4 and try again. Watch the Task Manager and try to find out when exactly it happens. If it happens with only PRT Loaders enabled in the scene and PFlow turned off or deleted completely, then we would be sure it is not related to PFlow at all. The PRT Loader lets you turn on and off each partition individually and also specify a percentage of particles to load. Use them to find the amount that starts causing problems.
Bobo,
thanks for the efforts here. Appreciated. Now, i will try loading sequentially less and less prt sequneces.
It would look like i need to get some more ram! perhaps another 6 gigs, as it appears that i have do not have enough swap space either perhaps… and that is what the issue is It would be nice if there was a way to by pass max altogether when useing prt files and rendering. Just a stand alone render. Just load the PRT files into the render and go.
perhaps something to look into… lol…
opps forgot to add this… but i am running version 1.5.2.39428
Well, Krakatoa used to be stand-alone, rendering just PRTs, but that was back in 2006 before we switched to 64 bit. We could render about 50 million particles with 2GB RAM.
I don’t think 10GB RAM are not enough. If you are loading a single partition from 500K particles, it will have about 44 MP on frame 200 and should need less than 2 GB of RAM. With PFlow off, Max should not take up much more (as Chad mentioned), and even if you had 4 GB already used before rendering, you should not go higher than 6GB.
I have the feeling you are underestimating the counts produced by the Spawning. Use the PRT Scanner Utility to check the total counts of your PRT Loader(s) before rendering as you claimed you had only 1.6 MP when you first posted, but the 500K birth version spawned over 40 million particles around the frames where you got the trouble. So do you know how many particles you really have on the peak frames? The PRT Scanner has the option to draw a red line at the memory limit, if you see frames going above that amount, you would be swapping to page file for sure.
You can of course update to the latest Beta, but it looks like the issue is not Krakatoa but memory usage. Windows 64 bit is supposed to automatically expand your page file when virtual memory is low and can address up to 4 TB of virtual memory, so I don’t know why you are getting lock ups. If paging works, rendering would become very slow but should still continue. Unless your Windows is set to fixed page size without the ability to adjust dynamically…
Hey chad,
I must be missing something then. Bobo was saying…that his system was eating up around 7 gigs or so, but Krakatoa was using only using 1.6 gigs on top of that. I was understanding that it was max eating up the rest. IF i only Max rendering with Krakatoa, and krakatoa is only using 1.6 then max must be using the rest no???
According to the task manager, i am now using 12.8 gigs of ram. According to Krakatoa it is using 1.6 gigs of ram…
that means that if max is only using 500kb then there is about 11 gigs unaccounted for.
Bobo,
my last attemp that ended with render time of 4:49 mins a frame was just with Krakatoa.
task manager saying 12.1 gig of ram used, and Krakatoa saying 1.8
Not ON TOP of 7GB, but INSIDE. According to Task Manager, the 3dsmax.exe process allocated 7.4 GB of memory during the test run yesterday, with Krakatoa using around 1.6 and PFlow+Max using the rest (around 5.8 GB). With PFlow disabled, I would expect Max to use around 500MB or up to 1GB depending on installed plugins and scene content.
Is the 12.8 GB figure the TOTAL used memory by Windows, or the 3dsmax.exe process’ memory?
If the former, it is possible that other apps have used memory. If the latter, what is in the scene?
It would be nice if there was a way to by pass max altogether when useing prt files and rendering. Just a stand alone render. Just load the PRT files into the render and go.