I have a Krakatoa job that I sent to my renderfarm to be PRT’ed. But when it hits the renderfarm it just hangs and doesn’t seem to processes the particles. It ends up crashing the deadline slave and I noticed that the 3ds max process is taking upwards to the tune of 15 gigs of RAM! Anyway, I don’t know what’s going on and was hoping someone could help shed some light on this. I have attached the slave log from one of the machines if that’s useful. The scene is using Kraktoa 2.1.3 and Thinking Particles 4.0.75. Thanks.
deadlineslave(Ibm002)-2012-08-29-0000.log (893 KB)
How much memory does it take if you make the partitions locally?
I think it’s something with the scene that I am working in. I tried it locally and it went to 20+ gigs and froze my computer. I took the whole particle system setup with it’s dependent geometry and forces and merged it into a brand new scene and now it seems to be PRT’ing ok on my local machine and on the renderfarm. However I don’t know what’s with the scene and I am not sure what’s going to happen when I want to render the prt’s with Krakatoa. Taking it one step at a time here…
Krakatoa has a calculator that lets you estimate your memory usage. It’s in the Memory Channels rollout at the bottom. You set up the render with the channels you will use, then enter an estimate on the number of particles. So if you allocate for Position, Color, Lighting, Velocity, Absorption, Emission, Eccentricity, Density, you will use ~46 bytes per point, and if you tell it to calculate for 350Mp, it will tell you you’ll need 15354MB of memory. Framebuffers and mattes will take up more memory, but it’s a good starting point.
Wishlist item: List the memory use in GB. I would rather see 15.4GB than 15354MB. 
When saving PRTs, the calculator plays no role since Krakatoa is not loading anything into memory (unless you are saving lighting as emission).
All the memory in this case seems to be going into the generation of the particles, Krakatoa has probably nothing to do with the issue.
As for the GB, sure, I can do that!
I was pointing out the calculator in response to "I am not sure what’s going to happen when I want to render the prt’s with Krakatoa. "
Based on the log file, I think these are the relevant portions:
2012-08-29 11:25:20: 0: INFO: >Partition 6 of 10 - Frame 680 (1/366) - Elapsed:00:00:01 - Remaining:00:08:41
2012-08-29 11:25:36: Scheduler Thread - Cancelling task because task filename "\\server-2\deadline_5\jobs\007_100_999_3d294db1\tasks\007_100_999_3d294db1_00005_6-6.Rendering.Ibm002" could not be found, it was likely requeued
2012-08-29 11:25:36: sending cancel task command to plugin
2012-08-29 11:25:38: 0: In the process of canceling current task: ignoring exception thrown by PluginLoader
2012-08-29 11:25:47: Scheduler Thread - In the process of canceling current tasks: ignoring exception thrown by render thread 0
2012-08-29 11:29:17: Info Thread - Using process affinity of 16 CPUs
... snip ...
2012-08-29 12:17:43: Scheduler Thread - Synchronizing job files
2012-08-29 12:17:48: Scheduler Thread - Synchronization time for job files: 4.883 s
2012-08-29 12:17:48: Scheduler Thread - Synchronizing plugin files
2012-08-29 12:17:49: Scheduler Thread - Synchronization time for plugin files: 1.232 s
2012-08-29 12:17:50: Constructor: 3dsmax2012-08-29 12:27:36: Slave - Exception: Failed to update slaveInfo: Could not find file '\\server-2\deadline_5\slaves\Ibm002\Ibm002.slaveInfo'.
2012-08-29 12:27:36: Slave - No slave update error notification address specified in Repository Options - cannot send notification
2012-08-29 12:28:16: Slave - Exception: Failed to update slaveInfo: Could not find file '\\server-2\deadline_5\slaves\Ibm002\Ibm002.slaveInfo'.
... snip ...
2012-08-29 12:37:21: Slave - Exception: Failed to update slaveInfo: Could not find file '\\server-2\deadline_5\slaves\Ibm002\Ibm002.slaveInfo'.
2012-08-29 12:37:42: Slave - Exception: Failed to update slaveInfo: Could not find file '\\server-2\deadline_5\slaves\Ibm002\Ibm002.slaveInfo'.
2012-08-29 12:38:13: Slave - Exception: Failed to update slaveInfo: Could not find file '\\server-2\deadline_5\slaves\Ibm002\Ibm002.slaveInfo'.
2012-08-29 12:42:35: Scheduler Thread - Cancelling task because task filename "\\server-2\deadline_5\jobs\007_100_999_510aba27\tasks\007_100_999_510aba27_00001_2-2.Rendering.Ibm002" could not be found, it was likely requeued
2012-08-29 12:42:35: sending cancel task command to plugin
2012-08-29 12:42:37: 0: In the process of canceling current task: ignoring exception thrown by PluginLoader
2012-08-29 12:42:47: Scheduler Thread - In the process of canceling current tasks: ignoring exception thrown by render thread 0What they seem to show is a Deadline issue causing a task to be cancelled (Scheduler Thread - Cancelling task because task filename “\server-2\deadline_5\jobs\007_100_999_3d294db1\tasks\007_100_999_3d294db1_00005_6-6.Rendering.Ibm002” could not be found) and then there is some sort of problem cancelling the task since we don’t see another job pick up until 50 minutes later.
Can you confirm that this is what you mean when you say it hangs? There were jobs the machine should have been picking up during this period right (ie. the job list wasn’t empty)?
No, that was the only job on the farm at the time, so the fact it didn’t pick anything else up after it canceled seems correct. I manually canceled at the time as well as it was sitting at .3% rendered and not going anywhere.