I was wondering about how people usually hand network traffic and space with PRT files. We have an clustered server for all our renders and scene files, space (and nightly backup speed) on this server is fairly precious and given PRT files can quickly add up to 100s of GB in even a short sequence we decided to build a dedicated server to hold our PRT files. This server is just a bog standard machine running Windows Server 2003, but we’ve noticed that Krakatoa PRT files are killing it.
We have a scene that we decided to smooth out by creating 50x PRT files each holding 200,000 particles. When we come to network render, we find that anything over 2-3 render machines trying to access these files just kills the server … in fact we see render times going from 2mins per frame with 2 computers, 13mins with 6 and then with 8 … just nothing comes back. I realise that these files are big … but to my mind it’s nothing compared to having a few artists running comps pulling in many layers of DPX files.
Based on our observations with this one setup … it seems it makes more economical sense to have artists write out PRTs to their local drives and render locally rather than on the farm … which seems a terrible waste of resources.
I’d be really keen to know how others handle the large amounts of data the PRT workflow creates.
From performance point of view, it makes more sense to create 5 partitions with 2MP each. The opening of multiple files adds some overhead as shown in the FAQ (although those tests were from a while ago AND on the local workstation): software.primefocusworld.com/sof … Partitions
Having multiple slaves hitting the network for PRTs is indeed a problem and there is a sweet spot in the number of machines that can access the network without congesting it. Since the loading of PRTs is a portion of the actual rendering job, if you are using Deadline you could limit the number of machines loading PRTs at the same time by setting the “Release Machine Limit Stip When Task Progress Reaches…%” option. For example, if the loading of PRTs happens in the first 20% of the rendering process, you could set the limit to 2 or 5 or whatever works and enable the “Release Limit” option and set the % to 20. This way, only the given number of machines will be loading particles in parallel, but once they have done so, they would release the limit so more machines could start loading…
In our past productions, we had several dedicated servers for simulation data and I believe the sweet spot was around 10 slaves loading at the same time.
I think we need to look more closely at that server and do more comparative tests between the clustered server, dedicated PRT server and saving PRTs locally. In your experience is there a bandwidth limit that Krakatoa’s processing of IO imposes to loading PRTs ? … ie would a local RAID or SSD be significantly faster than a local sata drive ?.
That’s a good idea about limiting machine limit to one part of the task, we’ll look into that.