I’ve discovered Deadline a few days ago and am testing its capabilities. However, at first only on my local machine (i7 870, 8gb ram) , later on the network rendering stuff… And I have some question/issue. Using it with after effects, it does seem a bit slow. I’ve read the manual, changed the chunks to 10 per task, 3 tasks parallel. It took 10 hours for 3500 frames. Same project, rendering via afterFX took 90 Minutes.
So what would be a solution?
Is there a reason for the “5-10” chunk size recommended in the guide?
Would bigger chunks slow it even more or what exactly would happen, if, let’s say, chunks were 500 frames?
The main reason why it’s taking longer is that there is a natural overhead per task that involves starting AE, loading the comp, rendering the frames, closing AE, and then getting the next task. When multiple machines are working on the tasks, this overhead isn’t a major factor in the wall clock render time because many tasks are being rendered in parallel. However, if you are rendering on a single machine, the overhead adds up quickly. In your case, AE would have had to be launched 350 times because you have 10 frames per task!
To compare the actual render time to local rendering time on a single machine, you would have to submit a single task with a chunk size of 3500. Now of course you wouldn’t do that once the render farm is in place, but it is a good test to confirm whether or not rendering through Deadline is taking longer than a local render.
If you’re frames only take a couple of second to render (or less), you would definitely benefit to bumping up the chunk size to 25, or maybe even 50. This helps offset the overhead, because it reduces the number of times AE needs to be launched.
Also, if you are rendering with Concurrent Tasks with AE, you should make sure the Multiprocess Rendering option is disabled. When MP rendering is enabled, AE automatically gives a separate frame to each available thread. Depending on the number of cores and number of frames per task, combining this with Deadline’s concurrent task feature can result in CPU competition that can be detrimental to the render time.
the frames at once [or chunks as you refer it to] are how the job is broken up across the farm. if you have one machine, you could set that to the size of the file - so 3500 frames. if you have 2 machines, i would set that to 1750 etc. that way each job would get half.
if you have more, you would reduce the size.
concurrency can help or hurt you depending on the CPU usage of the one job, the IO in use and the combination of those two factors.
My recommendation - if you are testing on one machine - set it to 3500 without concurrency.
if you are testing on 2, set it to 1750. if you want to test with concurrency, set to 1750 with concurrence at 2.
i’ve had jobs that are 1-2 frames per second to render, and the overhead of starting and stopping jobs on the farm means that splitting that into 100 tasks is going to be muchslower than having a few machines do a majority of the frames. as your frame times move up to 5-10minutes a frame [or hours] the seconds it takes to open/close tasks on slaves and corresponding applications becomes an insignificant amount of time. by my estimattion 3500 frames at 90 minutes = 1.5 seconds per job. if AE takes even 10 seconds to open and load then each frame would jump to 11.5 seconds or 10.5 hours roughly.
In many applications Deadline can keep the job open and change frames, but i’m not sure if that is possible with AE. in any case, increasing the frames at once mitigates this problem.
