Equal Priority Evenly Share Slaves?

Deadline
1

Is there a way that multiple jobs submitted with the same credentials/priority can evenly share slave nodes? For example… 5 jobs on the farm… 20 nodes, each job automatically gets 4 nodes and the farm tries to keep the distribution maintained so if one finished the remaining jobs are now using 5 nodes each? I believe back when I used Alfred they called this Round Robin mode…

It would also be good if you could limit how many simultaneous jobs can share the farm…

2

This currently can be controlled using machine limits for jobs. It’s not dynamic though, so if your 5 jobs have a machine limit of 4, and one job finishes, those jobs won’t spread over those 4 nodes that are now available. There is the benefit though that if another job gets submitted, those 4 nodes are immediately available.

There has been some discussion about supporting a round-robin system, but it would have to be a smart system to ensure that once a slave has started a job, it doesn’t move on to another. If it did, it would have to unload it’s current job and load a new one. If the job type supports keeping the scene loaded in memory between tasks, you lose this benefit. This would affect 3dsmax, Maya, Softimage, Lightwave, Nuke, Fusion, and modo jobs.

Cheers,

  • Ryan
3

Thanks for the quick reply… We’ve been juggling it with machine limits during the day so people can get equal progress. But as always someone kills a job then we have procs sitting idle. Would there be a way to poll the farm and if there are idle slaves automatically adjust machine limits, or even just evenly automatically set machine limits based on number of current jobs?

4

There may be a way to do this using deadlinecommand.exe. There are calls to get the slaves and jobs and if you had a program parse them, you might be able to achieve what you are looking for. However, you’d also have to handle the case when a new job gets submitted as well.

What’s the reason for wanting to share slaves like this? Is it to get preview frames quickly for multiple jobs? If so, you could submit 2 jobs for each render. The first would be a high priority job with a few frames, and the second would be the rest of the frames at normal priority. This way, you get feedback about jobs pretty quickly.

Cheers,

  • Ryan