Pardon a noob question but I just started with Deadline and am trying to figure out the best way to manage our new farm.
Current config:
14 - 24-core 2.4GHz, 32 GB RAM, Windows 7 64-bit blade servers (not sure what other tech elements might be necessary)
4 - 8-core 3.2GHz, 8 GB RAM, Windows 7 64-bit workstations (re-purposed from portable playback system)
Repository lives on a RAID6 array which is also the main data array for all project data.
We plan to use Deadline to render 3ds, Maya, Cinema4D, After Effects, and PremierePro. ~15 anticipated users currently.
What might be the best way to manage render jobs with different priorities? I have setup groups to delineate program installations on each slave but that is more of a stop gap measure as I keep running installs. At this point the main slaves (the 14 blades) are all clones.
I am trying to figure out what is the most efficient way to allow multiple users to render jobs from the same program? For example, if User A needs to render a job from Maya, they set their job to use the Maya group which encompasses all 14 blades. Those 14 slaves roar to life on User Job A. User B needs to render a job also in Maya. They set their job to render to the Maya group and their job now gets queued behind the other job since it is occupying all the slaves in the group. To compound the issue (I am sure this will happen eventually), User C has to render a 3ds job and it needs to happen right now. But the 3ds group is the same machines as the Maya group so now that job is queued behind User B’s job.
Basically, I guess I am asking what is the best way to setup the slaves so that multiple jobs can be run simultaneously but not that the expense of one job using all the slaves if nothing else is rendering?
In short, Pools control scheduling, Groups don’t. Groups are binary filters to allow / disallow a job from being processed on a machine, but the Pools, numeric Priority and Submission Date is what really controls when a job will be processed.
In most companies, Pools are created for projects that require render resources. So in your example, you could create three pools - say, Maya Project A, Maya Project B, 3ds Max Project C. You add all 14 blades to all three Pools, but in different order. For example, 5 blades would have Maya Project A, Maya Project B, 3ds Max Project C. Other 5 will have Maya Project B, Maya Project A, 3ds Max Project C. The last 4 would have 3ds Max Project C, Maya Project A, Maya Project B. As result, the first 5 blades will prefer Maya A jobs before other jobs, but still work on Maya B and Max C jobs if there are no Maya A jobs to work on. The other 5 blades will prefer B to A, and only work on Max jobs if no Maya jobs exist. The last 4 blades will work on Max jobs and only work on Maya A or, if none found, Maya B jobs if no Max jobs exist on the queue.
Within the pools, the job’s numeric priority will determine which job will go first. So if two artists working on Maya Project A submitted two jobs to the same pool, but the one job was set to Priority of 50 and the other to 60, the latter will render first. Unless the user specifically limited the job to render on only up to 3 machines. In that case, once 3 tasks are picked by the first 3 machines, all other machines will ignore it and potentially pick up the job with 50 Priority. If both jobs were submitted with the same Priority and no Machine Limits were specified, the submission time will determine the order - the earlier submission will go first.
These are the basics. Please read the article I posted and feel free to ask questions after reading it and my answer above!