Is there a way to set a machine limit, but then ignore it if there are no other jobs in the queue? Basically I don’t want the entire farm dedicated to one job…unless its the only job in the queue.
While this functionality is not currently available, you could use the priority setting to make sure not too many slaves focus on a particular job if other jobs are available.
-Cody
I was thinking…instead of machine limit, it would make more sense to implement it as a MinJobs variable for the repository. So if your minjobs was set to 3 and you have 15 servers. Each job would be given 5 servers, unless there are fewer than 3 jobs in the queue of coarse.
Jeff
Not totally sure what you mean… do you mean that the machine limit would be 5 for each job unless there are less than 3 jobs?
It wouldn’t really be all that advantageous to ensure that the slaves are spread out between the jobs since that would just mean that the individual jobs would take longer to finish.
For example, if there are 3 jobs with 5 tasks each and there are 5 slaves, distributing the slaves evenly to all jobs would simply make all jobs take longer and finish roughly at the same time. If all 5 slaves worked on one job at a time, it would theoretically take the same amount of time to complete all of them, but you would at least have individual jobs completed earlier.
If you have a job that shouldn’t take very long to complete and you would like slaves to pick it up instead of a much longer job that is currently in progress you can set its priority higher and give it a machine limit so that the other job still continues with any excess slaves.
Having using a min jobs could also actually result in less efficient distribution as well. For example, let’s say you had 2 jobs and 15 slaves. Job 1 has 15 tasks and Job 2 has 5. Min jobs is set to 1 so that machine limits are only in effect when there is more than 1 job and both jobs have a machine limit of 5. Job 1 would have 5 slaves working on it, and Job 2 would have 5 slaves, but another 5 slaves would be sitting Idle until Job2 finishes. When Job 2 finishes, Job 1 would then get all 15 slaves since MinJobs is 1. However, it would have been much more efficient to have the excess slaves on Job1 in the first place. The best thing to do here would be to not have a machine limit for Job1, and give Job2 a higher priority and perhaps a machine limit.
That being said, I certainly know what you’re getting at. Ryan, our lead developer, is out for the week and I’ll discuss this with him when he gets back.
-Cody
The point of my suggestion is so that if you have a big large job and a bunch of small ones, the big one doesn’t clog the pipe. Now true you could set machine limits and priorities…but you’d have to know in advance that one job is going to take awhile…or constantly monitor the queue. Unfortunately there isn’t a dedicated person that monitors the queue here. So my idea is an automatic way that the smaller jobs could get rendered and out of the way and then the big job would have all the machines to itself again.
I didn’t quite get what you meant by idle machines. There would never be any idle machines unless there were no more tasks to render on any job.
So for example lets go to my 15 machine farm again with a 3 job minimum. You start one job…all 15 machines start on it. A second job is submitted…as tasks finish on the first job…the second job starts getting machines. So now you have 8 and 7. A third is submitted. It would become become 5,5,5. A forth is job submitted…it doesn’t start, because your job minimum is met. Now say job 2 starts to finish before the other two. The machines that were on 2 start to go to 1 and 3. So you could have 6,3,6. Then once job 2 finishes…job four would start…and you would have 5,5,5 again.
Now this is a job minimum not maximum so if a machine was looking for a job and 3 jobs were already started…but it wasn’t part of any of the groups used for the first three it would find job in a group it was apart of and start that one.
So true the better way to do it would have a TD or render wrangler taking care of these things…but in a small shop like mine, you try to find automatic ways to do things 
But its just a suggestion, no need to take it seriously if its not practical.
Ahhh… I see what you mean now. I completely misinterpreted what you meant by minimum jobs. We’ll keep that in mind and I’ll discuss it with Ryan when he gets back.
Cheers
-Cody
Could this be possible already through use of seperate pools? Like a big job pool and a small job pool? It’s a second layer of prioritization.
The problem with using pools for this would be that if you had many jobs in the small job pool it could take a while until a big job is picked up. Unless you maybe dedicated a few slaves to prioritize big jobs. Using pools, like using priorities, would also require you to know whether a job was going to take a while when submitting.
-Cody