AWS Thinkbox Discussion Forums

Allow job to take resources from others

Is there a method of overriding a job that will allow it to take resources from other jobs in the queue? For instance if we set a Machine Limit of 100 but there are not that many available machines, can we setup a job to take from others until it hits 100? I think there is an option similar to this in Rush.

There is a hacky way to do it. Every job has an “is interuptible” flag. You could enable this by default. Or else run a monitor script that adds it to every currently running job. We have a pool called “Urgent” which is the top of every single machine’s pool priority. But we don’t by default use the “is Interruptible” it might be nice to have pools which are not just above other pools but are actually “interrupt” pools.

Are you wanting to interrupt tasks for other jobs that are mid-render, or do you want those tasks to complete, and then have the slaves move on to the higher priority job? The problem with interrupting mid-render is that you can lose potentially hours of rendering time. Theoretically, it would make sense to just interrupt the tasks that have been rendering the least amount of time, but that’s difficult with Deadline’s architecture because there is no central “master” application, and the slaves work independently of each other.

Gavin’s method of having a top priority is actually the recommended approach. We even did that back in the Frantic Films days (we had a pool called “crisis_friday”). If we absolutely had to get a job through immediately and couldn’t wait for slaves to finish their current tasks, we would then cancel tasks manually to ensure we weren’t killing other jobs that were still a high priority.

Cheers,
Ryan

Privacy | Site terms | Cookie preferences