AWS Thinkbox Discussion Forums

Ego-centric priority

Just wondering if it’s possible to implement a “personal priority” or something similar to that. This would allow each user to shuffle around machines amongst their own jobs without stealing resources from other users. To be more specific, let’s say a user has 10 jobs going, but they’re all different versions of the same thing. They would maybe want v01 to finish first before assigning any machines to v02-v10. Right now our farm is set up with the “Pool, Weighted, Balanced” logic, and sometimes a user might end up with maybe 10 machines on each of his 10 jobs. My question is, if a user wanted all of the 100 machines he got assigned to his jobs to be focused onto just 1 or 2 of those jobs until they are complete, how would that be possible without giving the job a priority bump which would in turn steal resources from all other jobs?

I don’t this would be possible. Keep in mind that if 10 slaves are working on a user’s job, it’s because for those 10 slaves, that job is next in their queue. There could be 20 other jobs in between that job and another job belonging to the same user. So it’s not just a matter of simply shuffling machines around, something would have to be done to change the queue order for those 10 slaves.

Perhaps some solutions that might help would be to suspend the original job, or set its machine limit to a low value like 1 or 2. Note though that if there are other jobs between the original job and the next job for that user, they will still have to wait until the slave processes those other jobs first.

Cheers,
Ryan

A bit kludgey perhaps, but you could make the latter jobs dependent on the ones you want to complete first. This would force the order without changing priority, but comes with a downside of possible inefficiency in some circumstances.

Yeah, while that seems like it would help in some circumstances, I think it would work against us in some other ways down the road. In the end, it’s just not how things are going to work. Everybody always thinks you can just “take the machines from this job and put them on that job” but it doesn’t work like that. I thought I’d ask anyway. Thanks for the followup.

Privacy | Site terms | Cookie preferences