AWS Thinkbox Discussion Forums

limit concurrent task by thread count?

Hi all,

I just set up my old linux box to render via Deadline. It’s a single threaded quad-core with 16GB RAM. My workstation, which is also a deadline slave, has a hyperthreade hex-core and 64GB RAM.

I’m trying to figure out how best to configure Deadline to be able to send Nuke jobs to the queue with settings optimised for my workstation without making the old box explode.
As an example, I have a Nuke job that I would like to submit with 3 concurrent tasks and 4 threads per task to get the best out of the 12 cores I have in my workstation.
I guess in this example I can set the old machine’s slave properties to have a thread limit of 1 and all is well. It will render away with 4 threads at a time.

However, what if I have a more lightweight task that I would like to run on 2 threads with 6 concurrent tasks? In that case I would like the old box to have an override value of 2 concurrent tasks so it doesn’t waste half it’s cpu power.

If I set the slave’s overrride value for concurrent tasks to 2 and forget to change it next time a heavy job comes along, which I submit with 4 threads and 3 concurrent tasks, surely that old box will grind to a halt trying to run 12 threads, right?!

What is the best solution for this?

EDIT: I just sent off a test job with 3 concurrent tasks with my old render slave set to use a task limit of 1. Still, it’s picking up 3 tasks at the same time. Is this a bug or am I not doing this right?

Cheers,
frank

Hi Frank,

I moved this to the beta forums, since you’re using Deadline 6.

I couldn’t reproduce this. Did you have the option enabled for the job to limit the tasks to the slave’s task limit? If not, that would explain this behavior. Also, the slave only reloads its settings from the database every 5 minutes, so after you changed it to 1, it might not have recognized it yet.

Unfortunately, there really isn’t a good solution to your concurrent tasks problem. This type of problem is something we plan to look at for 7.0. The idea would be that you could set up “slots” for the slaves, where a slot just represents some arbitrary resource limit (ie: a slot could be 2 cores and 4 GB of ram, or just 4 cores, or something else entirely). Then you would set a slot limit for each slave, and you would specify the number of slots the job needs during submission.

Cheers,

  • Ryan

Sorry, I forgot to mention this, but I id have that set.

Will test it again on Monday and see if it still happens.
Thanks for the info on the slots, sounds like what I was looking for.

Cheers,
frank

Privacy | Site terms | Cookie preferences