Idle Tasks

Deadline
1

Hi,

We’ve got some processes that we want to run that process unimportant information when our boxes are idle - is there any way of detecting when a slave is inactive? We can’t leave the process running continuously with a low priority as it affects render jobs detrimentally.

Basically what we’d like is either:

  1. The ability to set an exe to be run while no tasks are running on a slave, have it automatically killed when the slave is ready to start a task (no shutdown procedure necessary), and started back up when the box is idle (perhaps with a timeout to prevent it continuously spawning when the farm is busy)

or

  1. Have some kind of script hook when a slave starts/finishes running a task so we can implement it ourselves. We’ve used event plugins in the past, but this only works on entire jobs not individual tasks.

Is there any way of doing this at present?

Many thanks,
Jamie

2

Hello jamieh,

You can definitely use ‘deadlinecommand GetSlaveInfo SlaveStatus’ to obtain the status of a slave, or by using multiple commands, a series of slaves, and you could hypothetically use that to trigger the running of the executable, but sadly there is no built in structure within deadline to do as you wish.

Cheers,

Dwight

3

Jamie,
Out of curiosity, could you explain what the exe does that you need running during idle tasks?

4

There’s two - one that helps with distributed code compilation which will obviously slow the machines down. And also the VRay Render Slave, for distributed rendering - it can’t be run constantly as only one Max/Maya instance can be active on a machine at one time. So if a distributed render is running the render tasks will fail - this would be fine, but we want the jobs sent to the farm to take priority over the distributed rendering.

5

Hi,
Thanks for the info. Interesting stuff. Distributed code compiling. Cool. Sounds to me like the ideal situation for you, would be to have 2 plugins in Deadline, one for each type of job you wish to process. I don’t believe you would want either to execute in-between actual job tasks, but rather either type of job can run when there is either nothing else to do on the farm or if you purposefully, raised the priority of say, wanting to have DR based jobs go through the farm before ‘normal’ Deadline render jobs.

So, for the DR architecture, there isn’t really anything available right now…however, this is another +1 tick to the Deadline dev’s for why supporting DR based jobs (VRay DBR, VRay RT DBR, MR DBR, etc) is important in the future.

On the code compiling front, sounds like you need to write a custom Deadline plugin to support this executing and more importantly, allow you to control when it happens in Deadline. The other thing, is that as long as you are happy running say, code compiling at the same time as ‘processing’ any other type of job in Deadline, then you could run 2 x Slaves on each of your machines you want this to happen on. Of course, you will be using 2 Deadline licenses per machine, but it does mean, you can allocate different pools/groups to each slave instance and run them simultaneously.

You can also control a plugin in Deadline to not run if a certain “process.exe” is running as well. Example of this can be seen in some of the Deadline plugins. Finally, Deadline has the ability to ‘flag’ its jobs as ‘interruptible’, so if a higher priority job comes along, the slaves will drop what they are doing and jump onto this other job. Personally, I don’t like this, as it has the potential to introduce high levels of inefficiency. :frowning:

Can you tell us what compiler/platform you’re using? VS/Win? gcc/linux?

Mike