Can somebody please tell me how I can send job through the submitters which requires slaves that are members of more than one group simultaneously?
I.e. Job X requires a certain Plugin A and Plugin B to render
Slave 1 only has Plugin A (Group Plugin A)
Slave 2 only has Plugin B (Group Plugin B)
Slave 3 has Plugins A + B (Group Plugin A, Group Plugin B)
Job X should therefore only be picked up on Slave 3
If it’s not possible can I perhaps make that a feature request for future versions? We have a few hundred slaves and management will get quite complicated if we have to create and stay on top of some sort of manually created “union groups” to address this problem.
I think your best bet here is to look at the Limits aspect of Deadline, as a machine must have access to all of the limit groups assigned to a job in order to do it. You can read up on it at docs.thinkboxsoftware.com/produc … tml#limits
I can see how this might work, however it seems it’s a bit of an abuse of a function whose purpose is to deal with the number of licences one holds for a particular plugin, rather than what machines are available with the specs and software which I thought is what groups was intended for?
It’s also a bit more hidden away and obscure for the average user to get to or even know what it does, whereas groups are right there first thing in the submitter. Couldn’t this be redesigned more intuitively so that you can select multiple groups, and below that select the mode, I.e. all selected must be true, or any selected must be true?
Well, close anyway. Their initial purpose was for licensing certainly, but they can also be used for selecting specific machines. I even use them for Slave throttling.
The difference between limits and groups in this context is that limits take more computing power than straight groups (not much, but enough). We’ve had internal discussions here about deprecating groups and replacing them with limits exclusively (which was my idea), but the project manager for Deadline had a different approach in mind that I’m a bit fuzzy on.
All this to say, we will be overhauling groups at some point, but I don’t think we have a set version for that.
I see. I’ll have to have another look into limits I think.
From a UX point of view I wouldn’t advise you to deprecate groups and use limits in its place without addressing the usability issues that would entail. And “limits” might also be the wrong name for it in that case.
The beauty about groups is it’s very easy to assign them when a new slave is added, it’s super easy to manage, you can view each skave’s group membership in the monitor easily, and it’s easy to update. Limits is far more obscure and the process isn’t as straightforward by comparison. There is also the ease with which you can select a group in the submitters to suit your job, right next to pool selection. If only you could select multiple groups, this would be the perfect solution for us…
Hopefully you’ll reach a nice compromise. In my view the average user should have no real need for accessing limits, it’s something the admin can configure as part of each group if necessary to deal with the number of available licences. The user just needs to chose the right config to ensure the job is rendered in the right slaves, without having to worry about licensing numbers etc. Let deadline handle that transparently and efficiently in the background.
As Edwin mentioned, this is on the To-Do list. A Group essentially represents a set of resources. Since Groups are singular, it is currently necessary to matrix out the Groups (or what you termed “union” Groups) in order to represent different combinations of resources. One option we are considering is to replace Groups with what would more properly be called “resource lists”, such that arbitrary resource types can be defined and then each Slave can be assigned a list of the resources that it possesses. This is roughly equivalent to the idea of multiple Groups, where each Group could only represent a single resource.
Changes of this nature, of course, have broad-reaching impact throughout Deadline, so we still have a fair bit of internal discussion and consideration before deciding how Groups will be revised. I encourage anyone who wants to be more involved in this discussion to join our beta program.
We have machines with 32gb, 16gb and 12gb of RAM. We have groups for each of these memory amounts, but then the each machine which is assigned to the 32gb group is also assigned the 16gb and 12gb groups so that if we send a job which isn’t memory heavy it’ll go to the 12gb group which includes any 16gb and 32gb machines (sent on a lower priority to use available slow machines but to use faster machines if they are available), but if we send a job which requires lots of ram it’ll only go to the 32gb machines. We’ve previously had groups called “24_and_16_core”, but doing things like this increases the number of groups and makes things more complicated.
We use limit groups for managing plugin usage and licenses, and use the whitelist/blacklist feature in limit groups to prevent certain jobs going to workstation machines for instance.
What would be really nice to see is some intelligent memory grouping from Deadline where if a job was erroring out due to hitting the Ram limit to see Deadline automatically increase the job’s group limit. I know every studio will use their groups/pool differently, we don’t split our ram into departments for instance, we give 100% priority to comp and tile assembly jobs.
Your request is very much in line with some of the design work we are doing regarding enhancements to Deadline’s scheduling system. This would likely not be accomplished with nesting, but rather by a process of allocating resources.