GPU affinity with custom rez pipeline

Deadline
1

Hi,
We have many render servers with 8 gpus and want to use multpile workers on it where we set each worker to use a particular number of GPU’s i.e worker 1 use 4 gpu and worker 2 use the other 4 gpu’s. When using default mayabatch render it works but when using our own custome maya commands using rez environment it doesnt work. both workers use all the gpu’s. This results in one of the worker failing the render (as the gpu’s get taken over by the other worker). Can anyone shed some light on how they setup redshift with maya renders to use certain gpu’s but maintain the worker settings. For example a job can be set to 1 gpu but it uses the gpu’s set on the worker side.

cheers

2

Hello

Thanks for reaching out. Can you explain what are you own custom Maya commands?

If you are using Windows you may be able to set the GPU affinity on the OS level if it is not working in Deadline. But I would like to find out why is it not working.

If you are using Windows assuming the Workers are running already follow here: How to set default GPU for apps on Windows 10 - Pureinfotech I have not used it myself before so it may be a shot in the dark.

3

Hi @zainali
Thanks for the update. I can do the gpu for apps in windows but what we want to do is have 2 workers render maya + redshift renders but each worker use 4 gpus.
Worker A is set to use gpu (0 1 2 3) and worker B is set to use gpu (4 5 6 7) and both run the same maya + redshift render. As for our command i will get back to you.

4

There is a GPU affinity option in Deadline on the worker level and some plugins also handles the GPU affinity. This may override each other if the render has to account for both the affinity settings.

For Maya, the Deadline integrated submitter has an option to specify the GPU affinity and render application (Maya) will handle the affinity. You can leverage the submitter to limit the “GPU’s per task” to 4 and the worker that picks the task will only use 4 GPU for the render process.

5

Hi, Yes we have set the worker level gpu affinity but somehow it doesn’t work and the task uses all the gpu’s. As for the submitter level gpu option we have tested and it works whatever gpu amount we set on it. But we need to get the GPu affinity working on the custom rez environment.