AWS Thinkbox Discussion Forums

Deadline is only using one gpu? Cannot get it using other GPUs

Hi all,

I am submitting two separate jobs from Houdini/Redshift assigned to two different instances of Deadline workers. I have tried many combinations of GPU affinity overriden/not overridden, assigned each worker to different GPU slot, etc. No matter what I do, Deadline is always sending tasks to GPU 0, and NEVER uses my second GPU? What am I doing wrong? Using most recent version of all three software.

Thank you!

Is the worker’s Concurrent Task Limit Override set to 0?

Sorry, where would that be set? Is it supposed to be set to 0? I am not sure what it is set to currently. Would that be specified in Houdini or in the deadline monitor?

It would be set in the Deadline monitor by right clicking on the machine name in the machine list → Modify Worker Properties → General Tab → Concurrent Task Limit Override. (You’ll need to have Tools → Super User Mode enabled). You’d set this to however many GPUs are on that machine.

When you submit your job, you’d then set your “Concurrent Tasks” parameter to the highest amount of GPUs contained inside a single box on your farm. So if you have ten boxes on your farm, nine of them have two GPUs, and one of them has four GPUs, you’d set this parameter to four. The Concurrent Task Limit Override that you setup in the previous step would ensure that the three GPU machines would only take on three tasks at a time, while the four GPU box would take on four tasks at a time.

Lastly, you’d set your GPU affinity settings to 1, which essentially tells Deadline “Each GPU renders One Task”.

Hope that makes sense! There’s a lot of confusing parameters. Here’s a couple charts that I made to help out my coworkers that may help:


Thanks for explanation and charts - realy helpful.

I got a question: I am setuping render farm for C4D + Octane.
I have PC with octane benchmark score 1037 (2GPUs) and it renders faster than 4 GPUS with octane score 1574…i do not make any sense to me, (i used batch plugin, so project stays in memory and isnt loaded from disk every frame to minimalize CPU and RAM difference)
How can i get maximum performance from ma 4 GPUs setup?

Thanks for any information/clue.

I have the same problem, did you manage to find a solution?

yes i did!
I createt whole test to find out whats going on:
On multiple setups i run octane benchmark (as reference score), and 2 scenes represents our typical projects and measured render times using C4D picture viewer and also Deadline render farm setup.

  • First thing wrong with multiple gpu setup was raser cabel connecting GPU to motherboard - make sure you have 16bit (mining setups use only 4 pins and thats the bottle neck)

  • Second catch is motherboard GPU PIC Expres socket - make sure its full speed 16 bit (lanes) - you need to check it in manufacturer s documentation- PC stores has often missleading informations (we bought one only to find it out…)

During my test i found out octane benchmark is pretty good reprezentation of perfromance you can expect.
Here i need to mention- where you need adaptive render settings, for octane, render times and octane score relation ship isnt linear…

If your ocrane score and render time doesnt match expectations- probably there is bottle neck in data transition from GPU to Motherboard
(Octane benchmark sends scene to GPU and then shows you results, typical render proces is mutch more heavier to communication between GPU and motherboard).

Otoy tech support was a great help, on this debuging yourney. (and tech guys in our company)

Here are some graphs where you can see smaller render times on PCs with better Octane benchmark score.

  • Yelow line are expectations- 2 times more power should be 2 times faster

  • Oreange one is deadline farm performance - you can see some dellay - time difference there- its communication between server and worker

  • Gray line is C4D picture viewer - its pretty much meet the expectations…as i mentioned above, when you use adaptive render settings, render time and octane benchmark relationship is not linear anymore, but…well…adaptive…(it took me a lot of effort to figure it out why its behaves this way)

this scene has not linear behaviear due adaptive render settings was on

Hope thats helps!

Hi, Peter!
this is a great study!!! I’ll take a note and check everything on the points now!

Tell me, please, do you use the standalone version of Octane?

I do not use octane standalone- I have installed C4D and octane on each worker PC.
Since Deadline has script and jobs can be queed directly from C4D, it was more convinient option.

Useful tips:

  • Deadline calls C4D commandline - make sure you run c4d comnadline.exe and assign license (you have 3 commandline licenses to each standalone)
  • setup C4D plugins folder on NAS - to make sure each worker has the exact same octane version, here is how:

To provide 3rd party plugins over a plugins path, a folder you placed all your plugins, you can do the following:

  1. Please manually set the plugins folder path in GUI Cinema 4D Edit > Preferences > Plugins > Search Paths
  2. Open the preferences folder (by clicking on the button at the bottom left), copy the plugins.json
  3. On the machine you want to use the Commandline, navigate in the Windows Explorer to %AppData%\MAXON (just type this into the explorer window by clicking in the bar near the top of the window where it shows the file path), and open the folder with the "x" suffix (Cinema 4D R2023********_x) – it’s the prefs of the CLR instance. CLR must have started at least once before in order to have created the folder.
  4. Paste the plugins.json there.

Thanks so much for the tips, Peter!

Tell me, please, do you know how to make each card in a node render one frame at a time?

When I send a task to a node all cards render one small size task.
I would like all cards in the same node to render one task at a time. Is there such a possibility?

Hey Rendernaya,

I did not manage to run GPU afinity- if you truly need this feature, I would use .orbx format and octane standalone for rendering and contact otoy support…

I decided to use c4d + octane - since our team has multiple members with active license and nvidia gpu, i can use their pc as worker during meetings, lunch breaks, weekends, holudays etc
…pretty effective.
Also during my test i found out that 2 gpus was 2 times fadter then one, so i hed nobmotivation to make gpu afinity work.

PS: Make sure you use latest octane version (older ones supported only up tu 2 gpus in one Machine)

1 Like

i know this is old, but I’ve been trying to replicate this to no avail. I’ve set concurrent tasks to 4 (as the max GPU we have in a box is 4, but most have 2) in the job properties, and then on the c4dbench settings its set to 1 GPU per task… I then set the concurrent tasks limits on the individual workers according to the number of GPU they have. 2=2gpu box, 4=4gpu box… seems logical so far.

But the affinity part doesn’t make sense. You say to only use GPU affintity of 1, but you are now sending 2 or 4 tasks to those workers (defined by their concurrent task limit). If you set the ‘worker properties’ to GPU affinity of 1 then you are basically telling Deadline that that box only has 1 GPU… /boggle…

i’ve also tried experimenting w/ various GPU affinity settings but they really only seem to ‘work’ when you are attempting to split workers on the same box. (For instance a 4 GPU box, with 2 worker instances can have GPU affinity set to 0,1 for worker 1, and 2,3 for worker 2, and they behave as expected, but they still never seem to really utilize all the GPUs…

needless to say it doesnt work correctly when rendering redshift… the ‘secondary’ tasks just error out w/ out of memory errors or other (i cant access the GPU type errors)… :frowning:

Sorry to hear about the difficulties!

Part of the confusion might be my use of the phrase “GPU Affinity”, so let me restate things from scratch with better vocabulary:

In the Deadline Monitor we right click on a Worker machine → Modify Worker Properties:

  • In the “General” tab we set the “Concurrent Task Limit Override” to however many GPUs are in that box.
  • That’s the only setting we adjust per machine. We do not touch anything in the GPU Affinity tab - that stays completely disabled.

When submitting jobs, in the C4D Deadline Submitter dialog:

  • We set Concurrent Tasks to the maximum amount of GPUs in any particular Box (for us its 4).
  • In the Advanced Options tab, we set GPUs per Task to 1

The annoying thing is that this “GPUs per Task” parameter is listed under “GPU Affinity Overrides” and can easily be confused with the “GPU Affinity” settings in the Modify Worker Properties section… and that latter section has to be left completely disabled as otherwise it screws things up.

I’ll also add that this is a Redshift farm rather than Octane… not sure if that makes a difference.

Privacy | Site terms | Cookie preferences