I have an issue where my workers does not seem to see all CPUs avaliable. I have a machine with 128 cores, it’s a multi sockets machine so 64 per socket. When I fire up a worker it only shows 64 available CPUs.
Ideally I would like to run a multi worker setup where I dedicate CPU 0-63 on one worker and 64-127 on the other. Is this possible?
Currently I’ve tried to launch the workers with the /node 0 and /node 1 parameter but when running for example Houdini on one of the workers, it seems like all cores are being used. Basically I can have worker “RENDER1-a” running a job and having 90% CPU Usage and “RENDER1-b” being Idle but still having 90% CPU Usage.
Are you sure your renderer is not using all 128 cores? Deadline is known to mis-report the CPU count above 64 for various technical reasons, but Deadline itself does not consume the threads, it is the renderer that uses them. So if you can see all threads being loaded, you should not care what the Monitor claims it knows about. You might get funny CPU consumption reports like 200% CPU usage, but that would be a cosmetic issue.
Well, yes I believe all cores are used when rendering, to clarify, my problem is that I would like to use two workers that each have half of the available cores dedicated. I thought this would be achieved by clicking the “Check Individual CPUs to Use”.
Basically I want to render with half of the cores on one worker and half of the cores on the other worker. Currently all cores are being used even if I’m just rendering on one of the workers. Does this make sense or am I looking at this the wrong way?
You are thinking along the right lines, but if we cannot detect more than 64 cores, on a 128 cores machine you are out of luck setting up the required CPU affinity.
What you could do is go to the “Modify Worker Properties…” right-click menu in the Workers list, and set the first Worker to 0-63. Don’t modify the second Worker. If the first Worker is consuming the first 64 cores, the second one will try to use all 128, but will likely only load the 64-127 ones, as the rest are already busy. At least it is worth trying.
Alternatively, you could try Concurrent Tasks on a single Worker. Don’t look at the CPU load values, but pay attention to the total render time of the tasks. If rendering 100 tasks sequentially on a single Worker is slower than rendering the same 100 Tasks on two Workers or 2 Concurrent Tasks, then it is working. Even without the CPU affinity, you would be forcing the OS to squeeze out all possible cycles from all available CPUs. I have done some tests using Arnold and V-Ray on large machines with up to 96 cores, and there is a benefit.
Hey folks wondering, if deadline team managed to fix this issue. I have the same problem I have 128 logical processors and Deadline show only 64. This will get harder when we need to assign cpu affinity
@Arjun_Thekkumadathil this is reported issue and our devs are working on it internally. I am going to +1 on the internal ticket. Unfortunately, I would not be able to share the ETA and the roadmap information on this.
Hi All! I was wondering if anyone was able to create a suitable workaround for high core count machines? We have a few amd tr workstations with 64 cores /128 threads. and it would be great to run 2 or more workers with different settings.
Got a response via email from support that this wasn’t supported and the workaround was to manually assign affinity via the OS, though I don’t fancy manually changing 256 checkboxes in the task manager.
