Deadline and redshift GPU affinity broken since recently

Deadline
1

Wondering if someone can give some insight into this one.
Seems like something got broken so wondering how is it coming along…

This is reply from redshift admins:
“We’re contacting the Deadline developers to verify this, but it appears that Deadline is modifying Redshift’s preferences file directly in order to apply the GPU affinity. This file has changed format slightly in 3.0.31, which would explain the apparent breakage. All our plugins provide alternative means to set GPU affinity, so directly modifying the setting store is neither necessary, nor recommended. We’ll post an update when we know more.”

2

As far as I know, this is not correct. Here are some examples of how we set the GPU Affinity for Redshift in the various integration scripts:

In the Redshift Standalone’s Redshift.py integration plugin, we set the environment variable REDSHIFT_GPUDEVICES:

    gpus = self.getGPUList()
    if len(gpus) > 0:
        self.SetEnvironmentVariable('REDSHIFT_GPUDEVICES', ','.join(gpus))

Same is true for Redshift in 3ds Max, except we override the environment at process level because the plugin is launching a monitored process:

self.Plugin.SetProcessEnvironmentVariable( "REDSHIFT_GPUDEVICES", gpus ) # Redshift (newer env var method)

In Maya running Redshift, we add the override to the script file controlling the MayaBatch rendering as an additional argument

redshiftMiscArgs.append( 'redshiftSelectCudaDevices({' + gpus + '});' )

In Cinema4D, we pass a flag to the executable’s command line

        selectedGPUs = self.GetGpuOverrides()
        if selectedGPUs:
            for gpu in selectedGPUs:
                argument.append( "-redshift-gpu %s" % gpu )

So I cannot see any place where we touch any external files belonging to Redshift - we just pass info via environment variables, command line arguments, or command scripts.

3

I see.
Well since the last 2 versions of redshift 3.0.30 and 3.0.31 affinity isn’t working properly so something must have bin change in their setup that affected deadline as well.
Hope you guys are in contact with them as they mentioned to figure out what happened .

4

Hi guys, is this an ongoing issue?

i am actually new to redshift + houdini + deadline combination, and have been trying to get the affinity to work for a week, but was not able to.

SETUP

pcA with 3090 x 1 (concurrent task limit to 1)
pcB with 3090 x 2 (concurrent task limit to 2)

tried submitting job with [GPUs per task] (under GPU affinity overrides) set to 0, pcB both 3090 is activated.

however, when i submit the job with [GPUs per task] (under GPU affinity overrides) set to 1, only single 3090 seems to be activated (even though deadline do take 2 concurrent task, and do render them both, albit slowly than expected, leading me to think that both task is using a single GPU.

my understanding was that deadline will be using one GPU for each TASK?

any help will be appreciated

5

You could try running two Workers on pcB and set each Worker with an affinity to one card, this should ensure they’re both rendering on a different card. You won’t need to submit current tasks then.

You can use an app like MSI afterburner to check the card usage

6

current versions of redshift, 3.0.31 and 3.0.32 have issues with deadline and GPU affinity. At the moment the safest is just to use all GPUs until that is resolved.
Btw scaling across two GPUs is fine so with your configuration, I wouldn’t bother. You can simply use 1 concurrent task and 0 for GPU affinity ie to use all.

7

oh wow, this is a interesting workaround, never thought about this approach.

i have however, managed to get the affinity working with REDSHIFT 3.0.30, so i guess it is a problem with
3.0.31 & 32.

i have posted that i had a similar problem, but tinkering with some settings i managed to get it working.

1 Like