AWS Thinkbox Discussion Forums

Houdini RS selected GPU is always 0

Hi! Using Houdini 18.5.462 and Redshift 3.0.37, with 2 GPUs(RTX 3080). When submitting a job or modifying job properties, if i try to assign a single GPU, i can’t select GPU 1.

No matter which GPU i select within my job properties, Deadline always fires up the render with GPU 0.

I tried putting the GPUs per task to one even though the documentation says that selecting a GPU device or device list overwrites this parameter.

Here’s a log of a submission attempt.

2021-02-24 15:10:11:  0: Loading Job's Plugin timeout is Disabled
2021-02-24 15:10:11:  0: SandboxedPlugin: Render Job As User disabled, running as current user 'User'
2021-02-24 15:10:12:  All job files are already synchronized
2021-02-24 15:10:12:  Plugin Houdini was already synchronized.
2021-02-24 15:10:12:  0: nogui was not added to the CommandLineParser.
2021-02-24 15:10:12:  0: Executing plugin command of type 'Initialize Plugin'
2021-02-24 15:10:12:  0: INFO: Executing plugin script 'C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\plugins\6035d908c58ffd42c472e283\Houdini.py'
2021-02-24 15:10:13:  0: INFO: About: Houdini Plugin for Deadline
2021-02-24 15:10:13:  0: INFO: The job's environment will be merged with the current environment before rendering
2021-02-24 15:10:13:  0: Done executing plugin command of type 'Initialize Plugin'
2021-02-24 15:10:13:  0: Start Job timeout is disabled.
2021-02-24 15:10:13:  0: Task timeout is disabled.
2021-02-24 15:10:13:  0: Loaded job: seq_1_render_v001 - /out/Redshift_ROP_render (6035d908c58ffd42c472e283)
2021-02-24 15:10:13:  0: Executing plugin command of type 'Start Job'
2021-02-24 15:10:13:  0: DEBUG: S3BackedCache Client is not installed.
2021-02-24 15:10:13:  0: INFO: Executing global asset transfer preload script 'C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\plugins\6035d908c58ffd42c472e283\GlobalAssetTransferPreLoad.py'
2021-02-24 15:10:13:  0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer...
2021-02-24 15:10:13:  0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in C:/Program Files/Thinkbox/S3BackedCache/bin/task.py...
2021-02-24 15:10:13:  0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2021-02-24 15:10:13:  0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2021-02-24 15:10:13:  0: Done executing plugin command of type 'Start Job'
2021-02-24 15:10:13:  0: Plugin rendering frame(s): 682-691
2021-02-24 15:10:13:  0: Executing plugin command of type 'Render Task'
2021-02-24 15:10:13:  0: INFO: Starting Houdini Job
2021-02-24 15:10:13:  0: INFO: Stdout Redirection Enabled: True
2021-02-24 15:10:13:  0: INFO: Stdout Handling Enabled: True
2021-02-24 15:10:13:  0: INFO: Popup Handling Enabled: True
2021-02-24 15:10:13:  0: INFO: QT Popup Handling Enabled: False
2021-02-24 15:10:13:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2021-02-24 15:10:13:  0: INFO: Using Process Tree: True
2021-02-24 15:10:13:  0: INFO: Hiding DOS Window: True
2021-02-24 15:10:13:  0: INFO: Creating New Console: False
2021-02-24 15:10:13:  0: INFO: Running as user: User
2021-02-24 15:10:13:  0: INFO: Executable: "C:\Program Files\Side Effects Software\Houdini 18.5.462\bin\Hython.exe"
2021-02-24 15:10:13:  0: INFO: Argument: "C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\plugins\6035d908c58ffd42c472e283\hrender_dl.py" -f 682 691 1 -o "$JOB/render/seq_1/seq_1.$F4.exr" -g -d /out/Redshift_ROP_render -gpu 0 -tempdir "C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\jobsData\6035d908c58ffd42c472e283\0_tempKZ1d70" -arnoldAbortOnLicenseFail 1 "E:/projects/Invicare/hip/render/seq_1_render_v001.hiplc"
2021-02-24 15:10:13:  0: INFO: Full Command: "C:\Program Files\Side Effects Software\Houdini 18.5.462\bin\Hython.exe" "C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\plugins\6035d908c58ffd42c472e283\hrender_dl.py" -f 682 691 1 -o "$JOB/render/seq_1/seq_1.$F4.exr" -g -d /out/Redshift_ROP_render -gpu 0 -tempdir "C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\jobsData\6035d908c58ffd42c472e283\0_tempKZ1d70" -arnoldAbortOnLicenseFail 1 "E:/projects/Invicare/hip/render/seq_1_render_v001.hiplc"
2021-02-24 15:10:13:  0: INFO: Startup Directory: "C:\Program Files\Side Effects Software\Houdini 18.5.462\bin"
2021-02-24 15:10:13:  0: INFO: Process Priority: BelowNormal
2021-02-24 15:10:13:  0: INFO: Process Affinity: default
2021-02-24 15:10:13:  0: INFO: Process is now running
2021-02-24 15:10:16:  0: STDOUT: [Redshift] Redshift for Houdini plugin version 3.0.37 (Jan 29 2021 14:50:27)
2021-02-24 15:10:16:  0: STDOUT: [Redshift] Plugin compile time HDK version: 18.5.462
2021-02-24 15:10:16:  0: STDOUT: [Redshift] Houdini host version: 18.5.462
2021-02-24 15:10:16:  0: STDOUT: [Redshift] Plugin dso/dll and config path: C:/ProgramData/Redshift/Plugins/Houdini/18.5.462/dso
2021-02-24 15:10:16:  0: STDOUT: [Redshift] Core data path: C:\ProgramData\Redshift
2021-02-24 15:10:16:  0: STDOUT: [Redshift] Local data path: C:\ProgramData\Redshift
2021-02-24 15:10:16:  0: STDOUT: [Redshift] Procedurals path: C:\ProgramData\Redshift\Procedurals
2021-02-24 15:10:16:  0: STDOUT: [Redshift] Preferences file path: C:\ProgramData\Redshift\preferences.xml
2021-02-24 15:10:16:  0: STDOUT: [Redshift] License path: C:\ProgramData\Redshift
2021-02-24 15:10:17:  0: STDOUT: Detected Houdini version: (18, 5, 462)
2021-02-24 15:10:17:  0: STDOUT: ['C:\\ProgramData\\Thinkbox\\Deadline10\\workers\\DESKTOP-EVFHC5B\\plugins\\6035d908c58ffd42c472e283\\hrender_dl.py', '-f', '682', '691', '1', '-o', '$JOB/render/seq_1/seq_1.$F4.exr', '-g', '-d', '/out/Redshift_ROP_render', '-gpu', '0', '-tempdir', 'C:\\ProgramData\\Thinkbox\\Deadline10\\workers\\DESKTOP-EVFHC5B\\jobsData\\6035d908c58ffd42c472e283\\0_tempKZ1d70', '-arnoldAbortOnLicenseFail', '1', 'E:/projects/Invicare/hip/render/seq_1_render_v001.hiplc']
2021-02-24 15:10:17:  0: STDOUT: Start: 682
2021-02-24 15:10:17:  0: STDOUT: End: 691
2021-02-24 15:10:17:  0: STDOUT: Increment: 1
2021-02-24 15:10:17:  0: STDOUT: Ignore Inputs: True
2021-02-24 15:10:17:  0: STDOUT: Output: $JOB/render/seq_1/seq_1.$F4.exr
2021-02-24 15:10:17:  0: STDOUT: GPUs: 0
2021-02-24 15:10:17:  0: STDOUT: Driver: /out/Redshift_ROP_render
2021-02-24 15:10:17:  0: STDOUT: Input File: E:/projects/Invicare/hip/render/seq_1_render_v001.hiplc
2021-02-24 15:10:19:  0: STDOUT: Begin Path Mapping
2021-02-24 15:10:19:  0: STDOUT: End Path Mapping
2021-02-24 15:10:19:  0: STDOUT: ROP type: Redshift_ROP
2021-02-24 15:10:19:  0: STDOUT: This Slave is overriding its GPU affinity, so the following GPUs will be used by RedShift: 0
2021-02-24 15:10:19:  0: STDOUT: Rendering frame 682 to 691
2021-02-24 15:10:35:  Scheduler Thread - Task "15_682-691" has been requeued, because ChangedStatus. Cancelling task...
2021-02-24 15:10:35:  0: RenderThread CancelCurrentTask called, will transition from state None to None
2021-02-24 15:10:35:  0: In the process of canceling current task: ignoring exception thrown by PluginLoader
2021-02-24 15:10:35:  0: Executing plugin command of type 'Cancel Task'
2021-02-24 15:10:35:  0: Done executing plugin command of type 'Cancel Task'
2021-02-24 15:10:36:  0: Done executing plugin command of type 'Render Task'
2021-02-24 15:10:36:  0: Executing plugin command of type 'End Job'
2021-02-24 15:10:36:  0: Done executing plugin command of type 'End Job'
2021-02-24 15:10:36:  0: Error in EndJob: Process was canceled externally.
2021-02-24 15:10:36:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2021-02-24 15:10:38:  Scheduler Thread - In the process of canceling current tasks: ignoring exception thrown by render thread 0
2021-02-24 15:10:40:  Skipping pending job scan because it is not required at this time
2021-02-24 15:10:40:  Skipping repository repair because it is not required at this time
2021-02-24 15:10:40:  Skipping house cleaning because it is not required at this time
2021-02-24 15:10:40:  Could not obtain license because:
2021-02-24 15:10:40:  Cannot find license file.
2021-02-24 15:10:40:   The license files (or license server system network addresses) attempted are 
2021-02-24 15:10:40:  listed below.  Use LM_LICENSE_FILE to use a different license file,
2021-02-24 15:10:40:   or contact your software provider for a license file.
2021-02-24 15:10:40:  Feature:       deadline
2021-02-24 15:10:40:  Filename:      C:\flexlm\license.dat
2021-02-24 15:10:40:  License path:  C:\flexlm\license.dat;
2021-02-24 15:10:40:  FlexNet Licensing error:-1,359.  System Error: 2 "No such file or directory"
2021-02-24 15:10:40:  For further information, refer to the FlexNet Licensing documentation,
2021-02-24 15:10:40:  available at "www.flexerasoftware.com".
2021-02-24 15:10:40:  Failed to checkout a license using current configuration, checking for auto configuration...
2021-02-24 15:10:40:  Auto Configuration: Picking configuration based on: DESKTOP-EVFHC5B / 192.168.0.156
2021-02-24 15:10:40:  Auto Configuration: No auto configuration could be detected, using local configuration
2021-02-24 15:10:40:  Scheduler Thread - Job's Limit Groups: 
2021-02-24 15:10:40:  Could not obtain license because:
2021-02-24 15:10:40:  Cannot find license file.
2021-02-24 15:10:40:   The license files (or license server system network addresses) attempted are 
2021-02-24 15:10:40:  listed below.  Use LM_LICENSE_FILE to use a different license file,
2021-02-24 15:10:40:   or contact your software provider for a license file.
2021-02-24 15:10:40:  Feature:       deadline
2021-02-24 15:10:40:  Filename:      C:\flexlm\license.dat
2021-02-24 15:10:40:  License path:  C:\flexlm\license.dat;
2021-02-24 15:10:40:  FlexNet Licensing error:-1,359.  System Error: 2 "No such file or directory"
2021-02-24 15:10:40:  For further information, refer to the FlexNet Licensing documentation,
2021-02-24 15:10:40:  available at "www.flexerasoftware.com".
2021-02-24 15:10:40:  Failed to checkout a license using current configuration, checking for auto configuration...
2021-02-24 15:10:40:  Auto Configuration: Picking configuration based on: DESKTOP-EVFHC5B / 192.168.0.156
2021-02-24 15:10:40:  Auto Configuration: No auto configuration could be detected, using local configuration
2021-02-24 15:10:40:  0: Loading Job's Plugin timeout is Disabled
2021-02-24 15:10:40:  0: SandboxedPlugin: Render Job As User disabled, running as current user 'User'
2021-02-24 15:10:42:  All job files are already synchronized
2021-02-24 15:10:42:  Plugin Houdini was already synchronized.
2021-02-24 15:10:42:  0: nogui was not added to the CommandLineParser.
2021-02-24 15:10:42:  0: Executing plugin command of type 'Initialize Plugin'
2021-02-24 15:10:42:  0: INFO: Executing plugin script 'C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\plugins\6035d908c58ffd42c472e283\Houdini.py'
2021-02-24 15:10:42:  0: INFO: About: Houdini Plugin for Deadline
2021-02-24 15:10:42:  0: INFO: The job's environment will be merged with the current environment before rendering
2021-02-24 15:10:42:  0: Done executing plugin command of type 'Initialize Plugin'
2021-02-24 15:10:43:  0: Start Job timeout is disabled.
2021-02-24 15:10:43:  0: Task timeout is disabled.
2021-02-24 15:10:43:  0: Loaded job: seq_1_render_v001 - /out/Redshift_ROP_render (6035d908c58ffd42c472e283)
2021-02-24 15:10:43:  0: Executing plugin command of type 'Start Job'
2021-02-24 15:10:43:  0: DEBUG: S3BackedCache Client is not installed.
2021-02-24 15:10:43:  0: INFO: Executing global asset transfer preload script 'C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\plugins\6035d908c58ffd42c472e283\GlobalAssetTransferPreLoad.py'
2021-02-24 15:10:43:  0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer...
2021-02-24 15:10:43:  0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in C:/Program Files/Thinkbox/S3BackedCache/bin/task.py...
2021-02-24 15:10:43:  0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2021-02-24 15:10:43:  0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2021-02-24 15:10:43:  0: Done executing plugin command of type 'Start Job'
2021-02-24 15:10:43:  0: Plugin rendering frame(s): 682-691
2021-02-24 15:10:43:  0: Executing plugin command of type 'Render Task'
2021-02-24 15:10:43:  0: INFO: Starting Houdini Job
2021-02-24 15:10:43:  0: INFO: Stdout Redirection Enabled: True
2021-02-24 15:10:43:  0: INFO: Stdout Handling Enabled: True
2021-02-24 15:10:43:  0: INFO: Popup Handling Enabled: True
2021-02-24 15:10:43:  0: INFO: QT Popup Handling Enabled: False
2021-02-24 15:10:43:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2021-02-24 15:10:43:  0: INFO: Using Process Tree: True
2021-02-24 15:10:43:  0: INFO: Hiding DOS Window: True
2021-02-24 15:10:43:  0: INFO: Creating New Console: False
2021-02-24 15:10:43:  0: INFO: Running as user: User
2021-02-24 15:10:43:  0: INFO: Executable: "C:\Program Files\Side Effects Software\Houdini 18.5.462\bin\Hython.exe"
2021-02-24 15:10:43:  0: INFO: Argument: "C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\plugins\6035d908c58ffd42c472e283\hrender_dl.py" -f 682 691 1 -o "$JOB/render/seq_1/seq_1.$F4.exr" -g -d /out/Redshift_ROP_render -gpu 0 -tempdir "C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\jobsData\6035d908c58ffd42c472e283\0_tempg9lT20" -arnoldAbortOnLicenseFail 1 "E:/projects/Invicare/hip/render/seq_1_render_v001.hiplc"
2021-02-24 15:10:43:  0: INFO: Full Command: "C:\Program Files\Side Effects Software\Houdini 18.5.462\bin\Hython.exe" "C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\plugins\6035d908c58ffd42c472e283\hrender_dl.py" -f 682 691 1 -o "$JOB/render/seq_1/seq_1.$F4.exr" -g -d /out/Redshift_ROP_render -gpu 0 -tempdir "C:\ProgramData\Thinkbox\Deadline10\workers\DESKTOP-EVFHC5B\jobsData\6035d908c58ffd42c472e283\0_tempg9lT20" -arnoldAbortOnLicenseFail 1 "E:/projects/Invicare/hip/render/seq_1_render_v001.hiplc"
2021-02-24 15:10:43:  0: INFO: Startup Directory: "C:\Program Files\Side Effects Software\Houdini 18.5.462\bin"
2021-02-24 15:10:43:  0: INFO: Process Priority: BelowNormal
2021-02-24 15:10:43:  0: INFO: Process Affinity: default
2021-02-24 15:10:43:  0: INFO: Process is now running
2021-02-24 15:10:46:  0: STDOUT: [Redshift] Redshift for Houdini plugin version 3.0.37 (Jan 29 2021 14:50:27)
2021-02-24 15:10:46:  0: STDOUT: [Redshift] Plugin compile time HDK version: 18.5.462
2021-02-24 15:10:46:  0: STDOUT: [Redshift] Houdini host version: 18.5.462
2021-02-24 15:10:46:  0: STDOUT: [Redshift] Plugin dso/dll and config path: C:/ProgramData/Redshift/Plugins/Houdini/18.5.462/dso
2021-02-24 15:10:46:  0: STDOUT: [Redshift] Core data path: C:\ProgramData\Redshift
2021-02-24 15:10:46:  0: STDOUT: [Redshift] Local data path: C:\ProgramData\Redshift
2021-02-24 15:10:46:  0: STDOUT: [Redshift] Procedurals path: C:\ProgramData\Redshift\Procedurals
2021-02-24 15:10:46:  0: STDOUT: [Redshift] Preferences file path: C:\ProgramData\Redshift\preferences.xml
2021-02-24 15:10:46:  0: STDOUT: [Redshift] License path: C:\ProgramData\Redshift
2021-02-24 15:10:47:  0: STDOUT: Detected Houdini version: (18, 5, 462)
2021-02-24 15:10:47:  0: STDOUT: ['C:\\ProgramData\\Thinkbox\\Deadline10\\workers\\DESKTOP-EVFHC5B\\plugins\\6035d908c58ffd42c472e283\\hrender_dl.py', '-f', '682', '691', '1', '-o', '$JOB/render/seq_1/seq_1.$F4.exr', '-g', '-d', '/out/Redshift_ROP_render', '-gpu', '0', '-tempdir', 'C:\\ProgramData\\Thinkbox\\Deadline10\\workers\\DESKTOP-EVFHC5B\\jobsData\\6035d908c58ffd42c472e283\\0_tempg9lT20', '-arnoldAbortOnLicenseFail', '1', 'E:/projects/Invicare/hip/render/seq_1_render_v001.hiplc']
2021-02-24 15:10:47:  0: STDOUT: Start: 682
2021-02-24 15:10:47:  0: STDOUT: End: 691
2021-02-24 15:10:47:  0: STDOUT: Increment: 1
2021-02-24 15:10:47:  0: STDOUT: Ignore Inputs: True
2021-02-24 15:10:47:  0: STDOUT: Output: $JOB/render/seq_1/seq_1.$F4.exr
2021-02-24 15:10:47:  0: STDOUT: GPUs: 0
2021-02-24 15:10:47:  0: STDOUT: Driver: /out/Redshift_ROP_render
2021-02-24 15:10:47:  0: STDOUT: Input File: E:/projects/Invicare/hip/render/seq_1_render_v001.hiplc
2021-02-24 15:10:49:  0: STDOUT: Begin Path Mapping
2021-02-24 15:10:49:  0: STDOUT: End Path Mapping
2021-02-24 15:10:49:  0: STDOUT: ROP type: Redshift_ROP
2021-02-24 15:10:49:  0: STDOUT: This Slave is overriding its GPU affinity, so the following GPUs will be used by RedShift: 0
2021-02-24 15:10:49:  0: STDOUT: Rendering frame 682 to 691

In case it can help someone, I was able to make it work by overriding GPU Affinity to GPU 1 and setting my job properties to GPUs per task : 0 and Select Gpu Devices : 1.


2 Likes

You can make it work in Houdini by using a python pre-render script in the Redshift ROP :

and by using one of these instructions to select what GPU you want to use :

hou.hscript('setenv REDSHIFT_GPUDEVICES=0; varchange REDSHIFT_GPUDEVICES')    # use GPU 0
hou.hscript('setenv REDSHIFT_GPUDEVICES=1; varchange REDSHIFT_GPUDEVICES')    # use GPU 1
hou.hscript('setenv REDSHIFT_GPUDEVICES=0,1; varchange REDSHIFT_GPUDEVICES')  # use GPU 0  and 1

Basically the problem comes from

C:\ProgramData\redshift\preferences.xml

the line :

preference name=“SelectedComputeDevices” type=“string” value=“1:NVIDIA GeForce RTX 3090,”

override everything, thus the select GPU devices in deadline submitter doesn’t work and the GPU define by SelectedComputeDevices is always the one used by deadline.

Privacy | Site terms | Cookie preferences