AWS Thinkbox Discussion Forums

Submitting a Deadline job to Houdini with deadlinecommand

Hi, I’m using deadlinecommand from powershell and trying to figure out how to run Houdini for the rendering job from inside my venv.

The issue I encounter submitting the job (not touching venv yet) is the error I get in Deadline Monitor when running this line from C:\Program Files\Thinkbox\Deadline10\bin:

 .\deadlinecommand.exe "job_info.job" "plugin_info.job" "path_to_Houdini_file.hiplc"

powershell prints:

The job was submitted successfully.

Deadline Monitor however goes:

**Error: GetPluginInfoEntry: Script accessed non-existent plugin info key: OutputDriver (Deadline.Plugins.RenderPluginException)**

here are my test .job files:

  1. job_info.job
Plugin=Houdini
Name=DeadlineTest
Comment=
Department=
Pool=none
Group=none
Priority=50
TaskTimeoutMinutes=0
EnableAutoTimeout=False
ConcurrentTasks=1
LimitConcurrentTasksToNumberOfCpus=True
Whitelist=
LimitGroups=
JobDependencies=
OnJobComplete=Nothing
MachineLimit=0
Frames=1
ChunkSize=1
MachineName=<NAME_OF_MY_MACHINE_SETUP_AS_WORKER>
  1. plugin_info.job
Comp=test
Version=19.5
IgnoreMissingLayerDependenciesErrors=False
MemoryManagement=False
ImageCachePercentage=100
MaxMemoryPercentage=100
MultiProcess=False
ContinueOnMissingFootage=True

I actually never specify the OutputDriver. I added it to job_info.job at first but when I got this error I deleted that line. I keep getting this error nonetheless as if it’s in plugin_info.job, not job_info.job.

I’d like Deadline to submit a rendering job to Houdini to render eg. an exr image from a cone geometry in Solaris in Houdini:


It does so when I Save the USD Render ROP node to drive.

How do I submit the job from Deadline to Houdini? I don’t need to be using the actual .job files, I created those just for test purposes.

my Deadline:
Deadline Client Version: 10.3.0.13 Release (7883f0093)
Deadline Repository: 10.22.1.2:8080

I added these lines to plugin_info.job:

OutputDriver=/stage/layer_1
Output=C:\Test.0001.exr

Now Deadline Monitor doesn’t throw the OutputDriver error, but exports a .usda file with no geometry and still no .exr file.

Here’s the DeadlineWorker log:

=======================================================
Log
=======================================================
2024-05-10 14:55:07:  0: Loading Job's Plugin timeout is Disabled
2024-05-10 14:55:07:  0: SandboxedPlugin: Render Job As User disabled, running as current user 'm.zaras'
2024-05-10 14:55:09:  0: Executing plugin command of type 'Initialize Plugin'
2024-05-10 14:55:09:  0: INFO: Executing plugin script 'C:\ProgramData\Thinkbox\Deadline10\workers\KATANA-DK-36\plugins\663e1926f65215b1c3d71672\Houdini.py'
2024-05-10 14:55:09:  0: INFO: Plugin execution sandbox using Python version 3
2024-05-10 14:55:09:  0: INFO: About: Houdini Plugin for Deadline
2024-05-10 14:55:09:  0: INFO: The job's environment will be merged with the current environment before rendering
2024-05-10 14:55:09:  0: Done executing plugin command of type 'Initialize Plugin'
2024-05-10 14:55:09:  0: Start Job timeout is disabled.
2024-05-10 14:55:09:  0: Task timeout is disabled.
2024-05-10 14:55:09:  0: Loaded job: DeadlineTest (663e1926f65215b1c3d71672)
2024-05-10 14:55:09:  0: Executing plugin command of type 'Start Job'
2024-05-10 14:55:09:  0: DEBUG: S3BackedCache Client is not installed.
2024-05-10 14:55:09:  0: INFO: Executing global asset transfer preload script 'C:\ProgramData\Thinkbox\Deadline10\workers\KATANA-DK-36\plugins\663e1926f65215b1c3d71672\GlobalAssetTransferPreLoad.py'
2024-05-10 14:55:09:  0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer...
2024-05-10 14:55:09:  0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in C:/Program Files/Thinkbox/S3BackedCache/bin/task.py...
2024-05-10 14:55:09:  0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2024-05-10 14:55:09:  0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2024-05-10 14:55:09:  0: Done executing plugin command of type 'Start Job'
2024-05-10 14:55:09:  0: Plugin rendering frame(s): 1
2024-05-10 14:55:09:  0: Executing plugin command of type 'Render Task'
2024-05-10 14:55:09:  0: INFO: Starting Houdini Job
2024-05-10 14:55:09:  0: INFO: Stdout Redirection Enabled: True
2024-05-10 14:55:09:  0: INFO: Stdout Handling Enabled: True
2024-05-10 14:55:09:  0: INFO: Popup Handling Enabled: True
2024-05-10 14:55:09:  0: INFO: QT Popup Handling Enabled: False
2024-05-10 14:55:09:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2024-05-10 14:55:09:  0: INFO: Using Process Tree: True
2024-05-10 14:55:09:  0: INFO: Hiding DOS Window: True
2024-05-10 14:55:09:  0: INFO: Creating New Console: False
2024-05-10 14:55:09:  0: INFO: Running as user: m.zaras
2024-05-10 14:55:09:  0: INFO: Executable: "C:\Program Files\Side Effects Software\Houdini 19.5.805\bin\hython.exe"
2024-05-10 14:55:09:  0: INFO: Argument: "C:\ProgramData\Thinkbox\Deadline10\workers\KATANA-DK-36\plugins\663e1926f65215b1c3d71672\hrender_dl.py" -f 1 1 1 -o "C:/Projects/Houdini_running_OmniKit/testSceneHoudiniwLayers/render/Test.0001.exr" -d /stage/layer_1 -tempdir "C:\ProgramData\Thinkbox\Deadline10\workers\KATANA-DK-36\jobsData\663e1926f65215b1c3d71672\0_tempgncTE0" -arnoldAbortOnLicenseFail 1 "C:/ProgramData/Thinkbox/Deadline10/workers/KATANA-DK-36/jobsData/663e1926f65215b1c3d71672/KitTest.hiplc"
2024-05-10 14:55:09:  0: INFO: Full Command: "C:\Program Files\Side Effects Software\Houdini 19.5.805\bin\hython.exe" "C:\ProgramData\Thinkbox\Deadline10\workers\KATANA-DK-36\plugins\663e1926f65215b1c3d71672\hrender_dl.py" -f 1 1 1 -o "C:/Projects/Houdini_running_OmniKit/testSceneHoudiniwLayers/render/Test.0001.exr" -d /stage/layer_1 -tempdir "C:\ProgramData\Thinkbox\Deadline10\workers\KATANA-DK-36\jobsData\663e1926f65215b1c3d71672\0_tempgncTE0" -arnoldAbortOnLicenseFail 1 "C:/ProgramData/Thinkbox/Deadline10/workers/KATANA-DK-36/jobsData/663e1926f65215b1c3d71672/KitTest.hiplc"
2024-05-10 14:55:09:  0: INFO: Startup Directory: "C:\Program Files\Side Effects Software\Houdini 19.5.805\bin"
2024-05-10 14:55:09:  0: INFO: Process Priority: BelowNormal
2024-05-10 14:55:09:  0: INFO: Process Affinity: default
2024-05-10 14:55:09:  0: INFO: Process is now running
2024-05-10 14:55:13:  0: STDOUT: Detected Houdini version: (19, 5, 805)
2024-05-10 14:55:13:  0: STDOUT: ['C:\\ProgramData\\Thinkbox\\Deadline10\\workers\\KATANA-DK-36\\plugins\\663e1926f65215b1c3d71672\\hrender_dl.py', '-f', '1', '1', '1', '-o', 'C:/Projects/Houdini_running_OmniKit/testSceneHoudiniwLayers/render/Test.0001.exr', '-d', '/stage/layer_1', '-tempdir', 'C:\\ProgramData\\Thinkbox\\Deadline10\\workers\\KATANA-DK-36\\jobsData\\663e1926f65215b1c3d71672\\0_tempgncTE0', '-arnoldAbortOnLicenseFail', '1', 'C:/ProgramData/Thinkbox/Deadline10/workers/KATANA-DK-36/jobsData/663e1926f65215b1c3d71672/KitTest.hiplc']
2024-05-10 14:55:13:  0: STDOUT: Start: 1
2024-05-10 14:55:13:  0: STDOUT: End: 1
2024-05-10 14:55:13:  0: STDOUT: Increment: 1
2024-05-10 14:55:13:  0: STDOUT: Ignore Inputs: False
2024-05-10 14:55:13:  0: STDOUT: Output: C:/Projects/Houdini_running_OmniKit/testSceneHoudiniwLayers/render/Test.0001.exr
2024-05-10 14:55:13:  0: STDOUT: Driver: /stage/layer_1
2024-05-10 14:55:13:  0: STDOUT: Input File: C:/ProgramData/Thinkbox/Deadline10/workers/KATANA-DK-36/jobsData/663e1926f65215b1c3d71672/KitTest.hiplc
2024-05-10 14:55:13:  0: STDOUT: WARNING: Entered limited commercial session mode.
2024-05-10 14:55:14:  0: STDOUT: Unknown command: viewcharacteropts
2024-05-10 14:55:14:  0: STDOUT: Unknown command: viewcharacteropts
2024-05-10 14:55:14:  0: STDOUT: Unknown command: viewcharacteropts
2024-05-10 14:55:14:  0: STDOUT: Unknown command: viewcharacteropts
2024-05-10 14:55:14:  0: STDOUT: Warnings were generated during load.
2024-05-10 14:55:14:  0: STDOUT: Error loading: C:/ProgramData/Thinkbox/Deadline10/workers/KATANA-DK-36/jobsData/663e1926f65215b1c3d71672/KitTest.hiplc
2024-05-10 14:55:14:  0: STDOUT: Warning:     
2024-05-10 14:55:14:  0: STDOUT: /stage:
2024-05-10 14:55:14:  0: STDOUT:              Skipping unrecognized parameter "resolvercontextstringcount".
2024-05-10 14:55:14:  0: STDOUT:              Skipping unrecognized parameter "variantselectioncount".
2024-05-10 14:55:15:  0: STDOUT: /stage/layer_1:
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "filtertimesamples".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "filtertimesamplespadding".           
2024-05-10 14:55:15:  0: STDOUT: /stage/karmarendersettings:
2024-05-10 14:55:15:  0: STDOUT:              Unknown channel(s) "tonemapcurve1value, tonemapcurve2value, tonemapcurve3value, tonemapcurve4value, tonemapcurve5value, tonemapcurve6value, tonemapcurve7value, tonemapcurve8value, tonemapcurve9value, tonemapcurve10value, tonemapcurve11value, tonemapcurve12value, tonemapcurve13value, tonemapcurve14value, tonemapcurve15value, tonemapcurve16value, tonemapcurve17value, tonemapcurve18value, tonemapcurve19value, tonemapcurve20value, tonemapcurve21value" converted to spare parameter(s).
2024-05-10 14:55:15:  0: STDOUT:              
2024-05-10 14:55:15:  0: STDOUT: /stage/karmarendersettings:
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "samplesfromstage".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "blurstyle".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "autoraybias".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "dooutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "outputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "dobeautyoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "beautyoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "dobeautyunshadowedoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "beautyunshadowedoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "shadow".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "shadowperlpe".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "shadowfilter".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "shadowprecision".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "doshadowoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "shadowoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "docombineddiffuseoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "combineddiffuseoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "dodirectdiffuseoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "directdiffuseoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "doindirectdiffuseoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "indirectdiffuseoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "docombineddiffuseunshadowedoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "combineddiffuseunshadowedoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "dodirectdiffuseunshadowedoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "directdiffuseunshadowedoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "doindirectdiffuseunshadowedoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "indirectdiffuseunshadowedoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "sepparm39".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "combineddiffuseshadow".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "combineddiffuseshadowperlpe".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "combineddiffuseshadowfilter".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "combineddiffuseshadowprecision".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "docombineddiffuseshadowoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "combineddiffuseshadowoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "sepparm40".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "directdiffuseshadow".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "directdiffuseshadowperlpe".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "directdiffuseshadowfilter".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "directdiffuseshadowprecision".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "dodirectdiffuseshadowoutputcs".
2024-05-10 14:55:15:  0: STDOUT:              -- Too many errors, additional errors were suppressed. --
2024-05-10 14:55:15:  0: STDOUT: /stage/usdrender_rop1:
2024-05-10 14:55:15:  0: STDOUT:              Unknown channel(s) "outputimageshowwarning" converted to spare parameter(s).
2024-05-10 14:55:15:  0: STDOUT:              
2024-05-10 14:55:15:  0: STDOUT: /stage/usdrender_rop1:
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "outputimagewarning".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "husk_enable_headlight".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "husk_headlight".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "verbose".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "verbose_menu".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "vexprofile".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "dorenderexisting".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "renderexisting".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "resolvercontextstringcount".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "variantselectioncount".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "usdexport_group".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "filtertimesamples".
2024-05-10 14:55:15:  0: STDOUT:              Skipping unrecognized parameter "filtertimesamplespadding".
2024-05-10 14:55:15:  0: STDOUT:              
2024-05-10 14:55:15:  0: STDOUT: Begin Path Mapping
2024-05-10 14:55:15:  0: STDOUT: b''
2024-05-10 14:55:16:  0: STDOUT: b''
2024-05-10 14:55:16:  0: STDOUT: End Path Mapping
2024-05-10 14:55:16:  0: STDOUT: ROP type: usd_rop
2024-05-10 14:55:16:  0: STDOUT: Rendering frame 1
2024-05-10 14:55:16:  0: STDOUT: Finished Rendering
2024-05-10 14:55:17:  0: INFO: Process exit code: 0
2024-05-10 14:55:17:  0: INFO: Finished Houdini Job
2024-05-10 14:55:17:  0: Done executing plugin command of type 'Render Task'

=======================================================
Details
=======================================================
Date: 05/10/2024 14:55:18
Frames: 1
Job Submit Date: 05/10/2024 14:55:01
Job User: m.zaras
Average RAM Usage: 0 (0%)
Peak RAM Usage: 0 (0%)
Average CPU Usage: 0%
Peak CPU Usage: 0%
Used CPU Clocks (x10^6 cycles): 0
Total CPU Clocks (x10^6 cycles): 0

=======================================================
Worker Information
=======================================================
Worker Name: <WORKER_NAME>
Version: v10.3.0.13 Release (7883f0093)
Operating System: Windows 10 Pro
Running As Service: No
Machine User: m.zaras
IP Address: 10.12.1.67
MAC Address: 58:11:22:2A:0C:FC
CPU Architecture: x64
CPUs: 24
CPU Usage: 1%
Memory Usage: 26.7 GB / 127.7 GB (20%)
Free Disk Space: 155.303 GB 
Video Card: NVIDIA TITAN RTX

Our Houdini application plugin relies on OutputDriver being set in the plugin info to tell Houdini which renderer to use. So in your case I think you’d pass the usdrender_rop2 as the output driver.

I’d submit a job using the integrated submitter to and inspect the job’s properties->submission params to see how the submitter builds the job and go from there.

Thanks for your reply Justin_B. You’re right, I changed the node to usdrender_rop2 in the job info file.

  1. I modified the job.info file to reflect submission params of those that get created when I use Submit Houdini To Deadline plugin:
Denylist=
EventOptIns=
Frames=1
IsFrameDependent=True
MachineName=<worker name>
Name=KitTest **- /stage/usdrender_rop2**
OverrideTaskExtraInfoNames=False
Plugin=Houdini
Region=
UserName=<name>

The job gets submitted to Deadline, and completes but the exr file doesn’t get created from the USD render ROP node (which generates the exr file after I click “Save to disk” on it.

To recap, I do this with the powershell command:

.\deadlinecommand.exe "job_info.job" "plugin_info.job" "path_to_Houdini_file.hiplc"
  1. Alternatively, I’ve tried this from powershell:
 & "python" "<path to script in which I activate venv.py>"; & "C:\Program Files\Side Effects Software\Houdini 19.5.805\bin\Hython.exe" "<path to a copy of hrender_dl.py>" -f 1 1 1 -o "<path to exr file to be generated.exr>" -g -d /stage/usdrender_rop2 "<path to job info.job>" -tempdir "C:\tempDir" "<path to KitTest.hiplc>"

this renders the exr file, thought weird looking with no textures and doesn’t submit the job to Deadline Monitor at all. Houdini just renders in the background. And I’d like to submit the job to Houdini while adding it to Monitor:
image

and this log:
https://jumpshare.com/s/vSKkIAv1QOhGNlw8oJRF

Ah, that’s just emulating what Deadline is doing to run Houdini - you’re doing what the Worker is doing there. So no job is submitted, you’re just calling Houdini.

Your powershell command should be creating a new job though.

For the render issue, how does the render perform when you submit the scene with the integrated submitter instead of your manually created job? There’s a ton of Skipping unrecognized parameter callouts in there that make me think there’s an issue with your installation. That’s only based on what I’ve seen before, so I could be off the mark.

Hey Justin, thanks for the context. The Deadline submitter for Houdini works fine, and renders my .exr file without any errors re “Skipping unrecognized parameter”. It’s only when I use either

.\deadlinecommand.exe "job_info.job" "plugin_info.job" "path_to_Houdini_file.hiplc"

or

& "python" "<path to script in which I activate venv.py>"; & "C:\Program Files\Side Effects Software\Houdini 19.5.805\bin\Hython.exe" "<path to a copy of hrender_dl.py>" -f 1 1 1 -o "<path to exr file to be generated.exr>" -g -d /stage/usdrender_rop2 "<path to job info.job>" -tempdir "C:\tempDir" "<path to KitTest.hiplc>"

that I get these errors (and the 2nd command renders the textureless cone.

I wonder why my deadlinecommand doesn’t render an .exr file at all, is it something on my job files?

job.job

PreJobScript=C:\DeadlineLaunchingHoudiniFromVenv.py
Denylist=
EventOptIns=
Frames=1
IsFrameDependent=True
MachineName=<worker name>
Name=KitTest - /stage/usdrender_rop2
OverrideTaskExtraInfoNames=False
Plugin=Houdini
Region=
UserName=<name>

plugin.job

Comp=test
Version=19.5
IgnoreMissingLayerDependenciesErrors=False
MemoryManagement=False
ImageCachePercentage=100
MaxMemoryPercentage=100
MultiProcess=False
ContinueOnMissingFootage=True
OutputDriver=/stage/usdrender_rop2

I have path to the output .exr specified in both usd render rop node and karmarendersettings node:


If those job.info and plugin.info files are identical to what you’d get when you hit the export button on the job’s ‘Submission Params’ page then there shouldn’t be an issue, since that’d be an identical job.

If that’s not the case however - check and see what’s different. From what you’ve posted it looks okay, but what’s needed for a plugin file to be valid isn’t documented.

You’ve got to run it through a Worker to see how it behaves. The Houdini graph looks fine to me - but I can just about make a cube with a light in Houdini so don’t take my word on it.

What output do you get from the job you submit with deadlinecommand? I’m assuming a job gets submitted, as the error’s usually clear so I’m assuming something’s failing in a task report?

It could also be that your PreJobScript is causing trouble, but let’s leave that for your thread about venv - just for anyone in the future looking to learn from our work here. :slight_smile:

Privacy | Site terms | Cookie preferences