Errors running houdini on worker: ERROR: Encountered the following error while initializing the Plugin Sandbox: 'Value cannot be null. (Parameter 'input')'

I have recently updated to deadline 10.1.14.5, and I haven’t tested deadline for quite some time. When I tried to run some test houdini / hython tasks on a spot event worker today I got this error, and couldn’t determine a solution. Would anyone know how to fix this?

2021-05-02 05:24:20:  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021-05-02 05:24:21:  Spot: Spot Plugin - On Worker Info Updated
2021-05-02 05:24:21:  Spot: i-048e2c147c4b48b3f is rendering.
2021-05-02 05:24:22:  Scheduler Thread - Job's Limit Groups: 
2021-05-02 05:24:23:  Spot: Spot Plugin - On Worker Info Updated
2021-05-02 05:24:23:  Spot: i-048e2c147c4b48b3f is rendering.
2021-05-02 05:24:23:  0: Loading Job's Plugin timeout is Disabled
2021-05-02 05:24:23:  ERROR: Encountered the following error while initializing the Plugin Sandbox: 'Value cannot be null. (Parameter 'input')'.
2021-05-02 05:24:23:   Cannot render the current job [ID:"608e369f2ce4354bc907ccd0" / Name:"PDG TASKS"]. Contact Thinkbox Support for more information.
2021-05-02 05:24:24:  Scheduler Thread - Render Thread 0 threw a major error: 
2021-05-02 05:24:24:  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021-05-02 05:24:24:  Exception Details
2021-05-02 05:24:24:  RenderPluginException -- Failed to load the plugin because: Could not initialize the plugin sandbox (Deadline.Plugins.PluginException)
2021-05-02 05:24:24:  RenderPluginException.Cause: PluginLoadError (0)
2021-05-02 05:24:24:  RenderPluginException.Level: Major (1)
2021-05-02 05:24:24:  RenderPluginException.HasSlaveLog: True
2021-05-02 05:24:24:  RenderPluginException.SlaveLogFileName: /var/log/Thinkbox/Deadline10/deadlineslave_renderthread_0-ip-10-1-140-5-0000.log
2021-05-02 05:24:24:  Exception.TargetSite: Boolean e(System.String, Deadline.Jobs.Job, System.Threading.CancellationToken)
2021-05-02 05:24:24:  Exception.Data: ( )
2021-05-02 05:24:24:  Exception.Source: deadline
2021-05-02 05:24:24:  Exception.HResult: -2146233088
2021-05-02 05:24:24:    Exception.StackTrace: 
2021-05-02 05:24:24:     at Deadline.Slaves.SlaveRenderThread.e(String ajl, Job ajm, CancellationToken ajn
2021-05-02 05:24:24:     at Deadline.Slaves.SlaveRenderThread.b(TaskLogWriter ajh, CancellationToken aji)
2021-05-02 05:24:24:  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

I’ve progressed from that error, admittedly, I don’t know what occured there. I can get to the point now where hython launches, and the license server is picked up. however, further errors occured which I resolved only by attempting the raw command on the instance, because the deadline log did not show the true stdout/stderr that was responsible. Is there a way I can fix deadline’s logging?

This was the deceptive error:

2021-05-06 11:24:07:  Scheduler Thread - Job's Limit Groups: 
2021-05-06 11:24:07:  0: Loading Job's Plugin timeout is Disabled
2021-05-06 11:24:08:  0: WARNING: Python version for 'Houdini' plugin is not specified! Defaulting to Py2.
2021-05-06 11:24:08:  0: SandboxedPlugin: Render Job As User disabled, running as current user 'deadlineuser'
2021-05-06 11:24:09:  0: DEBUG: The Worker is also capable of decompressing responses using Brotli
2021-05-06 11:24:09:  Synchronization time for job files: 325.042 ms
2021-05-06 11:24:10:  Synchronizing Plugin Houdini from /home/deadlineuser/Thinkbox/Deadline10/cache/CYnKLgYt7a9TAQJ4j90qgAbBmRA/plugins/Houdini took: 0 seconds
2021-05-06 11:24:10:  0: Executing plugin command of type 'Initialize Plugin'
2021-05-06 11:24:10:  0: INFO: Executing plugin script '/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/plugins/6093ce69bb092c6bec669015/Houdini.py'
2021-05-06 11:24:10:  0: INFO: About: Houdini Plugin for Deadline
2021-05-06 11:24:10:  0: INFO: The job's environment will be merged with the current environment before rendering
2021-05-06 11:24:10:  0: Done executing plugin command of type 'Initialize Plugin'
2021-05-06 11:24:10:  0: Start Job timeout is disabled.
2021-05-06 11:24:10:  0: Task timeout is disabled.
2021-05-06 11:24:10:  0: Loaded job: untitled2 - /obj/ropnet1/mantra1 (6093ce69bb092c6bec669015)
2021-05-06 11:24:10:  0: Executing plugin command of type 'Start Job'
2021-05-06 11:24:10:  0: DEBUG: S3BackedCache Client is not installed.
2021-05-06 11:24:10:  0: INFO: Executing global asset transfer preload script '/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/plugins/6093ce69bb092c6bec669015/GlobalAssetTransferPreLoad.py'
2021-05-06 11:24:10:  0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer...
2021-05-06 11:24:10:  0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in /opt/Thinkbox/S3BackedCache/bin/task.py...
2021-05-06 11:24:10:  0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2021-05-06 11:24:10:  0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2021-05-06 11:24:10:  0: Done executing plugin command of type 'Start Job'
2021-05-06 11:24:10:  0: Plugin rendering frame(s): 1
2021-05-06 11:24:11:  0: Executing plugin command of type 'Render Task'
2021-05-06 11:24:11:  0: INFO: Starting Houdini Job
2021-05-06 11:24:11:  0: INFO: Stdout Redirection Enabled: True
2021-05-06 11:24:11:  0: INFO: Asynchronous Stdout Enabled: False
2021-05-06 11:24:11:  0: INFO: Stdout Handling Enabled: True
2021-05-06 11:24:11:  0: INFO: Popup Handling Enabled: True
2021-05-06 11:24:11:  0: INFO: QT Popup Handling Enabled: False
2021-05-06 11:24:11:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2021-05-06 11:24:11:  0: INFO: Using Process Tree: True
2021-05-06 11:24:11:  0: INFO: Hiding DOS Window: True
2021-05-06 11:24:11:  0: INFO: Creating New Console: False
2021-05-06 11:24:11:  0: INFO: Running as user: deadlineuser
2021-05-06 11:24:11:  0: INFO: Executable: "/opt/hfs18.5/bin/hython"
2021-05-06 11:24:11:  0: INFO: Argument: "/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/plugins/6093ce69bb092c6bec669015/hrender_dl.py" -f 1 1 1 -o "$HIP/render/$HIPNAME.$OS.$F4.exr" -g -d /obj/ropnet1/mantra1 -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/jobsData/6093ce69bb092c6bec669015/0_temphBG7G0" -arnoldAbortOnLicenseFail 1 "/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/jobsData/6093ce69bb092c6bec669015/untitled2.hip"
2021-05-06 11:24:11:  0: INFO: Full Command: "/opt/hfs18.5/bin/hython" "/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/plugins/6093ce69bb092c6bec669015/hrender_dl.py" -f 1 1 1 -o "$HIP/render/$HIPNAME.$OS.$F4.exr" -g -d /obj/ropnet1/mantra1 -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/jobsData/6093ce69bb092c6bec669015/0_temphBG7G0" -arnoldAbortOnLicenseFail 1 "/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/jobsData/6093ce69bb092c6bec669015/untitled2.hip"
2021-05-06 11:24:11:  0: INFO: Startup Directory: "/opt/hfs18.5/bin"
2021-05-06 11:24:11:  0: INFO: Process Priority: BelowNormal
2021-05-06 11:24:11:  0: INFO: Process Affinity: default
2021-05-06 11:24:11:  0: INFO: Process is now running
Success
2021-05-06 11:25:35:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'houdini' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/houdini'. Module will be skipped.
2021-05-06 11:25:35:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'partitioners' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/partitioners'. Module will be skipped.
2021-05-06 11:25:35:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'schedulers' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/schedulers'. Module will be skipped.
2021-05-06 11:25:35:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'utils' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/utils'. Module will be skipped.
2021-05-06 11:25:36:  0: STDOUT: Detected Houdini version: (18, 5, 563)
2021-05-06 11:25:36:  0: STDOUT: ['/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/plugins/6093ce69bb092c6bec669015/hrender_dl.py', '-f', '1', '1', '1', '-o', '$HIP/render/$HIPNAME.$OS.$F4.exr', '-g', '-d', '/obj/ropnet1/mantra1', '-tempdir', '/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/jobsData/6093ce69bb092c6bec669015/0_temphBG7G0', '-arnoldAbortOnLicenseFail', '1', '/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/jobsData/6093ce69bb092c6bec669015/untitled2.hip']
2021-05-06 11:25:36:  0: STDOUT: Start: 1
2021-05-06 11:25:36:  0: STDOUT: End: 1
2021-05-06 11:25:36:  0: STDOUT: Increment: 1
2021-05-06 11:25:36:  0: STDOUT: Ignore Inputs: True
2021-05-06 11:25:36:  0: STDOUT: Output: $HIP/render/$HIPNAME.$OS.$F4.exr
2021-05-06 11:25:36:  0: STDOUT: Driver: /obj/ropnet1/mantra1
2021-05-06 11:25:36:  0: STDOUT: Input File: /var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/jobsData/6093ce69bb092c6bec669015/untitled2.hip
2021-05-06 11:25:39:  0: STDOUT: Begin Path Mapping
2021-05-06 11:25:39:  0: STDOUT: End Path Mapping
2021-05-06 11:25:39:  0: STDOUT: ROP type: ifd
2021-05-06 11:25:39:  0: STDOUT: Enabled Alfred style progress
2021-05-06 11:25:39:  0: STDOUT: Set verbosity to 3
2021-05-06 11:25:39:  0: STDOUT: Rendering frame 1
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'houdini' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/houdini'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'partitioners' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/partitioners'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'schedulers' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/schedulers'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'utils' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/utils'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'houdini' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/houdini'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'partitioners' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/partitioners'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'schedulers' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/schedulers'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'utils' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/utils'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'houdini' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/houdini'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'partitioners' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/partitioners'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'schedulers' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/schedulers'. Module will be skipped.
2021-05-06 11:25:45:  0: STDOUT: PDG Type Registry: Failed to import duplicate module 'utils' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/utils'. Module will be skipped.
2021-05-06 11:25:50:  0: Sending kill command to process hython-bin with id: 3075
2021-05-06 11:25:50:  0: Done executing plugin command of type 'Render Task'
2021-05-06 11:25:50:  0: Executing plugin command of type 'End Job'
2021-05-06 11:25:50:  0: Done executing plugin command of type 'End Job'
2021-05-06 11:25:52:  Sending kill command to process deadlinesandbox.exe with id: 3044
2021-05-06 11:25:53:  Scheduler Thread - Render Thread 0 threw a major error: 
2021-05-06 11:25:53:  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021-05-06 11:25:53:  Exception Details
2021-05-06 11:25:53:  RenderPluginException -- FailRenderException : Error: Caught exception: The attempted operation failed.
2021-05-06 11:25:53:     at Deadline.Plugins.DeadlinePlugin.FailRender(String message) (Python.Runtime.PythonException)
2021-05-06 11:25:53:    File "/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/plugins/6093ce69bb092c6bec669015/Houdini.py", line 424, in HandleStdoutError
2021-05-06 11:25:53:      self.FailRender(self.GetRegexMatch(1))
2021-05-06 11:25:53:     at Python.Runtime.Dispatcher.Dispatch(ArrayList args)
2021-05-06 11:25:53:     at __FranticX_Processes_ManagedProcess_StdoutHandlerDelegateDispatcher.Invoke()
2021-05-06 11:25:53:     at FranticX.Processes.ManagedProcess.RegexHandlerCallback.CallFunction()
2021-05-06 11:25:53:     at FranticX.Processes.ManagedProcess.e(String cj, Boolean ck)
2021-05-06 11:25:53:     at FranticX.Processes.ManagedProcess.Execute(Boolean waitForExit)
2021-05-06 11:25:53:     at Deadline.Plugins.DeadlinePlugin.DoRenderTasks()
2021-05-06 11:25:53:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2021-05-06 11:25:53:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2021-05-06 11:25:53:  RenderPluginException.Cause: JobError (2)
2021-05-06 11:25:53:  RenderPluginException.Level: Major (1)
2021-05-06 11:25:53:  RenderPluginException.HasSlaveLog: True
2021-05-06 11:25:53:  RenderPluginException.SlaveLogFileName: /var/log/Thinkbox/Deadline10/deadlineslave_renderthread_0-ip-10-1-141-41-0000.log
2021-05-06 11:25:53:  Exception.TargetSite: Deadline.Slaves.Messaging.PluginResponseMemento d(Deadline.Net.DeadlineMessage, System.Threading.CancellationToken)
2021-05-06 11:25:53:  Exception.Data: ( )
2021-05-06 11:25:53:  Exception.Source: deadline
2021-05-06 11:25:53:  Exception.HResult: -2146233088
2021-05-06 11:25:53:    Exception.StackTrace: 
2021-05-06 11:25:53:     at Deadline.Plugins.SandboxedPlugin.d(DeadlineMessage bdb, CancellationToken bdc
2021-05-06 11:25:53:     at Deadline.Plugins.SandboxedPlugin.RenderTask(Task task, CancellationToken cancellationToken
2021-05-06 11:25:53:     at Deadline.Slaves.SlaveRenderThread.c(TaskLogWriter ajj, CancellationToken ajk)
2021-05-06 11:25:53:  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

But the true result by running the raw command was:

End Path Mapping
ROP type: ifd
Enabled Alfred style progress
Set verbosity to 3
Rendering frame 1
PDG Type Registry: Failed to import duplicate module 'houdini' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/houdini'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'partitioners' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/partitioners'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'schedulers' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/schedulers'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'utils' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/utils'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'houdini' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/houdini'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'partitioners' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/partitioners'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'schedulers' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/schedulers'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'utils' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/utils'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'houdini' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/houdini'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'partitioners' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/partitioners'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'schedulers' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/schedulers'. Module will be skipped.
PDG Type Registry: Failed to import duplicate module 'utils' which was previously imported from '/opt/hfs18.5/houdini/pdg/types/utils'. Module will be skipped.
Error: Caught exception: The attempted operation failed.
Error:       Unable to initialize rendering module with given camera
Traceback (most recent call last):
  File "/var/lib/Thinkbox/Deadline10/workers/ip-10-1-141-41/plugins/6093ce69bb092c6bec669015/hrender_dl.py", line 860, in <module>
    rop.render( frameTuple, resolution, ignore_inputs=ignoreInputs )
  File "/opt/hfs18.5/houdini/python2.7libs/hou.py", line 69067, in render
    return _hou.RopNode_render(self, *args, **kwargs)
hou.OperationFailed: The attempted operation failed.
Error:       Unable to initialize rendering module with given camera
[deadlineuser@ip-10-1-141-41 hfs18.5]$ 

Which is obvious since there was no camera… however what interests me is understanding how to get deadline to catch the error properly from stdout/stderr and present it in the monitor.

I can’t comment on the actual issues here other than we’ve seen a couple other cases but don’t have any solid leads on finding the root cause yet.

But I can shed some light on why Deadline didn’t catch that second error line and what could maybe be done about it.

The issue with Deadline’s error logging is as soon as it saw “Error: Caught exception: The attempted operation failed.” it sends a kill to Houdini and stops watching the output. In this case, right before catching the much more useful “Unable to initialize rendering module with given camera”.

The possibly short-sighted option would be to not recognize that first line as a render-failing error so we at least get the next line in the Job report. In a perfect world the regex that watches for stdout errors could be looking for multi-line errors before it triggers the failure.

Does Houdini commonly throw errors in this two line format? Or is this something that’s PDG specific. I ask just to flesh out a feature request with a little more context.

Thanks for your reply Justin. Actually this case is just using the standard deadline rop, rendering a mantra node in an empty scene. The PDG scheduler’s are not actually being used here.

…So it isn’t possible to design it so we wait for the process to yield a non-zero exit code perhaps? I would have thought normally this would occur after most failures (though I appreciate there might be some very rare exceptions where a exit code of 0 is inadequate, and in those cases a log scrape would be performed to return an error).

Tracebacks do often look like that, but I would think waiting for a non-zero exit code after the traceback would be be ther best start. What do you think?

Privacy | Site terms | Cookie preferences