Our renderfarm setup is as follows:
- Windows workstations with Maya 2023 and Arnold (mtoa) for rendering
- Deadline database and repository on a Linux host
- Renderfarm clients are Linux-based
That’s all working well, and now I’m adding a Windows host to the list of renderfarm clients. Put it in a new pool, for testing.
When I submit a job, the job is dispatched to the Windows renderfarm client host correctly, and almost immediately I see errors that look like this:
Error
Failed to load the plugin because: Could not initialize the plugin sandbox (Deadline Plugins PluginException)
Type
RenderPluginException
I do have the Launcher running as a service, and provided an appropriate username and password, which was accepted during the installation of the Deadline Client software.
In Maya, the “mtoa” plugin is set to auto-load.
In Deadline Monitor, a working render shows the MayaBatch plugin is used, and in this test render, under “Plugin” it says “(no plugin).”
I’m sure that I need to set something else up in Deadline… but aren’t sure what.
Update: the Deadline Launcher service is running - I just verified this.
This can happen e.g. if there is an error in one of the plugins.
There should be another log file in $ProgramData/Thinkbox/Deadline10/logs indicating this.
1 Like
Indeed there ARE log files on the renderfarm client.
I opened the log file for the worker as I prepared to send a render job to it, and after I hit “Submit Job” from within Maya, and before the “Submission Results” popup appeared, there was activity in the log file window.
0: Loading Job's Plugin timeout is Disabled
0: SandboxedPlugin: Render Job as User enabled, running as user 'spencer', which is defined in the User Settings for this job's user
ERROR: Encountered the following error while initializing the Plugin Sandbox: 'Error starting "C:\Program Files\Thinkbox\Deadline10\bin\deadlinesandbox.exe" for user "spencer" in "C:\Program Files\Thinkbox\Deadline10\bin" : The stub received bad data.
I do want to run the jobs as the user submitting the job. Not sure what this is telling me.
Huh, no idea. You should try running a command line job - check Deadline’s by searching for Full Command in the log, copy that over and adjust the paths (scenes, temp dir). Then run it in your terminal.
This will likely give you some insight.
I will see about running that job from the command line on that host.
In the meantime, here’s a more verbose log file output from the rendering host:
2024-02-01 14:38:33: Scheduler Thread - Scheduler State transition from = 'LicenseConfirmed' to = 'StartJob'
2024-02-01 14:38:33: 0: Render Thread - Render State transition from = 'WaitingForTask' to = 'ReceivedTask'
2024-02-01 14:38:33: Scheduler Thread - Scheduler State transition from = 'StartJob' to = 'PreRendering'
2024-02-01 14:38:34: 0: Got task!
2024-02-01 14:38:34: 0: Render Thread - Render State transition from = 'ReceivedTask' to = 'Other'
2024-02-01 14:38:34: 0: Plugin will be reloaded because a new job has been loaded.
2024-02-01 14:38:34: 0: Loading Job's Plugin timeout is Disabled
2024-02-01 14:38:34: 0: SandboxedPlugin: Render Job As User enabled, running as user 'spencer', which is defined in the User Settings for this job's user
2024-02-01 14:38:34: ERROR: Encountered the following error while initializing the Plugin Sandbox: 'Error starting "C:\Program Files\Thinkbox\Deadline10\bin\deadlinesandbox.exe" for user "spencer" in "C:\Program Files\Thinkbox\Deadline10\bin" : The stub received bad data.
2024-02-01 14:38:34: '.
2024-02-01 14:38:34: Cannot render the current job [ID:"65bc1d637a136ea0fcdeebb7" / Name:"figaro-painted"]. Contact Thinkbox Support for more information.
2024-02-01 14:38:34: ERROR: 0: An exception occurred: Failed to load the plugin because: Could not initialize the plugin sandbox (Deadline.Plugins.PluginException) (Deadline.Plugins.RenderPluginException)
2024-02-01 14:38:35: 0: Render Thread - Render State transition from = 'Other' to = 'WaitingForTask'
2024-02-01 14:38:35: Scheduler Thread - Render Thread 0 threw a major error:
2024-02-01 14:38:35: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024-02-01 14:38:35: Exception Details
2024-02-01 14:38:35: RenderPluginException -- Failed to load the plugin because: Could not initialize the plugin sandbox (Deadline.Plugins.PluginException)
2024-02-01 14:38:35: RenderPluginException.Cause: PluginLoadError (0)
2024-02-01 14:38:35: RenderPluginException.Level: Major (1)
2024-02-01 14:38:35: RenderPluginException.HasSlaveLog: True
2024-02-01 14:38:35: RenderPluginException.SlaveLogFileName: C:\ProgramData\Thinkbox\Deadline10\logs\deadlineslave_renderthread_0-dreams-0000.log
2024-02-01 14:38:35: Exception.TargetSite: Boolean e(System.String, Deadline.Jobs.Job, System.Threading.CancellationToken)
2024-02-01 14:38:35: Exception.Data: ( )
2024-02-01 14:38:35: Exception.Source: deadline
2024-02-01 14:38:35: Exception.HResult: -2146233088
2024-02-01 14:38:35: Exception.StackTrace:
2024-02-01 14:38:35: at Deadline.Slaves.SlaveRenderThread.e(String ajx, Job ajy, CancellationToken ajz)
2024-02-01 14:38:35: at Deadline.Slaves.SlaveRenderThread.b(TaskLogWriter ajt, CancellationToken aju)
2024-02-01 14:38:35: <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024-02-01 14:38:36: Scheduler Thread - Scheduler State transition from = 'PreRendering' to = 'PostRendering'
2024-02-01 14:38:36: Scheduler Thread - Scheduler State transition from = 'PostRendering' to = 'EndJob'
2024-02-01 14:38:36: Scheduler Thread - Scheduler State transition from = 'EndJob' to = 'WaitingForJob'
Here, it tells you the thread log
2024-02-01 14:38:35: RenderPluginException.SlaveLogFileName: C:\ProgramData\Thinkbox\Deadline10\logs\deadlineslave_renderthread_0-dreams-0000.log
That file does not exist. In that folder, I see log files that begin with
- deadlinelauncher-
- deadlinelauncherservice-
- deadlineslave-
but no deadlineslave_renderthread_0.
When I submit the job, yes, the file appears, ever so briefly, and then disappears. Over and over again. When I kill the job, the file stops appearing.
Well, I know (because I sent a job to one of our Linux-based renderfarm clients) what the deadlineslave_renderthread file is SUPPOSED TO contain…
2024-02-02 14:30:36: 0: SandboxedPlugin: Render Job As User enabled, running as user 'spencer', which is defined in the User Settings for this job's user
2024-02-02 14:30:40: 0: Loaded plugin MayaBatch
2024-02-02 14:30:43: 0: Executing plugin command of type 'Initialize Plugin'
2024-02-02 14:30:43: 0: INFO: Executing plugin script '/var/lib/Thinkbox/Deadline10/workers/wheezy/plugins/65bd6cb1133bed14411d6436/MayaBatch.py'
2024-02-02 14:30:43: 0: INFO: Plugin execution sandbox using Python version 3
2024-02-02 14:30:43: 0: INFO: About: Maya Batch Plugin for Deadline
2024-02-02 14:30:43: 0: INFO: The job's environment will be merged with the current environment before rendering
2024-02-02 14:30:43: 0: Done executing plugin command of type 'Initialize Plugin'
The _renderthread_ log gets folded into the slave log, so we can refer to that one instead.
Have you got ‘render as user’ enabled? This is a known problem using the Render Jobs as User setting on Windows. When impersonating the submitting user, the Deadline Worker does not update the temporary directory in the environment. It is suspected that the application tries to access the temp directory but gets permission or file-system errors. We have seen this on 3DS Max, After Effects and Nuke.
This can also happen if the “Render As User” (or possibly another field) is left blank in the users’ settings in Tools → “Manage Users”.
Yes, I have “render as user” enabled - I did that so that Maya can write the rendered output to the user’s directory, with appropriate ownership and permissions.
I did note that the Domain field was blank in the Tools → Manage Users → (user) → Render Job As User Settings, but putting the domain in there did not make a difference.
Good morning.
I made a change this morning, and something DID change, though I’m not sure that any progress is made.
In the Deadline Monitor application on the workstation I’m using (where I am submitting jobs), I entered my domain account’s password in the “Render Job As User Settings” section.
At this point, I submitted a job to the renderfarm, to the Windows renderfarm client, and see this in the deadlineslave log:
2024-02-06 09:40:28: 0: Got task!
2024-02-06 09:40:28: 0: Render Thread - Render State transition from = ‘ReceivedTask’ to = ‘Other’
2024-02-06 09:40:28: 0: Loading Job’s Plugin timeout is Disabled
2024-02-06 09:40:28: 0: SandboxedPlugin: Render Job As User enabled, running as user ‘spencer’, which is defined in the User Settings for this job’s user
2024-02-06 09:40:29: 0: Failed to properly create local user Deadline home folder ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10’ because: Access to the path ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10’ is denied. (System.UnauthorizedAccessException)
2024-02-06 09:40:29: 0: Failed to properly create Deadline settings folder ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10\settings’ because: Access to the path ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10\settings’ is denied. (System.UnauthorizedAccessException)
2024-02-06 09:40:29: 0: Failed to properly create Deadline cache folder ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10\cache’ because: Access to the path ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10\cache’ is denied. (System.UnauthorizedAccessException)
2024-02-06 09:40:29: 0: Failed to properly create Deadline plugin icons folder ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10\icons’ because: Access to the path ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10\icons’ is denied. (System.UnauthorizedAccessException)
2024-02-06 09:40:29: 0: Failed to properly create Deadline temp folder ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10\temp’ because: Access to the path ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10\temp’ is denied. (System.UnauthorizedAccessException)
2024-02-06 09:40:29: 0: System.UnauthorizedAccessException: Access to the path ‘C:\Users\Default\AppData\Local\Thinkbox\Deadline10’ is denied.
2024-02-06 09:40:29: 0: at System.IO.FileSystem.CreateDirectory(String fullPath, Byte[] securityDescriptor)
2024-02-06 09:40:29: 0: at System.IO.Directory.CreateDirectory(String path)
2024-02-06 09:40:29: 0: at Deadline.Configuration.DeadlineConfig.i(String dfp)
2024-02-06 09:40:29: 0: at Deadline.Configuration.DeadlineConfig.CheckConfigFile(Boolean allUsers)
2024-02-06 09:40:29: 0: at Deadline.Configuration.DeadlineConfig.GetIniFileSetting(String key, String defaultValue, Boolean errorOnNoConfig)
2024-02-06 09:40:29: 0: at Deadline.Configuration.DeadlineConfig.GetSlavePath(String defaultPath)
2024-02-06 09:40:29: 0: at Deadline.IO.DeadlineClientPath.get_DeadlineClientSlaveRoot()
2024-02-06 09:40:29: 0: at Deadline.IO.DeadlineClientPath.GetClientDirectory(DeadlineClientDir dirType, Boolean createIfMissing)
2024-02-06 09:40:29: 0: at Deadline.Configuration.DeadlineConfig.InitApplication(Boolean delayLoadPython, Boolean forceDirectConnection)
2024-02-06 09:40:29: 0: at DeadlinePython.DeadlineSandboxApp.a(CommandLineParser d)
2024-02-06 09:40:29: 0: at DeadlinePython.DeadlineSandboxAppStartup.Main(String[] args)
2024-02-06 09:40:29: ERROR: Encountered the following error while initializing the Plugin Sandbox: ‘Plugin Sandbox exited prematurely, see log for details.’.
2024-02-06 09:40:29: Cannot render the current job [ID:“65c26e83bf2d98da57ee7ab1” / Name:“figaro-painted”]. Contact Thinkbox Support for more information.
2024-02-06 09:40:29: ERROR: 0: An exception occurred: Failed to load the plugin because: Could not initialize the plugin sandbox (Deadline.Plugins.PluginException) (Deadline.Plugins.RenderPluginException)
2024-02-06 09:40:29: 0: Render Thread - Render State transition from = ‘Other’ to = ‘WaitingForTask’
2024-02-06 09:40:30: Scheduler Thread - Render Thread 0 threw a major error:
2024-02-06 09:40:30: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024-02-06 09:40:30: Exception Details
2024-02-06 09:40:30: RenderPluginException – Failed to load the plugin because: Could not initialize the plugin sandbox (Deadline.Plugins.PluginException)
2024-02-06 09:40:30: RenderPluginException.Cause: PluginLoadError (0)
2024-02-06 09:40:30: RenderPluginException.Level: Major (1)
2024-02-06 09:40:30: RenderPluginException.HasSlaveLog: True
2024-02-06 09:40:30: RenderPluginException.SlaveLogFileName: C:\ProgramData\Thinkbox\Deadline10\logs\deadlineslave_renderthread_0-dreams-0000.log
2024-02-06 09:40:30: Exception.TargetSite: Boolean e(System.String, Deadline.Jobs.Job, System.Threading.CancellationToken)
2024-02-06 09:40:30: Exception.Data: ( )
2024-02-06 09:40:30: Exception.Source: deadline
2024-02-06 09:40:30: Exception.HResult: -2146233088
2024-02-06 09:40:30: Exception.StackTrace:
2024-02-06 09:40:30: at Deadline.Slaves.SlaveRenderThread.e(String ajx, Job ajy, CancellationToken ajz)
2024-02-06 09:40:30: at Deadline.Slaves.SlaveRenderThread.b(TaskLogWriter ajt, CancellationToken aju)
2024-02-06 09:40:30: <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024-02-06 09:40:30: Scheduler Thread - Scheduler State transition from = ‘PreRendering’ to = ‘PostRendering’
2024-02-06 09:40:30: Scheduler Thread - Scheduler State transition from = ‘PostRendering’ to = ‘EndJob’
Why would Deadline be trying to create content in C:\Users\Default and not in C:\Users\spencer?
Is there ANY workaround for this, other than “…make sure your renderfarm clients are all Linux hosts?” I was trying to leverage some underused Windows hosts as renderfarm clients, but it doesn’t appear that they’re going to be usable as such.
Not sure why that is… there should be an association between user and home directory, either in the registry, or in a policy.