Inconsistent errors in StartJob with lightning 3dsmax Plugin

Deadline
1

We’re seeing some inconsistent errors in deadline with slaves getting an error “Error in StartJob: ExecuteScript: Timed out waiting for the lightning 3dsmax plugin to acknowledge the ExecuteScript command.” Based on some research, it looks like the only time the Lightning plugin is active is during the ‘Restart render between frames’ that we’ve enabled to prevent vray issues. This error does not occur consistently and with some nodes getting this error on a few models and other nodes getting the error on different models. Does anyone have any insight on what might be causing these errors or settings to investigate to try to eliminate the errors? Any help is appreciated
Deadline Version: 6.1
OS: Windows 7 and 8 (depending on the node)
Program: 3ds Max 2012
Render: V-Ray 2.4
submission method: deadlinecommand (from in-house software)
Example Log:

Full Log

0: Loaded plugin: 3dsmax
0: Task timeout is disabled.
0: Loaded job: nacre beauty 9565 - Metal _Christmas_Tree_Stand_v1_BeautyPass.max (540e23df4b943b0a3a84aa8c)
0: INFO: Executing plugin script C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\slave\KEVIN-3D\plugins\540e23df4b943b0a3a84aa8c\3dsmax.py
0: INFO: About: 3dsmax Plugin for Deadline
0: INFO: The current environment will be used for rendering
0: INFO: Start Job called - starting up 3dsmax plugin
0: INFO: Sys Env Var PATH: C:\Program Files\Thinkbox\Deadline\bin;C:\Program Files (x86)\AMD APP\bin\x86_64;C:\Program Files (x86)\AMD APP\bin\x86;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0;C:\Program Files (x86)\Windows Live\Shared;C:\Program Files (x86)\Common Files\Adobe\AGL;C:\Program Files\Common Files\Autodesk Shared;C:\Program Files (x86)\QuickTime\QTSystem;C:\Program Files (x86)\ATI Technologies\ATI.ACE\Core-Static;C:\Program Files (x86)\Autodesk\Backburner;C:\Program Files (x86)\Google\Google Apps Migration
0: INFO: Sys Env Var PATH length: 555
0: INFO: Backburner Path(s) Found in PATH: ‘C:\Program Files (x86)\Autodesk\Backburner’
0: INFO: Backburner server.exe version: 2014.0.0.1635
0: INFO: Rendering with 3dsmax version: 2012
0: INFO: Build of 3dsmax to force: 64bit
0: INFO: Rendering with executable: C:\Program Files\Autodesk\3ds Max 2012\3dsmax.exe
0: INFO: Slave 3dsmax.exe version: 14.12.508.0
0: INFO: Slave 3dsmax.exe description: 3dsMax 2012 + productupdate_pu12/advantage_pack_extension
0: INFO: Checking registry for 3dsmax language code
0: INFO: Found language code: 409
0: INFO: Language code string: enu
0: INFO: Fail on existing 3dsmax process: 0
0: INFO: Load 3dsmax timeout: 1000 seconds
0: INFO: Start job timeout: 1000 seconds
0: INFO: Progress update timeout: 8000 seconds
0: INFO: Progress update timeout disabled: 0
0: INFO: Kill ADSK WSCommCntr*.exe process: False
0: INFO: Slave mode enabled: 1
0: INFO: Silent mode enabled: 0
0: INFO: Local rendering enabled: 1
0: INFO: Running render sanity check using 3dsmaxcmd.exe
0: INFO: 3dsmaxcmd.exe returned: P9/10/2014 21:04:07 PM; Error opening scene file: “C:\Program Files\Autodesk\3ds Max 2012”
0: INFO: Render sanity check using 3dsmaxcmd.exe completed successfully. Please ignore the ‘Error opening…’ message thrown by 3dsmaxcmd.exe above.
0: INFO: 3dsmax start up file: C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\slave\KEVIN-3D\plugins\540e23df4b943b0a3a84aa8c\deadlineStartupMax2012.max
0: INFO: Using user profiles: 1
0: INFO: 3dsmax user profile path: C:\Users\kalimia\AppData\Local\Autodesk\3dsmax\2012 - 64bit\enu
0: INFO: 3dsmax data path: C:\Users\kalimia\AppData\Local\Autodesk\3dsMax\2012 - 64bit\enu
0: INFO: 3dsmax ini file: C:\Users\kalimia\AppData\Local\Autodesk\3dsmax\2012 - 64bit\enu\3dsmax.ini
0: INFO: Network log file: C:\Users\kalimia\AppData\Local\Autodesk\3dsmax\2012 - 64bit\enu\Network\Max.log
0: INFO: Plugin ini file: C:\Program Files\Autodesk\3ds Max 2012\plugin.ini
0: INFO: Including user profile plugin ini: C:\Users\kalimia\AppData\Local\Autodesk\3dsMax\2012 - 64bit\enu\Plugin.UserSettings.ini
0: INFO: Lightning connection plugin: C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\slave\KEVIN-3D\plugins\540e23df4b943b0a3a84aa8c\lightning64Max2012.dlx
0: INFO: 3dsmax socket connection port: 53860
0: INFO: Setting up startup environment
0: INFO: Copying C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\slave\KEVIN-3D\plugins\540e23df4b943b0a3a84aa8c\lightning64Max2012.dlx to C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\temp\lightning\lightning.dlx
0: INFO: Copying dl.ini to 3dsmax data path: C:\Users\kalimia\AppData\Local\Autodesk\3dsMax\2012 - 64bit\enu
0: INFO: If this fails, make sure that the necessary permissions are set on this folder to allow for this copy to take place
0: INFO: Starting monitored managed process 3dsmaxProcess
0: INFO: Stdout Handling Enabled: True
0: INFO: Popup Handling Enabled: True
0: INFO: Using Process Tree: True
0: INFO: Hiding DOS Window: False
0: INFO: Creating New Console: False
0: INFO: Executable: “C:\Program Files\Autodesk\3ds Max 2012\3dsmax.exe”
0: INFO: Argument: -p “dl.ini” -q -s “C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\slave\KEVIN-3D\plugins\540e23df4b943b0a3a84aa8c\deadlineStartupMax2012.max”
0: INFO: Startup Directory: “C:\Program Files\Autodesk\3ds Max 2012”
0: INFO: Process Priority: BelowNormal
0: INFO: Process Affinity: default
0: INFO: Process is now running
0: INFO: Waiting for connection from 3dsmax
0: STDOUT: Dark
0: STDOUT: Qt: Untested Windows version 6.2 detected!
0: INFO: Connected to 3dsmax plugin version Lightning 6.1.0.54665 Jan 27 2014 08:15:32 R
0: INFO: Scene file to render: “\10.10.0.25\edrive\freshwater\production\Files\Prod\E\ERTUHB\Metal _Christmas_Tree_Stand_v2_540a1f26e4077\Metal _Christmas_Tree_Stand_v1_BeautyPass.max”
0: INFO: Camera: no camera specified, rendering active viewport
0: INFO: Restarting renderer after each frame: 1
0: INFO: Showing frame buffer: 0
0: INFO: Overriding render output: “\10.10.0.25\edrive\freshwater\production\Files\Prod\E\ERTUHB\Metal _Christmas_Tree_Stand_v2_540a1f26e4077\psd\hero…png”
0: INFO: Frame number base: 0
0: INFO: Remove padding from output filename: 0
0: INFO: Ignore missing external file errors: 1
0: INFO: Ignore missing UVW errors: 1
0: INFO: Ignore missing XREF errors: 1
0: INFO: Ignore missing DLL errors: 0
0: INFO: Disabling Multipass: 0
0: INFO: Loading 3dsmax scene file
0: INFO: Scene loaded successfully
0: INFO: Executing script: C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\slave\KEVIN-3D\plugins\540e23df4b943b0a3a84aa8c\customize.ms
0: An exception occurred: Error in StartJob: ExecuteScript: Timed out waiting for the lightning 3dsmax plugin to acknowledge the ExecuteScript command.
2014/09/10 21:04:19 DBG: Starting network
2014/09/10 21:04:19 INF: [V-Ray] ============================================================
2014/09/10 21:04:19 INF: [V-Ray] Console created, V-Ray Adv 2.40.04 for x64 from Jun 8 2013, 01:16:36
2014/09/10 21:04:19 INF: [V-Ray] ============================================================
2014/09/10 21:04:19 INF: [V-Ray] Compiled with Intel C++ compiler, version 12.1
2014/09/10 21:04:19 INF: [V-Ray] Host is 3dsmax, version 14
2014/09/10 21:04:19 INF: [V-Ray] V-Ray DLL version is 2.00.01
2014/09/10 21:04:21 INF: Loaded C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\slave\KEVIN-3D\plugins\540e23df4b943b0a3a84aa8c\deadlineStartupMax2012.max
2014/09/10 21:04:21 INF: SYSTEM: Production renderer is changed to V-Ray Adv 2.40.04. Previous messages are cleared.
2014/09/10 21:04:21 WRN: Gamma Correction Settings are Being Changed
2014/09/10 21:04:21 INF: Job: C:\Users\kalimia\AppData\Local\Thinkbox\Deadline6\slave\KEVIN-3D\plugins\540e23df4b943b0a3a84aa8c\deadlineStartupMax2012.max
2014/09/10 21:04:21 INF: SYSTEM: Production renderer is changed to Default Scanline Renderer. Previous messages are cleared.
2014/09/10 21:04:21 INF: Loaded //10.10.0.25/edrive/freshwater/production/Files/Prod/E/ERTUHB/Metal _Christmas_Tree_Stand_v2_540a1f26e4077/Metal _Christmas_Tree_Stand_v1_BeautyPass.max
2014/09/10 21:04:21 WRN: Gamma Correction Settings are Being Changed
2014/09/10 21:04:21 INF: Job: //10.10.0.25/edrive/freshwater/production/Files/Prod/E/ERTUHB/Metal _Christmas_Tree_Stand_v2_540a1f26e4077/Metal _Christmas_Tree_Stand_v1_BeautyPass.max
2014/09/10 21:04:21 INF: SYSTEM: Production renderer is changed to V-Ray Adv 2.40.04. Previous messages are cleared.
at Deadline.Plugins.ScriptPlugin.StartJob(Job job, String& outMessage, AbortLevel& abortLevel) (Deadline.Plugins.RenderPluginException)

2

Hi,
Could you post the contents of a few files, I’d like to get an insight into this issue. I think I know how to fix it, but just want to get some evidence first. On the machine you provided the log previously from, can you get the following files, which should all be located somewhere under:

C:\Users\kalimia\AppData\Local\Autodesk\3dsMax\2012 - 64bit\enu\

Max.log
InfoCenter.log
infocenter.xml

If they are not present under the directory path given above, they might be located in your 3dsMax root directory:

C:\Program Files\Autodesk\3ds Max 2012\

Could you also confirm if the Windows Firewall client is running on this machine and if your running Deadline/3dsMax as a service?

Thanks,
Mike

3

Hello Mike,

I’ve attached the files you requested for the node in question. I can also confirm that windows firewall client is running and deadlines is running as a service. That’s not the case for our entire farm, but for this node and many others. Please let me know what you think.

-Mark
log_files.rar (11.8 KB)

4

Hi Mark,

OK, so the issue I was fishing for, turns out not to be the case in this circumstance anyway. I would look at disabling the firewall and taking the machine out of service mode and see if this resolves the issue. Are you running the ADSK Max 2012 Subscription Advantage Pack on your render nodes and are you using “Scene Sets”? If so, there was an issue and was fixed, then broken, then fixed again, in various versions since Max 2012. Essentially, if you remove all Scene Set information from your scene (even if your not using it), by using the UI in 3dsMax (it has some buttons which allow you to ‘reset’ & ‘remove’ all scene set information from the scene) and then try re-submitting the job to a machine which is NOT running in service-mode, then hopefully everything should be fine. As a test, you could also run a “Scene Set” based Max file through your farm and you should see that it renders fine on Deadline Slaves which are running in GUI mode, but fail on service running Deadline Slaves.

You could also be having secondary issues with the firewall being enabled and various exceptions such as 3dsMax not being allowed through, which can cause issues. In this case, I would try disabling the firewall as well during your tests. Here is our firewall/UAC/AV configuration for Deadline, which I would recommend checking everything is happy in your pipeline:
thinkboxsoftware.com/deadlin … worksetup/
thinkboxsoftware.com/deadlin … iderations

Hopefully, one of these things should help you narrow down the issue.

Let us know how it goes,

Mike

5

Mike,

 When you are referring to "Scene Sets", do you happen to mean either "Scene states" or "state sets". My tech artists artists weren't familiar with any scene sets. 

 Also because its a small world, Ryan Romans from Team Detroit says hello

-Mark

6

Hi Mark,
Oh man, why does Autodesk have to name things so similar to one another! I meant “State Sets”, introduced in Max2012 SAP and onwards.
Ooh, I guess Ryan has moved on. Greetings Ryan!
Mike