D6: Slaves stalling on "customize.ms"

im_thatoneguy · June 5, 2013, 5:48pm

0: STDOUT: WRN: Obsolete plugin C:\Program Files\Autodesk\3ds Max 2011\Network\max_server32.task found. Not loaded.
0: STDOUT: DBG: Looking for Adapters in C:\Program Files\Autodesk\3ds Max 2011\Network*.adp.exe.
0: STDOUT: DBG: Looking for plugins in C:\Program Files\Autodesk\3ds Max 2012\Network*.task.
0: STDOUT: WRN: Obsolete plugin C:\Program Files\Autodesk\3ds Max 2012\Network\max_server32.task found. Not loaded.
0: STDOUT: DBG: Looking for Adapters in C:\Program Files\Autodesk\3ds Max 2012\Network*.adp.exe.
0: STDOUT: DBG: Looking for plugins in C:\Program Files\Autodesk\3ds Max 2013\Network*.task.
0: INFO: Connected to 3dsmax plugin version Lightning 6.0.0.51561 May 31 2013 09:19:22 R
0: INFO: Scene file to render: “C:\Users\renderadmin\AppData\Local\Thinkbox\Deadline6\slave\RENDER-I7-04\jobsData\WK_CokeH_SlurpeeSucker_A02.01.max”
0: INFO: Camera: “Camera002”
0: INFO: Restarting renderer after each frame: 1
0: INFO: Showing frame buffer: 1
0: INFO: Overriding render output: “W:\13016 W+K Coke H\Shots\COKE\SlurpeeSucker\Renders\ANIM\WK_CokeH_SlurpeeSucker_A02.01_tile_4x4_4x4_.exr”
0: INFO: Overriding save file option: True
0: INFO: Frame number base: 0
0: INFO: Remove padding from output filename: 0
0: INFO: Ignore missing external file errors: 1
0: INFO: Ignore missing UVW errors: 1
0: INFO: Ignore missing XREF errors: 1
0: INFO: Ignore missing DLL errors: 0
0: INFO: Disabling Multipass: 0
0: INFO: The following elements will not be saved
0: INFO: MultiMatteElement
0: INFO: VRayDiffuseFilter
0: INFO: VRayGlobalIllumination
0: INFO: VRayLighting
0: INFO: VRayReflection
0: INFO: VRayRefraction
0: INFO: VRaySpecular
0: INFO: VRaySSS2
0: INFO: Region rendering enabled: left = 1686, top = 1686, right = 2250, bottom = 2250, padding = 0, type = CROP
0: INFO: Post frame script: “C:\Users\renderadmin\AppData\Local\Thinkbox\Deadline6\slave\RENDER-I7-04\jobsData\maxTileAssemblerCommand.ms”
0: INFO: Loading 3dsmax scene file
0: INFO: Scene loaded successfully
0: INFO: Executing script: C:\Users\renderadmin\AppData\Local\Thinkbox\Deadline6\slave\RENDER-I7-04\plugins\customize.ms

Still getting this error and stalling out slaves in infinite frozen loops.

rrussell · June 5, 2013, 8:02pm

We still don’t know what could be causing this. Just to refresh our memory, what do you have to do to “fix” this? Does restarting the slave application work, or do you have to restart the machine?

im_thatoneguy · June 6, 2013, 5:06pm

I don’t remember either. I usually just blacklist that slave from that job.

rrussell · June 6, 2013, 5:41pm

If you enable bad slave detection in Repository Options -> Job Settings -> Failure Detection, then at least your slaves can move on after X consecutive errors, instead of you having to blacklist them manually.

im_thatoneguy · June 6, 2013, 7:14pm

If I remember correctly and this seems to jive with what I’m seeing in practice. You have an auto-fail after like 10 minutes hardcoded in for this error. I don’t think it’s actually marking it as an “error” though that counts towards the failure detection. I do already have failure detection enabled.

rrussell · June 6, 2013, 8:09pm

The error for this timeout is just like any other error, so it should “count” like any other error. If you have the bad slave error limit in Failure Detection set to 5, then if the slave timeouts on customize.ms 5 times in a row, it should move on. I guess this would take 50 minutes to determine though. Maybe 10 minutes is still a bit high for the timeout for this script…