Hello
I cannot get DR slaves connected to a job with “workstation mode” option being turned off.
Configuration is like this :
Deadline 7.2.1.10 R
VRay 3.3
3dsmax 2014
I have one server which has sticky list of slaves , i am sending job to this server - it takes Distributed render slave list from VRay_DR.cfg file and renders.
With workstation mode being turned on this works fine.
Our scheme for now is like this :
we have 2 “main” servers which have VRay_DR.cfg sticky readonly files with slaves in it. Jobs are being sent to these main servers only , distributed after that to DR slaves.
We want to find a way to render without 3dsmax license being used for these two “main” servers.
I know about 3dsmax CMD submitter , though it lacks some of the features we are occasionally using , such as “merge xrefs” , “tile rendering” and couple more. So i thought to give a spin for “workstation mode” being off and came to this problem that none of the DR slaves are being connected. I am not really sure this is something related to VRay or Deadline , so thought to try to post here first. These are couple of lines from VRayLog :
[2016/Jan/18|10:50:43] Server 10.3.222.107:20204: Scene loaded; starting render
[2016/Jan/18|10:50:46] TM_SERVER_STARTED received from 10.3.222.101:20204
[2016/Jan/18|10:50:46] Render host 10.3.222.101:20204 added
[2016/Jan/18|10:50:46] Received TM_GET_CAPABILITIES from 10.3.222.101:20204
[2016/Jan/18|10:50:46] Scene transferred to pr101:20204
[2016/Jan/18|10:50:47] Server 10.3.222.101:20204: Scene loaded; starting render
[2016/Jan/18|10:50:47] warning: Render host pr104 (10.3.222.104:20204) is not responding
[2016/Jan/18|10:50:51] warning: Render host pr102 (10.3.222.102:20204) is not responding
And there are no errors in slave log.
Unfortunately, this is a limitation of VRay running with our Lightning plugin during off-load VRay DBR, which we are unable to work-around with the “3dsmax” plugin. This is why in Deadline 8.0 we have introduced a “Use 3dsCmd Plugin” checkbox option to SMTD, which allows you to submit via SMTD functionality but adding a 3dsCmd job to the queue, which won’t require a 3dsMax license for taskId:0. Note, in all use cases of VRay DBR, “tile rendering” isn’t supported.
Note, in all cases, the “vray_dr.cfg” file must have open both READ & WRITE access. If not, then the 3dsmax or 3dsCmd plugin whilst running on the slave rendering taskId:0, won’t be able to update the slave list. if you need to control which slaves pickup a DBR job, then you could use pools, groups, limits, white/black listing, etc.
Thing is that i am not sending job as DBR or DBR Offload. I am sending it as usual job on a server , which has VRay_DR.cfg sticky. Or that does not change anything ?
We are not using DBR or Offload because now amount of slaves is exact limit , if there are less slaves available than indicated in the submission - render will not start until that exact amount of slaves will be free.
Ah, ok, understood. So, slightly different situation, but my previous answer still stands. For a reason that we don’t fully understand (VRay when initiated with 3dsMax in Slave/Silent Mode) will require the pulling of a VRay GUI license for DBR rendering, which is only achievable if 3dsMax is running in GUI (workstation) mode, hence why it works for you when you force workstation mode on. This workflow is possible via Backburner, which in Thinkbox world, is essentially the basic “3dsCmd” plugin we provide (a few options are recently added to this plugin in 7.2). So if you use the in-app 3dsCmd submission UI, then you will have the equivalent BB workflow. However, I understand that SMTD has lots of advanced functionality, such as merge xref. The reason the “3dsCmd” plugin works, is it doesn’t use our Lightning c++ plugin, which in turn, loads 3dsMax & VRay, but rather relies on the basic CLI (command-line) 3dsmaxcmd.exe executable, which is known for being a bit odd/unreliable at times. However, if it’s working, then it should be stable for you.
So options for this exact workflow would be:
- Use 3dsCmd in-app submitter, but no merge xref. However, instead of submitting 3dsMax scene file, just reference it off your network file storage, thereby side-stepping any xref issues, as the scene file will open fine.
or
- Use Deadline 8.0 Beta + new “Use 3dsCmd Plugin” option in SMTD and allow write access to your vray_dr.cfg file and control slave job pickup via various Deadline controls, I mentioned previously.
Ah now i got that , thanks a lot for explaining.
For now we cannot jump on Deadline 8.0 because of steady project flow, though we will jump on it as soon as it will be released !
I will try to use 3dscmd , because of one nasty bug in Transfer Missing Assets and mapped drives being used in our office i was forced to make Merging Xrefs as a default.
Though with 3.3 out looks like they fixed that and i will give it a spin.
Thanks, Mike.
Tried 3dsmax cmd submitter , the same scene , settings and etc. , got this error , log attached.
Stopped at VRay reporting “transforming vertices”.
=======================================================
Error
=======================================================
Error: Renderer returned non-zero error code, 3. Check the log for more information.
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)
=======================================================
Type
=======================================================
RenderPluginException
=======================================================
Stack Trace
=======================================================
at Deadline.Plugins.Plugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.a(TaskLogWriter A_0)
=======================================================
Log
=======================================================
2016-01-18 14:01:26: BEGIN - PR112\Pikcells
2016-01-18 14:01:26: 0: Start Job timeout is disabled.
2016-01-18 14:01:26: 0: Task timeout is disabled.
2016-01-18 14:01:26: 0: Loaded job: S18_BU_SFS_Establishing_[F01]_3.3_3dscmd (569ce0737c0ec765dca2c787)
2016-01-18 14:01:26: 0: Skipping drive mapping because they have already been mapped for this job
2016-01-18 14:01:26: 0: INFO: Executing plugin script C:\Users\Pikcells\AppData\Local\Thinkbox\Deadline7\slave\pr112\plugins\569ce0737c0ec765dca2c787\3dsCmd.py
2016-01-18 14:01:26: 0: INFO: Sys Env Var PATH: C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files\Common Files\Autodesk Shared\;C:\Program Files (x86)\Autodesk\Backburner\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT
2016-01-18 14:01:26: 0: INFO: Sys Env Var PATH length: 583
2016-01-18 14:01:26: 0: INFO: Backburner Path(s) Found in PATH: 'C:\Program Files (x86)\Autodesk\Backburner\'
2016-01-18 14:01:26: 0: INFO: Backburner server.exe version: 2014.0.0.1635
2016-01-18 14:01:26: 0: INFO: About: 3dsCmd Plugin for Deadline
2016-01-18 14:01:26: 0: INFO: The job's environment will be merged with the current environment before rendering
2016-01-18 14:01:26: 0: Plugin rendering frame(s): 0
2016-01-18 14:01:26: 0: INFO: Stdout Redirection Enabled: True
2016-01-18 14:01:26: 0: INFO: Stdout Handling Enabled: True
2016-01-18 14:01:26: 0: INFO: Popup Handling Enabled: True
2016-01-18 14:01:26: 0: INFO: QT Popup Handling Enabled: False
2016-01-18 14:01:26: 0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2016-01-18 14:01:26: 0: INFO: Using Process Tree: True
2016-01-18 14:01:26: 0: INFO: Hiding DOS Window: True
2016-01-18 14:01:26: 0: INFO: Creating New Console: False
2016-01-18 14:01:26: 0: INFO: Running as user: Pikcells
2016-01-18 14:01:26: 0: INFO: Not enforcing a build of 3dsmax because version 2014 and later is 64 bit only
2016-01-18 14:01:26: 0: INFO: Slave 3dsmaxcmd.exe version: 16.5.277.0
2016-01-18 14:01:26: 0: INFO: Slave 3dsmaxcmd.exe description: 3ds Max 2014 + servicepack_sp5
2016-01-18 14:01:26: 0: INFO: Slave 3dsmax.exe version: 16.5.277.0
2016-01-18 14:01:26: 0: INFO: Slave 3dsmax.exe description: 3ds Max 2014 + servicepack_sp5
2016-01-18 14:01:26: 0: INFO: Submitted from 3dsmax.exe version: 16.5.277.0
2016-01-18 14:01:26: 0: INFO: Submitted from 3dsmax.exe description: 3ds Max 2014 + servicepack_sp5
2016-01-18 14:01:26: 0: INFO: Executable: "C:\Program Files\Autodesk\3ds Max 2014\3dsmaxcmd.exe"
2016-01-18 14:01:26: 0: INFO: Rendering to local drive, will copy files and folders to final location after render is complete
2016-01-18 14:01:26: 0: INFO: Argument: "C:\Users\Pikcells\AppData\Local\Thinkbox\Deadline7\slave\pr112\jobsData\569ce0737c0ec765dca2c787\S18_BU_SFS_Establishing_[F01]_3.3_NO.max" -v:4 -start:0 -end:0 -cam:"CAM Default" -w:5000 -h:3333 -pixelAspect:1.0 -atmospherics:true -renderHidden:false -effects:true -useAreaLights:false -displacements:true -force2Sided:false -videoColorCheck:false -superBlack:false -renderFields:false -fieldOrder:Odd -skipRenderedFrames:false -outputName:"C:\Users\Pikcells\AppData\Local\Thinkbox\Deadline7\slave\pr112\jobsData\569ce0737c0ec765dca2c787\3dsOutput_tempoOL5g0\S18_BU_SFS_Establishing.exr" -preRenderScript:"C:\Users\Pikcells\AppData\Local\Thinkbox\Deadline7\slave\pr112\jobsData\569ce0737c0ec765dca2c787\UNC_Converter_2.ms" -renderElements:true -useAdvLight:true -computeAdvLight:false -continueOnError -rfw:true -videopostJob:false
2016-01-18 14:01:26: 0: INFO: Startup Directory: "C:\Program Files\Autodesk\3ds Max 2014"
2016-01-18 14:01:26: 0: INFO: Process Priority: BelowNormal
2016-01-18 14:01:26: 0: INFO: Process Affinity: default
2016-01-18 14:01:26: 0: INFO: Process is now running
2016-01-18 14:01:26: 0: STDOUT: 18/01/2016 14:01:26; 1 frames initialized
2016-01-18 14:01:26: 0: STDOUT: 18/01/2016 14:01:26;
2016-01-18 14:01:26: 0: STDOUT: Max install location: C:\Program Files\Autodesk\3ds Max 2014\
2016-01-18 14:01:26: 0: STDOUT: 18/01/2016 14:01:26; Max file being rendered: C:\Users\Pikcells\AppData\Local\Thinkbox\Deadline7\slave\pr112\jobsData\569ce0737c0ec765dca2c787\S18_BU_SFS_Establishing_[F01]_3.3_NO.max
2016-01-18 14:01:26: 0: STDOUT: 18/01/2016 14:01:26; Renderer: V-Ray Adv 3.30.03
2016-01-18 14:01:38: 0: INFO: Ignoring popup "MAXScript Debugger"
2016-01-18 14:02:30: 0: STDOUT: 18/01/2016 14:02:30; Max is ready
2016-01-18 14:02:31: 0: STDOUT: 18/01/2016 14:02:31; Frame 0 assigned
2016-01-18 14:02:31: 0: STDOUT: 18/01/2016 14:02:31; [V-Ray] ============================================================
2016-01-18 14:02:31: 0: STDOUT: 18/01/2016 14:02:31; [V-Ray] Console created, V-Ray A for x64 from Dec 18 2015, 17:03:07
2016-01-18 14:02:31: 0: STDOUT: 18/01/2016 14:02:31; [V-Ray] ============================================================
2016-01-18 14:02:31: 0: STDOUT: 18/01/2016 14:02:31; [V-Ray] Compiled with Intel C++ compiler, version 14
2016-01-18 14:02:31: 0: STDOUT: 18/01/2016 14:02:31; [V-Ray] Host is 3ds Max, version 16
2016-01-18 14:02:31: 0: STDOUT: 18/01/2016 14:02:31; [V-Ray] V-Ray DLL version is 3.25.01
2016-01-18 14:06:23: 0: STDOUT: 18/01/2016 14:06:23; [V-Ray] UNHANDLED EXCEPTION: Updating material ()
2016-01-18 14:06:23: 0: STDOUT: 18/01/2016 14:06:23; An exception occurred while rendering with 3ds Max.
2016-01-18 14:06:23: 0: STDOUT: 18/01/2016 14:06:23; Canceled by User
2016-01-18 14:06:23: 0: STDOUT: 18/01/2016 14:06:23; Render Error (Render Frame)
2016-01-18 14:06:23: 0: STDOUT: 18/01/2016 14:06:23; Error rendering frame 0: Render Error (Render Frame)
2016-01-18 14:06:24: 0: STDOUT: 18/01/2016 14:06:24; Job Completed with Error(s) - see above
2016-01-18 14:06:34: 0: STDOUT: 18/01/2016 14:06:34; Error occured while rendering job.
2016-01-18 14:06:35: 0: INFO: Process exit code: 3
=======================================================
Details
=======================================================
Date: 01/18/2016 14:06:36
Frames: 0
Elapsed Time: 00:00:05:10
Job Submit Date: 01/18/2016 12:54:12
Job User: oleg
Average RAM Usage: 7752331264 (23%)
Peak RAM Usage: 11860062208 (35%)
Average CPU Usage: 15%
Peak CPU Usage: 33%
Used CPU Clocks: 2112021
Total CPU Clocks: 14080140
=======================================================
Slave Information
=======================================================
Slave Name: PR112
Version: v7.2.1.10 R (b8ff445b3)
Operating System: Windows 7 Professional (SP1)
Running As Service: No
Machine User: Pikcells
IP Address: 10.3.222.112
MAC Address: BC:EE:7B:98:E8:B5
CPU Architecture: x64
CPUs: 12
CPU Usage: 5%
Memory Usage: 11.0 GB / 31.9 GB (34%)
Free Disk Space: 58.658 GB
Video Card: NVIDIA GeForce 210
Strange. This very much looks to be a VRay renderer specific error:
Does it render locally on this machine ok?
We can test to see if this error occurs outside of Deadline. Grab the max scene file & maxscript file in question and copy it to say: “c:\test\S18_BU_SFS_Establishing_[F01]_3.3_NO.max” & “c:\test\UNC_Converter_2.ms”
Open up a command prompt on this machine and type in the following line and execute it:
"C:\Program Files\Autodesk\3ds Max 2014\3dsmaxcmd.exe" "c:\test\S18_BU_SFS_Establishing_[F01]_3.3_NO.max" -v:4 -start:0 -end:0 -cam:"CAM Default" -w:5000 -h:3333 -pixelAspect:1.0 -atmospherics:true -renderHidden:false -effects:true -useAreaLights:false -displacements:true -force2Sided:false -videoColorCheck:false -superBlack:false -renderFields:false -fieldOrder:Odd -skipRenderedFrames:false -outputName:"c:\test\S18_BU_SFS_Establishing.exr" -preRenderScript:"c:\test\UNC_Converter_2.ms" -renderElements:true -useAdvLight:true -computeAdvLight:false -continueOnError -rfw:true -videopostJob:false
If it still fails, then we know it’s an issue outside of Deadline.
Yeah that renders fine locally and with “workstation mode” turned on.
Will try to use that command , thanks.
Update : yep , exactly the same error. That is really weird, because it renders absolutely fine every way except 3dsmax cmd ! Looks like i will need to address Chaos Group this issue.
Thanks for help !
Mike , can you please explain this :
“1. Use 3dsCmd in-app submitter, but no merge xref. However, instead of submitting 3dsMax scene file, just reference it off your network file storage, thereby side-stepping any xref issues, as the scene file will open fine.”
What do you mean by " just reference it off your network file storage"
We are still on 7.2 and cannot switch to 8.0 beta because of constant production.
Thanks.
I’m referring to you saving your Max scene file onto a network file server and this file also references any external files including XREF objects/XREF scenes which are also saved to a network file server.
Ah… I thought there is a setting or something i’m missing. Yeah we have everything on network paths by default.
Small update for anyone having the same issue. 3DSmax cmd line is working incorrectly with any asset paths which length is bigger than 256 symbols. No solution yet from Autodesk support…
Hello from 1992 and “progra~1”