AWS Thinkbox Discussion Forums

VRay DBR feedback

Deprecated Deadline 6 Beta Discussion
#1

  1. Would be good if the submitted “VRay Spawner Job” job name was more like this: “VRay Spawner Job [Submitted Machine Name] [Deadline User Name] [Max/Maya Version Number]”. Need visibility of who’s submitting these spawner jobs and from where, using what version of 3dsMax or Maya. Could become chaos quickly!

  2. I like the auto-deletion of the spawner job. Nice touch. If another user suspends the job in the queue, the current DBR user isn’t made aware of this change? Notify user?

  3. “Max Servers=” UI element should be updated to read the same as the VRay DR dialog? - “Max servers (0 - all)” and function the same way?

  4. How about the other functionality “Restart slaves on render end” option in the DR dialog?

  5. You could add functionality in the “Setup VRay DBR With Deadline” UI to act the same as the normal 3dsMax built-in DR dialog. ie: Add ability to “add server”, “remove server”, “resolve IP” slaves in the Deadline UI as well as the normal 3dsMax one - they can just read/write to the file below and therefore the UI in both places would remain up to date. I jut tested this, by editing manually the “vray_dr.cfg” file and then re-opening the DR dialog in the currently running 3dsMax scene and it was updated.

See: [C:\max_root\en-US\plugcfg\vray_dr.cfg]

win7x64 1
restart_slaves 0
list_in_scene 0
max_servers 0

Taking this one stage further, provide a populated list of Deadline Slaves in the “Deadline VRay DBR” UI which can be allocated to this server list for DBR.
ie: (think “Deadline Machine List” here)

DDL: DBR Slaves Enabled:
deadline_slave01 > No
deadline_slave02 > No
deadline_slave03 > Yes

Also, you could provide filtering of the Deadline Slave machines by pool and/or group in the VRay DBR UI

  1. Think of a snappier name than “Setup VRay DBR With Deadline” as it’s a mouth full to keeping typing! - ie: “SMTD” equivalent!

  2. Can the VRay STDout from each spawner be piped into the individual Deadline Slaves for logging/reporting purposes? Awesome if possible, for tracking down an issue with one particular slave whilst doing DBR job.

  3. Sticky/defaults system, repository/server side, to allow studio wide control of various buttons/features in this UI? a.k.a: Bobo’s sticky/defaults system in SMTD.

  4. I wonder if a SMTD’type sanity checker system is also going to be required in this UI?

  5. Should I be getting quite so much monitor console logging as well? Seems excessive!!

2013-09-04 16:04:39: rendering!!
2013-09-04 16:04:44: rendering!!
2013-09-04 16:04:52: rendering!!
2013-09-04 16:04:57: rendering!!
2013-09-04 16:05:00: rendering!!
2013-09-04 16:05:06: rendering!!
2013-09-04 16:05:11: rendering!!
2013-09-04 16:05:13: rendering!!
2013-09-04 16:05:18: rendering!!
2013-09-04 16:05:24: rendering!!
2013-09-04 16:05:30: rendering!!
2013-09-04 16:05:35: rendering!!
2013-09-04 16:05:41: rendering!!
2013-09-04 16:05:46: rendering!!
2013-09-04 16:05:52: rendering!!
2013-09-04 16:05:58: rendering!!
2013-09-04 16:06:03: rendering!!
2013-09-04 16:06:09: rendering!!
2013-09-04 16:06:14: rendering!!
2013-09-04 16:06:20: rendering!!
2013-09-04 16:06:25: rendering!!
2013-09-04 16:06:31: rendering!!
2013-09-04 16:06:36: rendering!!
2013-09-04 16:06:41: rendering!!
2013-09-04 16:06:42: rendering!!
2013-09-04 16:06:48: rendering!!
2013-09-04 16:06:53: rendering!!
2013-09-04 16:06:59: rendering!!
2013-09-04 16:07:04: rendering!!
2013-09-04 16:07:10: rendering!!
2013-09-04 16:08:28: rendering!!
2013-09-04 16:08:35: rendering!!
2013-09-04 16:08:41: rendering!!
2013-09-04 16:08:46: rendering!!
2013-09-04 16:08:52: rendering!!
2013-09-04 16:08:57: rendering!!
2013-09-04 16:09:02: rendering!!
2013-09-04 16:09:08: rendering!!
2013-09-04 16:09:13: rendering!!
2013-09-04 16:09:18: rendering!!

  1. Finally, make this whole system work for VRay RT DBR as well?
#2

Thanks Mike!

  1. Yeah, I think we can give the job a better default name to include this information.

  2. Yeah, we should probably have some sort of check to see if the job has changed states or has been deleted.

  3. Do you think that’s necessary? Currently the behavior is to use all of the slaves that show up in that list. Would there ever be a case where you wouldn’t want this?

  4. Each DBR job will start up a fresh spawner instance, so I’m not sure this is necessary?

  5. Do we want to allow the addition of arbitrary servers in the Deadline UI? Like (3), the idea is you render on any slaves that show up in your list. By using Groups, you should always ensure you get the list of slaves you want. Also, we already have an option to use the slave’s IP address instead of host name. Finally, note that we are already currently modifying that file whenever the slave list in the UI is updated.

  6. SVDD? :slight_smile:

  7. Doesn’t look like it. The spawner doesn’t write anything to stdout. I guess there is the c:\vraylog.txt file though, right?

  8. Yeah, we could support this.

  9. Not sure, actually. The UI already does a bit of sanity checking (making sure VRay is the renderer, making sure DBR is enabled, etc). I’m sure we can add it if needed.

  10. This is just some debug output that made it into beta 3. It’s not specific to DBR, and it will be gone in beta 4. :slight_smile:

  11. Does VRay RT DBR work differently than normal VRay DBR? I’ve never actually played with it myself, so just curious…

#3
  1. I’m just thinking that it’s already quite a complicated workflow so to simplify/streamline everything for users, make the UI very similar to the current VRay one. By default, 0 means that all slaves are used by default. Obviously, the checkbox next to each machine name, helps to enable/disable certain machines.

That should be use ALL slaves that show up in the list which ALSO are checked ON, ignoring any that are un-checked (disabled)

Actually, I believe the recommendation from Chaos Group is to only allow a max. of 1 DBR spawner to run on 1 machine at any 1 time. Covers off any I/O, CPU, RAM limitation issues, I reckon.

  1. My vision was always let people define exact machines to use if they want (the Chaos Group way of thinking, which you have implemented), but how about another option, by default; it’s better to use the flexibility of Deadline to help you find X number of AVAILABLE NOW machines in the farm, which will deliver these machines back to the user’s workstation quicker. If just 1 of the selected slaves has just picked up another job, then it’s confusion/frustration for the user. So, I’m thinking, populate the SVDD(nice!) list if it contains anything from the VRay cfg file and then let the user fiddle, my populating machines from Deadline pools/groups. Easy for studio deadline admin to edit pools/groups configuration server side.

  2. An additional log is created in the same place as the standard VRay log. [%temp%\VRSpawner.log]. Unfortunately, there are no piping/logging commands available. In fact, the vrayspawner.exe only has 2 flags, which installs itself as a NT service OR remove the NT service! Annoying as the log file is useful. Can you think of anyway to get the stdout into the deadline slaves? Temporarily install it as a service on each deadline slave (to get at the logs?) and then remove at the end of each job? (I’ve seen issues of NT services sometimes not self un-installing when told to do so!) or maybe just try and parse the [%temp%\VRSpawner.log] file on each slave?

  3. VRay RT is quite different. It’s more like VRay Standalone as an cmd executable. Whereas, vrayspawner.exe effectively is a wrapper than executes 3dsMax in “-server” mode and passes it a “dummyMax2013.max” scene file. Which is very similar to Lightning. If you look at the properties of the “V-Ray RT render server” icon in the windows/start-menu/programs/…, you can see what parameters are being passed through:

V-Ray RT render server
C:\Windows\system32\cmd.exe /k vray.exe -server -rtEngine=1 -portNumber=20206
start-in directory: “C:\Program Files\Chaos Group\V-Ray\RT for 3ds Max 2014 for x64\bin”

So for VRay RT, we could get Deadline Slave to spawn the vray.exe, which then has a few cmd line options, which can be exposed via “Deadline Configure Plugins…”, namely, portNumber, VerboseLevel (excellent for debug) and also means that as a managed process, Deadline Slave can receive the STDout from the vray.exe into its log. :slight_smile: It does actually generate it’s own log as well: “VRRTSpawner.log”

“C:\Program Files\Chaos Group\V-Ray\RT for 3ds Max 2014 for x64\bin\vray.exe”

===============================================
V-Ray RT render server, version 2.20.02 for x64
Copyright © 2000-2010 Chaos Group Ltd. All rights reserved.
Use -credits option for additional third-party copyright notices.

Build from Jun 10 2013, 18:00:27
Compiled with Intel C++ compiler, version 12.1
Operating system is Microsoft™ Windows™, version 6.1, Service Pack 1

V-Ray core version is 2.00.01

Usage:
vray …

where option (case-sensitive) is one of the following:
([] means optional string, {} means the string can be repeated
zero or more times)

-help - print this help text and exit.

-version - print the V-Ray version and exit.

-credits - print copyright notices for V-Ray and available plugins.

-configFile - the path and file name to the vray config file
     Note that additional paths for V-Ray plugins can also be specified
     with the VRAY_PLUGINS_x64 environment variable.
     (default is vrayconfig.xml in the same folder as vray.exe).

-include="includePath{;includePath}" - specify path(s) for include files.
     More than one -include options can be specified.

-numThreads=nnn - set the number of rendering threads
     (default is 0 - automatic)

-portNumber=n - specifies the port number to use for distributed
     rendering. The port numbers of the render servers and the render
     client must match for DR to work. The port can also be overridden
     using the VRAY_DR_CONTROLPORT environment variable. This command
     line option overrides the environment variable.
     (default is 20204)

-verboseLevel=n - specifies the verbose level of information printed
     to the standard output: 0 - no information printed; 1 - only errors;
     2 - errors and warnings; 3 - errors, warnings and informational
     messages; 4 - all output
     (default is 3)

  1. Would be good to colour in GREEN the active servers in SVDD when they become READY to operate. Perhaps, colour code the spawner job ID to GREEN when ALL slaves are READY, white for idle and RED for error/not ready?

  2. Forgot to mention yesterday, could really do with a way to globally set a value in repository for MAX. number of VRay spawner jobs allowed to be ACTIVE at any one time in the queue. Otherwise, you could have artists taking down all the farm for lighting/rendering tests!

#4

3/4/5/12. Hmm, there might be a misunderstanding here on how the DBR jobs work in Deadline. The spawner job that is submitted to the farm is what starts the spawner process on the Deadline slaves. SVDD then shows the slaves that have picked up the spawner job in the server list, meaning that they are currently running the spawner process and are ready to go. So if we allowed the user to pick and choose slaves in SVDD, that would mean and any slaves not chosen will be sitting there doing nothing until the job is deleted. The idea is that any slaves in the chosen Deadline group should already be appropriate for DR rendering. If you want to pick and choose specific machines, you can use the whitelist/blacklist option in SVDD.

  1. Having the slave install it as a service probably isn’t a good idea. It would require admin privileges, and like you said, removing the service probably isn’t 100% reliable. It would probably be possible to open the log file at set intervals and grab what’s now and write that to the log.

  2. Thanks for this info. Looks like it should be pretty straightforward to support. Hopefully we can re-use the current SVDD script to support it as well.

  3. Could be part of the sticky settings and default ini files - maybe an option to set the maximum server count.

#5

3/4/5. Yep, you’re right. I’m trying to shoe-horn an unnecessary architecture into the mix here, when I should just be using what Deadline already provides! whitelist/blacklist with pools/groups will work fine :wink:

  1. I’m not finding the quality of ‘visibility’ of the slave status in SVDD brilliant. ie: Would be good to have colour-coding of the slave status? green/red?

  2. Cool. Grabbing the VRSpawner.log at set intervals is fine. Just want to capture the info for later debug if there is an issue. Thinking about this, if Deadline Slave from a ProcessExplorer point of view, is the parent of the VRaySpawner.exe, which in turn is the parent of the instance of 3dsMax running in “-server” mode, will Deadline Slave receive from 3dsMax its stdout from VRay? If not, then could we add the grabbing at set intervals the standard “vray.log” as well? This log will contain the answers to common issues such as “missing textures/path issues, etc”. Also in VRay v3.0, when “automatic asset transfer” is implemented for DBR, I can see us wanting to have visibility that internally VRay is copying textures/caches around. I assume this info will also be stored in the standard VRay.log file. Finally, RT DBR log streaming will much easier to support as previously mentioned. :slight_smile:

  3. No worries. I was hoping that VRay RT functionality could be merged into the existing SVDD (ie: no need to create a separate UI), although a separate plugin might be needed to spawn the different process?

  4. Hmm…sticky/default INI settings would help, but doesn’t stop MULTIPLE users on different machines from each submitting DBR jobs to the farm and taking over. Perhaps, expose “Limit Groups” functionality to SVDD UI to limit the number of overall slaves running?

[updated notes here as well] - viewtopic.php?f=106&t=10136

#6

  1. But what would the color indicate? In Deadline terms, those slaves will always be in the “rendering” state since they’ve picked up the spawner job and have launched the spawner. Note that they only show up in that list if they have picked up the spawner job that was submitted by SVDD.

  2. Yeah, I like the idea of using limit groups here. The more built-in Deadline features we can make use of, the better!

Thanks again for all the feedback!

#7
  1. Fair enough.

Interestingly, I did have an initial situation where the spawner was booting up an instance of Max 2014, which after booting up, crashed out and then the cycle begins again, where spawner attempts to re-launch Max 2014 again. The SVDD displays the server in its list for a second, before Max crashes out and the server entry disappears out of the list. It works fine in Max2013 and I’m pretty sure it’s something to do with Max 2014 or more importantly, Backburner and Max 2014. Haven’t quite figured it out yet. [I think it might be that AT LEAST 1 BB job must be processed on this machine before DBR will ever work…I’ll do more testing tomorrow hopefully]

#8
  1. Didn’t mention it exactly before, but of course, jobName should say which DCC app its for. ie: [3dsMax] as well as version/bit build would be useful. I know Max 2014 is only x64, but many studios are still running much older versions!
#9

Good point! Thanks!

Privacy | Site terms | Cookie preferences