Vray lightcache prerender?

Discussion and Support of Deadline, the Render Management System
squeakybadger
Posts: 80
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Wed Apr 26, 2017 11:32 am

Hi Bobo,

Log attached and listener output.

Code: Select all

Resolving Network Path: 1210ms
SMTD_AutoLoadSuccessful = true
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\RegionManipulator.ms"
Loading RegionManipulator.ms: 10ms
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\TileManipulator.ms"
Loading TileManipulator.ms: 6ms
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\SubmitMaxToDeadline_Functions.ms"
Loading SubmitMaxToDeadline_Functions.ms: 4159ms
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\SubmitMaxToDeadline_SanityCheck_Private.ms"
Loading SubmitMaxToDeadline_SanityCheck_Private.ms: 52ms
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\SubmitMaxToDeadline_SanityCheck_General.ms"
Loading SubmitMaxToDeadline_SanityCheck_General.ms: 6ms
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\SubmitMaxToDeadline_SanityCheck.ms"
Loading SubmitMaxToDeadline_SanityCheck.ms: 6ms
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\SubmitMaxToDeadline.ms"
Loading SubmitMaxToDeadline.ms: 8ms
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\SubmitMaxToDeadline_Defaults.ini"
Loading SubmitMaxToDeadline_Defaults.ini: 8ms
Running submission script "\\piknas003\DeadlineRepository9\submission/3dsmax/Main\SubmitMaxToDeadline_StickySettings.ini"
Loading SubmitMaxToDeadline_StickySettings.ini: 7ms
Total SMTD Launcher Time: 11746ms
--DEADLINE SUBMISSION CANCELED BY THE USER.


-- Return: not inside a function
Attachments
SubmitMaxToDeadline - [WKS18] - 26-04-2017-0014.log
(14.49 KiB) Downloaded 7 times

User avatar
Bobo
Posts: 5886
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Wed Apr 26, 2017 11:46 am

Oh, right, the latest version of SMTD has some code in it to allow canceling the job via the submit button. Originally, a variable SMTDIsCurrentlySubmitting was initialized to false, and set to true when the checkbutton is pressed. In your version, it remains false, and causes the WaitForCommandToComplete function to bail out early. So in your version the first job might submit, but the rest of the SMTD functions would never know.

Recently I changed this variable to start as undefined because it was causing exactly this premature canceling of the deadlineCommand wait in other scripts, including all non-Deadline integrations like Krakatoa, XMesh, Stoke etc. That fix might be in 9.0.2.0, but you don't have it yet. The fix makes the variable basically have 3 states - undefined, true, and false.
- undefined means "outside of submission process",
- true means "submission started and not canceled", and
- false means "user just canceled by unchecking the UI SUBMIT checkbutton"...

Here is a fix where we set the variable to true before submission, and to undefined when it is done...

Code: Select all

(--start local scope
   
--If SMTD has not been opened yet, or has been closed since the start of 3ds Max, launch its UI:
if SMTD_MainRollout == undefined or not SMTD_MainRollout.open do macros.run "Deadline" "SubmitMaxToDeadline"

local theRenderer = renderers.current               --get the current renderer
if isKindOf theRenderer VRay do                     --if it is VRay, we are good to go
(
   --Render final image is disabled, light cache is set to render in the output dir
   local oldRenderDialogState = renderSceneDialog.isopen()      --see if the render dialog was open
   renderSceneDialog.close()                        --close it before submission
   rendSaveFile = false                              --disable output file saving
   theRenderer.options_dontRenderImage = true         --disable the rendering
   local lcpath = getFileNamePath rendOutputFilename + getFileNameFile rendOutputFilename + ".vrlmap"   --build the LC path name
   theRenderer.lightcache_autoSave = true               --enable LC auto-save
   theRenderer.lightcache_autoSaveFileName = lcpath         --and set the file name to the cache path
   
   --Job sends to deadline
   ::SMTDIsCurrentlySubmitting = true
   SMTDSettings.PreviousJobsArray = #()                --clear the dependencies list
   SMTDSettings.SubmitAsDependent = false                --disable dependencies for first job submission

   SMTD_MainRollout.getNameFromMaxScene()            --update the job name in the UI
   local oldJobName = SMTDSettings.JobName             --store the current job name
   SMTDSettings.JobName = oldJobName+" [VRay LC]"         --set the LC job's name
   local batchName = oldJobName + " [VRay LC + Render]"      --a batch name to combine the two jobs
   local result = SMTDFunctions.SubmitJobFromUI batchName:batchName      --submit the job to Deadline as part of the batch
   format "%\n" result

   if SMTDSettings.PreviousJobsArray.count > 0 do
   (
   --Render final image is enabled, lc is switched to "from file" and uses the precalced light cache
   rendSaveFile = true                           --re-enable output file saving
   theRenderer.options_dontRenderImage = false            --re-enable rendering
   theRenderer.lightcache_autoSave = false               --disable the auto save mode
   local old_lightcache_mode = theRenderer.lightcache_mode   --store the old mode
   theRenderer.lightcache_mode = 2                  --set LC mode to From File
   theRenderer.lightcache_loadFileName = lcpath            --set the load file to the cached path from the previous submission

   --Job sends to deadline with the first as a dependency.
   SMTDSettings.SubmitAsDependent = true               --enable dependencies
   SMTDSettings.DependencyJobItems = #{1}               --set the dependency on the first entry from the list
   SMTDSettings.DependOnPreviousJobMode = #last         --set the mode to last, so only the last job is concidered
   SMTDSettings.JobName = oldJobName+" [VRay Render]"      --set the job name
   local result = SMTDFunctions.SubmitJobFromUI batchName:batchName      --submit to Deadline as part of the same batch
   format "%\n" result
   
   SMTDSettings.JobName = oldJobName               --restore the job name
   SMTDSettings.SubmitAsDependent = false               --disable dependencies
   SMTDSettings.DependencyJobItems = #{}               --and clear the dependency jobs bitarray flags
   theRenderer.lightcache_mode   = old_lightcache_mode       --restore the old cache mode
   )
   if oldRenderDialogState do renderSceneDialog.open()       --reopen the render dialog if needed
   ::SMTDIsCurrentlySubmitting = undefined
)--end if
)--end local scope
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

squeakybadger
Posts: 80
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Wed Apr 26, 2017 12:05 pm

Capture.JPG
Capture.JPG (19.07 KiB) Viewed 71 times


Well that looks like it is working!

Any idea about sending it as a tile job with padding whilst you are in the code zone? :)

Thanks again.

User avatar
Bobo
Posts: 5886
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Wed Apr 26, 2017 12:40 pm

  • Open SMTD from the Deadline menu before running the script
  • Set up Tile rendering (e.g. Single Frame Tile Rendering) as usual
  • Run the modified script below...

Code: Select all

(--start local scope
   
--If SMTD has not been opened yet, or has been closed since the start of 3ds Max, launch its UI:
if SMTD_MainRollout == undefined or not SMTD_MainRollout.open do macros.run "Deadline" "SubmitMaxToDeadline"

local theRenderer = renderers.current                     --get the current renderer
if isKindOf theRenderer VRay do                           --if it is VRay, we are good to go
(
   --Render final image is disabled, light cache is set to render in the output dir
   local oldRenderDialogState = renderSceneDialog.isopen()      --see if the render dialog was open
   renderSceneDialog.close()                           --close it before submission
   rendSaveFile = false                              --disable output file saving
   theRenderer.options_dontRenderImage = true               --disable the rendering
   local lcpath = getFileNamePath rendOutputFilename + getFileNameFile rendOutputFilename + ".vrlmap"   --build the LC path name
   theRenderer.lightcache_autoSave = true                  --enable LC auto-save
   theRenderer.lightcache_autoSaveFileName = lcpath         --and set the file name to the cache path
   
   --Handle Region Rendering - LC Job should always be full frame
   local oldRegionMode = SMTDSettings.RegionRenderingMode      --store whatever Region mode SMTD was in
   SMTDSettings.RegionRenderingMode = #none               --and force full frame submission
   
   --Job sends to deadline
   ::SMTDIsCurrentlySubmitting = true                     --raise the flag showing we started submission and it is not canceled
   SMTDSettings.PreviousJobsArray = #()                   --clear the dependencies list
   SMTDSettings.SubmitAsDependent = false                   --disable dependencies for first job submission

   SMTD_MainRollout.getNameFromMaxScene()                  --update the job name in the UI
   local oldJobName = SMTDSettings.JobName                --store the current job name
   SMTDSettings.JobName = oldJobName+" [VRay LC]"            --set the LC job's name
   local batchName = oldJobName + " [VRay LC + Render]"      --a batch name to combine the two jobs
   local result = SMTDFunctions.SubmitJobFromUI batchName:batchName      --submit the job to Deadline as part of the batch
   SMTDSettings.RegionRenderingMode = oldRegionMode         --restore the Region Rendering mode
   
   if SMTDSettings.PreviousJobsArray.count > 0 do            --if a previous job exists to depend on,
   (
      --Render final image is enabled, lc is switched to "from file" and uses the precalced light cache
      rendSaveFile = true                              --re-enable output file saving
      theRenderer.options_dontRenderImage = false            --re-enable rendering
      theRenderer.lightcache_autoSave = false               --disable the auto save mode
      local old_lightcache_mode = theRenderer.lightcache_mode   --store the old mode
      theRenderer.lightcache_mode = 2                     --set LC mode to From File
      theRenderer.lightcache_loadFileName = lcpath         --set the load file to the cached path from the previous submission

      --Job sends to deadline with the first as a dependency.
      SMTDSettings.SubmitAsDependent = true               --enable dependencies
      SMTDSettings.DependencyJobItems = #{1}               --set the dependency on the first entry from the list
      SMTDSettings.DependOnPreviousJobMode = #last         --set the mode to last, so only the last job is concidered
      SMTDSettings.JobName = oldJobName+" [VRay Render]"      --set the job name
      SMTDFunctions.SubmitJobFromUI batchName:batchName      --submit to Deadline as part of the same batch

      SMTDSettings.JobName = oldJobName                  --restore the job name
      SMTDSettings.SubmitAsDependent = false               --disable dependencies
      SMTDSettings.DependencyJobItems = #{}               --and clear the dependency jobs bitarray flags
      theRenderer.lightcache_mode   = old_lightcache_mode       --restore the old cache mode
   )
   ::SMTDIsCurrentlySubmitting = undefined                  --lower the cancel submission flag since we are done
   if oldRenderDialogState do renderSceneDialog.open()       --reopen the render dialog if needed
)--end if
)--end local scope


Note that I have removed the debug print from the Listener showing the submission result now that we know what the problem was.
The main change is that before submitting the LC job, we store the current region mode in a variable, and reset the SMTDSettings.RegionRenderingMode to #none to disable tile rendering. Then after the first submission we restore it to the original state. The function SMTDFunctions.SubmitJobFromUI() takes care of submitting region rendering if it is enabled, so we don't have to do anything more.

I am currently testing a single frame to confirm it works as it should.
I still need to see what happens when rendering an animation...
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

User avatar
Bobo
Posts: 5886
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Wed Apr 26, 2017 12:59 pm

Yes, as expected we are not handling animation correctly - if you have set it to 100 frames, the LC job will submit 100 tasks, which will overwrite each other's vrlmap files. This will require additional handling, but let's discuss how you intend to use it if you plan to do animation...
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

squeakybadger
Posts: 80
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Thu Apr 27, 2017 5:55 pm

Hi Bobo,

I've been testing this out today and it all looks like it is working well. I added in the components to save/load an irradiance map as well as the light cache and that works fine.

Will it be possible to integrate the tile rendering into your script as a one-click solution? All the tile jobs I send are split into the same number of tiles every time.

I'm thinking just being able to load a maxscript that sends a prepass, then sets the tile job and number of tiles as the dependency without having to call SMTD then run an additional script.

As for using it for animations, we usually just precalc the irr/lightcache and render a frame per slave on 1080p renders, but do have active jobs now that are 10k in size and have animated elements over a number of frames, so being able to just submit an animation as tiles with a precalc for each frame would work well if possible.

Thanks.

User avatar
Bobo
Posts: 5886
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Thu Apr 27, 2017 6:06 pm

Adding the Tiling to the script should be trivial.
Instead of the current line

Code: Select all

   SMTDSettings.RegionRenderingMode = oldRegionMode         --restore the Region Rendering mode

which restores the previous region rendering mode after the lighting cache pass has been submitted and before the render pass goes out, you have to set the mode to Tiles, and specify any other relevant options:

Code: Select all

   SMTDSettings.RegionRenderingMode = #singleFrameTiles --switch to tile mode
   SMTDSettings.TilesInX = 8 --number of tile columns
   SMTDSettings.TilesInY = 6 --number of tile rows
   TilesPadding = 5 --whatever padding you want to use
   SMTDSettings.SingleTileJobDependent = true --submit a dependent Draft Tile Assembly job
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

squeakybadger
Posts: 80
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Tue May 02, 2017 5:50 am

Hi Bobo,

I've been doing quite a few tests using this script over the long weekend here, and everything looks to be working really well.

Is it possible to add the camera name to the job submission name (or as a comment?) otherwise if you send a few renders from the same file they all get combined on the Deadline queue without knowing which is which.

Thanks.

User avatar
Bobo
Posts: 5886
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Tue May 02, 2017 11:10 am

squeakybadger wrote:Hi Bobo,

I've been doing quite a few tests using this script over the long weekend here, and everything looks to be working really well.

Is it possible to add the camera name to the job submission name (or as a comment?) otherwise if you send a few renders from the same file they all get combined on the Deadline queue without knowing which is which.

Thanks.


The line

Code: Select all

local batchName = oldJobName + " [VRay LC + Render]"

defines the Batch name that groups the jobs together in the Monitor. You can modify that, for example you could add the Camera, and/or the current time string to make each group unique. For example,

Code: Select all

local batchName = oldJobName + " [VRay LC + Render] " + localTime

You could get the current viewport's camera name if it exists, but you have to be mindful of cases where the view is locked in the render dialog, and handle that too...
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

squeakybadger
Posts: 80
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Tue May 02, 2017 12:02 pm

Thanks Bobo, thought it was that line. It seems to be quite picky about what commands it accepts so i'll have a bit of trial and error :)

Do you have any pointers on speeding up region rendering with this script/method?

At the moment the script sets the precalc to full frame rendering with this part:

Code: Select all

   local oldRegionMode = SMTDSettings.RegionRenderingMode      --store whatever Region mode SMTD was in
   SMTDSettings.RegionRenderingMode = #none               --and force full frame submission


The problem with this is that when I run Deadline and setup some jigsaw regions and run the script, the precalc has to calculate the whole image again, even when you just want a small region.

Setting it from #none to #maxRegion seems to get it to only precalc the specific 3dsmax region, but you lose the ability for multi regions with Jigsaw.

Usually any regions that we render would be feathered into the main image, so having the precalc being a bit different isn't a problem.

Any thought on this?


Return to “Thinkbox Software - Deadline”

Who is online

Users browsing this forum: No registered users and 1 guest