Vray lightcache prerender?

Discussion and Support of Deadline, the Render Management System
User avatar
Bobo
Posts: 5925
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Tue May 02, 2017 12:27 pm

You can either have one lighting precalc job and many region jobs, or one precalc job for each region. In the later case, you will have to make sure it actually produces consistent results. I would expect it to, but you have to be sure.

I suspect that if you are subdividing the image into N regions and rendering them all, submitting N+1 jobs is better than 2*N jobs.
But if you are rendering only one or two regions out of the whole image, then precalculating the lighting of the whole image would be a waste, unless you expect the rest of the regions to come in later anyway...

So it is up to you to decide how you want it - let me know if anything does not work right.
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

squeakybadger
Posts: 84
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Wed May 03, 2017 8:58 am

Thanks Bobo,

I see where you are coming from with that. If we had to render lots of little regions, then we would usually just rerender the whole image just to save on headaches.

The reason we needed the precalc submission was that you could see the tile joins on the final image, even when padding to the max. That doesn't happen anymore using this method where the slaves load the irradience map.

I've managed to get a bit of a working workflow by using the 3dsmax region as the precalc with DBR enabled (to speed it all up), then the script changes it to Jigsaw and creates the Jigsaw region from the 3dsmax region and submits (with DBR off)

This seems to work, but I can't get it to split into tiles - it just gets sent as a jigsaw job with 1 tile.

This is the code which I thought would work, but maybe a Jigsaw submission doesn't read it as tiles?

Code: Select all

   SMTDSettings.RegionRenderingMode = #singleFrameMultiRegion --switch to jigsaw
   SMTD_MultiRegionRendering.createFromMaxRegion  -- create region from max region
   SMTDSettings.TilesInX = 8 --number of tile columns
   SMTDSettings.TilesInY = 8 --number of tile rows
   TilesPadding = 0 --tile padding
   SMTDSettings.SingleTileJobDependent = true --submit a dependent Draft Tile Assembly job


Any idea on how it should be defined to get it to split into 8x8 tiles?

Thanks.

User avatar
Bobo
Posts: 5925
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Wed May 03, 2017 11:41 am

squeakybadger wrote:This is the code which I thought would work, but maybe a Jigsaw submission doesn't read it as tiles?

Code: Select all

   SMTDSettings.RegionRenderingMode = #singleFrameMultiRegion --switch to jigsaw
   SMTD_MultiRegionRendering.createFromMaxRegion  -- create region from max region
   SMTDSettings.TilesInX = 8 --number of tile columns
   SMTDSettings.TilesInY = 8 --number of tile rows
   TilesPadding = 0 --tile padding
   SMTDSettings.SingleTileJobDependent = true --submit a dependent Draft Tile Assembly job


Any idea on how it should be defined to get it to split into 8x8 tiles?


Of course :)

The TilesInX and TilesInY properties have nothing to do with Jigsaw, they define the grid when using the old Tile Rendering mode (which is mostly unrelated to Jigsaw, except the option to create Jigsaw regions from the Tile Grid settings).

What you really need to look at is the Jigsaw definition itself, which is created and stored in SMTDSettings.MultiRegionData
Also remember to add the () after SMTD_MultiRegionRendering.createFromMaxRegion() as it would not create anything otherwise.
If you execute the function and check out the content of the property, you will see something like

Code: Select all

SMTDSettings.MultiRegionData
#(#(true, 0.1, 0.1, 0.8, 0.8, 1, 1, "3ds Max Region Gizmo [X:64 Y:48 W:512 H:384]", true))

The value of the property is an array of arrays defining all current Jigsaw regions. The values inside the sub-array define:
1: boolean value for the enabled state
2,3: upper left corner X and Y in normalized screen space
4,5: width and height of the region in normalized screen space
6,7: the tiles along X and Y - this is what you want to modify!
8: the name of the region
9: the selected state of the region
10: (optional, added when keyframed) the keys array
11: (optional, added when keyframed) the lock boolean value

So once you have created the Jigsaw region from the Max region, you can say

Code: Select all

SMTDSettings.MultiRegionData[SMTDSettings.MultiRegionData.count][6]=8 --set the X tiles of the last region to 8
SMTDSettings.MultiRegionData[SMTDSettings.MultiRegionData.count][7]=6 --set the Y tiles of the last region to 6


If you look at the definition now, you will see

Code: Select all

SMTDSettings.MultiRegionData
#(#(true, 0.1, 0.1, 0.8, 0.8, 8, 6, "3ds Max Region Gizmo [X:64 Y:48 W:512 H:384]", true))

and in the Jigsaw rollout of the Tiles tab of the SMTD UI will see the actual tiles if you click again on the new region.

Remember that the Tiles tab must be selected in SMTD UI for the region creation function to work. If you prefer to create the Jigsaw from Max region without the Tiles tab selected, you can create your own function in your own script, and call that:

Code: Select all

      fn createJigsawFromMaxRegion =
      (
         local theRect = if viewport.activeViewport > 0 then
            viewport.getRegionRect (viewport.activeViewport)
         else
            Box2 0 0 RenderWidth RenderHeight
         append SMTDSettings.MultiRegionData #(true, (1.0*theRect.x/RenderWidth), (1.0*theRect.y/RenderHeight), (1.0*theRect.w/RenderWidth), (1.0*theRect.h/RenderHeight), 1, 1, "3ds Max Region Gizmo [X:"+ theRect.x as string + " Y:"+ theRect.y as string + " W:"+theRect.w as string + " H:"+theRect.h as string + "]" )
         ::SMTD_MultiRegionSettingsPersistentGlobal = SMTDSettings.MultiRegionData
      )


The above does the same as the function SMTD_MultiRegionRendering.createFromMaxRegion() you call, but it does not require the SMTD UI to be open at all or the Tiles tab to be selected. The ::SMTD_MultiRegionSettingsPersistentGlobal persistent global is used to remember the last Jigsaw setup between SMTD sessions. When you click on the Tiles tab of the SMTD UI, the content of this variable is copied into SMTDSettings.MultiRegionData, so if the persistent global is #(), it would destroy the region you just created. Thus we have to keep the two in sync in our function. Add the above function to the beginning of your script, and then use

Code: Select all

   SMTDSettings.RegionRenderingMode = #singleFrameMultiRegion --switch to jigsaw
   createJigsawFromMaxRegion()  -- create region from max region using own function
   SMTDSettings.MultiRegionData[SMTDSettings.MultiRegionData.count][6]=8 --set the X tiles of the last region to 8
   SMTDSettings.MultiRegionData[SMTDSettings.MultiRegionData.count][7]=6 --set the Y tiles of the last region to 6   
   SMTDSettings.TilesPadding = 0 --tile padding
   SMTDSettings.SingleTileJobDependent = true --submit a dependent Draft Tile Assembly job


Hope this helps.
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

squeakybadger
Posts: 84
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Mon May 08, 2017 11:53 am

Thanks for the detailed info Bobo, I've managed to make a few submitters that take care of single images, regions, jigsaw regions, brute force & lightcache, irradiance and lightcache and any other bits and bobs that people need thanks to this thread. :)

Did you have any ideas on how to get an animation working with the prebake script, it's the only thing missing now from my workflow.

Thanks again!

User avatar
Bobo
Posts: 5925
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Mon May 08, 2017 12:03 pm

squeakybadger wrote:Did you have any ideas on how to get an animation working with the prebake script, it's the only thing missing now from my workflow.


I am still unsure what you expect to happen, and what the technical issue is you need help with.

When rendering Animation Jigsaw regions, each job contains exactly one Max region, and could have a matching light cache job with the same region writing to a name that matches the tile signature.
Alternatively, one job with full frame calculations could be sent to calculate the cache for all regions, and all region jobs would be set to load its cache, and be dependent on it.

Both implementations should be trivial with the code you have, and it should be up to you to decide which of the two approaches you want. But that is a conceptual question, not a technical one. If you hit a snag in implementing the actual code, please let me know. :)
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

squeakybadger
Posts: 84
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Tue May 09, 2017 10:48 am

Hi Bobo,

Is it possible to force the submission script to only render the precalc animation on one slave?

At the moment when the lightcache prerender job is sent, it sends as 20 tasks, which then 20 slaves start having a go and so the irradiance/lightcache map is overwritten each time.

Can you specify that a slave holds onto the scene file and renders the lightcache file fully as an animation, so it adds to it, rather than overwriting?

Thanks Sensei :)

User avatar
Bobo
Posts: 5925
Joined: Tue Nov 15, 2005 10:26 am
Location: Vancouver, Canada
Contact:

Re: Vray lightcache prerender?

Postby Bobo » Tue May 09, 2017 11:03 am

squeakybadger wrote:Hi Bobo,

Is it possible to force the submission script to only render the precalc animation on one slave?

At the moment when the lightcache prerender job is sent, it sends as 20 tasks, which then 20 slaves start having a go and so the irradiance/lightcache map is overwritten each time.

Can you specify that a slave holds onto the scene file and renders the lightcache file fully as an animation, so it adds to it, rather than overwriting?

Thanks Sensei :)


There are a few approaches, but the best would be to simply set the chunk size to the number of frames, or just to a very large number. This way there will be just ONE task in the job, forcing a single Slave to go through all frames in order. If it fails in the middle, it or another Slave will start from the beginning.

Code: Select all

SMTDSettings.ChunkSize = 10000

Then for the actual render jobs, set it back to 1 or whatever chunk size you usually submit with.
Borislav "Bobo" Petrov
Product Specialist
Thinkbox Software Inc.

squeakybadger
Posts: 84
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Fri May 12, 2017 5:04 am

Hi Bobo,

I've managed to get everything working like it should now, but i've noticed it failing to report a task as failed if the precalc job crashes.

This is taken from the log, so everything has been marked complete and still tries to render without the actual irradiance map/lightcache file.


Code: Select all

2017-05-12 10:23:05:  0: INFO: V-Ray DBR: Updating cfg file for distributed render with the following machines:
2017-05-12 10:23:05:  0: INFO: V-Ray DBR:   10.3.222.107
2017-05-12 10:23:05:  0: INFO: V-Ray DBR:   10.3.222.105
2017-05-12 10:23:05:  0: INFO: V-Ray DBR:   10.3.222.110
2017-05-12 10:23:05:  0: INFO: V-Ray DBR:   10.3.222.112
2017-05-12 10:23:25:  0: STDOUT: 12/05/2017 10:23:24;  BEGIN FRAME 0; TASK 1 OF 1; (C:\Users\PIkcells\AppData\Local\Thinkbox\Deadline9\slave\r9\jobsData\59157e5c0eb9411b14575590\3dsOutput_temp0Rl220\BAT_LAR_Establishing_UNITS0000.exr);
2017-05-12 10:45:33:  0: STDOUT: 12/05/2017 10:45:33;  An unexpected exception has occurred in the network renderer and it is terminating.
2017-05-12 10:45:33:  0: STDOUT: 12/05/2017 10:45:33;  Error rendering frame 0: An unexpected exception has occurred in the network renderer and it is terminating.
2017-05-12 10:45:34:  0: INFO: Moving output files and folders from C:\Users\PIkcells\AppData\Local\Thinkbox\Deadline9\slave\r9\jobsData\59157e5c0eb9411b14575590\3dsOutput_temp0Rl220 to \\piknas003\Projects\2017\FOREMOST\2_BATHS\STILLS\RENDERS\LARKEN\MAINPOOL\REGIONS
2017-05-12 10:45:34:  0: INFO: V-Ray DBR: Restoring backup config file: C:\Users\PIkcells\AppData\Local\Thinkbox\Deadline9\slave\r9\jobsData\59157e5c0eb9411b14575590\vray_dr.cfg to original location: C:\Users\PIkcells\AppData\Local\Autodesk\3dsmax\2016 - 64bit\ENU\en-US\plugcfg\vray_dr.cfg
2017-05-12 10:45:34:  0: INFO: V-Ray DBR: Marking other incomplete tasks as complete


Any ideas?

squeakybadger
Posts: 84
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Fri May 12, 2017 12:12 pm

Also, i'm trying to get a sequence to render where it calcs a prepass/irradiance map for each frame (as objects go in/out on a 1 frame basis)

I thought I would just be able to set the rendering modes to animation prepass/animation rendering in the script, but it just renders 1 file (whereas the animation prepass should render separate precalcs for each frame)

Then (not surprisingly) Deadline errors out with vray saying the irradiance map cannot be used for this type...

This is the bulk of the code for creating the precalcs, which works for flythough animations. I can't seem to see what the problem is though.


Code: Select all

 --Render final image is disabled, light cache is set to render in the output dir
   local oldRenderDialogState = renderSceneDialog.isopen()      --see if the render dialog was open
   renderSceneDialog.close()                           --close it before submission
   rendSaveFile = false                              --disable output file saving
     theRenderer.options_dontRenderImage = true               --disable the rendering
   theRenderer.system_distributedRender = true   --turn on dr (j)

   
   local lcpath = getFileNamePath rendOutputFilename + getFileNameFile rendOutputFilename + ".vrlmap"   --build the LC path name
--theRenderer = renderers.current
   theRenderer.lightcache_mode =1  --set to flythrough (J)
   theRenderer.lightcache_dontDelete = true
   theRenderer.lightcache_autoSave = true                  --enable LC auto-save
   theRenderer.lightcache_autoSaveFileName = lcpath         --and set the file name to the cache path
   theRenderer.lightcache_saveFileName = lcpath
   
   
   SMTDSettings.ForceWorkstationMode = false
   SMTDSettings.MergeSceneXRefsOnSubmission = false
   SMTDSettings.MergeXRefsOnSubmission = false
   SMTDSettings.Priority = 100
   SMTDSettings.SingleTileJobDraftErrorOnMissing = true
   SMTDSettings.SingleTileJobCleanup = false
   SMTDSettings.RestartRenderer = true
   SMTDSettings.RemovePadding = false
   SMTDSettings.SubmitAsSuspended = false

   rendNThFrame = 1  --sets nth to 1
   

   
   local irpath = getFileNamePath rendOutputFilename + getFileNameFile rendOutputFilename + ".vrmap"   --build the IM path name


   
   theRenderer.adv_irradmap_mode = 1  --1 is multiframe inc (J)
   theRenderer.adv_irradmap_dontDelete = true
   theRenderer.adv_irradmap_autoSave = true
   theRenderer.adv_irradmap_autoSaveFileName = irpath
   theRenderer.adv_irradmap_saveFileName = irpath
   
   --Handle Region Rendering - LC Job should always be full frame
   local oldRegionMode = SMTDSettings.RegionRenderingMode      --store whatever Region mode SMTD was in
   SMTDSettings.RegionRenderingMode = #none               --and force full frame submission
   
           SMTDSettings.DBR = true               --turn on VrayDR (J)
   SMTDSettings.DBRUse3dsCmd = true   --turn on 3dscmd in deadline (J)
   SMTDSettings.DBRServers = 5            --set 4 dr servers (J)
   
   --Job sends to deadline
   ::SMTDIsCurrentlySubmitting = true                     --raise the flag showing we started submission and it is not canceled
   SMTDSettings.PreviousJobsArray = #()                   --clear the dependencies list
   SMTDSettings.SubmitAsDependent = false                   --disable dependencies for first job submission
      SMTDSettings.ChunkSize = 10000 --chunk it so only 1 slave does the calc (J)
   SMTDSettings.RemovePadding = false



Thanks for any pointers.

squeakybadger
Posts: 84
Joined: Tue Jul 28, 2009 9:50 am

Re: Vray lightcache prerender?

Postby squeakybadger » Wed May 17, 2017 10:19 am

Hi Bobo,

Just an update on this.

I've sorted out the Anim Prepass/Rendering issue and it looks like it works properly now.

I've just encountered a small bug when using the create from 3dsmax region script to add it to jigsaw - it doesn't seem to delete the new region data and hangs around even after deadline is closed.

So sending a single region after 2 other regions on different jobs have been sent, then next job will have 3 regions on the render.

I'm guessing the SMTDSettings.MultiregionData that adds it to Jigsaw needs deleting at the end of the submission?


Return to “Thinkbox Software - Deadline”

Who is online

Users browsing this forum: Bobo and 1 guest