Deep Images - Beta Tester Feedback

Deprecated Krakatoa SR Beta Discussion
1

Thinkbox is looking into providing support for deep images in Krakatoa. However, I am looking for some feedback for what you (the beta testers) want.

There are two aspects to this topic:

    1. Deep images for shadows and mattes:
      [list]
      []a) Import deep shadows from the geometry renderer into Krakatoa. This is to replace holdout masks, eliminating the need to export matte mesh files into Krakatoa:
      [list]
      [      ]Casting geometry shadows onto particles: A deep image of the matte geometry is rendered by the geometry renderer from each light in light-space. These images are imported into Krakatoa
  • Occluding particles from camera view: A deep image of the matte geometry is rendered by the geometry renderer from camera space. This image is imported into Krakatoa.
    [/:m]
    [    ]b) Import deep shadows from Krakatoa into the geometry renderer. This would cast shadows from the particles onto scene geometry.[/:m][/list:u][/:m]
    []2) Deep images as an output format for doing deep compositing. Only suitable for studios that do deep compositing.[/:m][/list:u]

Formats:
There are a lot of deep image file formats. Every rendering application seems to have its own format. Here are a few:

  • Pixar’s DTEX format.
    [list]
    [*]Proprietary format used by PRMan to read and write a deep shadow data.

  • Suitable to replace holdout masks.

  • Suitable as an output format for Krakatoa because DTEX images can be imported into Nuke. Studios such as AnimalLogic have used DTEX files in Nuke for full deep image compositing pipelines.

  • Unsuitable for any workflow that doesn’t render geometry using PRMan.
    [/:m]
    [    ]Houdini’s RAT format.

  • RAT files are deep images rendered by Mantra.

  • Suitable to replace holdout masks.

  • Unsuitable as a Krakatoa output format because no compositing engine currently supports them.

  • Unsuitable for any workflow that doesn’t render geometry using Mantra.
    [/:m]
    [    ]3Delight’s DSM format:

  • Proprietary format used by 3Delight to read and write a deep shadow data.

  • Suitable to replace holdout masks.

  • Unsuitable as a Krakatoa output format because no compositing engine currently supports them.

  • Unsuitable for any workflow that doesn’t render geometry using 3Delight.
    [/:m]
    [    ]OpenEXR 2.0:

  • OpenEXR 2.0 will include support for deep color images. The general consensus is that this will become the standard for deep rendering and compositing.

  • DOES NOT CURRENTLY EXIST!
    [/*:m][/list:u]

Deep Shadows Pipeline:

  • Could technically work with any geometry renderer that supports writing and reading deep shadow maps. Most likely PRMan’s DTEX format would be targeted first.
  • Which would be higher priority: casting geometry shadows onto particles, or casting particle shadows onto geometry?
  • Would implementing a workflow using DTEX or RAT for shadows be currently useful to anyone? Or are there other workflows that would be required?

Deep Image Compositing Pipeline:

  • Currently the only feasible workflow that I can see uses PRMan, Krakatoa, and Nuke. Beauty pass deep images of the geometry could be rendered in PRMan, and composited in Nuke using beauty pass deep images rendered in Krakatoa.
  • Are there any other deep compositing pipelines that are currently feasible?
  • Would implementing Krakatoa DTEX deep images output be currently useful to anyone?

Any feedback you have on the issue would be greatly appreciated!

2

Do any of you have a sample .dtex deep shadow map generated from RenderMan you could send me? I would like one for testing purposes and I do not have a version of RenderMan currently.

3

Hi,

Not sure if you already got this…but, attached is a deep shadow in the .dtex format…just some spherical particles with some opacity

devankpatel.com/files/upload … .0299.dtex

Hope it helps

-Devank

4

awesome, thank you! i shall test it out today.

5

Not to muddy things up, but there’s a 5th option, Deep Opacity Maps via OpenEXR 1.x which Krakatoa MX already supports. It at least it is using a standard file format, even if it is slightly less efficient than Deep Shadows for matte objects and geometry shadows.

  • Suitable to replace holdout masks.
  • Suitable as a Krakatoa output format because all compositing engines currently supports them.
  • Suitable for any workflow that can render geometry mattes or shadows to EXR as Deep Opacity Maps (see MX for shadow implementation?).
6

Hi Chad, Krakatoa SR will have the ability to load our own layered z-depth EXR files.

However, the issue is that we want to be able to load deep shadow maps from geometry renderers… and no geometry renderer uses our layered z-depth EXR format, so it’s not too useful for most people :frowning:

7

Agreed, but considering that the format is at least well understood, it may lower the bar for anyone interested in adding such a feature.

8

Hi Conrad,

It’s awesome to see this thread here.
I would like to send you a full set of deep images (rgba, 4 layers of objects, about 50mb/frame) for testing, but I’m not supposed to put this out in the open.
Can I contact you over email?

9

yes of course you may, my email address is conradwiebe@thinkboxsoftware.com

10

Hi Conrad, I’m curious what progress has been made regarding deep images and Krakatoa SR. I’m evaluating SR right now and it’s ability to write deep images is a factor in seeing how it might integrate in to our pipeline.

I’d be happy to give feedback on the questions posed here but want to understand where things are since I’m sure progress has been made since these postings. Thanks.

11

Hi,

Deep image output, as opposed to deep shadow output, is still a ways out. This is partly because the lack consensus on the expected format. We have anticipated adding this in a future version when for example OpenExr 2.0 is released. What format would like you use for the rendered deep image? Can you describe your expected workflow?

Thanks,
Ian

12

I’ll have to check with the higher ups to see how much I can say on the forum (Hello, Legal!) but mostly it deals with cutting out geometry rendered with our proprietary renderer.

What do people do now to cut out hard surface geometry with motion blur? Is it safe to assume to can bring in .obj files (previous and current) and use that? One great advantage of deep images is that you don’t have to worry about cutouts lining up.

13

Sure thing! We can move the discussion off the forum to avoid any concerns there if you would like. Krakatoa does support inputting deep images for matte holdouts that are for example a dtex file. You can also use depth maps and Krakatoa has its own geo renderer that supports at this time objs, and xmesh files (our geometry caching system). The maya exporter can export this geo for you in a form that will be acceptable for Krakatoa (it saves subframe samples where required to get consistent topology to generate the vertex motion blur). The xmesh saver for Maya is in alpha currently and is under active development. If you would like to try it, we can add you to the board easily enough.

You can email me at ianfraser@thinkboxsoftware.com from an email you are comfortable using. We are also willing to sign an NDA if needed (we already have mutual NDA’s with several testers).

Cheers,
Ian

14

I forgot to mention that in certain cases, you will need to follow the steps described in this thread: viewtopic.php?f=115&t=7561

That particular issue would be solved with deep compositing, but that gets us back to the expected output format for the deep compositing pipeline…

Cheers,
Ian

15

Ah, inputting a deep dm to cutout Krakatoa particles at render time would be excellent. I’ll look in to that.

I’ve been given the okay to discuss here what we would be looking for in regards to deep images. Here goes:

Much of our rendering happens in either our proprietary renderer or in Mantra. We have a custom goemetry format that we use, though we can, of course, convert that to .obj files inside of Houdini or Maya. That all said, being able to bring in a deep file at render time is very valuable because we (in the FX department) can grab deep dms that Lighting generates and use those as holdouts. Our production shots get very complicated and usually involve fur, crowds, and procedurally generated objects (like trees). So bringing in a deep dm as a holdout instead of .obj files is very beneficial for us. Another benefit of deep dms for cutouts is that we can store the deep dms that Lighting generates and use those, saving FX the hassle of having to render all the trees, fur, etc., which obviously can be a slow process.

I’ve been testing Krakatoa two-fold: (1) to see if it can achieve a very specific effect we are doing and (2) to ensure I can export our cameras and get cutouts/mblur/etc to line up. The effect I’m doing doesn’t require casting shadows on to Krakatoa particles but that would be important for more “traditional” effects (for example, a cloud of smoke).

In terms of casting geometry shadows on to particles, being able to load in a deep dm generated from a light in another renderer is very valuable. However we’d want to mix that deep dm with the Krakatoa light deep dm so the particles would receive shadows from geometry and self-shadow. Being able to add an arbitrary number of deep dms to a light would be ideal. This way the user can generate deep shadow maps for the crowds, trees, etc separately. That said, we do have an in-house tool to combine deep images in to a single one, but I don’t know if others have a similar tool. Might be valuable to offer something akin to:

ri.AttributeBegin() ri.LightSource("foo", "bar"...) ri.LightAddDeepShadowMap("bar", "/usr/pic1/chars_shadows.1.dm") ri.LightAddDeepShadowMap("bar", "/usr/pic1/trees_shadows.1.dm") ri.AttributeEnd()

In regards to casting Krakatoa particle shadows on to geometry, if you can output a deep dm from a Krakatoa light then that can be loaded as an ‘additional dm’ in many lighting packages (though I’ll admit I’m only familiar with Mantra and our in-house renderer. That said, I assume that feature is fairly standard in most renderers.)

Next, outputting deep images from Krakatoa would allow for deep compositing based pipelinse (that’s obvious) but also doing cutouts in command-line tools that can process deep images.

As for which format, that’s hard for me to comment on. We have our own internal implementation that we use now. We do use and process RAT files in the part of our pipeline that integrates Mantra with our in-house renderer. OpenEXR 2.0 is interesting but, as you said, not out yet.

For my needs now the main issue is holding out renders from our renderer in Krakatoa. Sounds like that may be possible. Next most important feature would be letting Krakatoa lights read deep dms generated in other packages so we can cast shadows from our characters on to Krakatoa particles. RAT would be good as we don’t use DTEX, but possibly there is an open-source converter we could use to get to DTEX in the meanwhile. I don’t know, I’ve never looked in to that.

I hope that helps. We’ve done quite some work to integrate Mantra in to our pipeline for FX so a lot of the questions you posed are hot topics for us these days. Hopefully this post illuminates some of what we’ve learned in that process.

16

Ian, is there documentation on how to pass a deep image (DTEX or otherwise) as a holdout in a render? Do I just pass a dtex file in place of an obj file? Thank you.

17

We will be posting some more documentation shortly! We are just pulling the examples together.

-i

18

Hey,
The documentation was a little out of date, but I went though it yesterday and updated everything. It includes details on how to import deep shadow maps and holdout mattes. It can be found here: viewtopic.php?f=118&t=6352&p=25511

There is also a video tutorial on how to import deep shadow maps and holdout mattes using RenderMan to generate the maps within Maya. I’m not sure if that will help you, but it is found at the bottom of the page here: viewtopic.php?f=118&t=6363

I haven’t looked into support for RAT format just yet, but it is something we could explore. We currently support Pixar’s DTEX, 3Delight’s DSM formats, and our own internal multi-layered EXR format. What input/output format would be most useful? We had figured that sticking with Pixar’s DTEX would be the best route for the moment, since there isn’t a good standard.

Here is a generic scene that demonstrates how to specify deep shadows and holdouts:

import KrakatoaSR

# Since the formats are not open, Krakatoa must use a DTEX or DSM loading library
# in order to load DTEX or DSM files. This call sets the location of that DLL.
KrakatoaSR.SetDtexLibraryPath( "C:/Program Files/3Delight/bin/3Delight.dll" )

ri = KrakatoaSR.Ri()

ri.Option( "render", "DensityPerParticle", 1 )
ri.Option( "render", "DensityExponent", -6 )

# This call imports a deep image to be used as a holdout matte
ri.Option( "render", "DeepMatteLoadingPath", "c:/pathto/my_deep_holdout_matte.dtex" )

ri.FrameBegin( 0 )
ri.Format( 1024, 768, 1.0 )
ri.Display( "my_output.exr", "file", "rgba" )
ri.Projection( "perspective", "fov", 45 )

ri.Transform( 1,0,0,0, 0,1,0,0, 0,0,1,0, 0,0,10,1 )

ri.WorldBegin()


ri.AttributeBegin()
ri.Transform( 1,0,0,0, 0,1,0,0, 0,0,1,0, 0,0,10,1 )
ri.LightSource( "spotlight", "my_spot_light",
	"ShadowMapWidth", 512,
	"ShadowsEnabled", True
	"Flux", (12.56636,12.56636,12.56636), 
	"InnerConeAngle", 90, 
	"OuterConeAngle", 90,	
	"DecayExponent", 0, 
	
	# This next parameter sets the loading path for the deep shadow map rendered for this light
	"c:/pathto/my_deep_shadow_map.dtex"
)
ri.AttributeEnd()
ri.Illuminate( "my_spot_light", True )

# add some random particles
ri.FractalPoints( 1000000, 9526, 4, 2 )
ri.FractalPoints( 1000000, 31527, 2, 2 )
ri.FractalPoints( 1000000, 58323, 2, 3 )
ri.FractalPoints( 1000000, 71903, 2, 2 )
ri.FractalPoints( 1000000, 76410, 5, 3 )
ri.FractalPoints( 1000000, 92082, 4, 2 )


ri.WorldEnd()

ri.FrameEnd()

For the time being, we only allow a single image for each light, and a single image for holdouts. So you would need to combine them for now, but I this to our wishlist.

Krakatoa does have the ability to export deep shadow maps from lights. However, the format we use is a multi-layered EXR file of our own format. Which basically saves attenuation at various z depths. Saving these types of files is described a little in the documentation. If you would like, I could provide further detail of how to create/use these shadow maps.

We do not currently support outputting the actual beauty pass data in various layers. Apart from the “two layer” approach (occluded/unoccluded) that is described in this post: viewtopic.php?f=115&t=7561

-Conrad

19

Excellent, thanks Conrad. I’ll try that out in the next couple of days and report back!

20

I’m testing using a .dtex file as a holdout now. What is the syntax for specifying a cutout .dtex file? Is it just:

ri.MeshFile(“myholdout.dtex”)

…or is there another command to use? Thanks.