Deadline 10 and Tile Assembly from Vrscenes?


#1

Hey all,

I just wanted to check in and see if the tile assembly issue from parsed vrscene files has been solved. We are running 9061 now and I want to upgrade to 10 soon, but after browsing the release notes, I can’t find if the issue has been reconciled or not.

For reference, one of my threads on it can be found here (forums.thinkboxsoftware.com/vie … 11&t=15418). I know its an ongoing and complex issue, but wanted to see if anyone has tested vrscene files with multiple render channels and the tile assembler to see if its fixed OR if anyone from the Deadline team knows it was handled.

Thanks!


#2

I don’t think it is yet resolved in Deadline 10. I filed the bug here:
forums.thinkboxsoftware.com/vie … hp?t=15469

Very much looking for an update that resolves this.


#3

gotcha - thanks, good to know. I’d have to take a look side by side, but I suspect that vrscene generated files are perhaps using different naming conventions for render channels whether they come from maya or sketchup etc, I wonder if that is part of the issue. Fingers crossed the Thinkbox guru’s hash this out!


#4

Well, there’s some initial work. It turns out there was a bug in how we handled double slashes. I’m thinking it may be faster to get you guys looped in closer to the integration team to remove barriers here, but just trying to find a good way to handle that.

There’s a reason why we don’t usually try and parse other company’s file formats! :smiley:

Still, hoping we can get this working for you both. I think you two are the ones who use this feature most, or at least find the best edge cases!


#5

That would be great!


#6

I’ve just installed D10 on our test environment and am in the process of testing all our workkflows. We do use VRayscens with tiling for our Revit workflow so will keep an eye on this thread


#7

Yeah, I suspect rendering via vrscene files will become more popular, especially as the interoperability of sharing vrscene files between programs works as it does in maya/max right now.

Its also a great way to work with cloud rendering and forgoing any plugins that may or may not work in virtual environments (ran into this issue with a 3ds Max plugin on my last project and WISHED I could have used vrscene files to render from)


#8

1051 seems to have fixed it! getting all of our material channels now, super excited!


#9

Whoo hoo! Thanks for keeping on us man.

How’s it going for others here?


#10

I found some specific settings that work for us. Rendering large images with 100 Tiles sure is nice.
I have to test some other settings that did not work before.

Cheers, Florian

Edit: Thanks and have a nice weekend!


#11

Ran a bunch of production tests last night, and still working great. Can’t tell you how excited I am to a) use all our computers again for rendering and b) not have to manually tile everything.

One request / bug (?): right now, the “single frame tile job enabled” seems bugged. It won’t actually submit the render job, just the tile assembly job. I’m HOPING by design (and when working) this gives us sub tasks on the job instead of sub jobs for the render regions. I saw this because I’m testing out scaling across our bad and very good machines. The quick fix is to launch multiple slaves on the good machine and have it render concurrently so that sub-jobs come out in about the same amount of time. However, ideally, I would like to submit a job with sub-tasks for the render regions. This way I could configure the stronger computers to run multiple tasks simultaneously from the same slave.

This is minor, but I would personally prefer it this way because running multiple slaves is somewhat of a pain when we have multiple pool and job assignments and I don’t want the machine to be running a tiled job WHILE its also trying to render a swarm job because they exist on separate slaves and pools.

But beyond that, haven’t run into any issues yet!


#12

Hey,
“single frame tile job enabled” doesn’t work on my end either. Sometimes it just creates one assembly job. But most of the times it creates one render job and one assembly job. The render job works fine but the assembly job will only merge the beauty pass. All render elements will not merge and no error will occur.

It would be great if this would work. I like to get notifications for a completed job but with something like 70 tiles, there are just too many pop up windows.

Cheers, Florian


#13

So close! Thanks for noticing guys. We’re going to have the devs helping out direct for a few days this week so I’ll make the dev issue and they can work with you on it.

You could always just submit an assembly job with the config file from the old assembly job. Thoughts?


#14

I guess? Honestly, we JUST got this functionality and I finally FINALLY got staff to submit tiled vrscenes because its a 1 button press submit process. They never warmed up to the 2 step process before (submit tiled vrscene and manually submit an assembly job), and I just can’t go back now that we’ve made some progress.

The change from a sub-job to sub-tasks for each tile render is more about balancing and admin usability, such as the concurrent tasks and pop up notifications. Functionally its about the same, just more of a preferance


#15

It looks like tile assembly is again broken in Deadline 10.0.5.1
The file parsing seems to work OK and shows the correct output path but the individual tiles end up with one of the render element names in each filename so the submit script will not find the correct files.

Wrong Tilename:
_tile2_Name_normals_multimatte.exr
Correct Name:
_tile2_Name_multimatte.exr

I reverted back to the old VrsceneParser.py and VraySubmission.py as a workaround.

Cheers, Florian


#16

I just updated Deadline to version 10.0.7.2 and the error is still there.

The assmbly job is looking for a file called:
_tile0_Name_normals.exr

But the file is (wrongly) named:
_tile0_Name_multimatte_normals.exr

The file parsing shows the correct names but it seems (from looking at the “Plugin Info Parameters”) that the output filname is wrongly configured by the Deadline submit script. The “multimatte” gets added to the output name.


#17

Hey guys,

I’ve got some changes that’ll be making their way into a future service pack of deadline that’ll address some of the issues mentioned in this thread. I’ll attach the changed files here. Feel free to test and let me know how they work for you/what I’ve missed. Ensure that you back up the two files before this and copy the files to these locations.

  • “DeadlineRepository/submission/VRay/Main/VrsceneUtils.py”
  • “DeadlineRepository/scripts/Submission/VraySubmission.py”

You’ll close/re-open the Monitor and re-run the job submission to see if the changes helped.

With regards to things like “multimatte” being added to the file name. I can’t say I’ve experienced that yet… Would you mind passing another vrscene file over that experiences it (or if the previous test file also experiences it, share your submission options)?

Cheers
Morgan
Vray.zip (12.4 KB)


#18

Hey Morgan
thanks for the update. I w ill be testing them right away.

I’l try to get you a test scene that shows this behaviour. Multimatte is just one example. What basically happens is that the name of one render element (Multimatte, GI, normals, etc…) gets added to all files.

I hacked the VraySubmission.py script of the current Deadline version yesterday and replaced this code:

            tempOutputFiles = [ outputFile ]
            if len(pulledOutputFiles) > 0:
                tempOutputFiles = pulledOutputFiles
            elif paddedOutputFile:
                tempOutputFiles = [ paddedOutputFile ]
            
            if regionRendering:
                for index, outputFile in enumerate(tempOutputFiles):
                    tileName = outputFile

                    splitFilename = os.path.split(tileName)
                    tileName = os.path.join(splitFilename[0], "_tile?_"+splitFilename[1])

                    if singleRegionJob:
                        for currTile in range(0, tilesInX*tilesInY):
                            regionOutputFileName = tileName.replace( "?", str(currTile) )
                            writer.WriteLine( "OutputFilename%sTile%s=%s" % ( index, currTile, regionOutputFileName ) )


                    else:
                        regionOutputFileName = tileName.replace( "?", str(jobNum) )
                        writer.WriteLine( "OutputFilename%s=%s" % ( index, regionOutputFileName ) )
            
            else:
                for index, outputFile in enumerate(tempOutputFiles):
                    writer.WriteLine( "OutputFilename%s=%s" % ( index, outputFile ) )

with this from an older version of the script:

            if singleRegionJob:

                tileName = paddedOutputFile
                paddingRegex = re.compile( "(#+)", re.IGNORECASE )
                matches = paddingRegex.findall( os.path.basename( tileName ) )
                if matches != None and len( matches ) > 0:
                    paddingString = matches[ len( matches ) - 1 ]
                    paddingSize = len( paddingString )
                    padding = str(scriptDialog.GetValue( "SingleJobFrameBox" ))
                    while len(padding) < paddingSize:
                        padding = "0" + padding
                    
                    padding = "_tile?_" + padding
                    tileName = RightReplace( tileName, paddingString, padding, 1 )
                else:
                    splitFilename = os.path.splitext(tileName)
                    tileName = splitFilename[0]+"_tile?_"+splitFilename[1]
                
                for currTile in range(0, tilesInX*tilesInY):
                    regionOutputFileName = tileName.replace( "?", str(currTile) )
                    writer.WriteLine( "OutputFilename0Tile%s=%s" % ( currTile, regionOutputFileName ) )

            elif paddedOutputFile != "":

                writer.WriteLine( "OutputFilename0=%s" % paddedOutputFile )

This does fix the bug for me. The code starts around line 630.

Cheers, Florian


#19

Ran a quick test and everything works fine! The option “Single Frame Tile Job Enabled”, which was broken previously, also now works.

Thanks! :smiley: :mrgreen: :smiley:

Cheers, Forian


#20

Good to hear!

Without modifying the submitter, are you still experiencing filenames writing out the render element type? I still haven’t been able to reproduce it with my own scene files.

Cheers