Layered Images with Tile-Rendering


#1

Hi, I’m trying to render layered images with tile rendering and assemble them in draft.

When rendering EXR or PSD files, the individual tiles have all layers but after the assembly with Draft everything is merged down to a single layer.

Is there a way to preserve the rendered Layers?

Jörg-Mark


#2

We should preserve the channels in EXRs but I’m not so sure about PSDs.

What is generating the images in the first place? Maya, Max, C4D? Also, what version of Deadline?

I believe we detect the channels from the first tile we open, then use that for the final output image. In fact, you should see it in the log for the tile assembler job. Here’s the code:

            if channelNames is not None:
                assembler.SetChannels(channelNames)
                print( "Total Number of Output Channels: " + str(len(channelNames)) )
                print( "Output Channels: " )
                for channelName in channelNames:
                    print( str(channelName) )

Having a job report from the assembly job would be great, but it will have file names in it so you may want to clean those out or submit a test job.


#3
=======================================================
Log
=======================================================
2018-09-11 09:44:58: 0: Loading Job's Plugin timeout is Disabled
2018-09-11 09:45:01: 0: Executing plugin command of type 'Sync Files for Job'
2018-09-11 09:45:02: 0: Synchronization time for job files: 129.092 ms
2018-09-11 09:45:02: 0: Synchronizing Plugin DraftTileAssembler from \\FILESERVER\jmk_server\DEADLINE_10_REPOSITORY\plugins\DraftTileAssembler took: 0 seconds
2018-09-11 09:45:02: 0: Done executing plugin command of type 'Sync Files for Job'
2018-09-11 09:45:02: 0: Executing plugin command of type 'Initialize Plugin'
2018-09-11 09:45:02: 0: INFO: Executing plugin script 'C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\plugins\5b9771c90c89bc2f68e21e0e\DraftTileAssembler.py'
2018-09-11 09:45:03: 0: INFO: Found Draft python module at: 'C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\Draft\Draft.pyd'
2018-09-11 09:45:03: 0: INFO: About: Draft Tile Assembler Plugin for Deadline
2018-09-11 09:45:03: 0: INFO: Render Job As User disabled, running as current user 'jmk_server'
2018-09-11 09:45:03: 0: INFO: The job's environment will be merged with the current environment before rendering
2018-09-11 09:45:03: 0: Done executing plugin command of type 'Initialize Plugin'
2018-09-11 09:45:03: 0: Start Job timeout is disabled.
2018-09-11 09:45:03: 0: Task timeout is disabled.
2018-09-11 09:45:03: 0: Loaded job: test_.lxo (Frame 0 - Draft Tile Assembly Job) (5b9771c90c89bc2f68e21e0e)
2018-09-11 09:45:03: 0: Skipping P: because it is already mapped
2018-09-11 09:45:03: 0: Executing plugin command of type 'Start Job'
2018-09-11 09:45:03: 0: INFO: Executing global asset transfer preload script 'C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\plugins\5b9771c90c89bc2f68e21e0e\GlobalAssetTransferPreLoad.py'
2018-09-11 09:45:03: 0: INFO: Looking for AWS Portal File Transfer...
2018-09-11 09:45:03: 0: INFO: Looking for File Transfer controller in C:/Program Files/Thinkbox/S3BackedCache/bin/task.py...
2018-09-11 09:45:03: 0: INFO: Could not find AWS Portal File Transfer.
2018-09-11 09:45:03: 0: INFO: AWS Portal File Transfer is not installed on the system.
2018-09-11 09:45:03: 0: Done executing plugin command of type 'Start Job'
2018-09-11 09:45:03: 0: Plugin rendering frame(s): 0
2018-09-11 09:45:03: 0: Executing plugin command of type 'Render Task'
2018-09-11 09:45:03: 0: INFO: Draft Tile Assembler job starting...
2018-09-11 09:45:04: 0: CheckPathMapping: Swapped "ImageFileName=P:\TEMP\DEADLINE_TEST\180911\test_tiled0000.psd
2018-09-11 09:45:04: 0: " with "ImageFileName=\\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled0000.psd
2018-09-11 09:45:04: 0: "
2018-09-11 09:45:04: 0: CheckPathMapping: Swapped "ImageWidth=P:\TEMP\DEADLINE_TEST\180911\test_tiled0000.psd
2018-09-11 09:45:04: 0: " with "ImageWidth=\\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled0000.psd
2018-09-11 09:45:04: 0: "
2018-09-11 09:45:04: 0: CheckPathMapping: Swapped "Tile0FileName=P:\TEMP\DEADLINE_TEST\180911\test_tiled_tile_1x1_2x2_0000.psd
2018-09-11 09:45:04: 0: " with "Tile0FileName=\\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled_tile_1x1_2x2_0000.psd
2018-09-11 09:45:04: 0: "
2018-09-11 09:45:04: 0: CheckPathMapping: Swapped "Tile1FileName=P:\TEMP\DEADLINE_TEST\180911\test_tiled_tile_2x1_2x2_0000.psd
2018-09-11 09:45:04: 0: " with "Tile1FileName=\\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled_tile_2x1_2x2_0000.psd
2018-09-11 09:45:04: 0: "
2018-09-11 09:45:04: 0: CheckPathMapping: Swapped "Tile2FileName=P:\TEMP\DEADLINE_TEST\180911\test_tiled_tile_1x2_2x2_0000.psd
2018-09-11 09:45:04: 0: " with "Tile2FileName=\\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled_tile_1x2_2x2_0000.psd
2018-09-11 09:45:04: 0: "
2018-09-11 09:45:04: 0: CheckPathMapping: Swapped "Tile3FileName=P:\TEMP\DEADLINE_TEST\180911\test_tiled_tile_2x2_2x2_0000.psd
2018-09-11 09:45:04: 0: " with "Tile3FileName=\\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled_tile_2x2_2x2_0000.psd
2018-09-11 09:45:04: 0: "
2018-09-11 09:45:04: 0: INFO: Stdout Redirection Enabled: True
2018-09-11 09:45:04: 0: INFO: Stdout Handling Enabled: True
2018-09-11 09:45:04: 0: INFO: Popup Handling Enabled: False
2018-09-11 09:45:04: 0: INFO: Using Process Tree: True
2018-09-11 09:45:04: 0: INFO: Hiding DOS Window: True
2018-09-11 09:45:04: 0: INFO: Creating New Console: False
2018-09-11 09:45:04: 0: INFO: Running as user: jmk_server
2018-09-11 09:45:04: 0: INFO: Looking for bundled python at: 'C:\Program Files\Thinkbox\Deadline10\bin\dpython.exe'
2018-09-11 09:45:04: 0: INFO: Executable: "C:\Program Files\Thinkbox\Deadline10\bin\dpython.exe"
2018-09-11 09:45:04: 0: INFO: Argument: "C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\plugins\5b9771c90c89bc2f68e21e0e\Assembler.py" "C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\jobsData\5b9771c90c89bc2f68e21e0e\thread0_tempuub2n0\test_tiled0000_config_18_09_11_09_41_57.txt" True False True "C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\jobsData\5b9771c90c89bc2f68e21e0e\AssembledImage_tempBMPO20"
2018-09-11 09:45:04: 0: INFO: Full Command: "C:\Program Files\Thinkbox\Deadline10\bin\dpython.exe" "C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\plugins\5b9771c90c89bc2f68e21e0e\Assembler.py" "C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\jobsData\5b9771c90c89bc2f68e21e0e\thread0_tempuub2n0\test_tiled0000_config_18_09_11_09_41_57.txt" True False True "C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\jobsData\5b9771c90c89bc2f68e21e0e\AssembledImage_tempBMPO20"
2018-09-11 09:45:04: 0: INFO: Startup Directory: "C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\Draft"
2018-09-11 09:45:04: 0: INFO: Process Priority: BelowNormal
2018-09-11 09:45:04: 0: INFO: Process Affinity: default
2018-09-11 09:45:04: 0: INFO: Process is now running
2018-09-11 09:45:04: 0: STDOUT: Draft 1.6.8
2018-09-11 09:45:04: 0: STDOUT: Checking for a license...
2018-09-11 09:45:05: 0: STDOUT: Error on missing tiles = True
2018-09-11 09:45:05: 0: STDOUT: Error on missing Background = True
2018-09-11 09:45:05: 0: STDOUT: Temporary Folder = "C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\jobsData\5b9771c90c89bc2f68e21e0e\AssembledImage_tempBMPO20"
2018-09-11 09:45:05: 0: STDOUT: Distance as pixels = False
2018-09-11 09:45:05: 0: STDOUT: Assembling Single File
2018-09-11 09:45:05: 0: STDOUT: Assembling File: \\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled0000.psd
2018-09-11 09:45:05: 0: STDOUT: Writing File to temporary location: C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\jobsData\5b9771c90c89bc2f68e21e0e\AssembledImage_tempBMPO20\test_tiled0000.psd
2018-09-11 09:45:05: 0: STDOUT: Tiles are not cropped
2018-09-11 09:45:05: 0: STDOUT: Image height and width not provided. Final image dimensions will be based off first tile provided.
2018-09-11 09:45:05: 0: STDOUT: TileCount given: 4
2018-09-11 09:45:05: 0: STDOUT: Using height and width of image: \\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled_tile_1x1_2x2_0000.psd
2018-09-11 09:45:05: 0: STDOUT: Final Image Dimensions = 6000x6000
2018-09-11 09:45:05: 0: STDOUT: Tile0 cropping area: [Left: 0, Right: 3000, Top: 6000, Bottom: 3000]
2018-09-11 09:45:05: 0: STDOUT: Tile0 position: 0, 3000
2018-09-11 09:45:05: 0: STDOUT: Tile0 dimension: 3000, 3000
2018-09-11 09:45:05: 0: STDOUT: Tile1 cropping area: [Left: 3000, Right: 6000, Top: 6000, Bottom: 3000]
2018-09-11 09:45:05: 0: STDOUT: Tile1 position: 3000, 3000
2018-09-11 09:45:05: 0: STDOUT: Tile1 dimension: 3000, 3000
2018-09-11 09:45:05: 0: STDOUT: Tile2 cropping area: [Left: 0, Right: 3000, Top: 3000, Bottom: 0]
2018-09-11 09:45:05: 0: STDOUT: Tile2 position: 0, 0
2018-09-11 09:45:05: 0: STDOUT: Tile2 dimension: 3000, 3000
2018-09-11 09:45:05: 0: STDOUT: Tile3 cropping area: [Left: 3000, Right: 6000, Top: 3000, Bottom: 0]
2018-09-11 09:45:05: 0: STDOUT: Tile3 position: 3000, 0
2018-09-11 09:45:05: 0: STDOUT: Tile3 dimension: 3000, 3000
2018-09-11 09:45:05: 0: STDOUT: Total Number of Output Channels: 3
2018-09-11 09:45:05: 0: STDOUT: Output Channels:
2018-09-11 09:45:05: 0: STDOUT: R
2018-09-11 09:45:05: 0: STDOUT: G
2018-09-11 09:45:05: 0: STDOUT: B
2018-09-11 09:45:16: 0: STDOUT: Assembling Progress: 16%
2018-09-11 09:45:16: 0: STDOUT: Assembling Progress: 32%
2018-09-11 09:45:17: 0: STDOUT: Assembling Progress: 49%
2018-09-11 09:45:26: 0: STDOUT: Assembling Progress: 65%
2018-09-11 09:45:27: 0: STDOUT: Assembling Progress: 81%
2018-09-11 09:45:27: 0: STDOUT: Assembling Progress: 97%
2018-09-11 09:45:28: 0: STDOUT: Assembling Progress: 100%
2018-09-11 09:45:29: 0: STDOUT: Copying output file from: C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\jobsData\5b9771c90c89bc2f68e21e0e\AssembledImage_tempBMPO20\test_tiled0000.psd to final output location: \\fileserver\jmk_server\TEMP\DEADLINE_TEST\180911\test_tiled0000.psd
2018-09-11 09:45:29: 0: STDOUT: Deleting: 'C:\Users\jmk_server\AppData\Local\Thinkbox\Deadline10\slave\fileserver\jobsData\5b9771c90c89bc2f68e21e0e\AssembledImage_tempBMPO20\test_tiled0000.psd'
2018-09-11 09:45:29: 0: STDOUT: Not cleaning up tiles
2018-09-11 09:45:29: 0: INFO: Process exit code: 0
2018-09-11 09:45:29: 0: INFO: Draft Tile Assembler Job Completed
2018-09-11 09:45:29: 0: Done executing plugin command of type 'Render Task'
=======================================================
Details
=======================================================
Date: 09/11/2018 09:45:30
Frames: 0
Job Submit Date: 09/11/2018 09:42:01
Job User: jmk_server
Average RAM Usage: 1054075136 (4%)
Peak RAM Usage: 1994989568 (6%)
Average CPU Usage: 2%
Peak CPU Usage: 2%
Used CPU Clocks (x10^6 cycles): 22967
Total CPU Clocks (x10^6 cycles): 1148306
=======================================================
Slave Information
=======================================================
Slave Name: Fileserver
Version: v10.0.19.0 Release (52defb510)
Operating System: Windows Server 2016 Essentials
Running As Service: No
Machine User: jmk_server
IP Address: 192.168.1.90
MAC Address: 34:97:F6:5B:1D:59
CPU Architecture: x64
CPUs: 40
CPU Usage: 2%
Memory Usage: 11.6 GB / 31.9 GB (36%)
Free Disk Space: 14.618 TB (111.915 GB on C:\, 14.509 TB on D:\)
Video Card: Microsoft Basic Display Adapter

#4

Hmm. It looks like there’s only the beauty channels in the PSD. I’d expect to see more than just these three:

2018-09-11 09:45:05: 0: STDOUT: Total Number of Output Channels: 3
2018-09-11 09:45:05: 0: STDOUT: Output Channels:
2018-09-11 09:45:05: 0: STDOUT: R
2018-09-11 09:45:05: 0: STDOUT: G
2018-09-11 09:45:05: 0: STDOUT: B

Could you send a test render with all the tiles and the config.txt out of the assembly job’s Aux folder (right-click the job to find that one)? You’ll want to use a tool that can view all the baked in layers here.

I guess a point of clarification: Does each tile have multiple files, or do the single files have all the layers? Given it’s PSD I assume it should all be baked in together.


#5

after a few attempts to reply I try it the way… I uploaded a little zip file on my server which contains all data you need - i.e. modo file, rendered tiles, reports etc.

[link removed by eamsler]


#6

Did zip files not work?! Can you PM me so we can work through why?


#7

I’m not allowed to upload files - I don#t know why… how do i PM you? is there a Button i can’t see possibly?

could you download the Zip file from my server?

chaars
Jörg-Mark


#8

You can click on my username then hit the “Message” button. It looks like this:

image


#9

no, I’t seems I#m not allowed to…

can we continue this conversation via e-mail?

for some reason my rights on this forum seem to be damaged or not working right… maybe you can fix this too?


#10

Hmm. Your account seems fine…

Feel free to send an e-mail to support@thinkboxsoftware.com in the mean time while we figure this out. There are trust levels to try and combat spam bots and it may be that the ability to post file uploads are impacted by this.


#11

Update here: We should have fixed the uploads. It seems there indeed was a limit for brand new users. It should work now.

Thinking through this, have you confirmed that it fails for EXRs as well? Are there more than three channels there? You’re the first person I’ve dealt with who is tile rendering PSDs so it may just not be supported yet and I’ll need to create a feature request. I also don’t have access to Photoshop so checking that there are in fact multiple layers is going to be tricky. :smiley:


#12

I tried the same file as EXR. The Main Render Works fine but the Draftjob does fail. Please find attched the report - If i understand it right Draft tries to use the pathmapping which is not right because i use Windows only in this case.

If you do not have Photoshop available you can use KRITA which is a free application…
Job_2018-09-19_12-36-39_5ba226b73016d40da0341d0f.zip (2.5 KB)

https://krita.org/en/

since zip is working now - here the files again (PSD VERSION // EXR VERSION)

test_.zip (845.9 KB)