Hey everyone,
I’m not new to the use of Deadline, but a little green with the setup for a new farm (so maybe that’s the source of my issue).
I have a small 3 machine setup that ultimately will be for AE and C4D, but at the moment, AE is failing on me and I’m not sure why. I’ve been digging all over this site and others and can’t seem to be able to nail down the problem.
I have a shared drive on one machine that all machine can see/read/write to. If I fire off a local render that writes frames to the network share, everything works as expected. However, if I submit the job to Deadline, 2 of the workers fail. The worker that is on the machine with the shared drive hardlined internally in the box, renders fine.
Here’s the log of the failure:
Error
Error: After Effects returned non-zero error code: -1073741819. Check the render log.
at Deadline.Plugins.PluginWrapper.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)
=======================================================
Type
RenderPluginException
=======================================================
Stack Trace
at Deadline.Plugins.SandboxedPlugin.d(DeadlineMessage bas)
at Deadline.Plugins.SandboxedPlugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.c(TaskLogWriter adz)
=======================================================
Log
2020-01-24 09:25:58: 0: Loading Job’s Plugin timeout is Disabled
2020-01-24 09:25:59: 0: Executing plugin command of type ‘Sync Files for Job’
2020-01-24 09:25:59: 0: All job files are already synchronized
2020-01-24 09:25:59: 0: Plugin AfterEffects was already synchronized.
2020-01-24 09:25:59: 0: Done executing plugin command of type ‘Sync Files for Job’
2020-01-24 09:26:00: 0: Executing plugin command of type ‘Initialize Plugin’
2020-01-24 09:26:00: 0: INFO: Executing plugin script ‘C:\Users\Stimulated Inc 001\AppData\Local\Thinkbox\Deadline10\slave\STMLTD-001\plugins\5e2b2893fdfdba18648615c9\AfterEffects.py’
2020-01-24 09:26:00: 0: INFO: Fail Without Finished Message set to: False
2020-01-24 09:26:00: 0: INFO: Fail on Existing After Effects process: False
2020-01-24 09:26:00: 0: INFO: About: After Effects Plugin for Deadline
2020-01-24 09:26:00: 0: INFO: Render Job As User disabled, running as current user ‘Stimulated Inc 001’
2020-01-24 09:26:00: 0: INFO: The job’s environment will be merged with the current environment before rendering
2020-01-24 09:26:00: 0: Done executing plugin command of type ‘Initialize Plugin’
2020-01-24 09:26:00: 0: Start Job timeout is disabled.
2020-01-24 09:26:00: 0: Task timeout is disabled.
2020-01-24 09:26:00: 0: Loaded job: AE_Test.aep - rockTitle (5e2b2893fdfdba18648615c9)
2020-01-24 09:26:00: 0: Executing plugin command of type ‘Start Job’
2020-01-24 09:26:00: 0: DEBUG: S3BackedCache Client is not installed.
2020-01-24 09:26:00: 0: INFO: Executing global asset transfer preload script ‘C:\Users\Stimulated Inc 001\AppData\Local\Thinkbox\Deadline10\slave\STMLTD-001\plugins\5e2b2893fdfdba18648615c9\GlobalAssetTransferPreLoad.py’
2020-01-24 09:26:00: 0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer…
2020-01-24 09:26:00: 0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in C:/Program Files/Thinkbox/S3BackedCache/bin/task.py…
2020-01-24 09:26:00: 0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2020-01-24 09:26:00: 0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2020-01-24 09:26:00: 0: Done executing plugin command of type ‘Start Job’
2020-01-24 09:26:00: 0: Plugin rendering frame(s): 20-29
2020-01-24 09:26:00: 0: Executing plugin command of type ‘Render Task’
2020-01-24 09:26:00: 0: INFO: Attempting to create “C:\Users\Stimulated Inc 001\Documents\ae_render_only_node.txt” to force After Effects to run in Render Engine mode
2020-01-24 09:26:00: 0: INFO: Stdout Redirection Enabled: True
2020-01-24 09:26:00: 0: INFO: Stdout Handling Enabled: True
2020-01-24 09:26:00: 0: INFO: Popup Handling Enabled: True
2020-01-24 09:26:00: 0: INFO: QT Popup Handling Enabled: False
2020-01-24 09:26:00: 0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2020-01-24 09:26:00: 0: INFO: Using Process Tree: True
2020-01-24 09:26:00: 0: INFO: Hiding DOS Window: True
2020-01-24 09:26:00: 0: INFO: Creating New Console: False
2020-01-24 09:26:00: 0: INFO: Running as user: Stimulated Inc 001
2020-01-24 09:26:00: 0: INFO: Executable: “C:\Program Files\Adobe\Adobe After Effects CC 2019\Support Files\aerender.exe”
2020-01-24 09:26:00: 0: INFO: Rendering to local drive, will copy files and folders to final location after render is complete
2020-01-24 09:26:00: 0: INFO: Validating the path: ‘Y:_PROJECTS_AE_Test’
2020-01-24 09:26:00: 0: INFO: Argument: -project “C:\Users\Stimulated Inc 001\AppData\Local\Thinkbox\Deadline10\slave\STMLTD-001\jobsData\5e2b2893fdfdba18648615c9\AE_Test.aep” -comp “rockTitle” -s 20 -e 29 -output “C:\Users\Stimulated Inc 001\AppData\Local\Thinkbox\Deadline10\slave\STMLTD-001\jobsData\5e2b2893fdfdba18648615c9\AfterEffectsOutput_temp5dICP0\rockTitle_[#####].png” -mem_usage 100 100 -v ERRORS_AND_PROGRESS -close DO_NOT_SAVE_CHANGES -sound OFF
2020-01-24 09:26:00: 0: INFO: Full Command: “C:\Program Files\Adobe\Adobe After Effects CC 2019\Support Files\aerender.exe” -project “C:\Users\Stimulated Inc 001\AppData\Local\Thinkbox\Deadline10\slave\STMLTD-001\jobsData\5e2b2893fdfdba18648615c9\AE_Test.aep” -comp “rockTitle” -s 20 -e 29 -output “C:\Users\Stimulated Inc 001\AppData\Local\Thinkbox\Deadline10\slave\STMLTD-001\jobsData\5e2b2893fdfdba18648615c9\AfterEffectsOutput_temp5dICP0\rockTitle_[#####].png” -mem_usage 100 100 -v ERRORS_AND_PROGRESS -close DO_NOT_SAVE_CHANGES -sound OFF
2020-01-24 09:26:00: 0: INFO: Startup Directory: “C:\Program Files\Adobe\Adobe After Effects CC 2019\Support Files”
2020-01-24 09:26:00: 0: INFO: Process Priority: BelowNormal
2020-01-24 09:26:00: 0: INFO: Process Affinity: default
2020-01-24 09:26:00: 0: INFO: Process is now running
2020-01-24 09:26:01: 0: STDOUT: Using DXGI: Device: “NVIDIA GeForce RTX 2080 Ti” has dedicated video RAM (MB): 11048
2020-01-24 09:26:01: 0: STDOUT: 2020-01-24 09:26:01:095 : ERROR *** IVCAutoReleasePool: Object 0000000022591960 of class vcfoundation::data::DataWrapper autoreleased with no pool in place - just leaking
2020-01-24 09:26:01: 0: STDOUT: 2020-01-24 09:26:01:095 : ERROR *** IVCAutoReleasePool: Object 0000000022591EA0 of class vcfoundation::data::DataWrapper autoreleased with no pool in place - just leaking
2020-01-24 09:26:02: 0: STDOUT: Unable to read VR Path Registry from C:\Users\Stimulated Inc 001\AppData\Local\openvr\openvrpaths.vrpath
2020-01-24 09:26:02: 0: STDOUT: InitializeFileLogging failed
2020-01-24 09:26:02: 0: STDOUT: name : NVIDIA GeForce RTX 2080 Ti
2020-01-24 09:26:02: 0: STDOUT: VendorId : 4318
2020-01-24 09:26:02: 0: STDOUT: DeviceId : 7687
2020-01-24 09:26:02: 0: STDOUT: SubSysId : 2255097923
2020-01-24 09:26:02: 0: STDOUT: Revision : 161
2020-01-24 09:26:02: 0: STDOUT: DedicatedVideoMemory : 11584667648
2020-01-24 09:26:02: 0: STDOUT: DedicatedSystemMemory : 0
2020-01-24 09:26:02: 0: STDOUT: SharedSystemMemory : 68692090880
2020-01-24 09:26:02: 0: STDOUT: MemoryBudget : 9846967500
2020-01-24 09:26:02: 0: STDOUT: AdapterLuid.LowPart : 55651
2020-01-24 09:26:03: 0: STDOUT: AdapterLuid.HighPart : 0
2020-01-24 09:26:03: 0: STDOUT: Flags : 0
2020-01-24 09:26:03: 0: STDOUT: DoublePrecisionFloatShaderOps : yes
2020-01-24 09:26:03: 0: STDOUT: ConservativeRasterizationTier : 3
2020-01-24 09:26:03: 0: STDOUT: CrossNodeSharingTier : 0
2020-01-24 09:26:03: 0: STDOUT: ResourceBindingTier : 3
2020-01-24 09:26:03: 0: STDOUT: ResourceHeapTier : 2
2020-01-24 09:26:03: 0: STDOUT: TiledResourcesTier : 3
2020-01-24 09:26:03: 0: STDOUT: NodeIndex : 0
2020-01-24 09:26:03: 0: STDOUT: TileBasedRenderer : no
2020-01-24 09:26:03: 0: STDOUT: UMA : no
2020-01-24 09:26:03: 0: STDOUT: CacheCoherentUMA : no
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x00009100
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x00009200
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x00009300
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x0000a000
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x0000a100
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x0000b000
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x0000b100
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x0000c000
2020-01-24 09:26:03: 0: STDOUT: feature level : 0x0000c100
2020-01-24 09:26:03: 0: STDOUT: MaxSupportedFeatureLevel : 0x0000c100
2020-01-24 09:26:03: 0: STDOUT: HighestShaderModel : 0x60
2020-01-24 09:26:03: 0: STDOUT: WaveOps : yes
2020-01-24 09:26:03: 0: STDOUT: WaveLaneCountMin : 32
2020-01-24 09:26:03: 0: STDOUT: WaveLaneCountMax : 32
2020-01-24 09:26:03: 0: STDOUT: *** GPU Warning: GPU3 failed (previous) sanity test ***
2020-01-24 09:26:03: 0: STDOUT: TotalLaneCount : 4352
2020-01-24 09:26:03: 0: STDOUT: ExpandedComputeResourceStates : yes
2020-01-24 09:26:03: 0: STDOUT: Int64ShaderOps : yes
2020-01-24 09:26:03: 0: STDOUT: RootSignature.HighestVersion : 0x2
2020-01-24 09:26:03: 0: STDOUT: 20-01-24 09:26:03 [INF] Prefs [session|0]: 1 preferences items found.
2020-01-24 09:26:03: 0: STDOUT: 20-01-24 09:26:03 [INF] Prefs [client|1]: 3 preferences items found.
2020-01-24 09:26:06: 0: INFO: Submitted from After Effects version: 16.1.3x5
2020-01-24 09:26:06: 0: STDOUT: aerender version 16.1.3x5
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 1/24/2020 9:26:06 AM: Starting composition rockTitle.
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Render Settings: Best Settings
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Quality: Best
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Resolution: Full
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Size: 430 x 80
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Proxy Use: Use No Proxies
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Effects: Current Settings
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Disk Cache: Read Only
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Color Depth: Current Settings
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Frame Blending: On for Checked Layers
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Field Render: Off
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Pulldown: Off
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Motion Blur: On for Checked Layers
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Solos: Current Settings
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Time Span: Custom
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Start: 00020
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: End: 00029
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Duration: 00010
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Frame Rate: 29.97 (comp)
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Guide Layers: All Off
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Skip Existing Files: Off
2020-01-24 09:26:06: 0: STDOUT: PROGRESS:
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Output Module: Lossless
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Output To: C:\Users\Stimulated Inc 001\AppData\Local\Thinkbox\Deadline10\slave\STMLTD-001\jobsData\5e2b2893fdfdba18648615c9\AfterEffectsOutput_temp5dICP0\rockTitle_[#####].png
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Format: PNG
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Output Info: -
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Start Frame: 20
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Output Audio: -
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Channels: RGB
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Depth: Millions of Colors
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Color: Premultiplied
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Resize: -
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Crop: -
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Final Size: 430 x 80
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Profile: -
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Post-Render Action: None
2020-01-24 09:26:06: 0: STDOUT: PROGRESS:
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00020 (1): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00021 (2): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00022 (3): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00023 (4): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00024 (5): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00025 (6): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00026 (7): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00027 (8): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00028 (9): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 00029 (10): 0 Seconds
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: 1/24/2020 9:26:06 AM: Finished composition rockTitle.
2020-01-24 09:26:06: 0: STDOUT: PROGRESS: Total Time Elapsed: 0 Seconds
2020-01-24 09:26:11: 0: INFO: Process exit code: -1073741819
2020-01-24 09:26:11: 0: Done executing plugin command of type ‘Render Task’
=======================================================
Details
Date: 01/24/2020 09:26:13
Frames: 20-29
Elapsed Time: 00:00:00:15
Job Submit Date: 01/24/2020 09:25:39
Job User: stimulated inc 001
Average RAM Usage: 7414321664 (6%)
Peak RAM Usage: 7414321152 (6%)
Average CPU Usage: 3%
Peak CPU Usage: 9%
Used CPU Clocks (x10^6 cycles): 38764
Total CPU Clocks (x10^6 cycles): 1292114
=======================================================
Worker Information
Worker Name: STMLTD-001
Version: v10.1.2.2 Release (d9f8db73b)
Operating System: Windows 10 Pro
Running As Service: No
Machine User: Stimulated Inc 001
IP Address: 2602:304:5ea2:e390::38
MAC Address: 00:D8:61:D2:CE:8E
CPU Architecture: x64
CPUs: 32
CPU Usage: 1%
Memory Usage: 6.9 GB / 127.9 GB (5%)
Free Disk Space: 6.947 TB (773.690 GB on C:, 5.458 TB on D:, 751.026 GB on E:)
Video Card: NVIDIA GeForce RTX 2080 Ti
I’ve seen this very error listed but the solution in the doc doesn’t seem to work. Even the doc seems to go with the assumption that files are still being written despite the error and that code is just to ignore the error message.
My network workers fail out. There isn’t anything produced by them.
Hopefully someone out there has some insight. Thanks in advance.