After Effects failed multicore render on one machine

[Mindaugas_Suchockas]

Hey all!

Just as a disclaimer - I’m totally green with deadline. I’ve been stuck on this error for couple of days now and would really appreciate your help!

I want to render heavier AE projects on multiple cores on one machine that I have.

Deadline renders the tasks but some have errors and get queued again, it trys to redo them and this goes on an on. Also the whole process is quite slow, I think I’d be faster running two copies of AE and rendering this way.

What am I doing wrong, and what’s causing the errors?

Here is the log:

=======================================================
Error

*** Error: InitializeWindowsML failed ***
at Deadline.Plugins.PluginWrapper.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type

RenderPluginException

=======================================================
Stack Trace

at Deadline.Plugins.SandboxedPlugin.d(DeadlineMessage bay)
at Deadline.Plugins.SandboxedPlugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.c(TaskLogWriter adv)

=======================================================
Log

2020-02-14 12:35:03: 1: Loading Job’s Plugin timeout is Disabled
2020-02-14 12:35:09: 1: Executing plugin command of type ‘Sync Files for Job’
2020-02-14 12:35:09: 1: Start Job timeout is disabled.
2020-02-14 12:35:09: 1: Task timeout is disabled.
2020-02-14 12:35:09: 1: Loaded job: test_02.aep - Comp 1 (5e4685beea581f617c14f304)
2020-02-14 12:35:09: 1: Plugin rendering frame(s): 50-99
2020-02-14 12:35:11: 1: All job files are already synchronized
2020-02-14 12:35:11: 1: Plugin AfterEffects was already synchronized.
2020-02-14 12:35:11: 1: Done executing plugin command of type ‘Sync Files for Job’
2020-02-14 12:35:11: 1: Executing plugin command of type ‘Initialize Plugin’
2020-02-14 12:35:11: 1: INFO: Executing plugin script ‘C:\Users\Mindaugas\AppData\Local\Thinkbox\Deadline10\slave\Mindes-PC\plugins\5e4685beea581f617c14f304\AfterEffects.py’
2020-02-14 12:35:11: 1: INFO: Fail Without Finished Message set to: False
2020-02-14 12:35:11: 1: INFO: Fail on Existing After Effects process: False
2020-02-14 12:35:11: 1: INFO: About: After Effects Plugin for Deadline
2020-02-14 12:35:11: 1: INFO: Render Job As User disabled, running as current user ‘Mindaugas’
2020-02-14 12:35:11: 1: INFO: The job’s environment will be merged with the current environment before rendering
2020-02-14 12:35:11: 1: Done executing plugin command of type ‘Initialize Plugin’
2020-02-14 12:35:11: 1: Executing plugin command of type ‘Start Job’
2020-02-14 12:35:11: 1: DEBUG: S3BackedCache Client is not installed.
2020-02-14 12:35:11: 1: INFO: Executing global asset transfer preload script ‘C:\Users\Mindaugas\AppData\Local\Thinkbox\Deadline10\slave\Mindes-PC\plugins\5e4685beea581f617c14f304\GlobalAssetTransferPreLoad.py’
2020-02-14 12:35:11: 1: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer…
2020-02-14 12:35:11: 1: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in C:/Program Files/Thinkbox/S3BackedCache/bin/task.py…
2020-02-14 12:35:11: 1: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2020-02-14 12:35:11: 1: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2020-02-14 12:35:11: 1: Done executing plugin command of type ‘Start Job’
2020-02-14 12:35:11: 1: Executing plugin command of type ‘Render Task’
2020-02-14 12:35:11: 1: INFO: Attempting to create “C:\Users\Mindaugas\Documents\ae_render_only_node.txt” to force After Effects to run in Render Engine mode
2020-02-14 12:35:11: 1: INFO: Stdout Redirection Enabled: True
2020-02-14 12:35:11: 1: INFO: Stdout Handling Enabled: True
2020-02-14 12:35:11: 1: INFO: Popup Handling Enabled: True
2020-02-14 12:35:11: 1: INFO: QT Popup Handling Enabled: False
2020-02-14 12:35:11: 1: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2020-02-14 12:35:11: 1: INFO: Using Process Tree: True
2020-02-14 12:35:11: 1: INFO: Hiding DOS Window: True
2020-02-14 12:35:11: 1: INFO: Creating New Console: False
2020-02-14 12:35:11: 1: INFO: Running as user: Mindaugas
2020-02-14 12:35:11: 1: INFO: Executable: “C:\Program Files\Adobe After Effects 2020\Support Files\aerender.exe”
2020-02-14 12:35:11: 1: INFO: Argument: -project “C:\Users\Mindaugas\Desktop\Render_Tests\test_02.aep” -comp “Comp 1” -s 50 -e 99 -mp -v ERRORS_AND_PROGRESS -close DO_NOT_SAVE_CHANGES -sound OFF
2020-02-14 12:35:11: 1: INFO: Full Command: “C:\Program Files\Adobe After Effects 2020\Support Files\aerender.exe” -project “C:\Users\Mindaugas\Desktop\Render_Tests\test_02.aep” -comp “Comp 1” -s 50 -e 99 -mp -v ERRORS_AND_PROGRESS -close DO_NOT_SAVE_CHANGES -sound OFF
2020-02-14 12:35:11: 1: INFO: Startup Directory: “C:\Program Files\Adobe After Effects 2020\Support Files”
2020-02-14 12:35:11: 1: INFO: Process Priority: BelowNormal
2020-02-14 12:35:11: 1: INFO: Process Affinity: default
2020-02-14 12:35:11: 1: INFO: Process is now running
2020-02-14 12:35:11: 1: INFO: Submitted from After Effects version: 17.0.2x26
2020-02-14 12:35:11: 1: STDOUT: aerender version 17.0.2x26
2020-02-14 12:35:11: 1: STDOUT: Using DXGI: Device: “NVIDIA GeForce GTX 780 Ti” has dedicated video RAM (MB): 3029
2020-02-14 12:35:11: 1: STDOUT: InitializeFileLogging failed
2020-02-14 12:35:11: 1: STDOUT: name : NVIDIA GeForce GTX 780 Ti
2020-02-14 12:35:11: 1: STDOUT: VendorId : 4318
2020-02-14 12:35:11: 1: STDOUT: DeviceId : 4106
2020-02-14 12:35:11: 1: STDOUT: SubSysId : 320084442
2020-02-14 12:35:11: 1: STDOUT: Revision : 161
2020-02-14 12:35:11: 1: STDOUT: DedicatedVideoMemory : 3176136704
2020-02-14 12:35:11: 1: STDOUT: DedicatedSystemMemory : 0
2020-02-14 12:35:11: 1: STDOUT: SharedSystemMemory : 17141514240
2020-02-14 12:35:11: 1: STDOUT: MemoryBudget : 2699716198
2020-02-14 12:35:11: 1: STDOUT: AdapterLuid.LowPart : 50122
2020-02-14 12:35:11: 1: STDOUT: AdapterLuid.HighPart : 0
2020-02-14 12:35:11: 1: STDOUT: Flags : 0
2020-02-14 12:35:11: 1: STDOUT: DoublePrecisionFloatShaderOps : yes
2020-02-14 12:35:11: 1: STDOUT: ConservativeRasterizationTier : 0
2020-02-14 12:35:11: 1: STDOUT: CrossNodeSharingTier : 0
2020-02-14 12:35:11: 1: STDOUT: ResourceBindingTier : 2
2020-02-14 12:35:11: 1: STDOUT: ResourceHeapTier : 1
2020-02-14 12:35:11: 1: STDOUT: TiledResourcesTier : 1
2020-02-14 12:35:11: 1: STDOUT: NodeIndex : 0
2020-02-14 12:35:11: 1: STDOUT: TileBasedRenderer : no
2020-02-14 12:35:11: 1: STDOUT: UMA : no
2020-02-14 12:35:11: 1: STDOUT: CacheCoherentUMA : no
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x00009100
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x00009200
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x00009300
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x0000a000
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x0000a100
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x0000b000
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x0000b100
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x0000c000
2020-02-14 12:35:11: 1: STDOUT: feature level : 0x0000c100
2020-02-14 12:35:11: 1: STDOUT: MaxSupportedFeatureLevel : 0x0000b000
2020-02-14 12:35:11: 1: STDOUT: HighestShaderModel : 0x60
2020-02-14 12:35:11: 1: STDOUT: WaveOps : yes
2020-02-14 12:35:11: 1: STDOUT: WaveLaneCountMin : 32
2020-02-14 12:35:11: 1: STDOUT: WaveLaneCountMax : 32
2020-02-14 12:35:11: 1: STDOUT: TotalLaneCount : 2880
2020-02-14 12:35:11: 1: STDOUT: ExpandedComputeResourceStates : yes
2020-02-14 12:35:11: 1: STDOUT: Int64ShaderOps : yes
2020-02-14 12:35:11: 1: STDOUT: RootSignature.HighestVersion : 0x2
2020-02-14 12:35:11: 1: STDOUT: *** GPU Warning: GPU3 failed (previous) sanity test ***
2020-02-14 12:35:11: 1: INFO: Checking for file “C:\Users\Mindaugas\Documents\ae_render_only_node.txt”
2020-02-14 12:35:11: 1: INFO: Attempting to delete “C:\Users\Mindaugas\Documents\ae_render_only_node.txt” to allow After Effects to run in the workstation environment
2020-02-14 12:35:11: 1: STDOUT: *** Error: InitializeWindowsML failed ***
2020-02-14 12:35:11: 1: STDOUT: 20-02-14 12:35:11 [INF] Prefs [session|0]: 1 preferences items found.
2020-02-14 12:35:11: 1: STDOUT: 20-02-14 12:35:11 [INF] Prefs [client|1]: 3 preferences items found.
2020-02-14 12:35:11: 1: Done executing plugin command of type ‘Render Task’

=======================================================
Details

Date: 02/14/2020 12:35:13
Frames: 50-99
Elapsed Time: 00:00:00:11
Job Submit Date: 02/14/2020 12:34:22
Job User: mindaugas
Average RAM Usage: 11493195776 (34%)
Peak RAM Usage: 12008124416 (36%)
Average CPU Usage: 56%
Peak CPU Usage: 100%
Used CPU Clocks (x10^6 cycles): 125733
Total CPU Clocks (x10^6 cycles): 224522

=======================================================
Worker Information

Worker Name: Mindes-PC
Version: v10.1.3.6 Release (871932b26)
Operating System: Windows 10 Pro
Running As Service: No
Machine User: Mindaugas
IP Address: 192.168.1.107
MAC Address: FC:AA:14:20:29:BA
CPU Architecture: x64
CPUs: 8
CPU Usage: 3%
Memory Usage: 9.5 GB / 31.9 GB (29%)
Free Disk Space: 1.037 TB (203.409 GB on C:, 554.299 GB on D:, 197.989 GB on E:, 105.690 GB on G:)
Video Card: NVIDIA GeForce GTX 780 Ti

[Jason_Ian_Yee]

I am doing tests with Adobe After Effects 2020 and getting that same error if concurrent tasks is more than 1. Earlier versions work great on the same nodes, so I am not sure.

I have the latest Deadline installed, and out of the box, the plugin path to aerender render executable was wrong as Adobe removed the “CC” from its path.

My only troubleshooting was to google the error and this is only post on the internet!

thanks

Jason

[kavi]

Hi,

There is a chance that this error is non-fatal and AE render could still complete. In that case,
add the following line with the other self.AddStdoutHandlerCallback lines in <Repo>/plugins/AfterEffects/AfterEffects.py :

self.AddStdoutHandlerCallback( ".*InitializeWindowsML failed.*" ).HandleCallback += self.HandleDummyInf

This will make Deadline skip this and continue rendering.

Regards,
Kavi

[Jason_Ian_Yee]

Hi Kavi,

I have noticed this error even with single task per machine renders. It will retry and eventually render ok, but it is adding to the error count.

I want to try your suggestion.

However, I do not have HandleDummyInf defined.

Initialize: Error: AttributeError : ‘AfterEffectsPlugin’ object has no attribute ‘HandleDummyInf’ (Python.Runtime.PythonException)

Should I just add something like?

def HandleDummyInf ( self ):
     pass

thanks,

Jason

[Jason_Ian_Yee]

Does not seem to make any difference. Basically the first tasks to a render node will fail with that error. Next attempts on that node will be ok.

Occasionally, just before the

2020-09-23 13:03:29: 1: STDOUT: *** Error: InitializeWindowsML failed ***

error, there would be this error

2020-09-23 13:03:29: 1: STDOUT: *** GPU Error: SafeLoad failed ***

thanks

Jason