After Effects Issues

We are having an issue with After Effects submissions. Of 40 nodes about 30% give this error at any one time.

I can’t find a pattern to the nodes but all nodes have given the error at one point.

Some render do still wok but other just give this error and never start. Others reach 100% but never finish or transfer files.

I know that a missing effect is listed in this log however I’ve enabled Ignoring missing effect references. It should not be an issue as far as I know.

Using AE 2015 V13.7.1.6

Thanks for any help.

=======================================================
Error

Error: Renderer returned non-zero error code, -1073741811. Check the log for more information.
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type

RenderPluginException

=======================================================
Stack Trace

at Deadline.Plugins.Plugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.a(TaskLogWriter A_0)

=======================================================
Log

2016-04-19 09:08:00: BEGIN - MT-RENDER24\mtrender
2016-04-19 09:08:00: 0: Start Job timeout is disabled.
2016-04-19 09:08:00: 0: Task timeout is disabled.
2016-04-19 09:08:00: 0: Loaded job: XXX
2016-04-19 09:08:00: 0: INFO: Fail on Existing After Effects process: 0
2016-04-19 09:08:00: 0: INFO: About: After Effects Plugin for Deadline
2016-04-19 09:08:00: 0: INFO: The job’s environment will be merged with the current environment before rendering
2016-04-19 09:08:00: 0: Plugin rendering frame(s): 380-389
2016-04-19 09:08:00: 0: INFO: Stdout Redirection Enabled: True
2016-04-19 09:08:00: 0: INFO: Stdout Handling Enabled: True
2016-04-19 09:08:00: 0: INFO: Popup Handling Enabled: True
2016-04-19 09:08:00: 0: INFO: QT Popup Handling Enabled: False
2016-04-19 09:08:00: 0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2016-04-19 09:08:00: 0: INFO: Using Process Tree: True
2016-04-19 09:08:00: 0: INFO: Hiding DOS Window: True
2016-04-19 09:08:00: 0: INFO: Creating New Console: False
2016-04-19 09:08:00: 0: INFO: Running as user: mtrender
2016-04-19 09:08:00: 0: INFO: Executable: “C:\Program Files\Adobe\Adobe After Effects CC 2015\Support Files\aerender.exe”
2016-04-19 09:08:00: 0: INFO: Rendering to local drive, will copy files and folders to final location after render is complete
2016-04-19 09:08:00: 0: INFO: Ignoring missing effect reference errors
2016-04-19 09:08:00: 0: INFO: Argument: XXX
2016-04-19 09:08:00: 0: INFO: Startup Directory: “C:\Program Files\Adobe\Adobe After Effects CC 2015\Support Files”
2016-04-19 09:08:00: 0: INFO: Process Priority: BelowNormal
2016-04-19 09:08:01: 0: INFO: Process Affinity: default
2016-04-19 09:08:01: 0: INFO: Process is now running
2016-04-19 09:08:01: 0: STDOUT: has video RAM(MB): 8
2016-04-19 09:08:04: 0: STDOUT: 2016-04-19 09:08:04 : INFO Environment: Scale factor from PP is 1.000000
2016-04-19 09:08:04: 0: STDOUT: 2016-04-19 09:08:04 : INFO Environment: m_scaleFactor is 1.000000
2016-04-19 09:08:21: 0: STDOUT: aerender version 13.5x347
2016-04-19 09:08:21: 0: STDOUT: WARNING:After Effects warning: Grouped_Message_{This project contains references to missing effects. Please install the following effects to restore these references.|||This project contains a reference to a missing effect. Please install the following effect to restore this reference.###@0 missing effects.}UnMult
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: XXX
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Render Settings: Best Settings
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Quality: Best
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Resolution: Full
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Size: 11300 x 2100
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Proxy Use: Use No Proxies
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Effects: Current Settings
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Disk Cache: Read Only
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Color Depth: Current Settings
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Frame Blending: On for Checked Layers
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Field Render: Off
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Pulldown: Off
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Motion Blur: On for Checked Layers
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Solos: Current Settings
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Time Span: Custom
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Start: 00380
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: End: 00389
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Duration: 00010
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Frame Rate: 30.00 (comp)
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Guide Layers: All Off
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Skip Existing Files: Off
2016-04-19 09:08:22: 0: STDOUT: PROGRESS:
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Output Module: Lossless
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: XXX
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Output Info: -
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Start Frame: 380
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Output Audio: -
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Channels: RGB
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Depth: Millions of Colors
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Color: Premultiplied
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Resize: -
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Crop: -
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Final Size: 11300 x 2100
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Profile: -
2016-04-19 09:08:22: 0: STDOUT: PROGRESS: Post-Render Action: None
2016-04-19 09:08:22: 0: STDOUT: PROGRESS:
2016-04-19 09:11:35: 0: STDOUT: PROGRESS: 00380 (1): 3 Min, 21 Sec
2016-04-19 09:14:50: 0: STDOUT: PROGRESS: 00381 (2): 3 Min, 22 Sec
2016-04-19 09:18:03: 0: STDOUT: PROGRESS: 00382 (3): 3 Min, 17 Sec
2016-04-19 09:21:47: 0: STDOUT: PROGRESS: 00383 (4): 3 Min, 46 Sec
2016-04-19 09:25:44: 0: STDOUT: PROGRESS: 00384 (5): 3 Min, 58 Sec
2016-04-19 09:29:19: 0: STDOUT: PROGRESS: 00385 (6): 3 Min, 34 Sec
2016-04-19 09:32:48: 0: STDOUT: PROGRESS: 00386 (7): 3 Min, 29 Sec
2016-04-19 09:36:16: 0: STDOUT: PROGRESS: 00387 (8): 3 Min, 28 Sec
2016-04-19 09:39:47: 0: STDOUT: PROGRESS: 00388 (9): 3 Min, 32 Sec
2016-04-19 09:43:20: 0: STDOUT: PROGRESS: 00389 (10): 3 Min, 33 Sec
2016-04-19 09:43:29: 0: INFO: Process exit code: -1073741811

=======================================================
Details

Date: 04/19/2016 09:43:31
Frames: 380-389
Elapsed Time: 00:00:35:52
Job Submit Date: 04/18/2016 15:16:29
Job User: XXX
Average RAM Usage: 38360301568 (75%)
Peak RAM Usage: 44153393152 (86%)
Average CPU Usage: 17%
Peak CPU Usage: 100%
Used CPU Clocks: 21110552
Total CPU Clocks: 124179720

=======================================================
Slave Information

Slave Name: MT-RENDER24
Version: v7.2.1.10 R (b8ff445b3)
Operating System: Windows 7 Professional (SP1)
Running As Service: No
Machine User: mtrender
IP Address: XXX
MAC Address: XXX
CPU Architecture: x64
CPUs: 24
CPU Usage: 4%
Memory Usage: 3.5 GB / 48.0 GB (7%)
Free Disk Space: 145.308 GB
Video Card: Standard VGA Graphics Adapter

That’s a big output resolution

is it only on certain machines that it fails on?

does it have the issue if you reduce the range to 1 frame?

Is your windows box up to date? or pending updates? ae cc on same version?

this error ‘1073741811’ could be a clue

It is a large output but not the largest we have completed recently.

Fearful of an AE mismatch i already upgraded everything to the latest version of 2015. Not checked windows upgrades yet.

Im not able to find a pattern of failures. Its across most of the farm as far as i can tell.

Ive not tried only one frame tasks. Ill try that next.

Thanks.

Ok. I just ran the deadline slave on my local workstation and got the same error(Log below). The PC is both up to date with windows 10 and After Effects.

I then opened the same project directly in AE and rendered the same frames without any issue.

That does look to point in Deadlines direction.

Any thoughts?

=======================================================
Error

Error: Renderer returned non-zero error code, -1073741811. Check the log for more information.
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type

RenderPluginException

=======================================================
Stack Trace

at Deadline.Plugins.Plugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.a(TaskLogWriter A_0)

=======================================================
Log

2016-04-20 09:36:03: BEGIN - WINPC04\XXX
2016-04-20 09:36:03: 0: Start Job timeout is disabled.
2016-04-20 09:36:03: 0: Task timeout is disabled.
2016-04-20 09:36:03: 0: Loaded job: XXX
2016-04-20 09:36:03: 0: INFO: Executing plugin script C:\Users\XXX\AppData\Local\Thinkbox\Deadline7\slave\winpc04\plugins\5717afdf619bac1a2831be71\AfterEffects.py
2016-04-20 09:36:03: 0: INFO: Fail on Existing After Effects process: 0
2016-04-20 09:36:03: 0: INFO: About: After Effects Plugin for Deadline
2016-04-20 09:36:03: 0: INFO: The job’s environment will be merged with the current environment before rendering
2016-04-20 09:36:03: 0: Plugin rendering frame(s): 349-373
2016-04-20 09:36:03: 0: INFO: Performing path mapping on aepx project file
2016-04-20 09:36:06: 0: INFO: Stdout Redirection Enabled: True
2016-04-20 09:36:06: 0: INFO: Stdout Handling Enabled: True
2016-04-20 09:36:06: 0: INFO: Popup Handling Enabled: True
2016-04-20 09:36:06: 0: INFO: QT Popup Handling Enabled: False
2016-04-20 09:36:06: 0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2016-04-20 09:36:06: 0: INFO: Using Process Tree: True
2016-04-20 09:36:06: 0: INFO: Hiding DOS Window: True
2016-04-20 09:36:06: 0: INFO: Creating New Console: False
2016-04-20 09:36:06: 0: INFO: Running as user: cedarconnor
2016-04-20 09:36:06: 0: INFO: Executable: “C:\Program Files\Adobe\Adobe After Effects CC 2015\Support Files\aerender.exe”
2016-04-20 09:36:06: 0: INFO: Rendering to network drive
2016-04-20 09:36:06: 0: INFO: Ignoring missing effect reference errors
2016-04-20 09:36:06: 0: INFO: Argument: -project XXX
2016-04-20 09:36:06: 0: INFO: Startup Directory: “C:\Program Files\Adobe\Adobe After Effects CC 2015\Support Files”
2016-04-20 09:36:06: 0: INFO: Process Priority: BelowNormal
2016-04-20 09:36:06: 0: INFO: Process Affinity: default
2016-04-20 09:36:06: 0: INFO: Process is now running
2016-04-20 09:36:06: 0: STDOUT: aerender version 13.7.1x6
2016-04-20 09:36:07: 0: STDOUT: Using DXGI: Device: “NVIDIA GeForce GTX 680” has dedicated video RAM (MB): 4026
2016-04-20 09:36:07: 0: STDOUT: e[1me[0mOptiX has 3 devices available for use.
2016-04-20 09:36:07: 0: STDOUT: e[1me[0mNon-default knobs:
2016-04-20 09:36:07: 0: STDOUT: e[1me[0m
2016-04-20 09:36:07: 0: STDOUT: e[1me[0mNon-default knobs:
2016-04-20 09:36:40: 0: STDOUT: WARNING:After Effects warning: Grouped_Message_{This project contains references to missing effects. Please install the following effects to restore these references.|||This project contains a reference to a missing effect. Please install the following effect to restore this reference.###@0 missing effects.}UnMult
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: 4/20/2016 9:36:44 AM: Starting composition CameraCenterRightFG_MediaServerRenders.
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Render Settings: Best Settings
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Quality: Best
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Resolution: Full
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Size: 1450 x 2100
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Proxy Use: Use No Proxies
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Effects: Current Settings
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Disk Cache: Read Only
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Color Depth: Current Settings
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Frame Blending: On for Checked Layers
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Field Render: Off
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Pulldown: Off
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Motion Blur: On for Checked Layers
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Solos: Current Settings
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Time Span: Custom
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Start: 00349
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: End: 00373
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Duration: 00025
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Frame Rate: 30.00 (comp)
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Guide Layers: All Off
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Skip Existing Files: Off
2016-04-20 09:36:44: 0: STDOUT: PROGRESS:
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Output Module: Photoshop
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: XXX
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Format: PNG
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Output Info: -
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Start Frame: 349
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Output Audio: -
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Channels: RGB + Alpha
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Depth: Millions of Colors+
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Color: Straight
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Resize: -
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Crop: -
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Final Size: 1450 x 2100
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Profile: -
2016-04-20 09:36:44: 0: STDOUT: PROGRESS: Post-Render Action: None
2016-04-20 09:36:44: 0: STDOUT: PROGRESS:
2016-04-20 09:36:56: 0: STDOUT: PROGRESS: 00349 (1): 12 Seconds
2016-04-20 09:36:59: 0: STDOUT: PROGRESS: 00350 (2): 3 Seconds
2016-04-20 09:37:03: 0: STDOUT: PROGRESS: 00351 (3): 4 Seconds
2016-04-20 09:37:07: 0: STDOUT: PROGRESS: 00352 (4): 3 Seconds
2016-04-20 09:37:10: 0: STDOUT: PROGRESS: 00353 (5): 4 Seconds
2016-04-20 09:37:14: 0: STDOUT: PROGRESS: 00354 (6): 3 Seconds
2016-04-20 09:37:17: 0: STDOUT: PROGRESS: 00355 (7): 4 Seconds
2016-04-20 09:37:21: 0: STDOUT: PROGRESS: 00356 (8): 3 Seconds
2016-04-20 09:37:24: 0: STDOUT: PROGRESS: 00357 (9): 4 Seconds
2016-04-20 09:37:28: 0: STDOUT: PROGRESS: 00358 (10): 3 Seconds
2016-04-20 09:37:31: 0: STDOUT: PROGRESS: 00359 (11): 4 Seconds
2016-04-20 09:37:35: 0: STDOUT: PROGRESS: 00360 (12): 3 Seconds
2016-04-20 09:37:38: 0: STDOUT: PROGRESS: 00361 (13): 4 Seconds
2016-04-20 09:37:42: 0: STDOUT: PROGRESS: 00362 (14): 3 Seconds
2016-04-20 09:37:45: 0: STDOUT: PROGRESS: 00363 (15): 4 Seconds
2016-04-20 09:37:49: 0: STDOUT: PROGRESS: 00364 (16): 3 Seconds
2016-04-20 09:37:52: 0: STDOUT: PROGRESS: 00365 (17): 4 Seconds
2016-04-20 09:37:56: 0: STDOUT: PROGRESS: 00366 (18): 4 Seconds
2016-04-20 09:37:59: 0: STDOUT: PROGRESS: 00367 (19): 3 Seconds
2016-04-20 09:38:03: 0: STDOUT: PROGRESS: 00368 (20): 3 Seconds
2016-04-20 09:38:06: 0: STDOUT: PROGRESS: 00369 (21): 4 Seconds
2016-04-20 09:38:10: 0: STDOUT: PROGRESS: 00370 (22): 4 Seconds
2016-04-20 09:38:13: 0: STDOUT: PROGRESS: 00371 (23): 3 Seconds
2016-04-20 09:38:17: 0: STDOUT: PROGRESS: 00372 (24): 4 Seconds
2016-04-20 09:38:21: 0: STDOUT: PROGRESS: 00373 (25): 4 Seconds
2016-04-20 09:38:32: 0: INFO: Process exit code: -1073741811

=======================================================
Details

Date: 04/20/2016 09:38:33
Frames: 349-373
Elapsed Time: 00:00:02:32
Job Submit Date: 04/20/2016 09:35:43
Job User: admin
Average RAM Usage: 22466150400 (33%)
Peak RAM Usage: 32875233280 (48%)
Average CPU Usage: 9%
Peak CPU Usage: 31%
Used CPU Clocks: 1423268
Total CPU Clocks: 15814081

=======================================================
Slave Information

Slave Name: WINPC04
Version: v7.2.1.10 R (b8ff445b3)
Operating System: Windows 10 Pro
Running As Service: No
Machine User: XXX
IP Address: XXX
MAC Address: XXX
CPU Architecture: x64
CPUs: 32
CPU Usage: 8%
Memory Usage: 15.4 GB / 64.0 GB (24%)
Free Disk Space: 291.397 GB (141.435 GB on C:, 149.962 GB on D:)
Video Card: NVIDIA GeForce GTX 680

That’s usually the case!

That exit code is pretty weird. It’s a standard Windows exit code which when turned to hex is “C000000D” which means “ERROR_INVALID_DATA”. Now, that can literally mean anything so I don’t know how much of a hint that is.

Since the error seems to be intermittent (or at least on different frames) this is going to be hard to solve. Does it fail if you run the command line program and args through the Command Line submitter in the Monitor? If so, I wonder if it’s somehow a .net issue (We’ve hit one that caused memory alignment problems in a 3DS Max plugin once before).

Update:

Forgot to ask, can you send us a project that could reproduce it? A synthetic example is fine, some test patterns or whatever should be good.

An update to our issues.

The final problem ended up being two different issues.

  1. The “saber” plugin from video copilot had a memory leak that only affected the command line version of After Effects. This crashed After effect mid-render. An update by video copilot fixed this.

  2. We also had a bug in deadline that was patched by an After effects submission script update.
    This bug allowed renders to finish but never released the node to the next job as noted in this thread.

forums.thinkboxsoftware.com/vie … cts#p63194

Cheers,

Well, both of those are a bit unexpected on our side.

How did you find that Saber was to blame? I’m trying to learn if that exit code just happens to go hand-in-hand with that problem or if it’s a more general exit code from AE.

I removed all AE plugins then ran a test comp through Deadline adding one plugin each time. Saber 1.38 crashed AE 50% of the time. When I updated to 1.39 all crashes stopped. I did not have any issues with Saber 1.38 when rendering through the UI.

Honestly the second issue I described is still happening to a small degree. On a few nodes each day.

This task finishes rendering the frames but never moves the frames or errors out. Before the Deadline update this problem was much worse.

“Success” was printed and another half hour has passed. Node still says 100%

2016-04-26 09:35:49: 0: STDOUT: PROGRESS: 4/26/2016 9:35:48 AM: Starting composition
XXX
.
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Render Settings: Best Settings
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Quality: Best
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Resolution: Full
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Size: 11300 x 2100
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Proxy Use: Use No Proxies
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Effects: Current Settings
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Disk Cache: Read Only
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Color Depth: Current Settings
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Frame Blending: On for Checked Layers
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Field Render: Off
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Pulldown: Off
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Motion Blur: On for Checked Layers
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Solos: Current Settings
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Time Span: Custom
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Start: 00275
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: End: 00299
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Duration: 00025
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Frame Rate: 29.97 (comp)
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Guide Layers: All Off
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Skip Existing Files: Off
2016-04-26 09:35:49: 0: STDOUT: PROGRESS:
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Output Module: PNG
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Output To: XXX
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Format: PNG
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Output Info: -
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Start Frame: 275
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Output Audio: -
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Channels: RGB + Alpha
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Depth: Millions of Colors+
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Color: Straight
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Resize: -
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Crop: -
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Final Size: 11300 x 2100
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Profile: -
2016-04-26 09:35:49: 0: STDOUT: PROGRESS: Post-Render Action: None
2016-04-26 09:35:49: 0: STDOUT: PROGRESS:
2016-04-26 09:37:40: 0: STDOUT: PROGRESS: 00275 (1): 1 Min, 51 Sec
2016-04-26 09:38:23: 0: STDOUT: PROGRESS: 00276 (2): 43 Seconds
2016-04-26 09:39:18: 0: STDOUT: PROGRESS: 00277 (3): 55 Seconds
2016-04-26 09:40:10: 0: STDOUT: PROGRESS: 00278 (4): 52 Seconds
2016-04-26 09:40:50: 0: STDOUT: PROGRESS: 00279 (5): 40 Seconds
2016-04-26 09:41:34: 0: STDOUT: PROGRESS: 00280 (6): 44 Seconds
2016-04-26 09:42:43: 0: STDOUT: PROGRESS: 00281 (7): 1 Min, 9 Sec
2016-04-26 09:43:21: Skipping thermal shutdown check because it is not required at this time
2016-04-26 09:43:37: 0: STDOUT: PROGRESS: 00282 (8): 54 Seconds
2016-04-26 09:44:26: 0: STDOUT: PROGRESS: 00283 (9): 49 Seconds
2016-04-26 09:45:07: 0: STDOUT: PROGRESS: 00284 (10): 41 Seconds
2016-04-26 09:45:55: 0: STDOUT: PROGRESS: 00285 (11): 48 Seconds
2016-04-26 09:46:43: 0: STDOUT: PROGRESS: 00286 (12): 48 Seconds
2016-04-26 09:47:29: 0: STDOUT: PROGRESS: 00287 (13): 46 Seconds
2016-04-26 09:48:15: 0: STDOUT: PROGRESS: 00288 (14): 46 Seconds
2016-04-26 09:48:56: 0: STDOUT: PROGRESS: 00289 (15): 41 Seconds
2016-04-26 09:49:37: 0: STDOUT: PROGRESS: 00290 (16): 41 Seconds
2016-04-26 09:50:33: 0: STDOUT: PROGRESS: 00291 (17): 56 Seconds
2016-04-26 09:51:19: 0: STDOUT: PROGRESS: 00292 (18): 46 Seconds
2016-04-26 09:52:02: 0: STDOUT: PROGRESS: 00293 (19): 43 Seconds
2016-04-26 09:52:50: 0: STDOUT: PROGRESS: 00294 (20): 48 Seconds
2016-04-26 09:53:31: 0: STDOUT: PROGRESS: 00295 (21): 41 Seconds
2016-04-26 09:53:33: Skipping thermal shutdown check because it is not required at this time
2016-04-26 09:54:18: 0: STDOUT: PROGRESS: 00296 (22): 48 Seconds
2016-04-26 09:55:04: 0: STDOUT: PROGRESS: 00297 (23): 45 Seconds
2016-04-26 09:55:46: 0: STDOUT: PROGRESS: 00298 (24): 42 Seconds
2016-04-26 09:56:28: 0: STDOUT: PROGRESS: 00299 (25): 42 Seconds
2016-04-26 10:04:00: Skipping thermal shutdown check because it is not required at this time
2016-04-26 10:14:28: Skipping thermal shutdown check because it is not required at this time
2016-04-26 10:24:59: Skipping thermal shutdown check because it is not required at this time
2016-04-26 10:35:21: Skipping thermal shutdown check because it is not required at this time
Success

Hmm… I was hoping for a smoking gun, but I’m still glad you’ve got the Sabre problem under control.

The next one is likely a problem with AE not closing correctly. Deadline by default is gracious to the apps it starts up and won’t kill them unless you tell us to (either by max task timeout, or you cancelling the task). Plus After Effects isn’t one we can cleanly terminate, so we’re relying on it to close itself once it’s done.

We’ve fought with AE not closing AERenderCore before, but that’s because aerender doesn’t send out the termination command properly.

Just to see if it’s AE, can you look on one of the stuck machines and see if closing aerender gets the Slave back on its way again?

Killed the aerender process on a node where the frames were done and “Success” was declared. The node reset the frames and logged this error.

Can I close the process more gently?

=======================================================
Error

aerender ERROR: After Effects can not render for aerender. Another instance of aerender, or another script, may be running; or, AE may be waiting for response from a modal dialog, or for a render to complete. Try running aerender without the -reuse flag to invoke a separate instance of After Effects
at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type

RenderPluginException

=======================================================
Stack Trace

at Deadline.Plugins.Plugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
at Deadline.Slaves.SlaveRenderThread.a(TaskLogWriter A_0)

=======================================================
Log

2016-04-26 13:37:45: BEGIN - XXX
2016-04-26 13:37:45: 0: Start Job timeout is disabled.
2016-04-26 13:37:45: 0: Task timeout is disabled.
2016-04-26 13:37:45: 0: Loaded job: XXX
2016-04-26 13:37:45: 0: INFO: Fail on Existing After Effects process: 0
2016-04-26 13:37:45: 0: INFO: About: After Effects Plugin for Deadline
2016-04-26 13:37:45: 0: INFO: The job’s environment will be merged with the current environment before rendering
2016-04-26 13:37:45: 0: Plugin rendering frame(s): 700-724
2016-04-26 13:37:45: 0: INFO: Stdout Redirection Enabled: True
2016-04-26 13:37:45: 0: INFO: Stdout Handling Enabled: True
2016-04-26 13:37:45: 0: INFO: Popup Handling Enabled: True
2016-04-26 13:37:45: 0: INFO: QT Popup Handling Enabled: False
2016-04-26 13:37:45: 0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2016-04-26 13:37:45: 0: INFO: Using Process Tree: True
2016-04-26 13:37:45: 0: INFO: Hiding DOS Window: True
2016-04-26 13:37:45: 0: INFO: Creating New Console: False
2016-04-26 13:37:45: 0: INFO: Running as user: mtrender
2016-04-26 13:37:45: 0: INFO: Executable: “C:\Program Files\Adobe\Adobe After Effects CC 2015\Support Files\aerender.exe”
2016-04-26 13:37:45: 0: INFO: Rendering to local drive, will copy files and folders to final location after render is complete
2016-04-26 13:37:45: 0: INFO: Continuing on missing footage
2016-04-26 13:37:45: 0: INFO: Ignoring missing layer dependency errors
2016-04-26 13:37:45: 0: INFO: Argument: XXX
2016-04-26 13:37:45: 0: INFO: Startup Directory: “C:\Program Files\Adobe\Adobe After Effects CC 2015\Support Files”
2016-04-26 13:37:45: 0: INFO: Process Priority: BelowNormal
2016-04-26 13:37:45: 0: INFO: Process Affinity: default
2016-04-26 13:37:45: 0: INFO: Process is now running
2016-04-26 13:37:46: 0: STDOUT: has video RAM(MB): 8
2016-04-26 13:37:49: 0: STDOUT: 2016-04-26 13:37:49 : INFO Environment: Scale factor from PP is 1.000000
2016-04-26 13:37:49: 0: STDOUT: 2016-04-26 13:37:49 : INFO Environment: m_scaleFactor is 1.000000
2016-04-26 13:38:01: 0: STDOUT: aerender version 13.5x347
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: 4/26/2016 1:38:01 PM: Starting composition XXX
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: Motion Blur: On for Checked Layers
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: Solos: Current Settings
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: Time Span: Custom
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: Start: 00700
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: End: 00724
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: Duration: 00025
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: Frame Rate: 30.00 (comp)
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: Profile: -
2016-04-26 13:38:01: 0: STDOUT: PROGRESS: Post-Render Action: None
2016-04-26 13:38:01: 0: STDOUT: PROGRESS:
2016-04-26 13:38:18: 0: STDOUT: PROGRESS: 00700 (1): 16 Seconds
2016-04-26 13:38:31: 0: STDOUT: PROGRESS: 00701 (2): 14 Seconds
2016-04-26 13:38:44: 0: STDOUT: PROGRESS: 00702 (3): 13 Seconds
2016-04-26 13:38:58: 0: STDOUT: PROGRESS: 00703 (4): 13 Seconds
2016-04-26 13:39:11: 0: STDOUT: PROGRESS: 00704 (5): 14 Seconds
2016-04-26 13:39:24: 0: STDOUT: PROGRESS: 00705 (6): 13 Seconds
2016-04-26 13:39:38: 0: STDOUT: PROGRESS: 00706 (7): 13 Seconds
2016-04-26 13:39:51: 0: STDOUT: PROGRESS: 00707 (8): 13 Seconds
2016-04-26 13:40:04: 0: STDOUT: PROGRESS: 00708 (9): 14 Seconds
2016-04-26 13:40:17: 0: STDOUT: PROGRESS: 00709 (10): 13 Seconds
2016-04-26 13:40:31: 0: STDOUT: PROGRESS: 00710 (11): 13 Seconds
2016-04-26 13:40:44: 0: STDOUT: PROGRESS: 00711 (12): 13 Seconds
2016-04-26 13:40:57: 0: STDOUT: PROGRESS: 00712 (13): 14 Seconds
2016-04-26 13:41:10: 0: STDOUT: PROGRESS: 00713 (14): 13 Seconds
2016-04-26 13:41:24: 0: STDOUT: PROGRESS: 00714 (15): 13 Seconds
2016-04-26 13:41:37: 0: STDOUT: PROGRESS: 00715 (16): 14 Seconds
2016-04-26 13:41:50: 0: STDOUT: PROGRESS: 00716 (17): 13 Seconds
2016-04-26 13:42:03: 0: STDOUT: PROGRESS: 00717 (18): 13 Seconds
2016-04-26 13:42:17: 0: STDOUT: PROGRESS: 00718 (19): 13 Seconds
2016-04-26 13:42:30: 0: STDOUT: PROGRESS: 00719 (20): 14 Seconds
2016-04-26 13:42:43: 0: STDOUT: PROGRESS: 00720 (21): 13 Seconds
2016-04-26 13:42:56: 0: STDOUT: PROGRESS: 00721 (22): 13 Seconds
2016-04-26 13:43:10: 0: STDOUT: PROGRESS: 00722 (23): 13 Seconds
2016-04-26 13:43:23: 0: STDOUT: PROGRESS: 00723 (24): 14 Seconds
2016-04-26 13:43:36: 0: STDOUT: PROGRESS: 00724 (25): 13 Seconds
2016-04-26 13:45:56: 0: STDOUT: aerender ERROR: After Effects can not render for aerender. Another instance of aerender, or another script, may be running; or, AE may be waiting for response from a modal dialog, or for a render to complete. Try running aerender without the -reuse flag to invoke a separate instance of After Effects

=======================================================
Details

Date: 04/26/2016 13:45:57
Frames: 700-724
Elapsed Time: 00:00:08:13
Job Submit Date: 04/26/2016 12:45:59
Job User: XXX
Average RAM Usage: 12185114624 (36%)
Peak RAM Usage: 25018200064 (73%)
Average CPU Usage: 7%
Peak CPU Usage: 79%
Used CPU Clocks: 1982662
Total CPU Clocks: 28323738

=======================================================
Slave Information

Slave Name: XXX
Version: v7.2.4.0 R (de7785a42)
Operating System: Windows 7 Professional (SP1)
Running As Service: No
Machine User: XXX
IP Address: XXX
MAC Address: XXX
CPU Architecture: x64
CPUs: 24
CPU Usage: 1%
Memory Usage: 3.2 GB / 32.0 GB (10%)
Free Disk Space: 151.972 GB
Video Card: Standard VGA Graphics Adapter

If you say “Success” from connecting via log streaming, that means the Launcher successfully connected and unfortunately has nothing to do with the state of After Effects.

I can’t tell given the lack of arguments, but if you are using multi-process rendering, it used to cause some issues with communication between aerender and aerendercore. I believe they fixed this in 2015 by doing some smarter child forking and communication code, but I’m still a little suspicious when that’s enabled.

As far as killing it more gently, there really isn’t a way. We don’t have any hooks into AE other than the command line, but you can have Deadline automate the closing of AE after it hasn’t seen log output for awhile. That could automate killing AE if it locks up for a few hours between frames. It’s a brute-force hack though, not a solution:
docs.thinkboxsoftware.com/produc … dc7fbf6c84

You’d add it to “[repo]/plugins/AfterEffects.py”:

        # Set the stdout handlers.
        self.SetUpdateTimeout(2 * 60 * 60) # Cancel the task if no progress for two hours
        self.AddStdoutHandlerCallback( "WARNING:(After Effects warning: .*)" ).HandleCallback += self.HandleStdoutWarning
        self.AddStdoutHandlerCallback( "INFO:This project contains .*" ).HandleCallback += self.HandleStdoutInfo

If you want to run some testing in the background, you can run what Deadline’s been trying by copying the ‘executable’ and ‘arguments’ lines into a Terminal / Command Prompt and letting that run a few times. I can write up a script example for looping over it indefinitely and you can sit it on one of the farm computers.

Unfortunately I’m not using multi-process rendering. Here is another log with a few driver errors I don’t understand. Perhaps helpful?

I’ll give your script a try.

Thanks,

UI Image
https://drive.google.com/open?id=0B0ZQdRaQ7xlcTTVhRm5rNmdRUFE

2016-04-27 14:46:14: Constructor: AfterEffects
2016-04-27 14:46:14: 0: Start Job timeout is disabled.
2016-04-27 14:46:14: 0: Task timeout is disabled.
2016-04-27 14:46:14: 0: Loaded job: XXX
2016-04-27 14:46:14: 0: INFO: Executing plugin script C:\Users\mtrender\AppData\Local\Thinkbox\Deadline7\slave\mt-render01\plugins\57212eaf6434d36948a7b1dc\AfterEffects.py
2016-04-27 14:46:14: 0: INFO: Fail on Existing After Effects process: 0
2016-04-27 14:46:14: 0: INFO: About: After Effects Plugin for Deadline
2016-04-27 14:46:14: 0: INFO: The job’s environment will be merged with the current environment before rendering
2016-04-27 14:46:14: 0: Plugin rendering frame(s): 100-124
2016-04-27 14:46:14: 0: INFO: Stdout Redirection Enabled: True
2016-04-27 14:46:14: 0: INFO: Stdout Handling Enabled: True
2016-04-27 14:46:14: 0: INFO: Popup Handling Enabled: True
2016-04-27 14:46:14: 0: INFO: QT Popup Handling Enabled: False
2016-04-27 14:46:14: 0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2016-04-27 14:46:14: 0: INFO: Using Process Tree: True
2016-04-27 14:46:14: 0: INFO: Hiding DOS Window: True
2016-04-27 14:46:14: 0: INFO: Creating New Console: False
2016-04-27 14:46:14: 0: INFO: Running as user: mtrender
2016-04-27 14:46:14: 0: INFO: Executable: “C:\Program Files\Adobe\Adobe After Effects CC 2015\Support Files\aerender.exe”
2016-04-27 14:46:14: 0: INFO: Argument: -project “XXX” -s 100 -e 124
2016-04-27 14:46:14: 0: INFO: Startup Directory: “C:\Program Files\Adobe\Adobe After Effects CC 2015\Support Files”
2016-04-27 14:46:14: 0: INFO: Process Priority: BelowNormal
2016-04-27 14:46:14: 0: INFO: Process Affinity: default
2016-04-27 14:46:14: 0: INFO: Process is now running
2016-04-27 14:46:14: 0: STDOUT: aerender version 13.6.1x6
2016-04-27 14:46:15: 0: STDOUT: has video RAM(MB): 8
2016-04-27 14:46:15: 0: STDOUT: e[1me[0mOptiX has 1 devices available for use.
2016-04-27 14:46:15: 0: STDOUT: e[1me[31me[1m[ERROR] e[0mNVAPI error (C:\code\rtsdk\adobe-ae\cc13.1\src\Util\DriverInfo.cpp:309):
2016-04-27 14:46:15: 0: STDOUT: e[1me[0mNon-default knobs:
2016-04-27 14:46:15: 0: STDOUT: e[1me[0m
2016-04-27 14:46:15: 0: STDOUT: e[1me[31me[1m[ERROR] e[0mNVAPI error (C:\code\rtsdk\adobe-ae\cc13.1\src\Util\DriverInfo.cpp:78):
2016-04-27 14:46:15: 0: STDOUT: e[1me[32me[1m[LOG 4] e[0mCPU fallback enabled
2016-04-27 14:46:15: 0: STDOUT: e[1me[0mNon-default knobs:
2016-04-27 14:46:15: 0: STDOUT: e[1me[0m
2016-04-27 14:46:15: 0: STDOUT: e[1me[31me[1m[ERROR] e[0mNVAPI error (C:\code\rtsdk\adobe-ae\cc13.1\src\Util\DriverInfo.cpp:78):
2016-04-27 14:48:22: 0: STDOUT: INFO: Skipping render queue item with correct comp name but marked to continue from a partly complete render.
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: 4/27/2016 2:48:29 PM: Starting composition XXX
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Render Settings: Best Settings
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Quality: Best
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Resolution: Full
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Size: 11300 x 2100
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Proxy Use: Use No Proxies
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Effects: Current Settings
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Disk Cache: Read Only
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Color Depth: Current Settings
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Frame Blending: On for Checked Layers
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Field Render: Off
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Pulldown: Off
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Motion Blur: On for Checked Layers
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Solos: Current Settings
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Time Span: Custom
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Start: 00100
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: End: 00124
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Duration: 00025
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Frame Rate: 30.00 (comp)
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Guide Layers: All Off
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Skip Existing Files: Off
2016-04-27 14:48:29: 0: STDOUT: PROGRESS:
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Output Module: PNG
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Output To: XXX
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Format: PNG
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Output Info: -
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Start Frame: 100
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Output Audio: -
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Channels: RGB + Alpha
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Depth: Millions of Colors+
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Color: Straight
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Resize: -
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Crop: -
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Final Size: 11300 x 2100
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Profile: -
2016-04-27 14:48:29: 0: STDOUT: PROGRESS: Post-Render Action: None
2016-04-27 14:48:29: 0: STDOUT: PROGRESS:
2016-04-27 14:49:28: 0: STDOUT: PROGRESS: 00100 (1): 1 Min, 1 Sec
2016-04-27 14:50:16: 0: STDOUT: PROGRESS: 00101 (2): 50 Seconds
2016-04-27 14:50:36: Skipping thermal shutdown check because it is not required at this time
2016-04-27 14:51:06: 0: STDOUT: PROGRESS: 00102 (3): 50 Seconds
2016-04-27 14:52:10: 0: STDOUT: PROGRESS: 00103 (4): 1 Min, 6 Sec
2016-04-27 14:53:03: 0: STDOUT: PROGRESS: 00104 (5): 54 Seconds
2016-04-27 14:53:52: 0: STDOUT: PROGRESS: 00105 (6): 50 Seconds
2016-04-27 14:54:43: 0: STDOUT: PROGRESS: 00106 (7): 51 Seconds
2016-04-27 14:55:34: 0: STDOUT: PROGRESS: 00107 (8): 52 Seconds
2016-04-27 14:56:25: 0: STDOUT: PROGRESS: 00108 (9): 51 Seconds
2016-04-27 14:57:17: 0: STDOUT: PROGRESS: 00109 (10): 53 Seconds
2016-04-27 14:58:09: 0: STDOUT: PROGRESS: 00110 (11): 52 Seconds
2016-04-27 14:59:03: 0: STDOUT: PROGRESS: 00111 (12): 55 Seconds
2016-04-27 14:59:58: 0: STDOUT: PROGRESS: 00112 (13): 54 Seconds
2016-04-27 15:00:46: Skipping thermal shutdown check because it is not required at this time
2016-04-27 15:00:50: 0: STDOUT: PROGRESS: 00113 (14): 53 Seconds
2016-04-27 15:01:43: 0: STDOUT: PROGRESS: 00114 (15): 53 Seconds
2016-04-27 15:02:36: 0: STDOUT: PROGRESS: 00115 (16): 53 Seconds
2016-04-27 15:03:28: 0: STDOUT: PROGRESS: 00116 (17): 53 Seconds
2016-04-27 15:04:21: 0: STDOUT: PROGRESS: 00117 (18): 53 Seconds
2016-04-27 15:05:14: 0: STDOUT: PROGRESS: 00118 (19): 53 Seconds
2016-04-27 15:06:07: 0: STDOUT: PROGRESS: 00119 (20): 54 Seconds
2016-04-27 15:07:01: 0: STDOUT: PROGRESS: 00120 (21): 54 Seconds
2016-04-27 15:07:56: 0: STDOUT: PROGRESS: 00121 (22): 55 Seconds
2016-04-27 15:08:49: 0: STDOUT: PROGRESS: 00122 (23): 53 Seconds
2016-04-27 15:09:42: 0: STDOUT: PROGRESS: 00123 (24): 54 Seconds
2016-04-27 15:10:35: 0: STDOUT: PROGRESS: 00124 (25): 53 Seconds
2016-04-27 15:21:41: Skipping thermal shutdown check because it is not required at this time
2016-04-27 15:32:13: Skipping thermal shutdown check because it is not required at this time
2016-04-27 15:42:40: Skipping thermal shutdown check because it is not required at this time
Success

Sadly, I’m still stumped. I did see some nVidia output and some other bits, but Google didn’t bring up any big clues.

In your After Effects project file can you go to the “Project Settings” dialog and at the top, make sure “Mercury Software Only” is selected as the “Renderer Type”, this will force all rendering to be CPU bound only. The previous NVAPI errors were indicating a CUDA GPU device was trying to be used for rendering, but I assume was not present or compliant with AE spec requirements to be used. Note, the above setting is saved in your AE project file, so you need to change this for each AE project file before submitting. I believe we are considering automating / adding a pre-flight check to our submitter to make this easier to handle.

Let us know the results!