AWS Thinkbox Discussion Forums

Should Path Mapping auto-replace forward-back slashes?

Hi! Running DL 8.0.13.

Just ran a C4D job where no errors reported but no files were written to the output path o the network. All slaves are Windows and job was submitted from a Mac.

I’ve setup a simple path mapping:
Replace Path: /Volumes/
Windows Path: \networkDrive\

The only thing I can see in the log which might be causing this is:
2017-04-13 13:23:24: 0: CheckPathMapping: Swapped “/Volumes/folder/tikviz/speed-test” with “\networkDrive\folder/tikviz/speed-test”

So it’s working but I thought DL also took care of the back/forward slash replacement.

From the docs:
docs.thinkboxsoftware.com/produc … %20mapping
“By default, Deadline just uses regular string replacement to swap out the paths. In this case, Deadline takes care of the path separators (‘/’ and ‘\’) automatically.”

Am I doing something wrong or do I need to use regex?

Thanks!
G

It may be that we missed some slash mangling there. C4D batch missed a few other path mapping niceties that have been logged, mainly if the destination path is blank, don’t do a replacement.

For this, I think we’re missing the slash replacement calls. I’ll double check and log it as a bug, but for the time being, on Windows Vista and higher, Windows only cares about those first three slashes on UNC paths, so I’m not sure this is the cause of the problem (though it may be).

I guess a good sanity check would be to make sure that UNC path isn’t asking for credentials when users connect.

Well, that was quick. We got the fix in place in the 9.0 branch, so you should see it in SP2. If not, feel free to chase me here.

Hey,

I’m having this same problem, trying to submit a job from a Windows machine to Media Encoder on a Mac. The start of the path replaces correctly, but the slashes in the rest of the path don’t.

Win: \Server\Share\folder\folder\file.
Mac: /Volumes/Share/folder\folder\file.

Media Encoder starts up but just hangs there, Deadline doesn’t report an error, think’s it’s rendering.

Deadline 9.0.3.0

Hi,
See attached patch for Media Encoder. Unzip and overwrite the same named files in your repo. (Take a backup first, just in case).
Let us know if that helps?
ame-patch-18-05-2017.zip (19.5 KB)

EDIT: Fixed some edge cases found during further testing (unrelated to your issue), so here is another patch (ignore the first patch):

ame-patch-18-05-2017-patch2.zip (20.1 KB)

Hi Mike,

Thanks for looking at this. I replaced these files in the repository with the second patch you posted and submitted a new job, but there’s no effect, getting the same result as before.

Let me know if I can get you more info.

Sam

Can you post the job log report / Slave log from the AME job?

Here’s the slave log:

2017-05-18 12:51:07:  Scheduler Thread - Seconds before next job scan: 7
2017-05-18 12:51:15:  Scheduler - Performing Job scan on Primary Pools with scheduling order Pool, Priority, First-in First-out
2017-05-18 12:51:15:  Scheduler - Successfully dequeued 1 task(s).  Returning.
2017-05-18 12:51:15:  Scheduler - Returning limit stubs not in use.
2017-05-18 12:51:15:  0: Got task!
2017-05-18 12:51:15:  0: Plugin will be reloaded because a new job has been loaded.
2017-05-18 12:51:15:  0: Loading Job's Plugin timeout is Disabled
2017-05-18 12:51:16:  0: Loaded plugin MediaEncoder
2017-05-18 12:51:17:  0: Executing plugin command of type 'Sync Files for Job'
2017-05-18 12:51:17:  0: All job files are already synchronized
2017-05-18 12:51:17:  0: Plugin MediaEncoder was already synchronized.
2017-05-18 12:51:17:  0: Done executing plugin command of type 'Sync Files for Job'
2017-05-18 12:51:17:  0: Executing plugin command of type 'Initialize Plugin'
2017-05-18 12:51:17:  0: INFO: Executing plugin script '/Users/producer/Library/Application Support/Thinkbox/Deadline9/slave/imac-3/plugins/591dcc02d18df12c14548e83/MediaEncoder.py'
2017-05-18 12:51:17:  0: INFO: Media Encoder Plugin Initializing...
2017-05-18 12:51:17:  0: INFO: About: Adobe Media Encoder Plugin for Deadline
2017-05-18 12:51:17:  0: INFO: Render Job As User disabled, running as current user 'producer'
2017-05-18 12:51:17:  0: INFO: The job's environment will be merged with the current environment before rendering
2017-05-18 12:51:17:  0: Done executing plugin command of type 'Initialize Plugin'
2017-05-18 12:51:17:  0: Start Job timeout is disabled.
2017-05-18 12:51:17:  0: Task timeout is disabled.
2017-05-18 12:51:17:  0: Loaded job: AME Final Test [Table_45_DC_v01_UNC_TEST.mov] (591dcc02d18df12c14548e83)
2017-05-18 12:51:17:  0: Executing plugin command of type 'Start Job'
2017-05-18 12:51:17:  0: INFO: Media Encoder Exe Name: Adobe Media Encoder CC 2015
2017-05-18 12:51:17:  0: INFO: Existing Adobe Media Encoder Process: Do Nothing
2017-05-18 12:51:17:  0: INFO: Auto Shutdown Adobe Media Encoder: True
2017-05-18 12:51:17:  0: INFO: Checking for an existing AME Web Service at 192.168.0.66:8080...
2017-05-18 12:51:18:  0: INFO: Unable to connect to an existing AME Web Service at 192.168.0.66:8080.
2017-05-18 12:51:18:  0: INFO: Starting AME Web Service...
2017-05-18 12:51:18:  0: INFO: Executable: /Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS/ame_webservice_agent
2017-05-18 12:51:18:  0: INFO: Host: 192.168.0.66
2017-05-18 12:51:18:  0: INFO: Port: 8080
2017-05-18 12:51:18:  0: INFO: Starting monitored managed process Media Encoder Web Service
2017-05-18 12:51:18:  0: INFO: Media Encoder Web Service starting...
2017-05-18 12:51:18:  0: INFO: Stdout Redirection Enabled: True
2017-05-18 12:51:18:  0: INFO: Asynchronous Stdout Enabled: False
2017-05-18 12:51:18:  0: INFO: Stdout Handling Enabled: True
2017-05-18 12:51:18:  0: INFO: Popup Handling Enabled: True
2017-05-18 12:51:18:  0: INFO: QT Popup Handling Enabled: False
2017-05-18 12:51:18:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2017-05-18 12:51:18:  0: INFO: Using Process Tree: True
2017-05-18 12:51:18:  0: INFO: Hiding DOS Window: True
2017-05-18 12:51:18:  0: INFO: Creating New Console: False
2017-05-18 12:51:18:  0: INFO: Running as user: producer
2017-05-18 12:51:18:  0: INFO: Executable: "/Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS/ame_webservice_agent"
2017-05-18 12:51:18:  0: INFO: Argument: 
2017-05-18 12:51:18:  0: INFO: Full Command: "/Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS/ame_webservice_agent"
2017-05-18 12:51:18:  0: INFO: Startup Directory: "/Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS"
2017-05-18 12:51:18:  0: INFO: Process Priority: BelowNormal
2017-05-18 12:51:18:  0: INFO: Process Affinity: default
2017-05-18 12:51:18:  0: INFO: Process is now running
2017-05-18 12:51:40:  0: STDOUT: Launching web services.  Use --help for options.
2017-05-18 12:51:40:  0: STDOUT: Starting HTTP service on ip 192.168.0.66, port 8080
2017-05-18 12:51:40:  0: STDOUT: History will be retained for the most recent 100 jobs.
2017-05-18 12:51:40:  0: STDOUT: Creating AMEServer
2017-05-18 12:51:40:  0: STDOUT: Creating AMEServer - Timeout
2017-05-18 12:51:40:  0: STDOUT: Creating AMEServer - Succeeded
2017-05-18 12:51:41:  0: INFO: AME Web Service started.
2017-05-18 12:51:41:  0: Done executing plugin command of type 'Start Job'
2017-05-18 12:51:41:  0: Plugin rendering frame(s): 0
2017-05-18 12:51:41:  0: Executing plugin command of type 'Render Task'
2017-05-18 12:51:41:  0: CheckPathMapping: Swapped "M:\test\stilltest\Table_45_DC_v01_UNC_TEST.mov" with "/Volumes/Projects/test\stilltest\Table_45_DC_v01_UNC_TEST.mov"
2017-05-18 12:51:41:  0: CheckPathMapping: Swapped "M:\test\stilltest\Table_45_DC_v01_ProRes_TEST.mov" with "/Volumes/Projects/test\stilltest\Table_45_DC_v01_ProRes_TEST.mov"
2017-05-18 12:51:41:  0: CheckPathMapping: Swapped "M:\test\stilltest\QT_ProRes_422HQ_1080p_23.976.epr" with "/Volumes/Projects/test\stilltest\QT_ProRes_422HQ_1080p_23.976.epr"
2017-05-18 12:51:51:  Connecting to Slave log: imac-3
Success

Interestingly I cannot connect to the slave log from my windows machine. Not sure if that’s a networking thing here or not. The job doesn’t generate a job log at all, it just hangs on Starting Up.

Sam

It looks like AME is failing to start-up. Is AME already running on this machine? I tested using CC 2017 and everything is fine. Perhaps, this is a CC 2015 issue. Can you try ensuring all instances of AME are already closed and also set the MediaEncoder plugin config setting: “Handle Existing AME Process” to either setting, just not the “Do Nothing” option, to see if that makes any difference here:
docs.thinkboxsoftware.com/produc … figuration

So yes this works with 2017! I had set it on Fail on Existing Process to start with, and had changed it for testing.

Neither worked before the patch, and 2015 still fails, but 2017 is working. Incidentally, it still displays the slashes incorrectly in the slave log on the mac, but it reads and writes the files without issue.

I can post logs from 2015 if you are interested.

Thanks,
Sam

I think the log message when it swaps paths via CheckPathMapping can be misleading. If the next few print statements for the various file paths are correct then you are good to go. Feel free to share both logs if you like. I guess something on the Adobe side is broken in CC 2015. Is that version patched to the latest SP? I will have to ask around if anyone has this version.

For 2015:

AME does start up when the job comes in, you can see it appear on the screen. It just doesn’t get any further than that. Deadline will throw an error if the software is running, saying it’s failing on an existing process, but it never manages to close the AME process when shutting down the slave, and or seemingly interact with it when its up, so it just sits there. I haven’t had it fail gracefully yet, AME always stays up even when cancelling the job by hand or closing the slave locally.

Here is the log with the Fail on Existing Process on, but with AME closed to start with. This shows it starting up, and at the end the network fails because the computer went to sleep from being unused for too long.

2017-05-22 11:47:15:  0: Got task!
2017-05-22 11:47:15:  0: Plugin will be reloaded because a new job has been loaded.
2017-05-22 11:47:15:  0: Loading Job's Plugin timeout is Disabled
2017-05-22 11:47:17:  0: Loaded plugin MediaEncoder
2017-05-22 11:47:17:  0: Executing plugin command of type 'Sync Files for Job'
2017-05-22 11:47:17:  0: All job files are already synchronized
2017-05-22 11:47:17:  0: Plugin MediaEncoder was already synchronized.
2017-05-22 11:47:17:  0: Done executing plugin command of type 'Sync Files for Job'
2017-05-22 11:47:17:  0: Executing plugin command of type 'Initialize Plugin'
2017-05-22 11:47:18:  0: INFO: Executing plugin script '/Users/producer/Library/Application Support/Thinkbox/Deadline9/slave/imac-3/plugins/5923077dd18df1521411a8e6/MediaEncoder.py'
2017-05-22 11:47:18:  0: INFO: Media Encoder Plugin Initializing...
2017-05-22 11:47:18:  0: INFO: About: Adobe Media Encoder Plugin for Deadline
2017-05-22 11:47:18:  0: INFO: Render Job As User disabled, running as current user 'producer'
2017-05-22 11:47:18:  0: INFO: The job's environment will be merged with the current environment before rendering
2017-05-22 11:47:18:  0: Done executing plugin command of type 'Initialize Plugin'
2017-05-22 11:47:18:  0: Start Job timeout is disabled.
2017-05-22 11:47:18:  0: Task timeout is disabled.
2017-05-22 11:47:18:  0: Loaded job: Test Again [Table_45_DC_v01_UNC_TEST.mov] (5923077dd18df1521411a8e6)
2017-05-22 11:47:18:  0: Executing plugin command of type 'Start Job'
2017-05-22 11:47:18:  0: INFO: Media Encoder Exe Name: Adobe Media Encoder CC 2015
2017-05-22 11:47:18:  0: INFO: Existing Adobe Media Encoder Process: Fail On Existing Process
2017-05-22 11:47:18:  0: INFO: Auto Shutdown Adobe Media Encoder: True
2017-05-22 11:47:18:  0: INFO: Checking for an existing AME Web Service at 192.168.0.66:8080...
2017-05-22 11:47:18:  0: INFO: Unable to connect to an existing AME Web Service at 192.168.0.66:8080.
2017-05-22 11:47:18:  0: INFO: Starting AME Web Service...
2017-05-22 11:47:18:  0: INFO: Executable: /Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS/ame_webservice_agent
2017-05-22 11:47:18:  0: INFO: Host: 192.168.0.66
2017-05-22 11:47:18:  0: INFO: Port: 8080
2017-05-22 11:47:18:  0: INFO: Starting monitored managed process Media Encoder Web Service
2017-05-22 11:47:18:  0: INFO: Media Encoder Web Service starting...
2017-05-22 11:47:18:  0: INFO: Stdout Redirection Enabled: True
2017-05-22 11:47:18:  0: INFO: Asynchronous Stdout Enabled: False
2017-05-22 11:47:18:  0: INFO: Stdout Handling Enabled: True
2017-05-22 11:47:18:  0: INFO: Popup Handling Enabled: True
2017-05-22 11:47:18:  0: INFO: QT Popup Handling Enabled: False
2017-05-22 11:47:18:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2017-05-22 11:47:18:  0: INFO: Using Process Tree: True
2017-05-22 11:47:18:  0: INFO: Hiding DOS Window: True
2017-05-22 11:47:18:  0: INFO: Creating New Console: False
2017-05-22 11:47:18:  0: INFO: Running as user: producer
2017-05-22 11:47:18:  0: INFO: Executable: "/Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS/ame_webservice_agent"
2017-05-22 11:47:18:  0: INFO: Argument: 
2017-05-22 11:47:18:  0: INFO: Full Command: "/Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS/ame_webservice_agent"
2017-05-22 11:47:18:  0: INFO: Startup Directory: "/Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS"
2017-05-22 11:47:18:  0: INFO: Process Priority: BelowNormal
2017-05-22 11:47:18:  0: INFO: Process Affinity: default
2017-05-22 11:47:18:  0: INFO: Process is now running
2017-05-22 11:47:32:  0: STDOUT: Launching web services.  Use --help for options.
2017-05-22 11:47:32:  0: STDOUT: Starting HTTP service on ip 192.168.0.66, port 8080
2017-05-22 11:47:32:  0: STDOUT: History will be retained for the most recent 100 jobs.
2017-05-22 11:47:32:  0: STDOUT: Creating AMEServer
2017-05-22 11:47:32:  0: STDOUT: Creating AMEServer - Succeeded
2017-05-22 11:47:32:  0: INFO: AME Web Service started.
2017-05-22 11:47:32:  0: Done executing plugin command of type 'Start Job'
2017-05-22 11:47:32:  0: Plugin rendering frame(s): 0
2017-05-22 11:47:33:  0: Executing plugin command of type 'Render Task'
2017-05-22 11:47:33:  0: CheckPathMapping: Swapped "M:\test\stilltest\Table_45_DC_v01_UNC_TEST.mov" with "/Volumes/Projects/test\stilltest\Table_45_DC_v01_UNC_TEST.mov"
2017-05-22 11:47:33:  0: CheckPathMapping: Swapped "M:\test\stilltest\Table_45_DC_v01_ProRes_TEST.mov" with "/Volumes/Projects/test\stilltest\Table_45_DC_v01_ProRes_TEST.mov"
2017-05-22 11:47:33:  0: CheckPathMapping: Swapped "M:\test\stilltest\QT_ProRes_422HQ_1080p_23.976.epr" with "/Volumes/Projects/test\stilltest\QT_ProRes_422HQ_1080p_23.976.epr"
2017-05-22 13:10:23:  Error occurred while updating network settings: An unexpected error occurred while interacting with the database (192.168.0.3:27090):
2017-05-22 13:10:23:  Network subsystem is down (System.Net.Sockets.SocketException)

Here’s a job report from when AME is running when it tries to start, and it fails as expected:

=======================================================
Error
=======================================================
Error: Fail On Existing Process is enabled, and a process 'Adobe Media Encoder CC 2015' with pid '575' exists! AME Web Service only works if Media Encoder application is not already running.
  at Deadline.Plugins.PluginWrapper.StartJob (System.String& outMessage, FranticX.Processes.ManagedProcess+AbortLevel& abortLevel) [0x0006c] in <92164420c75542ffabe6302e38083560>:0 

=======================================================
Type
=======================================================
RenderPluginException

=======================================================
Stack Trace
=======================================================
  at Deadline.Plugins.SandboxedPlugin.a (Deadline.Net.DeadlineMessage A_0) [0x0021c] in <92164420c75542ffabe6302e38083560>:0 
  at Deadline.Plugins.SandboxedPlugin.StartJob (Deadline.Jobs.Job job) [0x0006c] in <92164420c75542ffabe6302e38083560>:0 
  at Deadline.Slaves.SlaveRenderThread.a (Deadline.IO.TaskLogWriter A_0) [0x003ec] in <92164420c75542ffabe6302e38083560>:0 

=======================================================
Log
=======================================================
2017-05-18 12:37:14:  0: Plugin will be reloaded because a new job has been loaded.
2017-05-18 12:37:14:  0: Loading Job's Plugin timeout is Disabled
2017-05-18 12:37:15:  0: Loaded plugin MediaEncoder
2017-05-18 12:37:15:  0: Executing plugin command of type 'Sync Files for Job'
2017-05-18 12:37:15:  0: All job files are already synchronized
2017-05-18 12:37:15:  0: Synchronizing Plugin MediaEncoder from /Volumes/Projects/00 DeadlineRepository9/plugins/MediaEncoder took: 0 seconds
2017-05-18 12:37:15:  0: Done executing plugin command of type 'Sync Files for Job'
2017-05-18 12:37:15:  0: Executing plugin command of type 'Initialize Plugin'
2017-05-18 12:37:15:  0: INFO: Executing plugin script '/Users/producer/Library/Application Support/Thinkbox/Deadline9/slave/imac-3/plugins/591dcc02d18df12c14548e83/MediaEncoder.py'
2017-05-18 12:37:16:  0: INFO: Media Encoder Plugin Initializing...
2017-05-18 12:37:16:  0: INFO: About: Adobe Media Encoder Plugin for Deadline
2017-05-18 12:37:16:  0: INFO: Render Job As User disabled, running as current user 'producer'
2017-05-18 12:37:16:  0: INFO: The job's environment will be merged with the current environment before rendering
2017-05-18 12:37:16:  0: Done executing plugin command of type 'Initialize Plugin'
2017-05-18 12:37:16:  0: Start Job timeout is disabled.
2017-05-18 12:37:16:  0: Task timeout is disabled.
2017-05-18 12:37:16:  0: Loaded job: AME Final Test [Table_45_DC_v01_UNC_TEST.mov] (591dcc02d18df12c14548e83)
2017-05-18 12:37:16:  0: Executing plugin command of type 'Start Job'
2017-05-18 12:37:16:  0: INFO: Media Encoder Exe Name: Adobe Media Encoder CC 2015
2017-05-18 12:37:16:  0: INFO: Existing Adobe Media Encoder Process: Fail On Existing Process
2017-05-18 12:37:17:  0: WARNING: Found existing 'Adobe Media Encoder CC 2015' process
2017-05-18 12:37:17:  0: Done executing plugin command of type 'Start Job'
2017-05-18 12:37:17:  0: An exception occurred: Error: Fail On Existing Process is enabled, and a process 'Adobe Media Encoder CC 2015' with pid '575' exists! AME Web Service only works if Media Encoder application is not already running.
2017-05-18 12:37:17:    at Deadline.Plugins.PluginWrapper.StartJob (System.String& outMessage, FranticX.Processes.ManagedProcess+AbortLevel& abortLevel) [0x0006c] in <92164420c75542ffabe6302e38083560>:0  (Deadline.Plugins.RenderPluginException)

=======================================================
Details
=======================================================
Date: 05/18/2017 12:37:19
Frames: 0
Elapsed Time: 00:00:00:07
Job Submit Date: 05/18/2017 12:29:53
Job User: sam
Average RAM Usage: 4263758336 (50%)
Peak RAM Usage: 4275654656 (50%)
Average CPU Usage: 44%
Peak CPU Usage: 68%
Used CPU Clocks (x10^6 cycles): 9445
Total CPU Clocks (x10^6 cycles): 21465

=======================================================
Slave Information
=======================================================
Slave Name: imac-3
Version: v9.0.3.0 Release (c5f056bb6)
Operating System: Mac OS X 10.11.5
Machine User: producer
IP Address: 192.168.0.66
MAC Address: 20:c9:d0:32:ed:2f
CPU Architecture: x86_64
CPUs: 4
CPU Usage: 2%
Memory Usage: 4.0 GB / 8.0 GB (49%)
Free Disk Space: 716.623 GB 
Video Card: AMD Radeon HD 6770M

As for the versions - there is both a 2015 and a 2015.3. Both are installed. I tried editing the plugin setup to run 2015.3 on the mac, as so:
From: /Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.app/Contents/ame_webservice_console.app/Contents/MacOS/ame_webservice_agent
To: /Applications/Adobe Media Encoder CC 2015/Adobe Media Encoder CC 2015.3.app/Contents/ame_webservice_console.app/Contents/MacOS/ame_webservice_agent

Without success. It might be happier if the older one is removed, I can try that.

Failing to shut things down was an issue in 9.0.0 to 9.0.2 actually. If you’re not already, can you upgrade?

The issue was that we weren’t properly calling EndJob() in the script, so any cleanup that should have happened wasn’t being done.

You can use the link that you have onhand or e-mail support@thinkboxsoftware.com for that. The usual minor upgrade rules apply.

Hi Edwin,

We’re on 9.0.3.0.

Sam

Hmm. The good news is that I learned that the issue was not that EndJob isn’t called (that’s by design). It turns out that if the task or job is cancelled, we should be using an additional callback to clean things up. I’ll open an issue for the script to get fixed there. Sorry for the mis-information before.

Privacy | Site terms | Cookie preferences