[7.2.0.17][WINDOWS] Auto-update from prior build not working 100% of time.

[Tx-RxFx]

Hi there,

We’ve just pushed out build 7.2.0.17 onto our farm here. We thought the auto-update from 7.2.0.16 would be better behaved as we’re only one build up but it seems problematic for us on Windows here.

Would seem (to my lesser trained eye in these things) that it doesn’t have the ability to overwrite deadlinelauncher.exe and when you restart it that old build of it persists alongside the new binaries previously updated makes it crash on start every single time.

Most workstations don’t seem to have this peculiar problem and some of our nodes were able to update in an unattended manner but the bulk of it simply stalled when it tried to re-launch the launcher post update.

I can provide any more info, snapshot’s of the bin dir if you want.

This is the error from MS Windows crash report:

[code]Windows launcher crashing error:

Problem signature:
Problem Event Name: APPCRASH
Application Name: deadlinelauncher.exe
Application Version: 7.2.0.16
Application Timestamp: 55e5e3ba
Fault Module Name: KERNELBASE.dll
Fault Module Version: 6.1.7601.17514
Fault Module Timestamp: 4ce7c78c
Exception Code: e0434352
Exception Offset: 000000000000a49d
OS Version: 6.1.7601.2.1.0.256.48
Locale ID: 2057
Additional Information 1: 152d
Additional Information 2: 152d49a18dcb45231bb1c1efff527c4d
Additional Information 3: 1939
Additional Information 4: 1939f0e2c35703370b41f4b9742b71a2

Read our privacy statement online:
http://go.microsoft.com/fwlink/?linkid=104288&clcid=0x0409

If the online privacy statement is not available, please read our privacy statement offline:
C:\Windows\system32\en-US\erofflps.txt
[/code]

This is the log from one of the many nodes which we had to visit indivdually and fix…

2015-09-21 14:08:52: BEGIN - QUAD-xx\renderloginname 2015-09-21 14:08:52: Deadline Launcher 7.2 [v7.2.0.16 R (51587765f)] 2015-09-21 14:08:52: deadlinelaunchernew is running 2015-09-21 14:08:52: deadlinelauncher has exited, proceeding with self upgrade 2015-09-21 14:08:57: Reading in bootstrap ini settings... 2015-09-21 14:08:57: Bootstrap file: C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bootstrap.ini 2015-09-21 14:08:57: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\bin.zip to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip 2015-09-21 14:08:58: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\Version to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\Version 2015-09-21 14:08:58: Decompressing C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip to C:\appnet\applications\deadline\7.2\bin 2015-09-21 14:08:59: Error: The process cannot access the file 'C:\appnet\applications\deadline\7.2\bin\deadlinelauncher.exe' because it is being used by another process. (System.IO.IOException) 2015-09-21 14:08:59: Waiting 1 seconds to try again 2015-09-21 14:09:00: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\bin.zip to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip 2015-09-21 14:09:01: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\Version to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\Version 2015-09-21 14:09:01: Decompressing C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip to C:\appnet\applications\deadline\7.2\bin 2015-09-21 14:09:01: Error: The process cannot access the file 'C:\appnet\applications\deadline\7.2\bin\deadlinelauncher.exe' because it is being used by another process. (System.IO.IOException) 2015-09-21 14:09:01: Waiting 2 seconds to try again 2015-09-21 14:09:03: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\bin.zip to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip 2015-09-21 14:09:05: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\Version to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\Version 2015-09-21 14:09:05: Decompressing C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip to C:\appnet\applications\deadline\7.2\bin 2015-09-21 14:09:05: Error: The process cannot access the file 'C:\appnet\applications\deadline\7.2\bin\deadlinelauncher.exe' because it is being used by another process. (System.IO.IOException) 2015-09-21 14:09:05: Waiting 3 seconds to try again 2015-09-21 14:09:08: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\bin.zip to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip 2015-09-21 14:09:10: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\Version to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\Version 2015-09-21 14:09:10: Decompressing C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip to C:\appnet\applications\deadline\7.2\bin 2015-09-21 14:09:10: Error: The process cannot access the file 'C:\appnet\applications\deadline\7.2\bin\deadlinelauncher.exe' because it is being used by another process. (System.IO.IOException) 2015-09-21 14:09:10: Waiting 4 seconds to try again 2015-09-21 14:09:14: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\bin.zip to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip 2015-09-21 14:09:16: Copying \\xxx.xxx.xxx.xxx\repository\bin\Windows\64bit\Version to C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\Version 2015-09-21 14:09:16: Decompressing C:\Users\USERNAME_HERE\AppData\Local\Temp\LauncherBootstrap\bin.zip to C:\appnet\applications\deadline\7.2\bin 2015-09-21 14:09:17: Error: The process cannot access the file 'C:\appnet\applications\deadline\7.2\bin\deadlinelauncher.exe' because it is being used by another process. (System.IO.IOException) 2015-09-21 14:09:17: Launch Slave: True 2015-09-21 14:09:17: Launch Pulse: False 2015-09-21 14:09:17: Launch Monitor: False 2015-09-21 14:09:17: Launch Balancer: False 2015-09-21 14:09:17: Launch Web Service: False 2015-09-21 14:09:17: Launching launcher 2015-09-21 14:09:17: Self upgrade completed 2015-09-21 14:09:17: Shutting down
Cheers

[Tx-RxFx]

Just to confirm, there is a mix of 7.2.0.16 and 7.2.0.17 binaries in the client’s bin folder after a failed auto-update.

Cheers

[ryangagnon]

Hey Jamie,

I just tested out auto upgrading for 7.2.0.16 to 7.2.0.17 and it worked fine for me. Are any of your slaves/launchers running as services? Are the slaves running as a different user? Does the user the launcher is running as have write permissions on the deadline installation folder?

[Tx-RxFx]

Hi Ryan,

They are running as the same user and not as a service. I do believe there’s a mix of stale windows group policies on some of the nodes and that could be impacting permissions or not releasing things. Does this seem to yourselves it’s security/permission issues? If it is then that’s a viable source of the problem for me and I’d pursue this further on our side.

If anyone else has seen this fail (in the same manner) I’m sure you’d have heard about it before this
Cheer

[ryangagnon]

I think it’s probably worth looking into. It would be interesting to see if logging onto a render node (one that failed to upgrade) as a work station user and triggering the update succeeds. Could also try triggering the upgrade as the render node user and see if that works outside of Deadline.

[Tx-RxFx]

Hi there,

Will explore those ideas ahead of the next build update rollout. For us (I believe in isolation tbh) it’s a niggle, a time consuming one, but one we’ll try the above with and let you know how it goes. For now we’ll crack on testing 7.2.0.17 on Linux & Windows.

Cheers