Deadline suddenly stopped working

Hi, been using deadline pretty seamlessly for over one year. We have a RCS installed with repository on one machine, other machines are in different locations but able to connect to the RCS fine, until this week. I have just re-installed the latest version of deadline on repository and workers but still having issues. Below is the launcher log from one of the workers unable to connect. Does anyone have any suggestions?

Many thanks in advance.
Phil

2022-09-07 13:19:47: BEGIN - DESKTOP-EB4C3VA\tedwa
2022-09-07 13:19:47: Operating System: Windows 10 Pro
2022-09-07 13:19:47: CPU Architecture: x64
2022-09-07 13:19:47: CPUs: 64
2022-09-07 13:19:47: Video Card: Parsec Virtual Display Adapter
2022-09-07 13:19:47: Deadline Launcher 10.1 [v10.1.23.6 Release (773a6289d)]
2022-09-07 13:19:47: ERROR: UpdateClient.MaybeSendRequestNow caught an exception: POST https://(machine IP):4433/rcs/v1/update returned “One or more errors occurred. (A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.)” (Deadline.Net.Clients.Http.DeadlineHttpRequestException)
2022-09-07 13:20:08: ERROR: DataController threw a configuration exception during initialization: Failed to establish connection to machine IP:4433 due to a communication error. (Deadline.Configuration.DeadlineConfigException)
2022-09-07 13:20:08: Error encountered when connecting to repository machine ip:4433;C:\Users\tedwa\Documents\CERTS FOR RCS\Deadline10RemoteClient.pfx: Object reference not set to an instance of an object. (System.NullReferenceException)
2022-09-07 13:20:08: at Deadline.Launcher.Launcher.ChangeRepository(RepositoryConnectionSettings updatedSettings)
2022-09-07 13:20:08: at Deadline.Launcher.Launcher.CheckForConnectionUpdate(Boolean verbose)
2022-09-07 13:20:08: WARNING: PythonSync (Python 3) upgrade failed with the following error:
2022-09-07 13:20:08: Object reference not set to an instance of an object.
2022-09-07 13:20:08: WARNING: PythonSync (Python 2) upgrade failed with the following error:
2022-09-07 13:20:08: Object reference not set to an instance of an object.
2022-09-07 13:20:08: Launcher Thread - Launcher thread initializing…
2022-09-07 13:20:08: Launcher Thread - opening remote TCP listening port 17000
2022-09-07 13:20:08: Launcher Thread - creating local listening TCP socket on an available port…
2022-09-07 13:20:08: Launcher Thread - local TCP port bound to: [::1]:50340
2022-09-07 13:20:08: Launcher Thread - updating local listening port in launcher file: 50340
2022-09-07 13:20:08: Launcher Thread - Launcher thread listening on port 17000
2022-09-07 13:20:08: Launching Worker:
2022-09-07 13:20:08: Launching Monitor
2022-09-07 13:20:30: ERROR: UpdateClient.MaybeSendRequestNow caught an exception: POST https://machine ip:4433/rcs/v1/update returned “One or more errors occurred. (A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.)” (Deadline.Net.Clients.Http.DeadlineHttpRequestException)
2022-09-07 13:20:51: ERROR: DataController threw a configuration exception during initialization: Failed to establish connection to machine ip:4433 due to a communication error. (Deadline.Configuration.DeadlineConfigException)
2022-09-07 13:20:51: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 13:20:51: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 13:25:51: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 13:25:51: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 13:30:52: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 13:30:52: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 13:35:52: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 13:35:52: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 13:40:52: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 13:40:52: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 13:45:52: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 13:45:52: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 13:50:53: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 13:50:53: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 13:55:53: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 13:55:53: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 14:00:53: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 14:00:53: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 14:02:03: Launcher Thread - ::1 has connected
2022-09-07 14:02:03: Launcher Thread - Received command: LaunchMonitorPromptForUpgrade
2022-09-07 14:02:03: Launching Monitor
2022-09-07 14:02:03: Launcher Thread - Responded with: Success
2022-09-07 14:05:53: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 14:05:53: Launcher Thread - Remote Administration is now disabled since there is no database connection.
2022-09-07 14:10:54: Launcher Thread - No database connection - cannot update Repository Options
2022-09-07 14:10:54: Launcher Thread - Remote Administration is now disabled since there is no database connection.

Looks like a networking issue. Things to check:

  1. Is the RCS app running?
  2. Is the RCS IP ping-able?
  3. Is the RCS port 4433 open?
  4. Check the firewall entries.
  5. Is RCS running on linux – check the ulimit.
  6. Are the deadline.ini cfg files correct (Proxy* values, etc.)
  7. If you generated new certs – Are the certs valid / correct format / read-able?
1 Like