Can't connect to remote connection server with TLS

Jesse · October 3, 2019, 7:58pm

I’m trying to connect my Deadline Monitor to the Remote Connection Server using TLS. For testing purposes, they are both on the same machine. I was able to connect when TLS was disabled. I’m on Windows 10.

I initially tried using the certificates generated by the install and then I tried using the SSLGenerator (https://docs.thinkboxsoftware.com/products/deadline/10.0/1_User%20Manual/manual/proxy-sslgen.html#ssl-cert-gen-ref-label). Using those steps, I repeated step #4 to generate a pfx file for the server (even though the instructions don’t say to do this), as this seems to be the only format that I can supply for the ‘tls_cert’ argument for deadlinercs.exe.

I’m starting deadlinercs from a Windows command prompt with no arguments. These are the config options in the deadline.ini file:

TlsAuth=True
TlsListenPort=4434
TlsServerCert=C:\DeadlineDatabase10\certs\deadline.pfx
#TlsCaCert=C:\DeadlineDatabase10\certs\ca.crt

If I comment out the TlsCaCert option, I get the following error when starting the RCS:

C:\Program Files\Thinkbox\Deadline10\bin>deadlinercs.exe
Deadline Remote Connection Server 10.0 [v10.0.28.2 Release (31a4a2e50)]
Connected to “C:\Users\jlehrman\Documents\deadline_repo_10”
Exception Details
DeadlineConfigException – An error occurred while trying to load the specified CA certificate:
The system cannot find the file specified.
Exception.Data: ( )
Exception.TargetSite: Deadline.Configuration.HttpsServerSettings ParseSettings(FranticX.Applications.Consoles.CommandLineParser, Int32, Boolean)
Exception.Source: deadline
Exception.HResult: -2146233088
Exception.StackTrace:
at Deadline.Configuration.HttpsServerSettings.ParseSettings(CommandLineParser parser, Int32 previousListenPort, Boolean ignoreConfig)
at DeadlineRCS.DeadlineRCSApp.Main(String args)

These are the settings for the Monitor:

This is the error I recieve from the Monitor when trying to connect:

This is the output from the RCS:

C:\Program Files\Thinkbox\Deadline10\bin>deadlinercs.exe
Deadline Remote Connection Server 10.0 [v10.0.28.2 Release (31a4a2e50)]
Connected to “C:\Users\jlehrman\Documents\deadline_repo_10”
Listening for HTTP requests on 0.0.0.0 port 8080 loopbackOnly True…
Listening for TLS connections on 0.0.0.0:4434…
[tls_conn#1] Connection accepted from 127.0.0.1:58665; starting TLS negotiation. (1 total connections)
[tls_conn#1] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#1] Connection closed. (0 total connections)
[tls_conn#2] Connection accepted from 127.0.0.1:58667; starting TLS negotiation. (1 total connections)
[tls_conn#2] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#2] Connection closed. (0 total connections)
[tls_conn#3] Connection accepted from 127.0.0.1:58701; starting TLS negotiation. (1 total connections)
[tls_conn#3] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#3] Connection closed. (0 total connections)
[tls_conn#4] Connection accepted from 127.0.0.1:58702; starting TLS negotiation. (1 total connections)
[tls_conn#4] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#4] Connection closed. (0 total connections)
[tls_conn#5] Connection accepted from 127.0.0.1:58709; starting TLS negotiation. (1 total connections)
[tls_conn#5] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#5] Connection closed. (0 total connections)
[tls_conn#6] Connection accepted from 127.0.0.1:58710; starting TLS negotiation. (1 total connections)
[tls_conn#6] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#6] Connection closed. (0 total connections)
[tls_conn#7] Connection accepted from 127.0.0.1:58720; starting TLS negotiation. (1 total connections)
[tls_conn#7] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#7] Connection closed. (0 total connections)
[tls_conn#8] Connection accepted from 127.0.0.1:58721; starting TLS negotiation. (1 total connections)
[tls_conn#8] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#8] Connection closed. (0 total connections)
[tls_conn#9] Connection accepted from 127.0.0.1:58750; starting TLS negotiation. (1 total connections)
[tls_conn#9] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#9] Connection closed. (0 total connections)
[tls_conn#10] Connection accepted from 127.0.0.1:58751; starting TLS negotiation. (1 total connections)
[tls_conn#10] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#10] Connection closed. (0 total connections)
[tls_conn#11] Connection accepted from 127.0.0.1:58765; starting TLS negotiation. (1 total connections)
[tls_conn#11] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#11] Connection closed. (0 total connections)
[tls_conn#12] Connection accepted from 127.0.0.1:58766; starting TLS negotiation. (1 total connections)
[tls_conn#12] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#12] Connection closed. (0 total connections)
[tls_conn#13] Connection accepted from 127.0.0.1:58770; starting TLS negotiation. (1 total connections)
[tls_conn#13] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#13] Connection closed. (0 total connections)
[tls_conn#14] Connection accepted from 127.0.0.1:58772; starting TLS negotiation. (1 total connections)
[tls_conn#14] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#14] Connection closed. (0 total connections)
[tls_conn#15] Connection accepted from 127.0.0.1:58797; starting TLS negotiation. (1 total connections)
[tls_conn#15] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#15] Connection closed. (0 total connections)
[tls_conn#16] Connection accepted from 127.0.0.1:58800; starting TLS negotiation. (1 total connections)
[tls_conn#16] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#16] Connection closed. (0 total connections)
[tls_conn#17] Connection accepted from 127.0.0.1:58803; starting TLS negotiation. (1 total connections)
[tls_conn#17] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#17] Connection closed. (0 total connections)
[tls_conn#18] Connection accepted from 127.0.0.1:58805; starting TLS negotiation. (1 total connections)
[tls_conn#18] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#18] Connection closed. (0 total connections)
[tls_conn#19] Connection accepted from 127.0.0.1:58834; starting TLS negotiation. (1 total connections)
[tls_conn#19] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#19] Connection closed. (0 total connections)
[tls_conn#20] Connection accepted from 127.0.0.1:58835; starting TLS negotiation. (1 total connections)
[tls_conn#20] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#20] Connection closed. (0 total connections)
[tls_conn#21] Connection accepted from 127.0.0.1:58838; starting TLS negotiation. (1 total connections)
[tls_conn#21] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#21] Connection closed. (0 total connections)
[tls_conn#22] Connection accepted from 127.0.0.1:58839; starting TLS negotiation. (1 total connections)
[tls_conn#22] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#22] Connection closed. (0 total connections)
Update timeout has been set to 30 seconds
Stdout Redirection Enabled: True
Stdout Handling Enabled: False
Popup Handling Enabled: False
Using Process Tree: True
Hiding DOS Window: True
Creating New Console: False
Running as user: jlehrman
Executable: “C:\Program Files\Thinkbox\Deadline10\bin\deadlinecommand.exe”
Argument: -RunCommandForRepository Repository C:\Users\jlehrman\Documents\deadline_repo_10;C:\DeadlineDatabase10\certs\Deadline10Client.pfx -DoRepositoryRepair True False True
Full Command: “C:\Program Files\Thinkbox\Deadline10\bin\deadlinecommand.exe” -RunCommandForRepository Repository C:\Users\jlehrman\Documents\deadline_repo_10;C:\DeadlineDatabase10\certs\Deadline10Client.pfx -DoRepositoryRepair True False True
Startup Directory: “C:\Program Files\Thinkbox\Deadline10\bin”
Process Priority: BelowNormal
Process Affinity: default
Process is now running
Skipping repository repair because it is not required at this time
Process exit code: 0
Update timeout has been set to 30 seconds
Stdout Redirection Enabled: True
Stdout Handling Enabled: False
Popup Handling Enabled: False
Using Process Tree: True
Hiding DOS Window: True
Creating New Console: False
Running as user: jlehrman
Executable: “C:\Program Files\Thinkbox\Deadline10\bin\deadlinecommand.exe”
Argument: -RunCommandForRepository Repository C:\Users\jlehrman\Documents\deadline_repo_10;C:\DeadlineDatabase10\certs\Deadline10Client.pfx -DoHouseCleaning True False True
Full Command: “C:\Program Files\Thinkbox\Deadline10\bin\deadlinecommand.exe” -RunCommandForRepository Repository C:\Users\jlehrman\Documents\deadline_repo_10;C:\DeadlineDatabase10\certs\Deadline10Client.pfx -DoHouseCleaning True False True
Startup Directory: “C:\Program Files\Thinkbox\Deadline10\bin”
Process Priority: BelowNormal
Process Affinity: default
Process is now running
[tls_conn#23] Connection accepted from 127.0.0.1:58884; starting TLS negotiation. (1 total connections)
[tls_conn#23] Error: The remote certificate is invalid according to the validation procedure. (AuthenticationException)
[tls_conn#23] Connection closed. (0 total connections)
[tls_conn#24] Connection accepted from 127.0.0.1:58886; starting TLS negotiation. (1 total connections)
[tls_conn#24] Error: Authentication failed because the remote party has closed the transport stream. (IOException)
[tls_conn#24] Connection closed. (0 total connections)
Skipping house cleaning because it is not required at this time
Process exit code: 0

Any help would be appreciated. I’m at a loss about what to try next.

Cheers,

Jesse

Daniel_Neilson · October 7, 2019, 4:35pm

it looks like you’re configuring your RCS to use the database’s TLS certificates. If the client is configured to connect with the RCS client key (Deadline10RemoteClient.pfx), then that would explain the inability to connect. I’d have to see the ProxyRoot, ProxyUseSSL, and ProxySSLCertificate entries of your deadline.ini to confirm.

None-the-less, there are two solutions for this:

Option 1. Configure the RCS to use the TLS certificates that were generated for the RCS during installation.

If you selected the option to install the RCS with SSL/TLS, during the installation process, then the installer should have generated three files for you: Deadline10RemoteClient.pfx, <your hostname>.pfx (let’s just call this host.pfx below), and ca.crt. All of these should be in the same directory.

a) Modify your deadline.ini ( https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User Manual/manual/client-config.html ) on the RCS machine to have:

TlsListenPort=4433
TlsServerCert=<location of>\host.pfx
TlsCaCert=<location of>\ca.crt

b) Modify your deadline.ini on the client (Monitor) machine to have:

ProxyRoot=<IP of your RCS>:4433
ProxyUseSSL=True
ProxySSLCertificate=<location of>\Deadline10RemoteClient.pfx

Option 2. Generate a new keypair to use to authenticate TLS connections to the RCS.

a) Install Python. ( note: I did this with Python3.6 )
b) Run the following:

python -m venv ssl_env
.\ssl_env\Scripts\activate
curl https://raw.githubusercontent.com/ThinkboxSoftware/SSLGeneration/master/ssl_gen.py -o ssl_gen.py
python -m pip install pyOpenSSL
python ssl_gen.py --ca --cert-org Thinkbox --cert-ou Test
python ssl_gen.py --server --cert-name RCS_server
python ssl_gen.py --client --cert-name RCS_client
python ssl_gen.py --pfx --cert-name RCS_server
python ssl_gen.py --pfx --cert-name RCS_client

Note: The org (Thinkbox) and org unit (Test) in the above should be whatever you want, and that makes sense for your organization.

This should generate ca.crt, RCS_server.pfx, and RCS_client.pfx in the keys subdirectory of whereever you execute these.

c) Copy keys\RCS_server.pfx & keys\ca.crt to a directory on the RCS machine.
d) Copy keys\RCS_client.pfx to a directory on the client (Monitor) machine.
e) Delete the contents of the keys directory. (for security)
f) On the RCS machine, modify your deadline.ini:

TlsListenPort=4433
TlsServerCert=<location of>\RCS_server.pfx
TlsCaCert=<location of>\ca.crt

g) On the client (Monitor) machine, modify your deadline.ini:

ProxyRoot=<IP of RCS machine>:4433
ProxyUseSSL=True
ProxySSLCertificate=<location of>\RCS_client.pfx

Note about security: Secure all three of these files (both .pfx files, and ca.crt). They are the “keys to the kingdom”; someone with the client.pfx file can connect to your RCS, and anyone with both the server.pfx and ca.crt can impersonate your RCS (and generate new client keys).

That should do the trick.

-Daniel

Jesse · October 7, 2019, 8:19pm

Using the first method, I’m missing the host.pfx file. I thought I had skipped an installation step so I tried re-installing. It’s still missing (though the .pem file is there). This is the contents of the certs folder:

Cheers,

Jesse

Daniel_Neilson · October 7, 2019, 8:48pm

Hi Jesse,
Those look like the database certs, rather than the RCS certs. It’s a little confusing, but there are two different sets of certs at play in Deadline; one set to secure the database (mongo), and another to secure the RCS.

The database certs are created by the repository installer when you select the SSL/TLS option for the database (see: https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/quick-install-db-repo.html#installing-a-new-mongodb-database ).

The RCS certs are created by the Client installer when you select the option to generate new certificates ( see: https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/install-client.html#connection-server-setup-ref-label). There should only be three files generated here – one ca.crt, Deadline10RemoteClient.pfx, and <hostname>.pfx.

-Daniel

Jesse · October 8, 2019, 4:03pm

Thanks Daniel… that really cleared a lot up for me. I think when I originally did the installation, I told the installed to use the client DB certificates for the RCS. I re-installed the Repo, DB and Client (choosing to install new certificates) and all is good now.

Jesse · October 10, 2019, 4:43pm

Something I’ve noticed… what’s the deadline.ini /env variable, launcher setting, etc… to specify the client certificate pfx password on the worker?

Daniel_Neilson · October 11, 2019, 4:00pm

That’s a little trickier. The client certificate password is a secret that should be protected, and so we don’t allow storing it in a plain text file (like the deadline.ini file). Instead, it’s stored in an encrypted container on each individual client machine.

To set it, you’ll either have to use the launcher’s “Change Repository” option, or deadlinecommand SaveCertificatePassword. The launcher option, clearly, isn’t scriptable, so you’ll probably want to go the deadlinecommand route.

The command will be something like:

deadlinecommand SaveCertificatePassword <RCS Host IP/name>:<RCS port> <path to>/Deadline10RemoteClient.pfx <cert password>

Note that this will not change your deadline.ini. The value for <RCS Host IP/name>:<RCS port> must exactly match the value of ProxyRoot in your deadline.ini, and the value for <path to>/Deadline10RemoteClient.pfx must exactly match the value of ProxySSLCertificate in your deadline.ini.

This deadlinecommand is essentially setting up a lookup value within Deadline – we look in the encrypted storage to see if there is a password for the ProxyRoot + ProxySSLCertificate entries from your deadline.ini, and use the password if there is one.

(( you can run deadlinecommand help SaveCertificatePassword for information on the arguments ))