I needed to change the password for the deadline service account, and this resulted in connection problems with the database and RCS server. It seems like the credentials are stored in the .pem or .pfx files.
Is there a way to recreate the DeadlineDatabase10\certs?
Thanks!
Zach
2024-03-19 12:42:57: BEGIN - HOST-DEADLINE\deadline_super
2024-03-19 12:42:57: Operating System: Windows Server 2022 Datacenter
2024-03-19 12:42:57: CPU Architecture: x64
2024-03-19 12:42:57: CPUs: 12
2024-03-19 12:42:57: Video Card: Microsoft Basic Display Adapter
2024-03-19 12:42:57: Deadline Launcher 10.3 [v10.3.1.3 Release (2807fcb2a)]
2024-03-19 12:42:57: ERROR: DataController threw a configuration exception during initialization: Error encountered when loading the configured Client Certificate (D:\DeadlineDatabase10\certs\Deadline10Client.pfx). It is likely either an invalid x.509 certificate, or the wrong passphrase was provided.
2024-03-19 12:42:57: Full Error:
2024-03-19 12:42:57: The specified network password is not correct. (Deadline.Configuration.DeadlineConfigException)
2024-03-19 12:42:57: Error encountered when connecting to repository \\nas-server\common\Animation\Deadline\DeadlineRepository;D:\DeadlineDatabase10\certs\Deadline10Client.pfx: Object reference not set to an instance of an object. (System.NullReferenceException)
2024-03-19 12:42:57: at Deadline.Launcher.Launcher.ChangeRepository(RepositoryConnectionSettings updatedSettings)
2024-03-19 12:42:57: at Deadline.Launcher.Launcher.CheckForConnectionUpdate(Boolean verbose)
Though I think the text of that warning is misleading in this case. Usually when I see Object reference not set to an instance of an object in this case it’ll be because the file isn’t accessible.
The password on the certificate is baked into the certificate, so changing the service account’s password shouldn’t affect it. But if the service account isn’t able to access the D:\DeadlineDatabase10\certs\Deadline10Client.pfx then that’d explain the issue.
We’ve got a guide on replacing the database certificate, but it’s Unix centric, or involves installing the database on another machine to let the installer make new certs for you. If those work for you I’ve quoted it below.
Option 1
Generate new certificates and configure MongoDB to use those
Step 1: Run this script (note, this is Unix-centric but a customer can install openssl on Windows10 so this can be used on Windows with some minor syntax tweaks)
#!/bin/sh
# Fully-qualified domain name of the MongoDB server.
# The current <DatabaseRoot>/certs directory will have two .pem files:
# 1. mongo_client.pem
# 2. <FDQN>.pem
# Copy the FDQN from the second file's name to here.
FDQN=myhost.domain.net
# 3 years. Change if needed
EXPIRY_DAYS=1095
# TODO: Customer to change this to their desired password
export CLIENT_CERT_PW=somepassword
# Generate a new x.509 to use as a self-signed root CA.
openssl req -nodes -newkey rsa:2048 -x509 -days ${EXPIRY_DAYS} -extensions v3_ca -keyout ca.key -out ca.crt -subj "/CN=CA/O=Deadline10Installer/OU=Deadline10MongoDB"
# Use the root CA to generate a new server certificate
openssl req -nodes -newkey rsa:2048 -out server.csr -keyout server.key -subj "/CN=${FDQN}/O=Deadline10Installer/OU=Deadline10MongoDB"
openssl x509 -sha256 -req -days ${EXPIRY_DAYS} -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt
cat server.crt server.key > ${FDQN}.pem
# Use the root CA to generate a new client certificate, and package it.
openssl req -newkey rsa:2048 -days ${EXPIRY_DAYS} -nodes -out client.csr -keyout client.key -subj "/CN=Deadline10Client/O=Deadline10Installer/OU=Deadline10Client"
openssl x509 -sha256 -req -days ${EXPIRY_DAYS} -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt
openssl pkcs12 -export -nodes -passout env:CLIENT_CERT_PW -out Deadline10Client.pfx -inkey client.key -in client.crt
cat client.crt client.key > mongo_client.pem
# For security, we delete the parts that would allow an attacker to create their own key
rm -f ca.key ca.srl
# Cleanup the files we no longer require
rm -f server.key server.csr server.crt
rm -f client.key client.csr client.crt
Step 2: Replace the files Deadline10Client.pfx, ca.crt, and .pem in /certs with the files that Step 1 generated. They should have the same filenames.
Step 3: Restart the Deadline 10 Database service
Step 4: Configure the existing workers to use the new client certificate (Deadline10Client.pfx from the certs directory that you copied over)
Option 2
Involves installing a new repository on a temporary machine and copying the new certificates to the existing server. This process is as follows:
Install the Deadline Repository on a fresh temporary machine, choosing to install a new MongoDB database and setting a new certificate password
When the installation is complete, copy the /certs directory on the temporary machine to the /certs directory on the existing database server
In the copied certs directory on the existing database server, rename the .pem file to match the setting called “PEMKeyFile” in /mongo/data/config.conf
Restart the Deadline 10 Database service
Configure the existing workers to use the new client certificate (Deadline10Client.pfx from the certs directory that you copied over)
But if the service account isn’t able to access the D:\DeadlineDatabase10\certs\Deadline10Client.pfx then that’d explain the issue.
That makes sense - I’ll check this. But when I changed the service account pw back to what it was when I made the db and restarted the services, everything worked again. (I do need to change it and will follow your guide)
Notes for folks trying this in the future.
I used gitbash for the ssl and the syntax changes are starting the -subj strings with \\ and separating statements with /
Also, any special characters need to be escaped or the pw single quoted export CLIENT_CERT_PW=Password123\!
I did get two warnings, which from my reading I ?think? can be ignored: "Ignoring -days without -x509; not generating a certificate" "Warning: output encryption option -nodes ignored with -export"
However, this fix didn’t work and and the keys were rejected. I had to use option 2 and install on another machine and copy the certs over.
I also had to reinstall the RCS and point it to the new database keys, and use existing certs.
I have a custom CA I’m using for Deadline DB certificates. I have it all working, but the database is pinned to only accept a single client cert with the exact Subject CN = Deadline10Client, O = Deadline10Installer, OU = Deadline10Client How can I create additional client certs and take advantage of mongo’s CRLFile option to add and remove users?
