I recently upgraded my install to Deadline 10.1.17.4 from a previous 10.0 install, and it seems maybe enabling secrets management was a mistake. At least until I figure out what’s going wrong here.
What I’m trying to do:
- Render on a Maya instance using UBL for Maya and Arnold, using the latest 2022 Linux AMIs build by Thinkbox (AMI ID ami-0bc705167c064477e)
- Infrastructure gateway host seems to be using a recent AMI (AMI ID ami-0babaa21197fc1820). As far as I can tell these are all running 10.1.17.4 clients, so they should be compatible with secrets management.
- When the instances first boot up, I go to Manage Identities to register them and assign the Client role. I did this for both the gateway IP and the single worker I booted to test.
- When I first installed 10.1 I allowed the installer to migrate the secrets and remove the legacy secrets
What seems to happen:
On the worker node, I get an error pulling the secret for UBL:
2021-07-17 07:10:56: ERROR: Scheduler Thread - Unexpected Error Occurred 2021-07-17 07:10:56: Scheduler Thread - Failed to retrieve the secret (/admin/ublsettings/UsageBasedURL), this operation was forbidden. Please ensure you have been granted access to this resource, or contact your Administrator to ensure Secrets Management was correctly configured. Please see Server's application log for further information. (System.InvalidOperationException) 2021-07-17 07:10:58: POST https://10.128.2.4:4433/rcs/v1/getSecret returned Forbidden "" (Deadline.Net.Clients.Http.DeadlineHttpRequestException) 2021-07-17 07:10:58: at Deadline.Net.Clients.Http.HttpClient.b(HttpRequestMessage bka) 2021-07-17 07:10:58: at Deadline.Net.Clients.Http.HttpClient.SendRequestForStream(String method, String uri, String contentType, Dictionary`2 headers, HttpContent httpContent) 2021-07-17 07:10:58: at Deadline.Net.Clients.Http.HttpClient.SendRequest(String method, String uri, String contentType, Dictionary`2 headers, HttpContent httpContent) 2021-07-17 07:10:58: at Deadline.Net.Clients.Http.HttpClient.Post(String uri, Object body, String contentType, Dictionary`2 headers) 2021-07-17 07:10:58: at Deadline.Net.Clients.Http.HttpClient.Post[TRequest,TResponse](String uri, TRequest body, String contentType, Dictionary`2 headers) 2021-07-17 07:10:58: at Deadline.Controllers.RemoteSecretsManagementController.GetSecret(String secretId) 2021-07-17 07:10:58: ERROR: Scheduler Thread - Unexpected Error Occurred
I see a similar matching “Authentication Failed” error on the RCS:
Authentication failed! Failed authentication= POST /rcs/v1/getSecret Content-Type=application/json; charset=utf-8 Accept=application/json Accept-Encoding=br Authorization=DEADLINE-RSASSAPSS Credential=[REDACTED]/20210717/thinkboxrcs, SignedHeaders=accept-encoding;content-type;host;user-agent;x-amz-date;x-amz-deadline-rcs-api, Signature=[REDACTED] Host=10.128.2.4 User-Agent=DeadlineWorker10.1/10.1.17.4 Content-Length=47 x-amz-deadline-rcs-api=4 X-Amz-Date=20210717T071058Z
Thanks for any advice folks can share on how to further troubleshoot this. Additionally, it seems the docs (particularly the AWS Portal Quickstart) could benefit from an update to reference the secrets management stuff.