AWS portal link offline when building infrastructure

Deadline
1

Hi,
We have some large upcoming rendering requirement.
I have been looking into burst rendering with AWS.

I am following the video from Bobo
An intro to aws portal

So far I have installed the RCS, AwsPortalLink,and AWSassetserver
I have also created an appropriate AMI on my aws account.

I can connect to the RCS.

I setup the asset server to us port 4000 as this is what he does in the video.
This does not work but if I use port 4005 it does connect.

If I dial up infrastructure the aws portal link does not connect…most of the time.
A few times it has gone green but am not completely sure why.

This is the aws portallink log

1626845146.728000 2021-07-21 15:25:46,727 [root] [11104] [MainThread] [INFO] .log.style:deadline
1626845146.728000 2021-07-21 15:25:46,727 [root] [11104] [MainThread] [INFO] .log.level:20
1626845146.728000 2021-07-21 15:25:46,727 [root] [11104] [MainThread] [INFO] .ssh.hostkeys_path:C:\Users\oc-patrick\AppData\Local\Thinkbox\AWSPortalLink
1626845146.728000 2021-07-21 15:25:46,727 [root] [11104] [MainThread] [INFO] .ssh.client:C:\Program Files (x86)\Thinkbox\AWSPortalLink\openssh
1626845146.728000 2021-07-21 15:25:46,727 [root] [11104] [MainThread] [INFO] .miscellaneous.status_reporting.enable:True
1626845146.729000 2021-07-21 15:25:46,729 [root] [11104] [MainThread] [INFO] .miscellaneous.status_reporting.port:4005
1626845146.729000 2021-07-21 15:25:46,729 [root] [11104] [MainThread] [INFO] using INSTALL_LOCATION - ‘C:\Program Files (x86)\Thinkbox\AWSPortalLink’
1626845146.730000 2021-07-21 15:25:46,730 [root] [11104] [MainThread] [INFO] Using ssh private key: C:\Program Files (x86)\Thinkbox\AWSPortalLink\DashKey
1626845146.730000 2021-07-21 15:25:46,730 [root] [11104] [MainThread] [INFO] [AWSPortalLinkService] Starting AWS Portal Link service …
1626845146.732000 2021-07-21 15:25:46,732 [root] [11104] [Dummy-1] [INFO] [AWSPortalLinkService] has initialized
1626845146.733000 2021-07-21 15:25:46,733 [root] [11104] [Dummy-1] [INFO] [AWSPortalLinkService] try to start…
1626845146.734000 2021-07-21 15:25:46,733 [root] [11104] [Dummy-1] [INFO] [StatusListener] Listening on 0.0.0.0:4005
1626845146.735000 2021-07-21 15:25:46,734 [root] [11104] [Dummy-1] [INFO] [StatusListener] Listening on [::]:4005
1626845146.735000 2021-07-21 15:25:46,734 [root] [11104] [Dummy-1] [INFO] [SSHTunnelManager] Has been told to start.
1626845146.736000 2021-07-21 15:25:46,736 [root] [11104] [Dummy-1] [ERROR] [SSHTunnelManager] Unexpected exception getting tunnel parameters. Please check the GetTunnelParams logfile in this directory.
1626845156.737000 2021-07-21 15:25:56,736 [root] [11104] [Dummy-1] [ERROR] [SSHTunnelManager] Unexpected exception getting tunnel parameters. Please check the GetTunnelParams logfile in this directory.
1626845164.013000 2021-07-21 15:26:04,013 [root] [11104] [Thread-4] [ERROR] [StatusListener] Version mismatch between Deadline and AWSPortalLink

I am not sure what version mismatch the log refers to ?

I am unable to find the log file for getTunnelParam

We are on Deadline 10.1.12.1
FranticX client version: 2.4.0.0
AwsPortalLink 1.1.11.0

Any help would be much appreciated.

Thanks
Patrick

2

Installing the aws portal link and aws asset server on a different machine seems to have sorted it. Although Im not sure why?

I now have the aws portal link and aws asset server. I am able to dial up infrastructure.
I am not able to launch a spot fleet.
I get an error saying

Object reference not set to an instance of an object

This is from the console panel

2021-07-21 23:44:04: Fetched bucket region ap-southeast-2
2021-07-21 23:44:46: Starting Deadline AWS Resource Tracker
2021-07-21 23:44:46: Deadline AWS Resource Tracker Stack Already Exists.
2021-07-21 23:44:46: Updating the Deadline AWS Resource Tracker
2021-07-21 23:44:48: The Deadline AWS Resource Tracker Stack is already at the given version.
2021-07-21 23:45:08: Using the official ReverseDash.template
2021-07-21 23:48:34: Using the official PrivateSubnet.template
2021-07-21 23:48:36: Using the official PrivateSubnet.template
2021-07-21 23:48:38: Using the official PrivateSubnet.template
2021-07-21 23:48:40: The specified log group already exists
2021-07-21 23:48:40: The specified log group already exists
2021-07-21 23:56:21: Starting Deadline AWS Resource Tracker
2021-07-21 23:56:21: Deadline AWS Resource Tracker Stack Already Exists.
2021-07-21 23:56:21: Updating the Deadline AWS Resource Tracker
2021-07-21 23:56:22: The Deadline AWS Resource Tracker Stack is already at the given version.
2021-07-21 23:56:44: Amazon.EC2.AmazonEC2Exception: Parameter: SpotFleetRequestConfig.IamFleetRole is invalid.
2021-07-21 23:56:44: —> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type ‘Amazon.Runtime.Internal.HttpErrorResponseException’ was thrown.
2021-07-21 23:56:44: at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
2021-07-21 23:56:44: at Amazon.Runtime.Internal.HttpHandler1.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.EC2.Internal.AmazonEC2ResponseHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: --- End of inner exception stack trace --- 2021-07-21 23:56:44: at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleException(IExecutionContext executionContext, HttpErrorResponseException exception) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.ErrorHandler.ProcessException(IExecutionContext executionContext, Exception exception) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext) 2021-07-21 23:56:44: at Deadline.AWS.AWSUtils.i(AggregateException dsg, String dsh, String dsi, String dsj, String[] dsk) 2021-07-21 23:56:44: at Deadline.AWS.AWSUtils.f[n](Task1 drr, String drs, String drt, String dru, String[] drv)
2021-07-21 23:56:44: at Deadline.AWS.Wrappers.EC2Wrapper.a[n](Task`1 dvg, String dvh, String dvi, String dvj)
2021-07-21 23:56:44: at Deadline.AWS.Wrappers.EC2Wrapper.RequestSpotFleet(RequestSpotFleetRequest request)
2021-07-21 23:56:44: at Deadline.AWS.AWSUtils.StartSpotFleetRequest(String accessKey, String secretKey, DashFleetReqConfig config)

3

I created a new AWS account.
This has sorted out a few more steps.
I am finally getting spot instances to use a custom AMI and appear in deadline.

They are not rendering. They try and start but before they even start rendering they generate this error (not an error report on the job, as there is no error, this is from RCS server log)

I have run deadlinecommand a heap of times to try and understand what is going on but cannot figure it out. I believe I have configured the server to have access to the default key.

If I try and enter my flexnet instance id or activation code I get

image
image710×273 16.7 KB

This is from the server application log
INFO: [SecretsManagement] User(admin) Validated Admin authority. Result=True
2021-07-25 14:44:32: ERROR: Ref#8D94F26E533BE09 Encountered exception while handling request on endpoint ‘/rcs/v1/putSecret’: Could not retrieve valid master key. (System.InvalidOperationException)
2021-07-25 14:44:32: at Deadline.Controllers.SecretsManagementController.c(IIdentity dju)
2021-07-25 14:44:32: at Deadline.Controllers.SecretsManagementController.PutSecret(String secretId, String secretString, String description, String username, String password)
2021-07-25 14:44:32: at Deadline.RemoteConnectionServer.Http.Modules.SecretsManagementModule.PutSecret(PutSecretRequest request)
2021-07-25 14:44:32: at Deadline.RCS.Http.HttpServerModule.SerializationWrapper[TRequest,TResponse](Func`2 handlerFunction, DynamicDictionary urlParams)