AWS Thinkbox Discussion Forums

I try to use deadline on AWS but I can't

There are my assets on the NAS.
Client(Submitter)'s OS is windows OS.
Client can access the NAS. My assets path is \cgsv2021\project01\test/~~~~ for windows.
I wanna use the aws portal and start worker node(Linux OS).
The aws portal server 's OS is Linux OS.
So, my assets path is /mnt/cgsv2021/project01/test/~~~~ for Linux.
But status remain queued when I submit the job.
This is my scenario.

I answer your question.

  1. /mnt/cgsv2021 can be said or \cgsv2021 can be said.
  2. Yes. My submit Client is Windows.
  3. My Root directories of the Asset Server Settings in this scene belong to Linux.
  4. My Asset Server is running on Linux.

Please advice for me.

Best Regards
Taka.

Can you take a screenshot of the Configure Repo Options → Mapped paths? (You don’t have to include the bucketid)

I think Mois already asked you this, but do you have an entry towards the top:

Replace Path:  \\cgsv2021\
Linux Path:  /mnt/cgsv2021/

Otherwise the asset server should have log files if you want to check for errors:
/var/log/Thinkbox/AWSPortalAssetServer/

Thank you for the reply.

I have alerady add Mapped paths. But I can’t.

I realize that /var/log/Thinkbox/AWSPortalAssetServer/ is nothing.
Do you know how to check the assets server is running?
And why?

If there are no logs, it’s quite likely it is not running. Try e.g.
ps aux | grep AWSAsset
to find if there is a process.
Also a big help in debugging is running the programs directly in the shell, instead of from the service. This way you see everything in stdout and stderr.

Thank you for the advise.
I can check to install asset server.
I wonder if this is correct.

IML-PC is deadline repository not aws portal.
My environment is that deadline repository and aws portal is different server which has different IP.

  1. The aws portal server need to install RCS?

  2. If so, aws portal can access deadline repository directory?

  3. deadline repository can install in NAS?

Best regards
Takamitsu

You can see here
https://docs.thinkboxsoftware.com/products/deadline/10.1/1_User%20Manual/manual/aws-portal-setup-overview.html

Portal Link must be able to contact an RCS, and the Asset Server needs access to the File server. So this setting for the connection server is probably ok.

Can’t help you with the rest.

Thank you for your advice.
But, I can’t resolve this problem.
I submit the job, job status is still “queud”, and asset server log is not here in the aws portal server.
I don’t know next step.

  1. Yes, you need to install and run Remote Connection Server (RCS) to use AWS Portal.
  2. After RCS is running, then you can specify that as the “Connection Server”
    In the Configure AWS Portal Settings → Connection Server is the name of the RCS server.
  3. Deadline Repository is just a network share, so you don’t have to use a dedicated server for this.

In your previous posts, you said that the linux box was the aws portal server.

On that linux box, it would be simpler to install these of these on there:

  • Deadline Client
  • Deadline Remote Connection Server (RCS)
  • AWS Portal Link
  • AWS Portal Asset Server

And then:

  1. Start the Remote Connection Server (RCS)
  2. Launch the AWS Portal
  3. Create the AWS Portal Infrastructure

URLs:
Starting RCS:
https://docs.thinkboxsoftware.com/products/deadline/10.3/1_User%20Manual/manual/aws-portal-remote-connection-server.html

Check / verify connection:
https://docs.thinkboxsoftware.com/products/deadline/10.3/1_User%20Manual/manual/aws-portal-troubleshooting/verify-connection-to-rcs.html#aws-portal-verify-connection-to-rcs-ref-label

These (old) tutorials are pretty good:

I do see this command in the docs, but I personally have not run this command. It may (or may not) output an error that would be helpful in debugging.
deadlinecommand -AWSPortalPrecacheJob <Job ID(s)>

1 Like

Thank you for the deatil description.

My environment is special for the security. This is my architecture.

So, RCS and AWS portal is on the different server.

Thanks to your advice, I found out some point.

  • Infrastructure can’t connect to the RCS.

    • AWS portal server can’t resolve RCS hostname via DNS. Because my DNS is not set RCS hostname record. I try to fill in the hosts file, but this error didn’t dissapear. Maybe it cause this problem, I think. Do I need to record RCS hostname on DNS server?
  • What is this error? How do I resolve it?

From the AWSPortal server can you reach the RCS?
Try something like: curl -v http://10.111.80.51:8080
e.g.:

[root@test Deadline10]#  curl -v http://10.111.80.51:8080
* About to connect() to 10.111.80.51 port 8080 (#0)
*   Trying 10.111.80.51...
* Connected to 10.111.80.51 (10.111.80.51) port 8080 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.111.80.51:8080
> Accept: */*
>
* Empty reply from server
* Connection #0 to host 10.111.80.51 left intact

There is also this page (AWS Portal Infrastructure is Incorrectly Configured for Your Remote Connection Server) : AWS Portal Workers Don’t Connect to Repository — Deadline 10.3.1.4 documentation

The docs says (under Configure Infrastructure) Full RCS Hostname “if available” – but it doesn’t say “required”. Maybe someone from Thinkbox knows if you can leave it blank or not.

Not sure which server is complaining about the 502 Bad Gateway – the AWS Portal server or the DL Gateway instance. Was there something at the beginning of the log file?

No, the GetTunnelParams.py file doesn’t need to have an executable bit. It is being interpreted by python, so the python binary is the one that should be executable.

1 Like

Thank you for your advice.
I resolve this problem.
But I got new problem.
The worker that stood via aws portal has error log.

I have received the answer before. But I may not understand it.

This is the security rule of my infrastructure.

This is the security rule of my worker.

What does cause this problem?
And I want to know how to resolve this problem, if you know the cause.

“Address already in use”
Either hserver or aws portal is already running and you cannot start a new one on the same port.

Hi.

Did you install Houdini on your customized AMI? I’m asking because if you run the Houdini installer (linux), it installs “License Server” (sesinetd) by default. You don’t need sesinetd running on the worker.

The log is complaining about port 1715 which is sesinetd. As a test you should be able to disable sesinetd from running on the worker and see if the render progresses. You should be able to connect to the worker via aws console (or ssh into the worker via the GW instance) to check:

Check for sesinetd process:
ps ax | grep sesi

Check if port 1715 is open:

[ec2-user@test]$ ss -a | grep 1715
tcp   LISTEN      0       128		*:1715		*:*

Stop and disable the daemon:
sudo systemctl disable --now sesinetd.service

or if you’re using the older init style:

sudo /etc/init.d/sesinetd stop
sudo chkconfig sesinetd off

Run the ss -a command again to check that port 1715 has closed.

If all that worked you can modify your base AMI.

1 Like

Hi jarak,

Thank you.
I install houdini on my custom AMI.
I uninstall hserver, then job rendering can work.
But I have new error.

2024-03-15 01:48:01:  Scheduler Thread - Scheduler State transition from = 'PreRendering' to = 'PostRendering'
2024-03-15 01:48:01:  Scheduler Thread - Scheduler State transition from = 'PostRendering' to = 'EndJob'
2024-03-15 01:48:01:  Scheduler Thread - Scheduler State transition from = 'EndJob' to = 'WaitingForJob'
2024-03-15 01:48:01:  Scheduler Thread - Seconds before next job scan: 1
2024-03-15 01:48:03:  Scheduler Thread - Scheduler State transition from = 'WaitingForJob' to = 'LicenseCheck'
2024-03-15 01:48:03:  Scheduler Thread - Scheduler State transition from = 'LicenseCheck' to = 'LicenseConfirmed'
2024-03-15 01:48:03:  Scheduler - Returning limit stubs not in use.
2024-03-15 01:48:03:  Scheduler Thread - Job's Limit Groups: houdini, redshift
2024-03-15 01:48:03:  Scheduler Thread - Scheduler State transition from = 'LicenseConfirmed' to = 'LicenseCheck'
2024-03-15 01:48:03:  Scheduler Thread - Scheduler State transition from = 'LicenseCheck' to = 'LicenseConfirmed'
2024-03-15 01:48:03:  Scheduler Thread - Scheduler State transition from = 'LicenseConfirmed' to = 'StartJob'
2024-03-15 01:48:03:  0: Render Thread - Render State transition from = 'WaitingForTask' to = 'ReceivedTask'
2024-03-15 01:48:03:  Scheduler Thread - Scheduler State transition from = 'StartJob' to = 'PreRendering'
2024-03-15 01:48:03:  0: Got task!
2024-03-15 01:48:03:  0: Render Thread - Render State transition from = 'ReceivedTask' to = 'Other'
2024-03-15 01:48:03:  0: Loading Job's Plugin timeout is Disabled
2024-03-15 01:48:03:  0: SandboxedPlugin: Render Job As User disabled, running as current user 'ec2-user'
2024-03-15 01:48:04:  '/home/ec2-user/Thinkbox/Deadline10/pythonAPIs/8oI2s+G2DZfcng9zEWRA==' already exists. Skipping extraction of PythonSync.
2024-03-15 01:48:05:  0: Loaded plugin Houdini
2024-03-15 01:48:05:  All job files are already synchronized
2024-03-15 01:48:05:  Plugin Houdini was already synchronized.
2024-03-15 01:48:05:  0: Executing plugin command of type 'Initialize Plugin'
2024-03-15 01:48:05:  0: INFO: Executing plugin script '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/plugins/65f3a6d8874df71c50592ce4/Houdini.py'
2024-03-15 01:48:05:  0: INFO: Plugin execution sandbox using Python version 3
2024-03-15 01:48:05:  0: INFO: About: Houdini Plugin for Deadline
2024-03-15 01:48:05:  0: INFO: The job's environment will be merged with the current environment before rendering
2024-03-15 01:48:05:  0: Done executing plugin command of type 'Initialize Plugin'
2024-03-15 01:48:05:  0: Start Job timeout is disabled.
2024-03-15 01:48:05:  0: Task timeout is disabled.
2024-03-15 01:48:05:  0: Loaded job: B_rs_dl_render_test_full_path - /obj/ropnet1/Redshift_ROP (65f3a6d8874df71c50592ce4)
2024-03-15 01:48:06:  0: Executing plugin command of type 'Start Job'
2024-03-15 01:48:06:  0: INFO: Sending StartTaskRequest to S3BackedCacheClient.
2024-03-15 01:48:06:  0: DEBUG: Request:
2024-03-15 01:48:06:  0: DEBUG: 	JobId: 65f3a6d8874df71c50592ce4
2024-03-15 01:48:06:  0: DEBUG: 	JobUploadWhitelist: 
2024-03-15 01:48:06:  0: DEBUG: 	JobUploadWhitelistRe: ^.+\.abc$, ^.+\.avi$, ^.+\.bmp$, ^.+\.bw$, ^.+\.cin$, ^.+\.cjp$, ^.+\.cjpg$, ^.+\.cxr$, ^.+\.dds$, ^.+\.dpx$, ^.+\.dwf$, ^.+\.dwfx$, ^.+\.dwg$, ^.+\.dxf$, ^.+\.dxx$, ^.+\.eps$, ^.+\.exr$, ^.+\.fbx$, ^.+\.fxr$, ^.+\.hdr$, ^.+\.icb$, ^.+\.iff$, ^.+\.iges$, ^.+\.igs$, ^.+\.int$, ^.+\.inta$, ^.+\.iris$, ^.+\.jpe$, ^.+\.jpeg$, ^.+\.jpg$, ^.+\.jp2$, ^.+\.mcc$, ^.+\.mcx$, ^.+\.mov$, ^.+\.mxi$, ^.+\.pdf$, ^.+\.pic$, ^.+\.png$, ^.+\.prt$, ^.+\.ps$, ^.+\.psd$, ^.+\.rgb$, ^.+\.rgba$, ^.+\.rla$, ^.+\.rpf$, ^.+\.sat$, ^.+\.sgi$, ^.+\.stl$, ^.+\.sxr$, ^.+\.targa$, ^.+\.tga$, ^.+\.tif$, ^.+\.tiff$, ^.+\.tim$, ^.+\.vda$, ^.+\.vrimg$, ^.+\.vrmesh$, ^.+\.vrsm$, ^.+\.vrst$, ^.+\.vst$, ^.+\.wmf$, ^.+\.ass$, ^.+\.gz$, ^.+\.ifd$, ^.+\.mi$, ^.+\.mi2$, ^.+\.mxi$, ^.+\.rib$, ^.+\.rs$, ^.+\.vrscene$
2024-03-15 01:48:06:  0: DEBUG: S3BackedCache Client Returned Sequence: 1
2024-03-15 01:48:06:  0: INFO: Executing global asset transfer preload script '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/plugins/65f3a6d8874df71c50592ce4/GlobalAssetTransferPreLoad.py'
2024-03-15 01:48:06:  0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer...
2024-03-15 01:48:06:  0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in /opt/Thinkbox/S3BackedCache/bin/task.py...
2024-03-15 01:48:06:  0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2024-03-15 01:48:06:  0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2024-03-15 01:48:06:  0: Done executing plugin command of type 'Start Job'
2024-03-15 01:48:06:  0: Plugin rendering frame(s): 1
2024-03-15 01:48:06:  0: Render Thread - Render State transition from = 'Other' to = 'Rendering'
2024-03-15 01:48:06:  0: Executing plugin command of type 'Render Task'
2024-03-15 01:48:06:  0: INFO: Sending StartTaskRequest to S3BackedCacheClient.
2024-03-15 01:48:06:  0: DEBUG: Request:
2024-03-15 01:48:06:  0: DEBUG: 	JobId: 65f3a6d8874df71c50592ce4
2024-03-15 01:48:06:  0: DEBUG: 	JobUploadWhitelist: 
2024-03-15 01:48:06:  0: DEBUG: 	JobUploadWhitelistRe: ^.+\.abc$, ^.+\.avi$, ^.+\.bmp$, ^.+\.bw$, ^.+\.cin$, ^.+\.cjp$, ^.+\.cjpg$, ^.+\.cxr$, ^.+\.dds$, ^.+\.dpx$, ^.+\.dwf$, ^.+\.dwfx$, ^.+\.dwg$, ^.+\.dxf$, ^.+\.dxx$, ^.+\.eps$, ^.+\.exr$, ^.+\.fbx$, ^.+\.fxr$, ^.+\.hdr$, ^.+\.icb$, ^.+\.iff$, ^.+\.iges$, ^.+\.igs$, ^.+\.int$, ^.+\.inta$, ^.+\.iris$, ^.+\.jpe$, ^.+\.jpeg$, ^.+\.jpg$, ^.+\.jp2$, ^.+\.mcc$, ^.+\.mcx$, ^.+\.mov$, ^.+\.mxi$, ^.+\.pdf$, ^.+\.pic$, ^.+\.png$, ^.+\.prt$, ^.+\.ps$, ^.+\.psd$, ^.+\.rgb$, ^.+\.rgba$, ^.+\.rla$, ^.+\.rpf$, ^.+\.sat$, ^.+\.sgi$, ^.+\.stl$, ^.+\.sxr$, ^.+\.targa$, ^.+\.tga$, ^.+\.tif$, ^.+\.tiff$, ^.+\.tim$, ^.+\.vda$, ^.+\.vrimg$, ^.+\.vrmesh$, ^.+\.vrsm$, ^.+\.vrst$, ^.+\.vst$, ^.+\.wmf$, ^.+\.ass$, ^.+\.gz$, ^.+\.ifd$, ^.+\.mi$, ^.+\.mi2$, ^.+\.mxi$, ^.+\.rib$, ^.+\.rs$, ^.+\.vrscene$
2024-03-15 01:48:06:  0: DEBUG: S3BackedCache Client Returned Sequence: 1
2024-03-15 01:48:06:  0: INFO: Set HOUDINI_PATHMAP to {"/mnt/cgsv2021/":"/mnt/Data/mntcgsv2021386d7e3204ea8557cff0f48192a0df69/", "//cgsv2021":"/mnt/cgsv2021"}
2024-03-15 01:48:06:  0: INFO: Redshift Path Mapping...
2024-03-15 01:48:06:  0: INFO: source: "/mnt/cgsv2021/" dest: "/mnt/Data/mntcgsv2021386d7e3204ea8557cff0f48192a0df69/"
2024-03-15 01:48:06:  0: INFO: source: "\\cgsv2021" dest: "/mnt/cgsv2021"
2024-03-15 01:48:06:  0: INFO: [REDSHIFT_PATHOVERRIDE_FILE] now set to: "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/jobsData/65f3a6d8874df71c50592ce4/RSMapping_tempPDE5M0/RSMapping.txt"
2024-03-15 01:48:06:  0: INFO: Starting Houdini Job
2024-03-15 01:48:06:  0: INFO: Stdout Redirection Enabled: True
2024-03-15 01:48:06:  0: INFO: Asynchronous Stdout Enabled: False
2024-03-15 01:48:06:  0: INFO: Stdout Handling Enabled: True
2024-03-15 01:48:06:  0: INFO: Popup Handling Enabled: True
2024-03-15 01:48:06:  0: INFO: QT Popup Handling Enabled: False
2024-03-15 01:48:06:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2024-03-15 01:48:06:  0: INFO: Using Process Tree: True
2024-03-15 01:48:06:  0: INFO: Hiding DOS Window: True
2024-03-15 01:48:06:  0: INFO: Creating New Console: False
2024-03-15 01:48:06:  0: INFO: Running as user: ec2-user
2024-03-15 01:48:06:  0: INFO: Executable: "/opt/hfs19.5.640/bin/hython"
2024-03-15 01:48:06:  0: CheckPathMapping: Swapped "//cgsv2021/Projects02/AWS_Test/RS_deadline_test/render/Redshift_ROP/$Redshift_ROP.exr" with "/mnt/cgsv2021/Projects02/AWS_Test/RS_deadline_test/render/Redshift_ROP/$Redshift_ROP.exr"
2024-03-15 01:48:06:  0: CheckPathMapping: Swapped "//cgsv2021/projects02/AWS_Test/RS_deadline_test/B_rs_dl_render_test_full_path.hip" with "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/B_rs_dl_render_test_full_path.hip"
2024-03-15 01:48:06:  0: INFO: Argument: "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/plugins/65f3a6d8874df71c50592ce4/hrender_dl.py" -f 1 1 1 -o "/mnt/cgsv2021/Projects02/AWS_Test/RS_deadline_test/render/Redshift_ROP/$Redshift_ROP.exr" -d /obj/ropnet1/Redshift_ROP -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/jobsData/65f3a6d8874df71c50592ce4/0_tempVBK4L0" -arnoldAbortOnLicenseFail 1 "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/B_rs_dl_render_test_full_path.hip"
2024-03-15 01:48:06:  0: INFO: Full Command: "/opt/hfs19.5.640/bin/hython" "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/plugins/65f3a6d8874df71c50592ce4/hrender_dl.py" -f 1 1 1 -o "/mnt/cgsv2021/Projects02/AWS_Test/RS_deadline_test/render/Redshift_ROP/$Redshift_ROP.exr" -d /obj/ropnet1/Redshift_ROP -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/jobsData/65f3a6d8874df71c50592ce4/0_tempVBK4L0" -arnoldAbortOnLicenseFail 1 "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/B_rs_dl_render_test_full_path.hip"
2024-03-15 01:48:06:  0: INFO: Startup Directory: "/opt/hfs19.5.640/bin"
2024-03-15 01:48:06:  0: INFO: Process Priority: BelowNormal
2024-03-15 01:48:06:  0: INFO: Process Affinity: default
2024-03-15 01:48:06:  0: INFO: Process is now running
2024-03-15 01:48:09:  0: STDOUT: Detected Houdini version: (19, 5, 640)
2024-03-15 01:48:09:  0: STDOUT: ['/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/plugins/65f3a6d8874df71c50592ce4/hrender_dl.py', '-f', '1', '1', '1', '-o', '/mnt/cgsv2021/Projects02/AWS_Test/RS_deadline_test/render/Redshift_ROP/$Redshift_ROP.exr', '-d', '/obj/ropnet1/Redshift_ROP', '-tempdir', '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/jobsData/65f3a6d8874df71c50592ce4/0_tempVBK4L0', '-arnoldAbortOnLicenseFail', '1', '/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/B_rs_dl_render_test_full_path.hip']
2024-03-15 01:48:09:  0: STDOUT: Start: 1
2024-03-15 01:48:09:  0: STDOUT: End: 1
2024-03-15 01:48:09:  0: STDOUT: Increment: 1
2024-03-15 01:48:09:  0: STDOUT: Ignore Inputs: False
2024-03-15 01:48:09:  0: STDOUT: Output: /mnt/cgsv2021/Projects02/AWS_Test/RS_deadline_test/render/Redshift_ROP/$Redshift_ROP.exr
2024-03-15 01:48:09:  0: STDOUT: Driver: /obj/ropnet1/Redshift_ROP
2024-03-15 01:48:09:  0: STDOUT: Input File: /mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/B_rs_dl_render_test_full_path.hip
2024-03-15 01:48:09:  0: Sending kill command to process hython-bin with id: 11933
2024-03-15 01:48:09:  0: INFO: Sending EndTaskRequest to S3BackedCacheClient.
2024-03-15 01:48:09:  0: DEBUG: Request:
2024-03-15 01:48:09:  0: DEBUG: 	JobId: 65f3a6d8874df71c50592ce4
2024-03-15 01:48:09:  0: Done executing plugin command of type 'Render Task'
2024-03-15 01:48:09:  ERROR: 0: An exception occurred: FailRenderException : Error: Caught exception: The attempted operation failed.
2024-03-15 01:48:09:     at Deadline.Plugins.DeadlinePlugin.FailRender(String message) (Python.Runtime.PythonException)
2024-03-15 01:48:09:    File "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/plugins/65f3a6d8874df71c50592ce4/Houdini.py", line 412, in HandleStdoutError
2024-03-15 01:48:09:      self.FailRender(self.GetRegexMatch(1))
2024-03-15 01:48:09:     at Python.Runtime.Dispatcher.Dispatch(ArrayList args)
2024-03-15 01:48:09:     at __FranticX_Processes_ManagedProcess_StdoutHandlerDelegateDispatcher.Invoke()
2024-03-15 01:48:09:     at FranticX.Processes.ManagedProcess.RegexHandlerCallback.CallFunction()
2024-03-15 01:48:09:     at FranticX.Processes.ManagedProcess.e(String cj, Boolean ck)
2024-03-15 01:48:09:     at FranticX.Processes.ManagedProcess.Execute(Boolean waitForExit)
2024-03-15 01:48:09:     at Deadline.Plugins.DeadlinePlugin.DoRenderTasks()
2024-03-15 01:48:09:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2024-03-15 01:48:09:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel) (Deadline.Plugins.RenderPluginException)
2024-03-15 01:48:09:  0: Unloading plugin: Houdini
2024-03-15 01:48:09:  0: Executing plugin command of type 'End Job'
2024-03-15 01:48:09:  0: INFO: Sending EndTaskRequest to S3BackedCacheClient.
2024-03-15 01:48:09:  0: DEBUG: Request:
2024-03-15 01:48:09:  0: DEBUG: 	JobId: 65f3a6d8874df71c50592ce4
2024-03-15 01:48:09:  0: Done executing plugin command of type 'End Job'
2024-03-15 01:48:10:  Port Forwarder (houdini:1715): Client connected to port forwarder.
2024-03-15 01:48:11:  Sending kill command to process deadlinesandbox.exe with id: 11896
2024-03-15 01:48:12:  0: Render Thread - Render State transition from = 'Rendering' to = 'WaitingForTask'
2024-03-15 01:48:12:  Scheduler Thread - Render Thread 0 threw a major error: 
2024-03-15 01:48:12:  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024-03-15 01:48:12:  Exception Details
2024-03-15 01:48:12:  RenderPluginException -- FailRenderException : Error: Caught exception: The attempted operation failed.
2024-03-15 01:48:12:     at Deadline.Plugins.DeadlinePlugin.FailRender(String message) (Python.Runtime.PythonException)
2024-03-15 01:48:12:    File "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/plugins/65f3a6d8874df71c50592ce4/Houdini.py", line 412, in HandleStdoutError
2024-03-15 01:48:12:      self.FailRender(self.GetRegexMatch(1))
2024-03-15 01:48:12:     at Python.Runtime.Dispatcher.Dispatch(ArrayList args)
2024-03-15 01:48:12:     at __FranticX_Processes_ManagedProcess_StdoutHandlerDelegateDispatcher.Invoke()
2024-03-15 01:48:12:     at FranticX.Processes.ManagedProcess.RegexHandlerCallback.CallFunction()
2024-03-15 01:48:12:     at FranticX.Processes.ManagedProcess.e(String cj, Boolean ck)
2024-03-15 01:48:12:     at FranticX.Processes.ManagedProcess.Execute(Boolean waitForExit)
2024-03-15 01:48:12:     at Deadline.Plugins.DeadlinePlugin.DoRenderTasks()
2024-03-15 01:48:12:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2024-03-15 01:48:12:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2024-03-15 01:48:12:  RenderPluginException.Cause: JobError (2)
2024-03-15 01:48:12:  RenderPluginException.Level: Major (1)
2024-03-15 01:48:12:  RenderPluginException.HasSlaveLog: True
2024-03-15 01:48:12:  RenderPluginException.SlaveLogFileName: /var/log/Thinkbox/Deadline10/deadlineslave_renderthread_0-ip-10-128-18-28-0000.log
2024-03-15 01:48:12:  Exception.TargetSite: Deadline.Slaves.Messaging.PluginResponseMemento d(Deadline.Net.DeadlineMessage, System.Threading.CancellationToken)
2024-03-15 01:48:12:  Exception.Data: ( )
2024-03-15 01:48:12:  Exception.Source: deadline
2024-03-15 01:48:12:  Exception.HResult: -2146233088
2024-03-15 01:48:12:    Exception.StackTrace: 
2024-03-15 01:48:12:     at Deadline.Plugins.SandboxedPlugin.d(DeadlineMessage bgm, CancellationToken bgn
2024-03-15 01:48:12:     at Deadline.Plugins.SandboxedPlugin.RenderTask(Task task, CancellationToken cancellationToken
2024-03-15 01:48:12:     at Deadline.Slaves.SlaveRenderThread.c(TaskLogWriter ajt, CancellationToken aju)

I want to have the advice.

Best Regards
Taka.

You’re making progress.

This error is coming from Houdini - “The attempted operation failed.”
You should check your scene, try to render it locally.

Often this error is caused by a locked node that needs to be modified or something similar.
Unfortunately the way the Deadline plugin works, it stops at the first line showing an error, thus not reaching the actual error in the output.

Can you please try disabling “Enable Path Mapping” in the Configure Plugins → Houdini. Set it to “False”. And maybe unset REDSHIFT_PATHOVERRIDE_FILE.

You want Deadline global rules to handle the Mapped Paths (not Houdini and Redshift), so the “Full Command” would have the S3 bucket paths, not your local paths. In post #22 the Global Mapped Paths look correct.

I see in your logs:
2024-03-15 01:48:06: 0: INFO: Set HOUDINI_PATHMAP to {"/mnt/cgsv2021/":"/mnt/Data/mntcgsv2021386d7e3204ea8557cff0f48192a0df69/", "//cgsv2021":"/mnt/cgsv2021"}
and

2024-03-15 01:48:06:  0: INFO: Redshift Path Mapping...
2024-03-15 01:48:06:  0: INFO: source: "/mnt/cgsv2021/" dest: "/mnt/Data/mntcgsv2021386d7e3204ea8557cff0f48192a0df69/"
2024-03-15 01:48:06:  0: INFO: source: "\\cgsv2021" dest: "/mnt/cgsv2021"

The logs show the command that the AWS Deadline worker is using are local paths, not remote bucket paths:
2024-03-15 01:48:06: 0: INFO: Full Command: "/opt/hfs19.5.640/bin/hython" "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/plugins/65f3a6d8874df71c50592ce4/hrender_dl.py" -f 1 1 1 -o "/mnt/cgsv2021/Projects02/AWS_Test/RS_deadline_test/render/Redshift_ROP/$Redshift_ROP.exr" -d /obj/ropnet1/Redshift_ROP -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-18-28/jobsData/65f3a6d8874df71c50592ce4/0_tempVBK4L0" -arnoldAbortOnLicenseFail 1 "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/B_rs_dl_render_test_full_path.hip"

Thank you for your advices.

I checked “submit Houdini Scenes” and changed “Enable Path Mapping” to false.
image (4)

The rendering is working.
But output folder(local) is nothing and the worker log is down below.

2024-03-18 05:35:41:  0: INFO: Starting Houdini Job
2024-03-18 05:35:41:  0: INFO: Stdout Redirection Enabled: True
2024-03-18 05:35:41:  0: INFO: Asynchronous Stdout Enabled: False
2024-03-18 05:35:41:  0: INFO: Stdout Handling Enabled: True
2024-03-18 05:35:41:  0: INFO: Popup Handling Enabled: True
2024-03-18 05:35:41:  0: INFO: QT Popup Handling Enabled: False
2024-03-18 05:35:41:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2024-03-18 05:35:41:  0: INFO: Using Process Tree: True
2024-03-18 05:35:41:  0: INFO: Hiding DOS Window: True
2024-03-18 05:35:41:  0: INFO: Creating New Console: False
2024-03-18 05:35:41:  0: INFO: Running as user: ec2-user
2024-03-18 05:35:41:  0: INFO: Executable: "/opt/hfs19.5.640/bin/hython"
2024-03-18 05:35:41:  0: CheckPathMapping: Swapped "//cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr" with "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr"
2024-03-18 05:35:41:  0: INFO: Argument: "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/plugins/65f7cc28874df71c50593303/hrender_dl.py" -f 238 238 1 -o "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr" -d /out/Redshift_ROP1 -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/jobsData/65f7cc28874df71c50593303/0_tempkHIL50" -arnoldAbortOnLicenseFail 1 "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/jobsData/65f7cc28874df71c50593303/rs_dl_render_test_B.hip"
2024-03-18 05:35:41:  0: INFO: Full Command: "/opt/hfs19.5.640/bin/hython" "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/plugins/65f7cc28874df71c50593303/hrender_dl.py" -f 238 238 1 -o "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr" -d /out/Redshift_ROP1 -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/jobsData/65f7cc28874df71c50593303/0_tempkHIL50" -arnoldAbortOnLicenseFail 1 "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/jobsData/65f7cc28874df71c50593303/rs_dl_render_test_B.hip"
2024-03-18 05:35:41:  0: INFO: Startup Directory: "/opt/hfs19.5.640/bin"
2024-03-18 05:35:41:  0: INFO: Process Priority: BelowNormal
2024-03-18 05:35:41:  0: INFO: Process Affinity: default
2024-03-18 05:35:41:  0: INFO: Process is now running
2024-03-18 05:35:43:  0: STDOUT: Detected Houdini version: (19, 5, 640)
2024-03-18 05:35:43:  0: STDOUT: ['/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/plugins/65f7cc28874df71c50593303/hrender_dl.py', '-f', '238', '238', '1', '-o', '/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr', '-d', '/out/Redshift_ROP1', '-tempdir', '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/jobsData/65f7cc28874df71c50593303/0_tempkHIL50', '-arnoldAbortOnLicenseFail', '1', '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/jobsData/65f7cc28874df71c50593303/rs_dl_render_test_B.hip']
2024-03-18 05:35:43:  0: STDOUT: Start: 238
2024-03-18 05:35:43:  0: STDOUT: End: 238
2024-03-18 05:35:43:  0: STDOUT: Increment: 1
2024-03-18 05:35:43:  0: STDOUT: Ignore Inputs: False
2024-03-18 05:35:43:  0: STDOUT: Output: /mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr
2024-03-18 05:35:43:  0: STDOUT: Driver: /out/Redshift_ROP1
2024-03-18 05:35:43:  0: STDOUT: Input File: /var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/jobsData/65f7cc28874df71c50593303/rs_dl_render_test_B.hip
2024-03-18 05:35:43:  0: STDOUT: Warnings were generated during load.
2024-03-18 05:35:43:  0: STDOUT: Error loading: /var/lib/Thinkbox/Deadline10/workers/ip-10-128-29-182/jobsData/65f7cc28874df71c50593303/rs_dl_render_test_B.hip
2024-03-18 05:35:43:  0: STDOUT: Warning:     Bad node type found: Redshift_ROP in /out.
2024-03-18 05:35:43:  0: STDOUT:              
2024-03-18 05:35:43:  0: STDOUT: /out/Redshift_ROP1:
2024-03-18 05:35:43:  0: STDOUT:              Unknown channel(s) "f[12]" converted to spare parameter(s).
2024-03-18 05:35:43:  0: STDOUT:              
2024-03-18 05:35:43:  0: STDOUT: /out/Redshift_ROP1:
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_IPR".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_IPRRV".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_IPRVP".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "renderpreview".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "trange".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "f".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "take".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_separator_rrs2".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_rrs2".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_mainSwitcher".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_group_mainTabSwitcherCameras".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_renderCamera".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_group_mainTabSwitcherCamerasRO".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_overrideCameraRes".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_overrideResScale".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_overrideRes".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "override_camerares".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "res_fraction".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "res_overridex".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "res_overridey".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_group_mainTabSwitcherCamerasOver".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_overscanMode".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_overscanData".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_group_mainTabSwitcherNetworks".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_renderNetworks".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_group_mainTabSwitcherMaterialsOvr".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_matOverride_enable".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_matOverride_mat".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_matOverride_exclude".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_defaultMaterialOverride".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_apply_stylesheets".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_group_mainTabSwitcherSeqRendering".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_updateMeshDeform".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_updateMeshNoReload".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_updateMeshNoReload_list".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_group_mainTabSwitcherOptions".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_initSimulations".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_multihreadLoader".
2024-03-18 05:35:43:  0: STDOUT:              Skipping unrecognized parameter "RS_multihreadPacked".
2024-03-18 05:35:44:  0: STDOUT:              Skipping unrecognized parameter "RS_fastPackedGeoExpansion".
2024-03-18 05:35:44:  0: STDOUT:              -- Too many errors, additional errors were suppressed. --
2024-03-18 05:35:44:  0: STDOUT:              The following node types are using incomplete asset definitions:
2024-03-18 05:35:44:  0: STDOUT:     Object/rslightdome::2.0
2024-03-18 05:35:44:  0: STDOUT:     Object/rslight
2024-03-18 05:35:44:  0: STDOUT:     Driver/Redshift_IPR
2024-03-18 05:35:44:  0: STDOUT: Begin Path Mapping
2024-03-18 05:35:45:  0: STDOUT: b''
2024-03-18 05:35:45:  0: STDOUT: End Path Mapping
2024-03-18 05:35:45:  0: STDOUT: ROP type: merge
2024-03-18 05:35:45:  0: STDOUT: Rendering frame 238
2024-03-18 05:35:45:  0: STDOUT: Finished Rendering
2024-03-18 05:35:45:  0: INFO: Process exit code: 0
2024-03-18 05:35:45:  0: INFO: Finished Houdini Job
2024-03-18 05:35:45:  0: INFO: Sending EndTaskRequest to S3BackedCacheClient.
2024-03-18 05:35:45:  0: DEBUG: Request:
2024-03-18 05:35:45:  0: DEBUG: 	JobId: 65f7cc28874df71c50593303
2024-03-18 05:35:45:  0: Done executing plugin command of type 'Render Task'
2024-03-18 05:35:45:  0: Render time for frame(s): 3.831 s
2024-03-18 05:35:45:  0: Total time for task: 4.353 s
2024-03-18 05:35:46:  0: Saving task log...
2024-03-18 05:35:46:  0: Render Thread - Render State transition from = 'Rendering' to = 'WaitingForTask'
2024-03-18 05:35:46:  Scheduler Thread - Render Thread 0 completed its task
2024-03-18 05:35:46:  Scheduler Thread - Scheduler State transition from = 'PreRendering' to = 'PostRendering'
2024-03-18 05:35:46:  Scheduler Thread - Scheduler State transition from = 'PostRendering' to = 'EndJob'
2024-03-18 05:35:46:  Scheduler Thread - Scheduler State transition from = 'EndJob' to = 'WaitingForJob'
2024-03-18 05:35:46:  Scheduler Thread - Seconds before next job scan: 1
2024-03-18 05:35:46:  Port Forwarder (houdini:1715): Client connected to port forwarder.
2024-03-18 05:35:47:  Scheduler Thread - Scheduler State transition from = 'WaitingForJob' to = 'LicenseCheck'
2024-03-18 05:35:47:  Scheduler Thread - Scheduler State transition from = 'LicenseCheck' to = 'LicenseConfirmed'
2024-03-18 05:35:47:  Scheduler - Returning limit stubs not in use.
2024-03-18 05:35:47:  Scheduler Thread - Job's Limit Groups: houdini, redshift
2024-03-18 05:35:47:  Scheduler Thread - Scheduler State transition from = 'LicenseConfirmed' to = 'LicenseCheck'
2024-03-18 05:35:47:  Scheduler Thread - Scheduler State transition from = 'LicenseCheck' to = 'LicenseConfirmed'
2024-03-18 05:35:47:  Scheduler Thread - Scheduler State transition from = 'LicenseConfirmed' to = 'StartJob'
2024-03-18 05:35:47:  0: Render Thread - Render State transition from = 'WaitingForTask' to = 'ReceivedTask'
2024-03-18 05:35:47:  Scheduler Thread - Scheduler State transition from = 'StartJob' to = 'PreRendering'
2024-03-18 05:35:47:  0: Got task!
2024-03-18 05:35:47:  0: Render Thread - Render State transition from = 'ReceivedTask' to = 'Other'
2024-03-18 05:35:47:  0: Plugin already loaded: Houdini
2024-03-18 05:35:47:  0: Start Job timeout is disabled.
2024-03-18 05:35:47:  0: Task timeout is disabled.
2024-03-18 05:35:47:  0: Job already loaded: rs_dl_render_test_B - /out/Redshift_ROP1
2024-03-18 05:35:47:  0: Plugin rendering frame(s): 239
2024-03-18 05:35:47:  0: Render Thread - Render State transition from = 'Other' to = 'Rendering'
2024-03-18 05:35:48:  0: Executing plugin command of type 'Render Task'
2024-03-18 05:35:48:  0: INFO: Sending StartTaskRequest to S3BackedCacheClient.
2024-03-18 05:35:48:  0: DEBUG: Request:
2024-03-18 05:35:48:  0: DEBUG: 	JobId: 65f7cc28874df71c50593303
2024-03-18 05:35:48:  0: DEBUG: 	JobUploadWhitelist: testB.####.exr
2024-03-18 05:35:48:  0: DEBUG: 	JobUploadWhitelistRe: ^.+\.abc$, ^.+\.avi$, ^.+\.bmp$, ^.+\.bw$, ^.+\.cin$, ^.+\.cjp$, ^.+\.cjpg$, ^.+\.cxr$, ^.+\.dds$, ^.+\.dpx$, ^.+\.dwf$, ^.+\.dwfx$, ^.+\.dwg$, ^.+\.dxf$, ^.+\.dxx$, ^.+\.eps$, ^.+\.exr$, ^.+\.fbx$, ^.+\.fxr$, ^.+\.hdr$, ^.+\.icb$, ^.+\.iff$, ^.+\.iges$, ^.+\.igs$, ^.+\.int$, ^.+\.inta$, ^.+\.iris$, ^.+\.jpe$, ^.+\.jpeg$, ^.+\.jpg$, ^.+\.jp2$, ^.+\.mcc$, ^.+\.mcx$, ^.+\.mov$, ^.+\.mxi$, ^.+\.pdf$, ^.+\.pic$, ^.+\.png$, ^.+\.prt$, ^.+\.ps$, ^.+\.psd$, ^.+\.rgb$, ^.+\.rgba$, ^.+\.rla$, ^.+\.rpf$, ^.+\.sat$, ^.+\.sgi$, ^.+\.stl$, ^.+\.sxr$, ^.+\.targa$, ^.+\.tga$, ^.+\.tif$, ^.+\.tiff$, ^.+\.tim$, ^.+\.vda$, ^.+\.vrimg$, ^.+\.vrmesh$, ^.+\.vrsm$, ^.+\.vrst$, ^.+\.vst$, ^.+\.wmf$, ^.+\.ass$, ^.+\.gz$, ^.+\.ifd$, ^.+\.mi$, ^.+\.mi2$, ^.+\.mxi$, ^.+\.rib$, ^.+\.rs$, ^.+\.vrscene$
2024-03-18 05:35:48:  0: DEBUG: S3BackedCache Client Returned Sequence: 3

I don’t know next step and struggle.
Please advice for this problem.

Best Regards
Taka

This error shows Redshift is not being loaded in Houdini 19.5.640, the nodes are replaced with generic ones.

Do you have redshift configured in the ‘packages’ json or houdini.env?

2 Likes

Thank you for the advice.

Then I installed redshift ver.3.5.19 on the worker and make ~/houdini19.5/houdini.env and rewrite houdini.env.

HOUDINI_PATH="/usr/redshift/redshift4houdini/19.5.640;&"

But rendering status is queued and waiting start.
This is a worker log error.

2024-03-19 05:53:49:  Scheduler Thread - Scheduler State transition from = 'PreRendering' to = 'PostRendering'
2024-03-19 05:53:49:  Scheduler Thread - Scheduler State transition from = 'PostRendering' to = 'EndJob'
2024-03-19 05:53:49:  Scheduler Thread - Scheduler State transition from = 'EndJob' to = 'WaitingForJob'
2024-03-19 05:53:49:  Scheduler Thread - Seconds before next job scan: 1
2024-03-19 05:53:50:  Listener Thread - ::ffff:10.128.2.4 has connected
2024-03-19 05:53:50:  Listener Thread - Received message: StreamLog
2024-03-19 05:53:51:  Scheduler Thread - Scheduler State transition from = 'WaitingForJob' to = 'LicenseCheck'
2024-03-19 05:53:51:  Scheduler Thread - Scheduler State transition from = 'LicenseCheck' to = 'LicenseConfirmed'
2024-03-19 05:53:51:  Scheduler - Returning limit stubs not in use.
2024-03-19 05:53:51:  Scheduler Thread - Job's Limit Groups: houdini, redshift
2024-03-19 05:53:51:  Scheduler Thread - Scheduler State transition from = 'LicenseConfirmed' to = 'LicenseCheck'
2024-03-19 05:53:51:  Scheduler Thread - Scheduler State transition from = 'LicenseCheck' to = 'LicenseConfirmed'
2024-03-19 05:53:51:  Scheduler Thread - Scheduler State transition from = 'LicenseConfirmed' to = 'StartJob'
2024-03-19 05:53:51:  0: Render Thread - Render State transition from = 'WaitingForTask' to = 'ReceivedTask'
2024-03-19 05:53:51:  Scheduler Thread - Scheduler State transition from = 'StartJob' to = 'PreRendering'
2024-03-19 05:53:51:  0: Got task!
2024-03-19 05:53:51:  0: Render Thread - Render State transition from = 'ReceivedTask' to = 'Other'
2024-03-19 05:53:51:  0: Loading Job's Plugin timeout is Disabled
2024-03-19 05:53:51:  0: SandboxedPlugin: Render Job As User disabled, running as current user 'ec2-user'
2024-03-19 05:53:52:  '/home/ec2-user/Thinkbox/Deadline10/pythonAPIs/8oI2s+G2DZfcng9zEWRA==' already exists. Skipping extraction of PythonSync.
2024-03-19 05:53:53:  0: Loaded plugin Houdini
2024-03-19 05:53:53:  All job files are already synchronized
2024-03-19 05:53:53:  Plugin Houdini was already synchronized.
2024-03-19 05:53:53:  0: Executing plugin command of type 'Initialize Plugin'
2024-03-19 05:53:53:  0: INFO: Executing plugin script '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/plugins/65f9274b874df71c505934ff/Houdini.py'
2024-03-19 05:53:53:  0: INFO: Plugin execution sandbox using Python version 3
2024-03-19 05:53:53:  0: INFO: About: Houdini Plugin for Deadline
2024-03-19 05:53:53:  0: INFO: The job's environment will be merged with the current environment before rendering
2024-03-19 05:53:53:  0: Done executing plugin command of type 'Initialize Plugin'
2024-03-19 05:53:53:  0: Start Job timeout is disabled.
2024-03-19 05:53:53:  0: Task timeout is disabled.
2024-03-19 05:53:53:  0: Loaded job: rs_dl_render_test_B - /out/Redshift_ROP1 (65f9274b874df71c505934ff)
2024-03-19 05:53:53:  0: Executing plugin command of type 'Start Job'
2024-03-19 05:53:53:  0: INFO: Sending StartTaskRequest to S3BackedCacheClient.
2024-03-19 05:53:53:  0: DEBUG: Request:
2024-03-19 05:53:53:  0: DEBUG: 	JobId: 65f9274b874df71c505934ff
2024-03-19 05:53:53:  0: DEBUG: 	JobUploadWhitelist: testB.####.exr
2024-03-19 05:53:53:  0: DEBUG: 	JobUploadWhitelistRe: ^.+\.abc$, ^.+\.avi$, ^.+\.bmp$, ^.+\.bw$, ^.+\.cin$, ^.+\.cjp$, ^.+\.cjpg$, ^.+\.cxr$, ^.+\.dds$, ^.+\.dpx$, ^.+\.dwf$, ^.+\.dwfx$, ^.+\.dwg$, ^.+\.dxf$, ^.+\.dxx$, ^.+\.eps$, ^.+\.exr$, ^.+\.fbx$, ^.+\.fxr$, ^.+\.hdr$, ^.+\.icb$, ^.+\.iff$, ^.+\.iges$, ^.+\.igs$, ^.+\.int$, ^.+\.inta$, ^.+\.iris$, ^.+\.jpe$, ^.+\.jpeg$, ^.+\.jpg$, ^.+\.jp2$, ^.+\.mcc$, ^.+\.mcx$, ^.+\.mov$, ^.+\.mxi$, ^.+\.pdf$, ^.+\.pic$, ^.+\.png$, ^.+\.prt$, ^.+\.ps$, ^.+\.psd$, ^.+\.rgb$, ^.+\.rgba$, ^.+\.rla$, ^.+\.rpf$, ^.+\.sat$, ^.+\.sgi$, ^.+\.stl$, ^.+\.sxr$, ^.+\.targa$, ^.+\.tga$, ^.+\.tif$, ^.+\.tiff$, ^.+\.tim$, ^.+\.vda$, ^.+\.vrimg$, ^.+\.vrmesh$, ^.+\.vrsm$, ^.+\.vrst$, ^.+\.vst$, ^.+\.wmf$, ^.+\.ass$, ^.+\.gz$, ^.+\.ifd$, ^.+\.mi$, ^.+\.mi2$, ^.+\.mxi$, ^.+\.rib$, ^.+\.rs$, ^.+\.vrscene$
2024-03-19 05:53:53:  0: DEBUG: S3BackedCache Client Returned Sequence: 1
2024-03-19 05:53:53:  0: INFO: Executing global asset transfer preload script '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/plugins/65f9274b874df71c505934ff/GlobalAssetTransferPreLoad.py'
2024-03-19 05:53:53:  0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer...
2024-03-19 05:53:53:  0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in /opt/Thinkbox/S3BackedCache/bin/task.py...
2024-03-19 05:53:53:  0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2024-03-19 05:53:53:  0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2024-03-19 05:53:53:  0: Done executing plugin command of type 'Start Job'
2024-03-19 05:53:53:  0: Plugin rendering frame(s): 1
2024-03-19 05:53:53:  0: Render Thread - Render State transition from = 'Other' to = 'Rendering'
2024-03-19 05:53:53:  0: Executing plugin command of type 'Render Task'
2024-03-19 05:53:53:  0: INFO: Sending StartTaskRequest to S3BackedCacheClient.
2024-03-19 05:53:53:  0: DEBUG: Request:
2024-03-19 05:53:53:  0: DEBUG: 	JobId: 65f9274b874df71c505934ff
2024-03-19 05:53:53:  0: DEBUG: 	JobUploadWhitelist: testB.####.exr
2024-03-19 05:53:53:  0: DEBUG: 	JobUploadWhitelistRe: ^.+\.abc$, ^.+\.avi$, ^.+\.bmp$, ^.+\.bw$, ^.+\.cin$, ^.+\.cjp$, ^.+\.cjpg$, ^.+\.cxr$, ^.+\.dds$, ^.+\.dpx$, ^.+\.dwf$, ^.+\.dwfx$, ^.+\.dwg$, ^.+\.dxf$, ^.+\.dxx$, ^.+\.eps$, ^.+\.exr$, ^.+\.fbx$, ^.+\.fxr$, ^.+\.hdr$, ^.+\.icb$, ^.+\.iff$, ^.+\.iges$, ^.+\.igs$, ^.+\.int$, ^.+\.inta$, ^.+\.iris$, ^.+\.jpe$, ^.+\.jpeg$, ^.+\.jpg$, ^.+\.jp2$, ^.+\.mcc$, ^.+\.mcx$, ^.+\.mov$, ^.+\.mxi$, ^.+\.pdf$, ^.+\.pic$, ^.+\.png$, ^.+\.prt$, ^.+\.ps$, ^.+\.psd$, ^.+\.rgb$, ^.+\.rgba$, ^.+\.rla$, ^.+\.rpf$, ^.+\.sat$, ^.+\.sgi$, ^.+\.stl$, ^.+\.sxr$, ^.+\.targa$, ^.+\.tga$, ^.+\.tif$, ^.+\.tiff$, ^.+\.tim$, ^.+\.vda$, ^.+\.vrimg$, ^.+\.vrmesh$, ^.+\.vrsm$, ^.+\.vrst$, ^.+\.vst$, ^.+\.wmf$, ^.+\.ass$, ^.+\.gz$, ^.+\.ifd$, ^.+\.mi$, ^.+\.mi2$, ^.+\.mxi$, ^.+\.rib$, ^.+\.rs$, ^.+\.vrscene$
2024-03-19 05:53:53:  0: DEBUG: S3BackedCache Client Returned Sequence: 1
2024-03-19 05:53:53:  0: INFO: Set HOUDINI_PATHMAP to {"/mnt/cgsv2021/":"/mnt/Data/mntcgsv2021386d7e3204ea8557cff0f48192a0df69/", "//cgsv2021":"/mnt/cgsv2021"}
2024-03-19 05:53:53:  0: INFO: Redshift Path Mapping...
2024-03-19 05:53:53:  0: INFO: source: "/mnt/cgsv2021/" dest: "/mnt/Data/mntcgsv2021386d7e3204ea8557cff0f48192a0df69/"
2024-03-19 05:53:53:  0: INFO: source: "\\cgsv2021" dest: "/mnt/cgsv2021"
2024-03-19 05:53:53:  0: INFO: [REDSHIFT_PATHOVERRIDE_FILE] now set to: "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/jobsData/65f9274b874df71c505934ff/RSMapping_temphnvI50/RSMapping.txt"
2024-03-19 05:53:53:  0: INFO: Starting Houdini Job
2024-03-19 05:53:53:  0: INFO: Stdout Redirection Enabled: True
2024-03-19 05:53:53:  0: INFO: Asynchronous Stdout Enabled: False
2024-03-19 05:53:53:  0: INFO: Stdout Handling Enabled: True
2024-03-19 05:53:53:  0: INFO: Popup Handling Enabled: True
2024-03-19 05:53:53:  0: INFO: QT Popup Handling Enabled: False
2024-03-19 05:53:53:  0: INFO: WindowsForms10.Window.8.app.* Popup Handling Enabled: False
2024-03-19 05:53:53:  0: INFO: Using Process Tree: True
2024-03-19 05:53:53:  0: INFO: Hiding DOS Window: True
2024-03-19 05:53:53:  0: INFO: Creating New Console: False
2024-03-19 05:53:53:  0: INFO: Running as user: ec2-user
2024-03-19 05:53:54:  0: INFO: Executable: "/opt/hfs19.5.640/bin/hython"
2024-03-19 05:53:54:  0: CheckPathMapping: Swapped "//cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr" with "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr"
2024-03-19 05:53:54:  0: INFO: Argument: "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/plugins/65f9274b874df71c505934ff/hrender_dl.py" -f 1 1 1 -o "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr" -d /out/Redshift_ROP1 -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/jobsData/65f9274b874df71c505934ff/0_tempBvHAZ0" -arnoldAbortOnLicenseFail 1 "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/jobsData/65f9274b874df71c505934ff/rs_dl_render_test_B.hip"
2024-03-19 05:53:54:  0: INFO: Full Command: "/opt/hfs19.5.640/bin/hython" "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/plugins/65f9274b874df71c505934ff/hrender_dl.py" -f 1 1 1 -o "/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr" -d /out/Redshift_ROP1 -tempdir "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/jobsData/65f9274b874df71c505934ff/0_tempBvHAZ0" -arnoldAbortOnLicenseFail 1 "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/jobsData/65f9274b874df71c505934ff/rs_dl_render_test_B.hip"
2024-03-19 05:53:54:  0: INFO: Startup Directory: "/opt/hfs19.5.640/bin"
2024-03-19 05:53:54:  0: INFO: Process Priority: BelowNormal
2024-03-19 05:53:54:  0: INFO: Process Affinity: default
2024-03-19 05:53:54:  0: INFO: Process is now running
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/usr/redshift/redshift4houdini/19.5.640/dso/redshift4houdini.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/COP2_EnableGPU.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/CHOP_Mouse3D.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/COP2_GPUFog.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/COP2_GPUEnvironment.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/COP2_GPUZComposite.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/COP2_GPULighting.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/SOP_VDBUI.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/OBJ_ReLight.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/ROP_OpenGL.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/SOP_Substance.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:54:  0: STDOUT: Houdini DSO error on '/opt/hfs19.5.640/houdini/dso/SOP_InstantMeshes.so'
2024-03-19 05:53:54:  0: STDOUT: 	libXrandr.so.2: cannot open shared object file: No such file or directory
2024-03-19 05:53:55:  0: STDOUT: Detected Houdini version: (19, 5, 640)
2024-03-19 05:53:55:  0: STDOUT: ['/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/plugins/65f9274b874df71c505934ff/hrender_dl.py', '-f', '1', '1', '1', '-o', '/mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr', '-d', '/out/Redshift_ROP1', '-tempdir', '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/jobsData/65f9274b874df71c505934ff/0_tempBvHAZ0', '-arnoldAbortOnLicenseFail', '1', '/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/jobsData/65f9274b874df71c505934ff/rs_dl_render_test_B.hip']
2024-03-19 05:53:55:  0: STDOUT: Start: 1
2024-03-19 05:53:55:  0: STDOUT: End: 1
2024-03-19 05:53:55:  0: STDOUT: Increment: 1
2024-03-19 05:53:55:  0: STDOUT: Ignore Inputs: False
2024-03-19 05:53:55:  0: STDOUT: Output: /mnt/cgsv2021/projects02/AWS_Test/RS_deadline_test/render/testB/testB.$F4.exr
2024-03-19 05:53:55:  0: STDOUT: Driver: /out/Redshift_ROP1
2024-03-19 05:53:55:  0: STDOUT: Input File: /var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/jobsData/65f9274b874df71c505934ff/rs_dl_render_test_B.hip
2024-03-19 05:53:55:  0: Sending kill command to process hython-bin with id: 9464
2024-03-19 05:53:55:  0: INFO: Sending EndTaskRequest to S3BackedCacheClient.
2024-03-19 05:53:55:  0: DEBUG: Request:
2024-03-19 05:53:55:  0: DEBUG: 	JobId: 65f9274b874df71c505934ff
2024-03-19 05:53:55:  0: Done executing plugin command of type 'Render Task'
2024-03-19 05:53:55:  ERROR: 0: An exception occurred: FailRenderException : Error: Caught exception: The attempted operation failed.
2024-03-19 05:53:55:     at Deadline.Plugins.DeadlinePlugin.FailRender(String message) (Python.Runtime.PythonException)
2024-03-19 05:53:55:    File "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/plugins/65f9274b874df71c505934ff/Houdini.py", line 412, in HandleStdoutError
2024-03-19 05:53:55:      self.FailRender(self.GetRegexMatch(1))
2024-03-19 05:53:55:     at Python.Runtime.Dispatcher.Dispatch(ArrayList args)
2024-03-19 05:53:55:     at __FranticX_Processes_ManagedProcess_StdoutHandlerDelegateDispatcher.Invoke()
2024-03-19 05:53:55:     at FranticX.Processes.ManagedProcess.RegexHandlerCallback.CallFunction()
2024-03-19 05:53:55:     at FranticX.Processes.ManagedProcess.e(String cj, Boolean ck)
2024-03-19 05:53:55:     at FranticX.Processes.ManagedProcess.Execute(Boolean waitForExit)
2024-03-19 05:53:55:     at Deadline.Plugins.DeadlinePlugin.DoRenderTasks()
2024-03-19 05:53:55:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2024-03-19 05:53:55:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel) (Deadline.Plugins.RenderPluginException)
2024-03-19 05:53:55:  0: Unloading plugin: Houdini
2024-03-19 05:53:55:  0: Executing plugin command of type 'End Job'
2024-03-19 05:53:55:  0: INFO: Sending EndTaskRequest to S3BackedCacheClient.
2024-03-19 05:53:55:  0: DEBUG: Request:
2024-03-19 05:53:55:  0: DEBUG: 	JobId: 65f9274b874df71c505934ff
2024-03-19 05:53:55:  0: Done executing plugin command of type 'End Job'
2024-03-19 05:53:57:  Port Forwarder (houdini:1715): Client connected to port forwarder.
2024-03-19 05:53:57:  Sending kill command to process deadlinesandbox.exe with id: 9406
2024-03-19 05:53:59:  0: Render Thread - Render State transition from = 'Rendering' to = 'WaitingForTask'
2024-03-19 05:53:59:  Scheduler Thread - Render Thread 0 threw a major error: 
2024-03-19 05:53:59:  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024-03-19 05:53:59:  Exception Details
2024-03-19 05:53:59:  RenderPluginException -- FailRenderException : Error: Caught exception: The attempted operation failed.
2024-03-19 05:53:59:     at Deadline.Plugins.DeadlinePlugin.FailRender(String message) (Python.Runtime.PythonException)
2024-03-19 05:53:59:    File "/var/lib/Thinkbox/Deadline10/workers/ip-10-128-62-183/plugins/65f9274b874df71c505934ff/Houdini.py", line 412, in HandleStdoutError
2024-03-19 05:53:59:      self.FailRender(self.GetRegexMatch(1))
2024-03-19 05:53:59:     at Python.Runtime.Dispatcher.Dispatch(ArrayList args)
2024-03-19 05:53:59:     at __FranticX_Processes_ManagedProcess_StdoutHandlerDelegateDispatcher.Invoke()
2024-03-19 05:53:59:     at FranticX.Processes.ManagedProcess.RegexHandlerCallback.CallFunction()
2024-03-19 05:53:59:     at FranticX.Processes.ManagedProcess.e(String cj, Boolean ck)
2024-03-19 05:53:59:     at FranticX.Processes.ManagedProcess.Execute(Boolean waitForExit)
2024-03-19 05:53:59:     at Deadline.Plugins.DeadlinePlugin.DoRenderTasks()
2024-03-19 05:53:59:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2024-03-19 05:53:59:     at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)
2024-03-19 05:53:59:  RenderPluginException.Cause: JobError (2)
2024-03-19 05:53:59:  RenderPluginException.Level: Major (1)
2024-03-19 05:53:59:  RenderPluginException.HasSlaveLog: True
2024-03-19 05:53:59:  RenderPluginException.SlaveLogFileName: /var/log/Thinkbox/Deadline10/deadlineslave_renderthread_0-ip-10-128-62-183-0000.log
2024-03-19 05:53:59:  Exception.TargetSite: Deadline.Slaves.Messaging.PluginResponseMemento d(Deadline.Net.DeadlineMessage, System.Threading.CancellationToken)
2024-03-19 05:53:59:  Exception.Data: ( )
2024-03-19 05:53:59:  Exception.Source: deadline
2024-03-19 05:53:59:  Exception.HResult: -2146233088
2024-03-19 05:53:59:    Exception.StackTrace: 
2024-03-19 05:53:59:     at Deadline.Plugins.SandboxedPlugin.d(DeadlineMessage bgm, CancellationToken bgn
2024-03-19 05:53:59:     at Deadline.Plugins.SandboxedPlugin.RenderTask(Task task, CancellationToken cancellationToken
2024-03-19 05:53:59:     at Deadline.Slaves.SlaveRenderThread.c(TaskLogWriter ajt, CancellationToken aju)

I read down below error. So, I do installed yum install libXrandr.i686. But there is not change.
‘/opt/hfs19.5.640/houdini/dso/COP2_GPULighting.so’
2024-03-19 05:53:54: 0: STDOUT: libXrandr.so.2: cannot open shared object file: No such file or directory

My worker has /opt/hfs19.5.640/houdini/dso/COP2_GPULighting.so and so on.

Please advice for me.

The issue is with the plugin configuration,
https://help.maxon.net/r3d/houdini/en-us/Content/html/Houdini+Plugin+Configuration.html

Your Houdini.env should contain the following…

HOUDINI_DSO_ERROR = 2
PATH = "/usr/redshift/bin:$PATH"
HOUDINI_PATH = "/usr/redshift/redshift4houdini/19.5.640;&"
PXR_PLUGINPATH_NAME="/usr/redshift/redshift4solaris/19.5.640;&"

i686 is the 32bit version of this and won’t be used

Privacy | Site terms | Cookie preferences