3Delight getting all errors when rendering via Deadline 10.3.0.9

Hello, all my renders return errors when rendering on the farm.
All shaders are flagged Bad node type

2023-08-15 16:32:01: 0: STDOUT: Detected Houdini version: (19, 5, 640)
2023-08-15 16:32:01: 0: STDOUT: [’/var/lib/Thinkbox/Deadline10/workers/render-1/plugins/64dc08e2741e340f6ea5bb0a/hrender_dl.py’, ‘-f’, ‘1’, ‘1’, ‘1’, ‘-g’, ‘-d’, ‘/out/3Delight1’, ‘-tempdir’, ‘/var/lib/Thinkbox/Deadline10/workers/render-1/jobsData/64dc08e2741e340f6ea5bb0a/0_temp97Ohq0’, ‘-arnoldAbortOnLicenseFail’, ‘1’, ‘/mnt/qnap/Clients/_BlackCrown/TEST_FARMuBUNTU_3delight_2.hiplc’]
2023-08-15 16:32:01: 0: STDOUT: Start: 1
2023-08-15 16:32:01: 0: STDOUT: End: 1
2023-08-15 16:32:01: 0: STDOUT: Increment: 1
2023-08-15 16:32:01: 0: STDOUT: Ignore Inputs: True
2023-08-15 16:32:01: 0: STDOUT: No output specified. Output will be handled by the driver
2023-08-15 16:32:01: 0: STDOUT: Driver: /out/3Delight1
2023-08-15 16:32:01: 0: STDOUT: Input File: /mnt/qnap/Clients/_BlackCrown/TEST_FARMuBUNTU_3delight_2.hiplc
2023-08-15 16:32:02: 0: STDOUT: Traceback (most recent call last):
2023-08-15 16:32:02: 0: STDOUT: File “/var/lib/Thinkbox/Deadline10/workers/render-1/plugins/64dc08e2741e340f6ea5bb0a/hrender_dl.py”, line 481, in
2023-08-15 16:32:02: 0: STDOUT: Error: Caught exception: The attempted operation failed.
2023-08-15 16:32:02: 0: STDOUT: Error loading: /mnt/qnap/Clients/_BlackCrown/TEST_FARMuBUNTU_3delight_2.hiplc
2023-08-15 16:32:02: 0: STDOUT: Warning: Bad node type found: 3Delight::dlMaterialBuilder in /mat.
2023-08-15 16:32:02: 0: STDOUT: Error: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlTerminal1.init.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlTerminal1.def.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlTerminal1.parm.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlTerminal1.userdata.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlCarPaint1.init.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlCarPaint1.def.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlCarPaint1.spareparmdef.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlCarPaint1.chn.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlCarPaint1.parm.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder1/dlCarPaint1.userdata.
2023-08-15 16:32:02: 0: STDOUT: Warning: Bad node type found: 3Delight::dlMaterialBuilder in /mat.
2023-08-15 16:32:02: 0: STDOUT: Error: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlTerminal1.init.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlTerminal1.def.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlTerminal1.parm.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlTerminal1.userdata.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlCarPaint1.init.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlCarPaint1.def.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlCarPaint1.spareparmdef.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlCarPaint1.chn.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlCarPaint1.parm.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder2/dlCarPaint1.userdata.
2023-08-15 16:32:02: 0: STDOUT: Warning: Bad node type found: 3Delight::dlMaterialBuilder in /mat.
2023-08-15 16:32:02: 0: STDOUT: Error: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlTerminal1.init.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlTerminal1.def.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlTerminal1.parm.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlTerminal1.userdata.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlCarPaint1.init.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlCarPaint1.def.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlCarPaint1.spareparmdef.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlCarPaint1.chn.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlCarPaint1.parm.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder3/dlCarPaint1.userdata.
2023-08-15 16:32:02: 0: STDOUT: Warning: Bad node type found: 3Delight::dlMaterialBuilder in /mat.
2023-08-15 16:32:02: 0: STDOUT: Error: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlTerminal1.init.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlTerminal1.def.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlTerminal1.parm.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlTerminal1.userdata.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlCarPaint1.init.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlCarPaint1.def.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlCarPaint1.spareparmdef.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlCarPaint1.chn.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlCarPaint1.parm.
2023-08-15 16:32:02: 0: STDOUT: Bad parent found (parent is not a network): mat/dlMaterialBuilder4/dlCarPaint1.userdata.
2023-08-15 16:32:02: 0: STDOUT: Warning: Bad node type found: 3Delight::dlMaterialBuilder in /mat.
2023-08-15 16:32:02: 0: STDOUT: Bad node type found: 3Delight in /out.

Never happened before, looks like a 10.3.09 thing, anyone ?

Tanguy

If it was a Deadline 10.3 issue I wouldn’t expect the issue to get past the plugin. This looks like a scene or render node issue. Is this a new workflow or a new version of Houdini or 3delight?

It does look like Houdini’s not happy about the 3Delight node being in the /out network. What kind of output do you get running the render from the Houdini UI?

Hey Justin, thank to look into it. I am outputting exr and the 3delight ROP is set as usual.
I am doing that for a while. I crashed my previous setup (linux) after a electric shenanigan at my house.
I am using the latest 3Delight. But I see the issue either on ubuntu or rocky.
I used 10.3.0.9 earlier but I had even more problems so I switch back to 10.2.1.1 (both on server and client)
My render goes as expected within Houdini / 3Delight.

When it renders successfully on 10.2.1.1. what do the task reports look like? Do you see the same warnings about the /out network?

Hey @Justin_B , I am running with 10.3.0.9 right now and it is working good, BUT 3delight.
It still gives me error when using the submitter and no error when running via terninal / deadline command

@Justin_B I just did a render via command line and deadline gives me the bad node type as well.
Hmmm, let me make the same test with Karma just to be sure it is not system wide.

@Justin_B Yes it works with karma both manual and via houdini submitter.
So it is 3delight installation in the system which gives me error. I i wll re-install it and will give you an update

Thank you Justin for taking time

@Justin_B
So here we are… the same machine, same os same scene, nothing change BUT restart between both.
before restart: FAIL
after restart: SUCCESS

I place the log in copy
Job_2023-08-17_16-42-19_64deb05b5e12d7306068994f_fail.zip (2.3 KB)
Job_2023-08-17_16-48-04_64deb1b4886284ed0fec17d1_success.zip (2.7 KB)

Any idea what would create this behavior ?

No clue to be honest, but based on what we’re seeing maybe the reboot is needed to finalize 3Delight installation? Or maybe the mismatch of Houdini versions is creating issues?

The failing log fails with a lot of ‘Skipping unrecognized parameter’ errors from /out/3Delight1. And the failing log doesn’t output a version check from 3Delight, like the successful one does but warns of issues when mixing versions:

2023-08-17 16:47:57:  0: STDOUT: 3Delight for Houdini: This 3Delight package is configured to be used with Houdini 19.5.640.
2023-08-17 16:47:57:  0: STDOUT: Using it with another version of Houdini (currently, 19.5.682) might result in crashes and other problems.

Otherwise the logs look identical.