Houdini/redshift, $HIP and $JOB paths no longer work

[Phil_Jackson]

Hi, Ive just done a total reinstall of deadline to 10.1.17.4 and Im having issues rendering a simple test scene in houdini/redshift - this is doing something that im pretty sure used to work.

It no longer seems to work when i use $HIP or $JOB, so if i use a default render output of
$HIP/render/$HIPNAME.$OS.$F4.exr
The render fails and I get an error-

FailRenderException : Error: Caught exception: [Error 5] Access is denied: ‘$HIP’
at Deadline.Plugins.DeadlinePlugin.FailRender(String message) (Python.Runtime.PythonException)

And the log says-

2021-07-20 19:45:19: 2: STDOUT: Creating the output directory “$HIP/render”
2021-07-20 19:45:19: 2: STDOUT: Failed to create output directory “$HIP/render”. The path may be invalid or permissions may not be sufficient.

If I change the output path to something on my filesystem that does not use $HIP or $JOB it works fine.

Any Idea why it would be doing this?

[mirkoj]

I can confirm, I was just gonna post about it when I saw this. I confirm that $JOB and $HIP are no longer working at all when used in the output directory. Using absolute path makes it work again.
There is onw more thing not working but will get that on other post. This is sepparate issue as it seems

[mois.moshev]

Interesting, even in the old plugin I could not find where the variables are expanded/evaluated. Could be Houdini that does it internally, though.
Is it the same version of Houdini before and after?

[anthonygelatka]

I’m running 10.1.17.4 on Win10 and just tested this with 18.5.596

Worked fine using V-Ray 5 but failed with Redshift 3.0.49 so maybe something changed with the redshift ROP node?

It is specifying the variable beforehand, not sure if this is an issue with the slashes or recognising the variable

2021-07-20 13:30:32:  0: STDOUT: Setting variable "HIP" to \\scala\share\jobs/houdini
2021-07-20 13:30:32:  0: STDOUT: Setting variable "HIPFILE" to \\scala\share\jobs/houdini/18.5.596_rs3.0.49_simple.hip
2021-07-20 13:30:32:  0: STDOUT: End Path Mapping
2021-07-20 13:30:32:  0: STDOUT: ROP type: Redshift_ROP
2021-07-20 13:30:32:  0: STDOUT: Creating the output directory "$HIP/render"
2021-07-20 13:30:32:  0: STDOUT: Failed to create output directory "$HIP/render". The path may be invalid or permissions may not be sufficient.

I wonder if the issue is with Windows looking for %HIP% rather than $HIP , I’ll try on a linux node when I get chance

[mirkoj]

seems like it is working fine with 10.1.16 so must be something different in the latest deadline…
Anyone from deadline support to plugin, please?

[anthonygelatka]

@Phil_Jackson @mois.moshev @mirkoj

I think I found a bug with evaluating variables

Hey, You seem to have added some new code in the hrender_dl.py for Deadline 10.17.1.4 that breaks renders which use variables in the output path in the redshift node. (I updated from 10.1.3.6) I now get this error: 2021-07-21 12:30:34: 0: STDOUT: Creating the output directory “//server/jobs/$GW_JOB/films/$GW_FILM/shots/$GW_SHOT/renders/$GW_ELEMENT/$OS/$GW_VERSION” 2021-07-21 12:30:34: 0: STDOUT: Failed to create output directory “//server/jobs/$GW_JOB/films/$GW_FILM/shots/$GW_SHOT/renders/…

This bug is found and resolved here, hopefully it (and the other bugs) will be included on the next release

[mois.moshev]

That is why we should have a git history of the Deadline repo. Would be so much easier to see what went wrong.

[anthonygelatka]

+1 for that idea, although since the AWS acquisition things aren’t as visible.

I wonder if Microsoft acquiring GitHub causes issues with the godlike cloud super powers?

[mois.moshev]

Doesn’t have to be GitHub, you could host a git repo anywhere, and there are various ways of securing it if you want to keep it accessible only to customers.
But there has never been a git repo of the Deadline (customer-accessible) code, also before the acquisition.

[Justin_B]

You folks figured this out before I could finish figuring out what was blowing up!

I added a step to run hou.expandString(output) on the output directory before it checks it per @mois.moshev 's suggestion in this file.

Just added the following:

# create the output directory
output_folder_unexpanded = os.path.dirname(output)
output_folder = hou.expandString(output_folder_unexpanded)

hrender_dl.py (34.9 KB)

As for why we added this code we weren’t handling the case where the output directory Redshift is using didn’t exist and would skip the frame with [Redshift] Output file locked or path not available for writing. Skipping frame..

I’ve got a bug report in the system now and I’ll be chasing to be sure we make use of Houdini variables in our testing paths in the future.

[mois.moshev]

Awesome, quick reaction, Justin, thanks.

[mois.moshev]

Justin quick related question - aren’t rops supposed to create their own directories? There seems to be an inconsistent story where some nodes create them, and some don’t. Which is a Houdini+plugins problem, but…

[Justin_B]

I’m honestly not sure - it could be that the Houdini+Redshift scene that we used for testing has issues.

I keep meaning to test it but it keeps falling off my to-do list, but it’s on there. I did poke around the Houdini docs and I don’t see directory creation called out. But I also don’t work in Houdini so you’ll likely know better than I.

[antoinedurr]

I was under the impression that renderers, e.g. Mantra, created their own directories, not the ROPs. Any ROPs creating folders is typically due to pipeline glue.

[mirkoj]

got same old $JOB issue with houdini 19.0.455, deadline 10.1.19… $JOB is not working proerply

[mirkoj]

Just did a bit more testing and can only confirm. All my textures with $JOB are not loaded (Houdini 19.0.455, redshift 3.0.61), and replacing $JOB with full path fixes thing. But that is not really a solution. Seems like this fix posted before is not working anymore as well.
This is a kind of big deal not having $JOB variable working.

[mirkoj]

Wondering if there is anyone working on deadline anymore? having couple different issues running here and none of them receive any look at all. I remember before when devs were active and solving issues rather fast. Now I see things being ignored for over a year

[Justin_B]

Have you cut a ticket to awsthinkbox.zendesk.com on this issue before? I’m not finding much chatter about this issue outside of this thread. I assume you’re testing with the file I updated?

We’re working to get more active in the forums, but for the moment it’s an early experiment in how much person-time gets put into the forums and what kind of value we get from the work.