AWS Thinkbox Discussion Forums

Path mapping with Houdini plugin

Hi,
We have some mapped path global rules (tools->configure repo options->MappedPaths) to support Windows and Macs submitting to our Windows farm. But, on the Houdini plugin (tools->configure plugins->Houdini , there is an option at the bottom – Enable Path Mapping – which I set to False. It seems the global rules apply, as in the logs I see ‘Begin Path Mapping’ and 'End Path Mapping. There’s a Houdini bug where rbdfracturematerial SOPs cook themselves to death when the DL code calls hou.fileReferences() (and rbdbulletsolver SOPs don’t seem immune either).
So my question: is there a way to force path mapping to ‘off’ for Houdini jobs?

– Antoine

As you have discovered, the option in the Houdini Plugin configuration does not affect the path mapping of the file references. Here is an overview how things fit together:

  • The option in the Plugin configuration controls whether the environment variable HOUDINI_PATHMAP will be populated with the rules from the Tools > Configure Repository Options > Mapped Paths dialog. When the option is set to False, the env. var. will not be set, so Houdini will not be able to perform its own path mapping.
  • The “Begin Path Mapping” and “End Path Mapping” log messages come from the hrender_dl.py script (around line 341) in the function perform_pathmapping(). It is not dependent on the Plugin config. property, currently it is always applied to all file references and Houdini env. variables.
  • The perform_pathmapping() function calls the function gather_params_to_map() to collect parameters that require path mapping. This is where the hou.fileReferences() is called to collect the references that need remapping.
  • Then it calls pathmap_params() with the temp. dir. and the params list collected in the previous step as arguments to perform the mapping.
  • After the job parameters are remapped, it deals with the Houdini environment variables by calling gather_envs_to_map() to collect a list of the Houdini env., and then calling pathmap_envs() to remap them.

As mentioned, there is currently no control to disable the path mapping of the Houdini file references. If you need to disable the call to hou.fileReferences(), you will have to suppress the call to gather_parms_to_map() around line 353 in hrender_dl.py.

An elegant way to do this would be to introduce an additional control to the (Repo)/plugins/Houdini/houdini.params file. For example, you could add

[EnablePathMappingFileRef]
Type=boolean
Category=Path Mapping (For Mixed Farms)
CategoryOrder=2
CategoryIndex=1
Label=Enable File References Path Mapping
Default=true
Description=If enabled, Deadline will remap the Houdini file references.

Then edit the hrender_dl.py perform_pathmapping() function to look something like this:

def perform_pathmapping(tempdir):
    """
    Perform pathmapping on all input parameters and other select parameters
    :param tempdir: the temporary location that a text file will be written to to aid pathmapping.
    :return: None
    """
    
    if not tempdir:
        print("Temporary Directory has not been set. Skipping Pathmapping.")
        return

    print("Begin Path Mapping")
    
    if (self.GetBooleanConfigEntryWithDefault( "EnablePathMappingFileRef", True )):
        # gather a list of all parameters that need to be pathmapped
        parms = gather_parms_to_map()
        if parms:
            pathmap_parms(tempdir, parms)
    else:
        print("Path Mapping of File References has been disabled in the Configure Plugins > Houdini dialog.")

    envs = gather_envs_to_map()
    if envs:
        pathmap_envs(tempdir, envs)

    print("End Path Mapping")

Now if you disable the new option in the Houdini plugin configuration, no path mapping will be performed on the file references, but will be applied to environment variables. If you want to add a separate control for the environment, you can do it with another parameter and if statement…

I have not tested this, but it is likely to work. You can copy the whole Houdini plugin folder into (Repo)/custom/plugins/Houdini/ first, then modify it there. This way your own version of the plugin will never be overwritten by Repository updates.

Wow, thanks for the super-comprenensive rundown of how pathmapping is implemented, incredibly helpful! In addition to how to add our own feature, I especially like the insight on (Repo)/custom/plugins to avoid repository updates overwriting work we’ve put in.

I’ve submitted a bug to SESI r.e. the crazy cooking caused by hou.fileReferences(), ticket #105440 / bug #112903.

BTW, only /somewhat/ likely to work :wink: – ‘self’ doesn’t exist in houdini_dl.py, only in Houdini.py. But I get the idea and can figure it out.

In retrospect, it seems like it might be a good idea to formalize having two path mapping options for plugins: native pathmapping and Deadline pathmapping. If the app, Houdini in this case, supports native (internal to the app) pathmapping, use that. If not, do it via Deadline code.

Good point.

You might have to get the parameter’s value in the Houdini.py and make it accessible to the houdini_dl.py via an environment variable or even a temporary text file.

Check out the reply by Grant Bartel in this thread, and test his updated houdini_dl.py:

1 Like

Hi @antoinedurr
could you kindly give me some pointers on how to pass the “self” from Houdini.py to houdini_dl.py. Were you able to make it work?
Thanks in advance!

You cannot, because they are separate processes. Your best bet would be to extract whatever you need from the plugin and pass it down as command-line argument to houdini_dl.

Privacy | Site terms | Cookie preferences