On-prem and UBL licensing for Nuke

Deadline
1

Hello,
We are running into an issue while attempting to use a limit that will use UBL licensing once our on-prem nuke license count is reached.
We do not store any of our env vars for licensing on the system or user level. We typically set these when we run our pipeline bootstrap/apps/render. For Nuke farm submissions, we had been appending foundry_LICENSE to the actual job with an event.
ex) job.SetJobEnvironmentKeyValue(“foundry_LICENSE”, “4101@onpremserver”)

This has been working just fine for our on-prem rendering. However we recently have begun using our own license forwarder with nuke UBL usage.

We can’t append foundry_LICENSE to the job environment because it will override the value set for the UBL usage when the limit’s “Enable Existing License Usage” count is met.

I then tried using a JobPreload.py in the nuke plugin folder.

.....
job = deadlinePlugin.GetJob()
deadlinePlugin.SetProcessEnvironmentVariable("foundry_LICENSE", "4101@onpremserver")
....

If I set the value here it also overrides the value set for the UBL usage when the limit’s “Enable Existing License Usage” count is met.

Has anyone else come up against this issue or found a way around it?

Thanks,
-Jake

2

The UBL environment changes get set before the render sandbox is created, and SetProcessEnvironmentVariable (and any other way of setting the environment) is run when the sandbox is created which is why you’re seeing the environment variable getting overridden.

The usual tactic I’ve seen is to set the on-premise license server settings on the system. If you’re really unable to do that, I’d maybe try setting them for the system in an event plugin that runs with OnSlaveStartedCallback. That way when the Worker starts your on-premise license server settings are set and can’t override what’s set by the UBL system.

3

Hi Justin,
Thanks for your suggestions.
I tried all of the following, one at a time, within the event:

    def OnSlaveStarted(self, slavename):

        Environment.SetEnvironmentVariable("foundry_LICENSE", "4101@onpremserver")
        os.environ["foundry_LICENSE"] = "4101@onpremserver"
        self.SetProcessEnvironmentVariable("foundry_LICENSE", "4101@onpremserver")

I’m guessing because of the sandboxing that is happening during the render process this env variable is ignored?

Thanks,
-Jake

4

Hello,

RLM will take multiple servers in its env var and try them in order for example:

export foundry_LICENSE=4101@mylicserver:4101@anotherserver

If linux, use a colon as a separator; if windows, use a semicolon – not sure which OS you’re using on your nodes.

Maybe you can try to append the UBL foundry_LICENSE e.g.:

onpremlic='4101@onpremserver'
flic = os.getenv('foundry_LICENSE', '')
os.environ['foundry_LICENSE'] = onpremlic + ":" + flic

IDK if it will work, but easy enough to try.

5

Hi Jarak,
Thanks for your reply.
Yes I tried this initially with the JobPreload.py
This does allow both license servers to pick up. However it causes the on-prem “License Count” to be ignored and maxes out all our on-prem lics before moving onto the UBL licenses.
For example if our limit has “Enable Existing Licenses Usage” checked, and our License Count is set to 5, but we actually have 20 nuke r licenses on prem, deadline will use all 20 on-prem lics before moving onto our UBL licensing.
Maybe there’s a way around this using groups or pools to specify a machine limit in conjunction with the license limits. I’ll have a think about it.
Thanks,
-Jake

6

Hi Jake,

I was under the impression if using both “Enable Existing Licenses Usage” AND “Use UBL”, the license count was the actual total of on-prem licenses you own. Hopefully someone from Deadline can clarify!

7

Hi Jarak,
Could be the case. Although, some facilities might not want to use their entire nuke r license count in a single limit. Maybe split it up across a couple of different regions or repos, etc.
Would be good to get clarification from Deadline for sure.
-Jake

8

@jarak That’s absolutely correct - the issue here is that if you use Deadline to set your on-premise license server environment variables you’ll overwrite what UBL is doing. Since the on-premise server is a valid option it’ll get used even if you’re expecting UBL to be used if listing all servers.

Here’s how UBL environment variables work. Assume we’ve got an Arnold limit like this:

image
image961×529 19.5 KB

For the first 10 Workers dequeuing tasks from jobs with this limit, nothing is done and your usual environment changes are applied via JobPreLoad.py, event plugins or what-have-you.

Worker 11 engages the UBL system which means:

  • A local port forwarder is started so we know when Arnold starts and stops talking to a license server. This way we can charge you by the minute of actual license usage versus just how long the task ran for.
  • foundry_license=4101@license-forwarder is set on the render sandbox as it’s created. (this varies per DCC, but it’s usually just this. SideFX is more complicated).
  • Your usual environment changes are applied via JobPreLoad.py, event plugins or what-have-you. This is what’s causing the 11th Worker to go to your on-premise server as you’re overwriting foundry_license.

You could either add a if foundry_license is set do nothing or you could set foundry_license on the system. You both already kind of know all this, but hopefully it’s a little clearer all lain out how things are going awry.

1 Like
9

I was surprised that the os.environ isn’t affecting the system, but that’ll be only affecting the event plugin sandbox. Sorry about that, I wasn’t thinking on that front!

10

Hi Justin,
Thanks for clarifying all of this. I now have a solution that’s working for us!
Using the JobPreload.py - something like this:

def __main__(deadlinePlugin):
    job = deadlinePlugin.GetJob()
    deadlinePlugin.LogInfo("JobName: %s" % job.JobName)
    deadlinePlugin.LogInfo("JobId: %s" % job.JobId)

    nukefoundrylicense=Environment.GetEnvironmentVariable("foundry_LICENSE")
    if not nukefoundrylicense:
        deadlinePlugin.SetProcessEnvironmentVariable("foundry_LICENSE", "4101@onpremserver")
        print("Setting foundry_LICENSE to on-prem license server: 4101@onpremserver")
    else:
        print("Using UBL. foundry_LICENSE is set to license forwarder port and ip: " + nukefoundrylicense)

@jarak thanks for your input as well.

-Jake

2 Likes