Shotgun Pipeline Toolkit support

Certainly the choice back when we implemented the bitness check was the best decision. Back then, plugins for 3DS Max chose sides. Some chose to move forward, and some didn’t have time to make the move, or clients couldn’t update everything. Having two installs, one for each, wasn’t uncommon. It worked great, but people move on from those plugins that don’t support 64bit, so now it seems almost funny that someone would have multiple copies installed (I had eight versions of max, and eight versions of Maya on my workstation when I started here in 2010).

Now, lets move into the problem again. You say “Second, the bitness check is done even though it’s set to None”. Let’s take a look at the code here. It’s python, so it’s somewhat readable hopefully even for non-programmers.

(from [repo]\plugins\Nuke\Nuke.py)

[code] ## Called by Deadline to get the render executable.
def RenderExecutable( self ):
build = self.deadlinePlugin.GetPluginInfoEntryWithDefault( “Build”, “None” ).lower()

    nukeExeList = self.deadlinePlugin.GetConfigEntry( "RenderExecutable" + str(self.Version).replace( ".", "_" ) )
    if build == "32bit":
        nukeExe = FileUtils.SearchFileListFor32Bit( nukeExeList )
        if( nukeExe == "" ):
            self.deadlinePlugin.FailRender( "32 bit Nuke %s render executable could not be found in the semicolon separated list \"%s\". The path to the render executable can be configured from the Plugin Configuration in the Deadline Monitor." % (self.Version, nukeExeList) )
    elif build == "64bit":
        nukeExe = FileUtils.SearchFileListFor64Bit( nukeExeList )
        if( nukeExe == "" ):
            self.deadlinePlugin.FailRender( "64 bit Nuke %s render executable could not be found in the semicolon separated list \"%s\". The path to the render executable can be configured from the Plugin Configuration in the Deadline Monitor." % (self.Version, nukeExeList) )
    else:
        nukeExe = FileUtils.SearchFileList( nukeExeList )
        if( nukeExe == "" ):
            self.deadlinePlugin.FailRender( "Nuke %s render executable could not be found in the semicolon separated list \"%s\". The path to the render executable can be configured from the Plugin Configuration in the Deadline Monitor." % (self.Version, nukeExeList) )
    
    return nukeExe

[/code]

The way this works, it that Deadline Slave reads in all of the possible paths, then depending on build, it’ll to a bitness check. Yu’ll see here there are two ‘if build =’, one for 64, one for 32. In both of those there’s a “SearchFileListForXXBit()” function. That’s what’s exploding because within it (in the C# code) we’re doing those tests. We can only say if it passed or failed, so if you say you want a bitness, we just say “nope, didn’t find it”.

You’ll also see at the bottom here there is an ‘else’. That’s for catching the case that you didn’t say you wanted a 32 or 64 bit build. We don’t do any testing there whatsoever, we just see if the path exists. This is why it works when choosing ‘none’. The choice by the user is usually saved on subsequent submissions, so whichever version they are using, it shouldn’t be too tedious to swap between them.

If it’s actually not working for you, can you e-mail my personal address (edwinamsler@thinkboxsoftware.com) so we can set a time to troubleshoot this? I’d probably need your copy of the above Nuke plugin file. We’ll get the technical stuff sorted for you pretty quickly.

Now on to the wonders of software politics! Much more complicated and touchy situation.

I want to start out by saying I completely agree with you that this feature is antiquated and has, at this time, likely a vanishingly small use case. I also want it gone because of code duplication: I’m bothered an unhealthy amount about seeing nearly the same source code copied over and over with little change. That’s because I’m strange and really need to get over that :smiley:

Now, why hasn’t this been done yet? The short of it is that the Deadline team is FANTASTIC at balancing opportunity costs and keeping people happy. We don’t always do things because they should be done, we do them because the greatest number of people will be served for the most efficient use of time. Because few people have come saying the bitness feature has caused them problems (I should be able to solve your case with a phone call and some custom scripting), the effort of updating (and testing!!!) 40-plus plugins and 80-plus submission scripts to avoid supporting bitness checks hasn’t been done.

Internally here, there have been great tug-of-wars between myself and the Deadline lead over these sorts of problems (we’ve already had one over this!). Both sides usually learn something, and for complex problems, occasionally better features and implementations fall out.

All of this to distill the current problem down to the following:

  1. Bitness check is failing for you. We both feel it should be removed entirely.
  2. Deadline team feels it’s not yet worth the effort. This is frustrating, but can change.
  3. I can fix this problem for you. Even if I have to re-code the darn plugin. It’s only deleting ~8 lines per plugin.
  4. Making a change for one is easier than changing for all. We need to weigh the possible consequences.

So, I’m kind of done typing now. Can we set up something so I can fix these problems for you, then I can go off and talk with the Deadline lead when he gets back from vacation?

Hi Edwin,

Thanks again for taking the time to dig into this.
From the code, it’s clear that the bitness check isn’t what’s failing here, which suggests it’s failing in the routine that tries to actually executed the provided path. In

, you see what happens if a shell script is provided in place of a binary executable, when executed on a an OSX/posix system (sorry for the bad quality).

On the plugin side, you will see what we initially used in place of a nuke executable when looking at

(sorry again for the bad image quality).

What we might see is a mix of incorrect error handling - it claims the bitness check went wrong, even though the error message states the provided file is not a valid Mach-O file. The statement is true, because it’s a shell script with executable bit set, therefore you can execute it natively on a posix system.

This is what I tried to be talking about all the time, maybe it’s a bit clearer now.

The technical debt you spoke about in the second part of the text is something entirely different, and I guess it’s something the deadline team should really try to work on. Bugs have been duplicated that way, and simple changes will now take many times as long as they should. Even though this is off-topic, my recommendation is to prohibit copy-paste and simply not allow that kind of code. After all, it makes the development process more cumbersome and error prone, which will be visible in the overall product quality. When I had to work on deadline submission scripts years ago, the first thing I did was to deduplicate the code of all submission scripts I had to touch.

Look at the vast amounts of text you and I produced here to dig down to a simple issue, which shouldn’t have been an issue in the first place. It’s a great example on how things shouldn’t be in many ways.

If there are open questions on your side, please message Timor or me to setup a quick call as we’d be glad to help. As mentioned previously, we worked around the issue by providing binary executables for deadline using cxfreeze, which is fine for now and easy enough to maintain.

Cheers,
Sebastian

Speaking of that, can you send us your Nuke.py plugin script? I want to see if it errors the same for us.

Hi Edwin,

You can create a file called nuke and put in the following lines.

#!/usr/bin/env python
#-*-coding:utf-8-*-
print "it works"

On a posix system, you should make the file executable like so

chmod +x nuke

As you can see, this is just a standard posix executable script, which should work similarly with bash or ruby scripts with an adjusted shebang.
As mentioned before, I wouldn’t expect this to work on a windows system, but it would be useful of deadline if it could execute .bat files for example. If this is implemented correctly, .py files should be runnable too in case a python interpreter was installed correctly and associated with the file extension.

Thanks