Deadline doesn't upload to Shotgun

mepp · November 2, 2018, 4:46pm

Hey Marko_Juice,

Did you ever end up figuring out what was going wrong here?

Cheers

Marko_Juice · November 14, 2018, 1:19pm

hey everyone,
Sorry for absent period but we had many other things to sort out.

Basically we never fixed this issue.
We narrowed it down to deadline not populating tasks list properly, or rather we get random tasks listed for users.

As we are getting back into this right now i am more than open for any idea!
Cheers

Marko_Juice · November 14, 2018, 3:29pm

And now same old error is back.
We managed to get deadline listing tasks properly and users.

Weird thing is that every other connection is working, we can create tasks from Deadline, we can read all of the data from submission scripts etc. But it simply doesn’t upload.

Marko_Juice · November 14, 2018, 5:27pm

and on job submitted i get this error now:

kwatts · November 14, 2018, 6:11pm

Took a peak at your error…

looks like the main cause for the error looks correct, python2.6 std lib for htmllib, doesnt have a “HTTPSConnection” class.

Though i suspect, you might be hitting a problem with an older version of python and shotgun.
There was a point a number of months ago, where they deprecated support for some versions of python2.6, because the version doesnt support the new type of ssl certs, something like that ill have to find that link.

But in a nut shell, see if you can figure out why its trying to load python 2.6, most inbuilt deadline jobs are running in its native 2.7 env.

if possible adding some extra debug to the job, might help narrow down what is occuring.

hope this helpes.

eamsler · November 14, 2018, 8:23pm

Looks like Kym’s definitely onto something there:

Just throw “module has no attribute HTTPSConnection” into Google and find a few more examples.

Actually… Deadline has shipoed with 2.7 for awhile now. It seems like whatever’s running that code isn’t using our bundled Python which may have been caused by a bug in one of our builds where “LD_LIBRARY_PATH” wasn’t being set properly. What version of Deadline are you running?

Marko_Juice · November 14, 2018, 8:44pm

Interesting find. Didnt cross my mind because i am pretty sure we dont have 2.6 anywhere.
Deadline version right now is 10.0.10.4.

Will upgrade tomorrow.

Thanks for the help. Will keep you posted.

Marko_Juice · November 15, 2018, 12:18pm

so after updating to Deadline 10.0.21.5

Same problem but different error this time.

Marko_Juice · November 15, 2018, 12:37pm

and resubmiting a job gets these errors OnJobsSubmitted

kwatts · November 15, 2018, 6:54pm

Hey Marko,

So if you look at the shotgun error, the line that contains:

'Shotgun' : Fault : API create() Version.tasks expected [Array] data type(s) but got Hash.

Chances are that your passing in a single task entity dictionary.
The tasks field on a shotgun task, is a multi entity field, its expecting a list of entities, which for you is a list of task dicts.

The simplest way to fix this , is in the data dict in ShotgunUtils.py, where you are declaring the tasks key, you want to add the incoming as a list. example

my guess is your code looks like:

task = sg.find_one('Task', [['id','is',taskId]], ['entity', 'project'])
data['tasks'] = task

Should be:

task = sg.find_one('Task', [['id','is',taskId]], ['entity', 'project'])
data['tasks'] = [task]

If it still doesnt work after this, add a print data line above the line:

new_version = sg.create('Version', data)

then we can inspect the results.

Hope this helps

cheers
Kym

Marko_Juice · November 16, 2018, 9:44am

Actualy my code looks like

task = sg.find_one(‘Task’, [[‘id’,‘is’,taskId]], [‘entity’, ‘project’])
data[‘project’] = task[‘project’]
data[‘entity’] = task[‘entity’]

changing it into

task = sg.find_one(‘Task’, [[‘id’,‘is’,taskId]], [‘entity’, ‘project’])
data[‘tasks’] = [task]
data[‘project’] = task[‘project’]
data[‘entity’] = task[‘entity’]

gives me different error

Error

=======================================================

Event Error (OnJobFinished): IndentationError : (‘unexpected indent’, (‘\\192.168.1.196\x\deadlinerepository\events\Shotgun\ShotgunUtils.py’, 293, 5, "\t data[‘tasks’] = [task]\n"))

=======================================================

Type

=======================================================

PythonException

=======================================================

Stack Trace

=======================================================

File "none", line 228, in OnJobFinished

so i have tried a few other things but nothing seems to work, i am running in circles here, and honestly getting tired of this error.
We have created a new project on SG with default template to check if there is something wrong in our template\setup. But same error appears.

kwatts · November 16, 2018, 2:38pm

Hi Marko,

This is better, your no longer getting a shotgun error.
Your now getting a python indentation error.
python needs to either have all the indentation as 4 spaces or a tab.
in this case, it looks like your shotgunUtils.py is all 4 spaces indentation and the error is letting you know that the new line added , has indentation of a tab.

python needs things to be consistent.

Most* code editors/ text editors have a menu function that allows you to change the indentation of the selected code.
example:

this image is from an editor called sublime, super easy to fix the indentation of any code your editing, especially if your new to python.

Here is a copy of the ShotgunUtils.py with the new lines added and the indentation set correctly:
ShotgunUtils.zip (4.2 KB)

I added an extra print line, just before the sg.create(‘Version’,data) line.
so in case there is still another error from shotgun, it will be easier to see what is occuring in the data set given to shotgun.

Hope this helps.
Kym

Marko_Juice · November 16, 2018, 3:28pm

First of all thanks a lot for your help.
Right now onJobSubmitted i get following error:

=======================================================

Error

=======================================================

Event Error (OnJobSubmitted): Fault : API create() Version.tasks expected [Array] data type(s) but got Hash:

{"type"=>"Task", "id"=>22580}

=======================================================

Type

=======================================================

PythonException

=======================================================

Stack Trace

=======================================================

File "none", line 194, in OnJobSubmitted

File "none", line 132, in CreateShotgunVersion

File "\192.168.1.196\x\deadlinerepository\events\Shotgun\ShotgunUtils.py", line 300, in AddNewVersion

print ‘Creating new version with data:’, data

File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 1220, in create

record = self._call_rpc("create", params, first=True)

File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 3084, in _call_rpc

self._response_errors(response)

File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 3375, in _response_errors

raise Fault(sg_response.get("message", "Unknown Error"))

=======================================================

Log

=======================================================

2018-11-16 16:23:40: INFO: Importing Shotgun API from ‘C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z’…

2018-11-16 16:23:40: INFO: Output path: W:\0000_sg_developmenttemplate\shots\sc01\sh0050\comp\render\sc01_sh0050_scene2_Rnd_v008\sc01_sh0050_scene2_Rnd_v008_.@@@@.jpg

2018-11-16 16:23:42: An error occurred in the "OnJobSubmitted" function in events plugin ‘Shotgun’: Fault : API create() Version.tasks expected [Array] data type(s) but got Hash:

2018-11-16 16:23:42: {"type"=>"Task", "id"=>22580} (Python.Runtime.PythonException)

2018-11-16 16:23:42: File "none", line 194, in OnJobSubmitted

2018-11-16 16:23:42: File "none", line 132, in CreateShotgunVersion

2018-11-16 16:23:42: File "\192.168.1.196\x\deadlinerepository\events\Shotgun\ShotgunUtils.py", line 300, in AddNewVersion

2018-11-16 16:23:42: print ‘Creating new version with data:’, data

2018-11-16 16:23:42: File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 1220, in create

2018-11-16 16:23:42: record = self._call_rpc("create", params, first=True)

2018-11-16 16:23:42: File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 3084, in _call_rpc

2018-11-16 16:23:42: self._response_errors(response)

2018-11-16 16:23:42: File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 3375, in _response_errors

2018-11-16 16:23:42: raise Fault(sg_response.get("message", "Unknown Error"))

2018-11-16 16:23:42: (Deadline.Events.DeadlineEventPluginException)

2018-11-16 16:23:42: at Deadline.Events.DeadlineEventPlugin.b(String aon, Exception aoo)

2018-11-16 16:23:42: at Deadline.Events.DeadlineEventPlugin.OnJobSubmitted(Job job)

2018-11-16 16:23:42: at Deadline.Events.DeadlineEventManager.OnJobSubmitted(Job job, DataController dataController)

2018-11-16 16:23:42: ---------- Inner Stack Trace (Python.Runtime.PythonException) ----------

2018-11-16 16:23:42: File "none", line 194, in OnJobSubmitted

2018-11-16 16:23:42: File "none", line 132, in CreateShotgunVersion

2018-11-16 16:23:42: File "\192.168.1.196\x\deadlinerepository\events\Shotgun\ShotgunUtils.py", line 300, in AddNewVersion

2018-11-16 16:23:42: print ‘Creating new version with data:’, data

2018-11-16 16:23:42: File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 1220, in create

2018-11-16 16:23:42: record = self._call_rpc("create", params, first=True)

2018-11-16 16:23:42: File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 3084, in _call_rpc

2018-11-16 16:23:42: self._response_errors(response)

2018-11-16 16:23:42: File "C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z\shotgun_api3\shotgun.py", line 3375, in _response_errors

2018-11-16 16:23:42: raise Fault(sg_response.get("message", "Unknown Error"))

=======================================================

Details

=======================================================

And onJobFinished:

=======================================================

Error

=======================================================

Event Error (OnJobFinished): SyntaxError : (‘invalid syntax’, (‘\\192.168.1.196\x\deadlinerepository\events\Shotgun\ShotgunUtils.py’, 300, 43, " print ‘Creating new version with data:’, data\n"))

=======================================================

Type

=======================================================

PythonException

=======================================================

Stack Trace

=======================================================

File "none", line 228, in OnJobFinished

=======================================================

Log

=======================================================

2018-11-16 16:32:37: INFO: Found Shotgun Version info

2018-11-16 16:32:37: INFO: Event Plugin Mode: UploadThumbnail

2018-11-16 16:32:37: INFO: Importing Shotgun API from ‘C:\Users\m.zaric\AppData\Local\Thinkbox\Deadline10\pythonAPIs\2018-09-28T133245.0000000Z’…

2018-11-16 16:32:37: An error occurred in the "OnJobFinished" function in events plugin ‘Shotgun’: SyntaxError : (‘invalid syntax’, (‘\\192.168.1.196\x\deadlinerepository\events\Shotgun\ShotgunUtils.py’, 300, 43, " print ‘Creating new version with data:’, data\n")) (Python.Runtime.PythonException)

2018-11-16 16:32:37: File "none", line 228, in OnJobFinished

2018-11-16 16:32:37: (Deadline.Events.DeadlineEventPluginException)

2018-11-16 16:32:37: at Deadline.Events.DeadlineEventPlugin.b(String aon, Exception aoo)

2018-11-16 16:32:37: at Deadline.Events.DeadlineEventPlugin.OnJobFinished(Job job)

2018-11-16 16:32:37: at Deadline.Events.DeadlineEventManager.OnJobsFinished(Job jobs, DataController dataController)

2018-11-16 16:32:37: ---------- Inner Stack Trace (Python.Runtime.PythonException) ----------

2018-11-16 16:32:37: File "none", line 228, in OnJobFinished

kwatts · November 16, 2018, 5:46pm

We ended out resolving this offline.

In the end no edits needed to be done to the “ShotgunUtils.py” file at all.

The problem was coming from the event config, where the task field was incorrect for the shotgun instance.
example:

The field mappings need to match how your shotgun entities are setup and how your shotgun hierarchy works.
The current config for deadline is set for a stock , unchanged shotgun, so if you have changed or added new fields or entities to your hierarchy, you will need to modify this.

All is good in the world again.

Marko_Juice · November 17, 2018, 6:52pm

Hello everyone. So to follow up properly on this i will explain few things that happend.

Version of deadline we had before updating was pointing to python 2.6. We never figured out why but latest update fixed that issue.
Our shotgun fields are customized but our Shotgun is not displaying them properly. We reported this issue to SG and they confirmed something was wrong. That led us to changing fields back to stock or sg_task in this case and as said previously all is good in the world again.

Thanks everyone for help.
Cheers
M