AWS Thinkbox Discussion Forums

Upload to Shotgun not working with Draft submitter

Draft
#1

hey draftFolk,

When I choose “DraftSubmission” from the “Submit” menu in the Deadline Monitor, I can add Shotgun data (User, Entity, Version) with the “Use Shotgun Data…” button and process an image sequence with Draft, all without issue. But if I enable “Upload Output to Shotgun” button, then when I click “Submit,” no Deadline job is created. I get no response at all from the gui. If I disable “Upload Output to Shotgun,” I can click “Submit” and it works fine. Also, I am able to upload Draft results to Shotgun through other means (“QuicktimeSubmission” from the right click menu, and from the After Effects submitter), so I suspect the issue is with the “DraftSubmission” script specifically.

It would be great to get this functionality! For us, it would mean anyone rendering image sequences not on our renderfarm could still process quicktimes / upload to Shotgun.

Cheers,
Raphe

#2

Which version of Deadline are you using? If you’re not sure, you can find out by selecting the ‘About Deadline Monitor…’ in the Monitor’s ‘Help’ menu.

#3

Thanks for replying Jon. Here’s the info from my Monitor’s help menu:

Deadline Version: 6.1.0.54584 R
FranticX Version: 2.0.0.54577 R

The original install was a 6.0 & we upgraded to 6.1.

Cheers,
Raphael

#4

Thanks! I fixed the bug that was causing this, and attached a fixed version of the Draft submitter.

To apply this to your repo, simply extract the attached file (DraftSubmission.py) to “scripts/Submission/DraftSubmission.py” in your Deadline repository.

Let me know how it goes!

Cheers,

#5

Hi Jon,

Thanks for the quick fix! Okay, new problem: now the job submits fine, and the Draft folder / output file are created correctly on disk (not sure you need to have the confirm dialog for creating the Draft folder … maybe just throw a warning if it can’t create the folder?) … BUT: the results aren’t uploaded to Shotgun. I’ve clicked on the “Use Shotgun Data” button, and also enable the “Upload Output to Shotgun” option. Upload to Shotgun works from other scripts, like the afterEffects Client script, and the right-click menu QuicktimeSubmission script, so I don’t think the issue is with Deadline’s connection to Shotgun. I’ve attached a screenshot of what the submission gui looks like right before I click Submit … and here’s the output from the Deadline message confirm dialog & Draft Render log:

testDraftUploadToShotgun job has now completed.

Job Details
name: testDraftUploadToShotgun
id: 52f8e6496a55113e80874e9e
comment:
plugin: Draft
frames: 1-4
pool: h5g109
group: none
output location: file:///Z:/Public/Users/rmatto/afterEffects/deadlineTest/renderedImages/deadlineTest_cc_v06/Comp1/Draft

Submission Details
username: rmatto
machine: H5G109
department: pipeline
time: Feb 10/14 09:46:33

Results
frames/chunk: 1000000
total chunks: 1
queued: 0
rendering: 0
completed: 1
suspended: 0

Timings
submitted date/time: Feb 10/14 09:46:33
started date/time: Feb 10/14 09:46:38
completed date/time: Feb 10/14 09:46:43
elapsed running time: 00d 00h 00m 00s

total task time: 00d 00h 00m 04s
average task time: 00d 00h 00m 04s
median task time: 00d 00h 00m 04s
total task startup time: 00d 00h 00m 01s
average task startup time: 00d 00h 00m 01s
median task startup time: 00d 00h 00m 01s
total task render time: 00d 00h 00m 03s
average task render time: 00d 00h 00m 03s
median task render time: 00d 00h 00m 03s

wasted time due to errors: 00d 00h 00m 00s
wasted time due to requeues: 00d 00h 00m 00s

Miscellaneous Info
errors reported: 0
average number of tasks rendering concurrently: 1

Image Size Info
total image file size: 196847
average image file size: 196847
median image file size: 196847

Task Info
peak task RAM usage: 87.766 MB
average task RAM usage: 32.760 MB
median task RAM usage: 32.760 MB
peak task CPU utilisation: 24%
average task CPU utilisation: 10%
median task CPU utilisation: 10%

message auto generated by Deadline Slave
version: v6.1.0.54584 R
machine: H5G109
date/time: 02/10/2014 09:46

//------------------------------------------------------------------------------------------------------------------
… and here’s the Draft Render Log:
//------------------------------------------------------------------------------------------------------------------

=======================================================
Log

0: Task timeout is disabled.
0: Loaded job: testDraftUploadToShotgun (52f8e6496a55113e80874e9e)
0: INFO: Executing plugin script C:\Users\rmatto\AppData\Local\Thinkbox\Deadline6\slave\H5G109\plugins\52f8e6496a55113e80874e9e\Draft.py
0: INFO: Found Draft python module at: ‘C:\Users\rmatto\AppData\Local\Thinkbox\Deadline6\slave\H5G109\Draft\Draft.pyd’
0: INFO: About: Draft Plugin for Deadline
0: INFO: The current environment will be used for rendering
0: Plugin rendering frame(s): 1-4
0: INFO: Draft job starting…
0: INFO: Stdout Handling Enabled: True
0: INFO: Popup Handling Enabled: False
0: INFO: Using Process Tree: True
0: INFO: Hiding DOS Window: True
0: INFO: Creating New Console: False
0: INFO: Looking for bundled python at: ‘C:\Program Files\Thinkbox\Deadline6\bin\dpython.exe’
0: INFO: Executable: “C:\Program Files\Thinkbox\Deadline6\bin\dpython.exe”
0: INFO: Argument: -u “T:\tools\deadline\common\draftTemplates\createQuicktime_v01.py” username=“rmatto” entity=“testAssetA” version=“rapheTest_Draw something (52d814996a55111fa82fec86)” frameList=1-4 startFrame=1 endFrame=4 outFolder=“Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\Draft” outFile=“Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\Draft\deadlineTest_cc_v06_Comp1.mov” inFile=“Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\deadlineTest_cc_v06_Comp1_#####.png” deadlineRepository=“X:\Pipeline\DeadlineRepository6” taskStartFrame=1 taskEndFrame=4
0: INFO: Startup Directory: “C:\Users\rmatto\AppData\Local\Thinkbox\Deadline6\slave\H5G109\Draft”
0: INFO: Process Priority: BelowNormal
0: INFO: Process Affinity: default
0: INFO: Process is now running
0: STDOUT: Checking for license at @H5G109
0: STDOUT: Draft 1.1.0.54257
0: STDOUT: Command line args:
0: STDOUT: username=rmatto
0: STDOUT: entity=testAssetA
0: STDOUT: version=rapheTest_Draw something (52d814996a55111fa82fec86)
0: STDOUT: frameList=1-4
0: STDOUT: startFrame=1
0: STDOUT: endFrame=4
0: STDOUT: outFolder=Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\Draft
0: STDOUT: outFile=Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\Draft\deadlineTest_cc_v06_Comp1.mov
0: STDOUT: inFile=Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\deadlineTest_cc_v06_Comp1_#####.png
0: STDOUT: deadlineRepository=X:\Pipeline\DeadlineRepository6
0: STDOUT: taskStartFrame=1
0: STDOUT: taskEndFrame=4
0: STDOUT: [libx264 @ 00000000003BB4E0] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
0: STDOUT: [libx264 @ 00000000003BB4E0] profile Constrained Baseline, level 4.0
0: STDOUT: [libx264 @ 00000000003BB4E0] 264 - core 125 - H.264/MPEG-4 AVC codec - Copyright 2003-2012 - videolan.org/x264.html - options: cabac=0 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=0 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=3 keyint_min=1 scenecut=40 intra_refresh=0 rc_lookahead=3 rc=crf mbtree=1 crf=18.3 qcomp=0.60 qpmin=4 qpmax=51 qpstep=4 ip_ratio=1.40 aq=1:1.00
0: STDOUT: Output #0, mov, to ‘Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\Draft\deadlineTest_cc_v06_Comp1.mov’:
0: STDOUT: Metadata:
0: STDOUT: encoder : Lavf54.29.104
0: STDOUT: Stream #0:0: Video: h264 (avc1 / 0x31637661), yuvj420p, 1920x1080, q=4-51, 24 tbn, 24 tbc
0: STDOUT: [libx264 @ 00000000003BB4E0] frame I:2 Avg QP:19.24 size: 94376
0: STDOUT: [libx264 @ 00000000003BB4E0] frame P:2 Avg QP:18.80 size: 3329
0: STDOUT: [libx264 @ 00000000003BB4E0] mb I I16…4: 74.3% 0.0% 25.7%
0: STDOUT: [libx264 @ 00000000003BB4E0] mb P I16…4: 1.7% 0.0% 0.0% P16…4: 13.8% 1.9% 0.8% 0.0% 0.0% skip:81.8%
0: STDOUT: [libx264 @ 00000000003BB4E0] coded y,uvDC,uvAC intra: 22.6% 41.5% 15.0% inter: 0.1% 6.4% 0.0%
0: STDOUT: [libx264 @ 00000000003BB4E0] i16 v,h,dc,p: 43% 15% 4% 37%
0: STDOUT: [libx264 @ 00000000003BB4E0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 34% 14% 3% 5% 4% 7% 3% 7%
0: STDOUT: [libx264 @ 00000000003BB4E0] i8c dc,h,v,p: 47% 10% 5% 38%
0: STDOUT: [libx264 @ 00000000003BB4E0] ref P L0: 90.7% 9.3%
0: STDOUT: [libx264 @ 00000000003BB4E0] kb/s:9379.68
0: INFO: Process exit code: 0
0: INFO: Draft job complete!

=======================================================
Details

Date: 02/10/2014 09:46:42
Frames: 1-4
Job Submit Date: 02/10/2014 09:46:33
Job User: rmatto
Average RAM Usage: 0 (0%)
Peak RAM Usage: 0 (0%)
Average CPU Usage: 0%
Peak CPU Usage: 0%

=======================================================
Slave Information

Slave Name: H5G109
Version: v6.1.0.54584 R
Operating System: Windows 7 Professional (SP1)
Machine User: rmatto
IP Address: fe80::2d8c:126c:318:543%11
MAC Address: D4:AE:52:C0:0B:DE
CPU Architecture: x64
CPUs: 4
CPU Usage: 3%
Memory Usage: 5.1 GB / 6.0 GB (85%)
Free Disk Space: 481.470 GB (210.439 GB on C:, 271.031 GB on D:)
Video Card: ATI FirePro 2260

Let me know if you need any more information!

Cheers,
Raphael

draftSubmitter.jpg
draftSubmitter.jpg573×732 123 KB

#6

Hmmm, so the Draft plugin and the Quicktime one handle uploading to Shotgun a little bit differently, but they should both work – I just re-tested the Draft one here to confirm. The problem might be that the Draft plugin is trying to do this upload asynchronously, and might end up getting killed by the Slave before it can finish the upload. I’ll tinker with the plugin a bit, and give you a new version to test.

In the meantime, could you open up the Job Properties of your test job, and paste the Plugin Info Parameters for it here (Job Properties -> Submission Params -> Plugin Info Parameters)? Should help me figure out what’s wrong, if it it does end up being something else.

#7

Sure – thanks for looking into this, Jon:

scriptFile=T:/tools/deadline/common/draftTemplates/createQuicktime_v01.py
ScriptArg0=username=“rmatto”
ScriptArg1=entity=“testAssetA”
ScriptArg2=version=“rapheTest_Draw something (52d814996a55111fa82fec86)”
ScriptArg3=frameList=1-4
ScriptArg4=startFrame=1
ScriptArg5=endFrame=4
ScriptArg6=outFolder=“Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\Draft”
ScriptArg7=outFile=“Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\Draft\deadlineTest_cc_v06_Comp1.mov”
ScriptArg8=inFile=“Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\deadlineTest_cc_v06_Comp1_#####.png”
ScriptArg9=
postRenderScript=“X:\Pipeline\DeadlineRepository6\events\Shotgun\ShotgunUtils.py” Upload 795 “Z:\Public\Users\rmatto\afterEffects\deadlineTest\renderedImages\deadlineTest_cc_v06\Comp1\Draft\deadlineTest_cc_v06_Comp1.mov”

#8

Alright, here’s a new version of the submission script, that actually employs the Qt method of uploading the output; it will actually go through the Shotgun event plugin instead of trying to invoke a separate script to do it. Same thing as last time to apply the update, just extract the attached script to the ‘scripts/Submission’ folder in your repo.

Let me know how it goes!

Cheers,

#9

That works!

And, I have to apologize here – the submitter you sent earlier today worked too. For various sundry reasons I missed it (sent the .mov file to the wrong sg version, when I found the right version it was visually identical to the previous version & there was no datestamp to tell me it was new). So, my fault! But I guess a suggestion comes out of this: if a shotgun email could be generated pointing to the new uploaded content (like the ae client submitter does) – that would have tipped me off. Or if the asset overview stream in Shotgun had been modified I would also have noticed; for example, the last modification to the asset overview stream is 9:38am, even though I just updated new media at 4:06pm. At the moment, updating media in a version happens super quietly.

I do prefer the second submitter you sent only because it generates a separate OnJobFinished event log – i.e., its log output looks clearly different than a draft job I send without the upload to shotgun option enabled.

Cheers,
Raphael

#10

Ahhh, gotcha, no worries! Like you said, the new way is definitely better, so that change needed to be made anyways :slight_smile:

Privacy | Site terms | Cookie preferences