AWS Thinkbox Discussion Forums

Draft Submission script error

Deprecated Deadline 6 Bug Reports
#1

Hi,
Submitting a Draft job via the py Draft Submission UI with the Frame Range field empty, causes an out-of-range array index error.
I was trying to submit a *.mov to be converted into another movie format. Should the frame range be empty? What happens when I don’t know how long the move is?!
Can it be left blank?
Mike

#2

That’s actually a complaint I have with the existing draft submission. I think the default behavior should be to not submit a frame-range and just process all frames. That way if I re-render a sub-range of frames it still creates a quicktime of the entire sequence. If you specify a frame range… THEN it should only process the frame range. But by default it should just look at the files and determine the range from that. That’s how I’ve setup my templates.

#3

I thought I had fixed this, but apparently I never did

I’ve attached an updated submitter that shouldn’t explode without a Frame Range. Just drop it in ‘scripts/Submission/DraftSubmission/DraftSubmission.py’ in your repo.

Cheers,

  • Jon

NOTE: I based this updated submitter off the latest version, so if you’re not on the latest 5.2 beta, it might explode when selecting an input file. If so, updating to the latest beta should fix things :slight_smile:
DraftSubmission.zip (4.63 KB)

#4

Thanks Jon.
I made some additional tweaks. see attached file. (update job name when input images or output file-name/extension is changed)

Also, would you be happy to put together a Draft BATCH submission script which automatically handles both incoming image sequences (different file #/##/###/#### padding) but also movie based files as well of various different frame lengths and resolution. I have a lot of image sequences and also movie files (mov/mp4), which I would like to batch submit to Draft and have them all processed as separate Draft jobs but ALL against the same 1 Draft py script.

I really think this would be a big advantage to the current Draft submission system.

Could this be developed quite soon, or should I look towards building this in-house? I need something in place by the end of next week!

Thanks,
Mike

DraftSubmission.zip (4.66 KB)

#5

Mike, I have something that you could take a look at. I’ll email you off-list. It should be able to handle all of your needs.

#6

Cool.
Thanks,
M

#7

Jon,
Forgot to mention, that the auto-calculate the frame list function isn’t working.
I gave it an input image sequence of [mike0000.png to mike0700.png] and the framelist remained “0-0” after the inputImagesModified event function had executed…
Mike

#8

Hey Mike,

Which version of Deadline are you currently running (are you on the latest beta)? If not, that’s likely why the Frame Range stuff isn’t working. If you are on the latest beta, how are you entering the path to your frames? Are you using the file-picker, entering manually, or copy/pasting?

Anywho, regarding your earlier question, I might be able to get you something by tomorrow; sorry I didn’t get to this earlier! The batch thing is definitely something I’ve wanted to look at since someone else had brought it up.

Cheers,

  • Jon
#9

While we’re on the subject, how would you expect the batch submission to handle values such as Output Name/Folder? I suppose the Frame List should just grab whatever is there, and the other values are probably OK to just reuse as-is for the rest of them…

#10

Hi Jon,
Ah right. I was testing using v5.2. Didn’t realise you guys had made a tweak to one of the API calls to make frameRange calculation work differently.

[My vision of Batch Draft submission - which is completely different from DraftBatch or any concept of multiple Draft Template py scripts working together. That’s a whole different world of interesting things…]

I see the batch submission being a ‘switcher’ between either ‘normal’ Draft submission with all the usual settings available or a ‘batch’ mode (both existing within the current Draft submission dialog), where it will handle multiple image sequences and movies either by individual selection or via browsing and selecting a ‘root’ directory with an option to recursively search sub-directories for input image sequences and movies. I see all output names and folders being handled the same way as Draft currently handles output. A Draft directory is created in the directory of each input sequence or movie. This will stop any clobbering of identical files during processing. Simply, the Batch mode allows multiple images sequences/movies to be picked up such as (take note the different number padding on purpose!):

/path1/mike_01.png to mike_10.png
/path1/mike_0002.exr to mike_1000.exr
/path1/sub-path2/mike_review_01.mov
/path3/mike_final.mov
/mike_003.jpg to mike_999.jpg

The Draft submission script should auto work out the frame ranges for each input sequence or ignore for movie files and submit a Draft job for each image sequence or movie against 1 identified Draft Python script. Of course, you could get more fancy by allowing more than 1 python script & subsequent Draft job to be submitted against each image seq/movie, but that’s not something I personally need.

I see all other values just being re-used as is in the submission script dialog.

I actually wrote a similar interface to batch submit Combustion comps back in the day, which allowed artists to just submit a bunch of *.cws files into Deadline and just submit them with whatever they built in render settings were, against their default render output node setting. Of course, this had reduced functionality of pulling various bits of information/operators out of the ASCII based Combustion file using your CombustionCWSFile class in the Deadline submission script in its normal submission mode, but Combustion BATCH submit it did allow us to submit 250+ Combustion files very fast!

I have some image sequence py code to auto-calculate frame ranges(list) per image sequence if that would help you?

Thanks,
Mike

#11

Hey Mike,

That’s a bit more involved than what I’m planning on doing. I was essentially just going to turn the input file picker into a multi-file picker, allowing you to specify multiple inputs; it would then create a separate Draft job for each of these as you describe. The catch is obviously that you’d have to pick them all manually, here.

Once I have that working though, it shouldn’t be a big stretch to have a “Scan Directory” button or something to scan subfolders for media items, and just populate the Output Box with whatever it finds.

And yeah, the submitter I posted was based on the latest beta one which depended on a new function I added in the core Deadline code for Frame Parsing.

Cheers,

  • Jon
#12

Here’s a version that functions as I described in my post above (with the multi-file picker). Extract to ‘scripts/submission/DraftSubmission’ in the repo.

I also updated the frame # parsing to be a bit smarter (hopefully), and it should no longer rely on the latest Deadline Beta build (I just do the parsing directly in the submitter now).

Let me know if you find any bugs in it (I haven’t done a good testing pass on it yet, so it’s not so unlikely that you do…)

Cheers,

  • Jon

EDIT: Removed attachment, see post below for updated version

#13

Hi Jon,
I’m getting a variable referenced before its been declared kind of issue.
See attached error message.

Untitled.png

Mike

#14

Oops, sorry about that; I was just testing with local files so this hadn’t cropped up >_<. Attached a fixed version.

Cheers,

  • Jon
#15

Hi Jon,
Much better. Working now. More feedback for you:

  1. I was expecting it to auto-create on multiple file input to save itself into a “Draft” directory as a sub-directory in the directory of each input file. Instead it does nothing. Should we change it to be identical to the individual file input behaviour?

  2. If I select an input file sequence of say, 100 tga files named something like: “B999_mikes_renders_0001.tga” to “B999_mikes_renders_0001.tga”, unfortunately the frame list gets incorrectly populated with the frame range “999-999”. The frame padding number needs to be extracted by reading it from the right hand end of the filename and not the first numbers it gets to when it reads the filename from the left!

  3. See attached screen-grab of an interface I regularly re-use to allow a switcher interface between selecting multiple files by “file” or by “directory”. I was thinking of something similar to this for the Draft submission, which grabs all the files and then sorts them all out, through the use of your new extra code?

Capture.PNG

Here’s a snippet of the kind of way I do this:

[code] scriptDialog.AddControl( “Separator3”, “SeparatorControl”, “Directory / File Options”, dialogWidth - 16, -1 )

scriptDialog.AddRow()
scriptDialog.AddControl( "DirFileChooserLabel", "LabelControl", "Selection Method", labelWidth, -1 )
DirFileChooserRC1 = scriptDialog.AddRadioControl( "Select by Directory", "RadioControl", True, "DirFileChooserRadioCtrl", labelWidth + 8, -1 )
DirFileChooserRC1.ValueModified += DirFileChooserChanged
DirFileChooserRC2 = scriptDialog.AddRadioControl( "Select by File", "RadioControl", False, "DirFileChooserRadioCtrl", labelWidth + 8, -1 )
DirFileChooserRC2.ValueModified += DirFileChooserChanged
scriptDialog.EndRow()
scriptDialog.AddRow()
scriptDialog.AddControl( "PNGDirectoryLabel", "LabelControl", "PNG Directory", labelWidth, -1 )
InputDirectoryButton = scriptDialog.AddSelectionControl( "InputDirectoryBox", "FolderBrowserControl", "", "", dialogWidth - labelWidth - 24, -1 )
InputDirectoryButton.ValueModified += InputDirectoryButtonPressed
scriptDialog.EndRow()
scriptDialog.AddRow()
SubDirectoryButton = scriptDialog.AddSelectionControl( "SubDirectories", "CheckBoxControl", True, "Process Sub-Directories", dialogWidth - 290, -1 )
SubDirectoryButton.ValueModified += SubDirectoryButtonPressed
scriptDialog.EndRow()

scriptDialog.AddRow()
scriptDialog.AddControl( "PNGFilesLabel", "LabelControl", "PNG Files", labelWidth, -1 )
PNGFilesButton = scriptDialog.AddSelectionControl( "PNGFiles", "MultiFileBrowserControl", "", "PNG (*.png)|*.png", dialogWidth - labelWidth - 24, -1 )
PNGFilesButton.ValueModified += PNGFilesToProcessChanged
scriptDialog.SetEnabled( "PNGFilesLabel", False )
scriptDialog.SetEnabled( "PNGFiles" , False )
scriptDialog.EndRow()

def DirFileChooserChanged( *args ):
global scriptDialog
global DirFileChooser

radioControl = args[0]
if( bool( radioControl.TheValue ) == True ):
	DirFileChooser = radioControl.Name

if DirFileChooser == "Select by Directory":
	scriptDialog.SetEnabled( "PNGDirectoryLabel", True )
	scriptDialog.SetEnabled( "InputDirectoryBox" , True )
	scriptDialog.SetEnabled( "SubDirectories", True )
	scriptDialog.SetEnabled( "PNGFilesLabel", False )
	scriptDialog.SetEnabled( "PNGFiles", False )
elif DirFileChooser == "Select by File":
	scriptDialog.SetEnabled( "PNGDirectoryLabel", False )
	scriptDialog.SetEnabled( "InputDirectoryBox" , False )
	scriptDialog.SetEnabled( "SubDirectories", False )
	scriptDialog.SetEnabled( "PNGFilesLabel", True )
	scriptDialog.SetEnabled( "PNGFiles", True )

def InputDirectoryButtonPressed( *args ):
global scriptDialog

InputPNGFiles = scriptDialog.GetValue( "InputDirectoryBox" )

if InputPNGFiles != "":
	InputPNGFiles = InputPNGFiles.replace( "\\", "/" )
	scriptDialog.SetValue( "InputDirectoryBox", InputPNGFiles )

def SubmitButtonPressed( *args ):

Get ALL PNG Files by Directory - Check whether Process Sub-Directories is Enabled

if( DirFileChooser == "Select by Directory" ):
	rootdirectory = scriptDialog.GetValue( "InputDirectoryBox" )
	if ( scriptDialog.GetValue( "SubDirectories" ) ):
		try:
			PNGFiles = Directory.GetFiles( rootdirectory, "*.png", SearchOption.AllDirectories )
		except:
			scriptDialog.ShowMessageBox( "Do NOT select symbolic path shortcuts OR UNC Paths", "Error - Access Denied" )
			return
	else:
		try:
			PNGFiles = Directory.GetFiles( rootdirectory, "*.png", SearchOption.TopDirectoryOnly )
		except:
			scriptDialog.ShowMessageBox( "Do NOT select symbolic path shortcuts OR UNC Paths", "Error - Access Denied" )
			return[/code]
#16

  1. You’re right, forgot about that.

  2. Oops, I thought I was handling that, turns out I had a ‘0’ where I thought there was a ‘1’… doh. It should now only populate the Frame Range if there’s more than one thing in what it would consider to be the ‘sequence’ (so it shouldn’t put anything in there for single images or movies). I also changed it to go from right to left as well, just in case.

  3. I feel like this might make things a bit too easy for me to put in the generally released Draft script, if that makes sense… I can see someone accidentally submitting a bazillion Draft jobs by selecting the wrong folder. There’s also the problem of not knowing what kind of file should be considered valid input for the current Draft script. Should it just be looking for images? Movies? Both? What if someone makes a Draft script that requires XML files?

I might be able to make a one-time super special version to upload here, but Ryan might not like it if I put off my 6.0 work for much longer :slight_smile:

Updated version attached.

Cheers,

  • Jon
#17

OK. Understood. Don’t worry about the special version for recursively crawling directories. I’ll take care of this internally. (Don’t want to go upsetting RR now…:slight_smile:)
Last couple of things I forgot to mention earlier…

(1) Shouldn’t the “Output File Name” field also be disabled when we ‘switch’ into batch submission mode?

(2) The “Output Folder Name” doesn’t update itself to “…/Draft” when the InputImages gets modified anymore.

Otherwise, I think we are good to go :slight_smile:

Thanks for your time,
Mike

#18

  1. I went back and forth on this one. I figured someone might want to name all their outputs the same (they’d be in different folders though, obviously). I think I will disable it after all though, it seems like kindof a weird thing to do…

  2. So this is actually intentional. I changed the Output Folder to be relative to the input’s folder if it’s not an absolute (rooted) path. So setting the value as ‘Draft’ is equivalent to the old behaviour of explicitly having the full input directory + the additional ‘Draft’ at the end. I feel this is better than the old way, since if you’re using a different subfolder name (ie, something other than ‘Draft’), you don’t have to change it every time, due to sticky settings. Similarly, if you always want your output to go to the same (absolute) path, changing the input won’t muck with it. If you think the old way was better, I’d love to hear your reasoning, though :slight_smile: I guess it might suck if there are already sticky settings kicking around from before… I might just change its name slightly so it doesn’t pick up old values.

#19
  1. OK
  2. OK, agreed. Let’s keep it the way you have it now.

Can I get a copy of the final version and I’ll put it through its paces tomorrow?

Thanks,
Mike

#20

I’ve spent some time this morning playing with the code and I found a couple more bugs.
I’m just implementing the recursive directory search function now. I’ll send you an updated file shortly.
Mike

Privacy | Site terms | Cookie preferences