I have a phython script that downloads a sequence of images from the Nasa site.
I need to run a bunch of these and thought to use deadline to do this.
However, I’m not sure where to start.
When I look at the ‘Submit Command Line Job’ this also includes a frame list, argument tags and frame tag padding options.
From the help documentation i’m not sure how to approach this in relation to the script i wish to put into it.
What the script does is download images used in HelioViewer.org at certain date-time ranges and by itself creates frame numbers etc.
I modified a testing script I use to show how to do this. The usage would be, for example:
python SubmitHgJobs.py 2015-03-07 10
This would submit 10 jobs starting from date 2015-03-07. I didn’t do much testing, so you may need to refine it a little to get it to work as you expect. SubmitHgJobs.zip (1.46 KB)
I believe a command script job(docs.thinkboxsoftware.com/produc … cript.html) would be easiest, as it will automatically make one task for each command and then multiple slaves can do the work. I was also maybe a scheduled job would be good to grab the previous day’s images in future? You could set it to run daily and grab images, unless the command for each day would be unique. Just a thought.
Ok… I fear I’m going to have to do something horrible… ask for another submit Phython script
I had another phython script to be made to slice up 1 image into 15 parts at the same time.
I showed the guy that wrote it the Batch script that was previously made to submit the Helio script to deadline… but it was beyond his understanding (of phython & deadline).
would it maybe be an option to adapt it for this new image slicing script.
It would be awesome if there would be comments to which parts of the script are deadline specific and which parts can be adapted for a new script in the future (if that is easily separable ?)
Then I can give my scripter that version as a guideline and we should be able to manage in the future.
The current script is attached and basically loads a sequence of images from a location ( -i ) with a start framenumber ( -sf ) and outputs that image sequence in a directory ( -o ) until a certain end frame (- ef ) .
What I hoped to do is feed a 9000 frame range command of this script to deadline and have deadline divide it into smaller tasks (eg. 10) for all nodes to process.
This would be the expected input for the script
-i Z:\input -o Z:\output -sf 0 -ef 100
Which also brings me to a suggestion. At the moment it’s possible to assembly tiles within deadline.
Perhaps an idea to also make a slice module that could slice up big images in a desired division? iss.rar (3.31 KB)
I’m at Siggraph this week, but if you ping me next week to remind me, I should be able to take a look. In the mean time, I think the example script provides the clues needed, if you don’t want to wait. Just walk through the function calls and it should become apparent.
I should have come with this a bit earlier, I’ll try and have a look here again.
As we couldn’t figure out how to change the date setup with just a ‘simple’ frame range.
For now I’m using the command line function with big chunks (and just thought to have an auto-generate-list-script of commands per desired frame-chunk)
Okay, I pushed the other scripting guy a bit and introduced him to this forum and he fixed it!
It’s working at the moment.
Is it also possible to generate the tasks within a batch group?
I did find the new batch feature, but would be great to implement this into the generation script.
As it’s nice to have it in a group instead of 900 jobs in the monitor
i tried to find it in the reference, but don’t seem to find it
(actually… I’d love there to be a grouping feature in the monitor to group certain jobs as it’s easier to manage when running multiple projects at the same time)
I found the batch option in the monitor too… it rocks… somehow i missed it (thought it could only be done by the 3dsmax tile plugin) SubmitIssJobs.rar (1.87 KB)
Okay, having messed around with it for a bit I run into the following things:
When submitting a Python script that generates jobs it is basically generated each task as a job with the name of the script + a suffix
eg.
iss.py 01
iss.py 02
etc.
Which is freaky when I have 10 different sections that each generate 100 jobs based on the same iss.py script
Is it possible to automatically assign / inherit the below properties of a spawing job to it’s spawned child jobs?
inherit name + suffix
inherit priority
inherit white/blacklist
inherit batch name
Would these things ‘simply’ need to be scripted or does it need to be hardcoded?
would it be possible to add a batch name input in the Python submit menu?
would it maybe an idea to have an “inherit properties” button when submitting a python script so that, if selected, all this jobs properties are automatically transfer to it’s spawned children?
