- For already submitted jobs - how do we edit its “Job Info Parameters” as well as its “Plugin Info Parameters”?
If the property you want to change isn’t exposed in the Job’s Modify Job Properties window you’ll have to resubmit the job with the desired settings.- How can we edit the already submitted Gaffer job to add the “-threads 16” parameter to it? and how can we get the respective job’s execution also to pick it up and respect that flag for the actual command line execution?
You’re referring to this Gaffer plugin?. It looks like you’d have to either edit the plugin or reach out to the author.
- Can we have the Job’s “Concurrent Tasks” be a dynamic or programmatically calculated value? For example, for the Deadline-Gaffer job, we want to drive this through the job’s submission parameter “-threads int” value. And hoping to implement it in a way like the one below:
You can modify the concurrent tasks programmatically, the issue is that’ll apply to the whole job and not just tasks picked up by individual Workers. Depending on how high the --threads
option can go you might be able to interrogate the Worker for its CPU thread count and build that argument then. The nitty-gritty on how to do that I’m not certain of, but it should be possible.
You could also always submit with --threads 16
and set the maximum concurrent tasks for each size of worker. Then on the job set the concurrent tasks to some high number, and the Workers will bring it down to their local maximum set in ‘Concurrent Task Limit Override’.