At the moment, it’s a bit awkward and you do need to adjust those pieces.
I did a quick dive into the Draft plugin, and a creative workaround might be to submit your template with the Draft job and throw your additional arguments in a file and ship it off as an additional aux file. With that, your template could check where the file is being run from and look for your extra auxiliary file along with it.
I think that’s fairly awkward, and this is a common enough issue that I’ll make sure it’s bubbled up the team.
Update: I did some more digging here and chatting with the dev team, and one option would be to provide addtional arguments through the “arguments” field that’s in the Draft job’s PluginInfo file. It won’t allow you to pass along overly complex data, but you could use argparse to provide some future proofed way to pull the data you need.