Hello - I wanted to make a dedicated post for this issue.
I am attempting to submit a python job to Deadline where I want to use subprocess.popen (or equivalent) to spawn a detached process that runs an executable.
Running on Windows and struggling to get an independent process to spawn with my Python job.
When the task completes, any spawned processes from the task terminate. I’ve tried a slew of flags for subprocess.popen as well as other Pythonic ways to start a program with no avail.
Is there an accessible way to force Deadline NOT to terminate all child processes spawned by the main process? Again, trying to launch an executable via Python job that stays open after the Python job finishes.
Thank you,
Clayton
I think Deadline is going to keep you from doing this, since processes started by the Worker should all belong to its process tree and it’ll terminate everything when the task completes. I’m not finding anything obvious from python’s subprocess that’d let it create and release a process.
What are you looking to do? Maybe there’s another way to go about this the folks in here can help with 
Quick on the draw, Justin. Thank you for the response.
I am trying to launch a persistent executable on a target worker machine via Houdini PDG/TOPs through Deadline.
Houdini PDG/TOPs lets me push a task to Deadline that is just a Python script. I took Houdini out of the equation completely (for simplicity) and focused on getting this to work with a Misc Deadline Python job.
Hoping for a way to have Deadline not simply terminate everything after the task completes.
A good (yet useless) example is opening a session of VSCode on the worker machine via a Deadline Submission (through Houdini PDG or vanilla Python Job). I can get the application to open, but as soon as the job finishes, it terminates.
PLAN B: Would be to create a standalone, running Python process and use sockets to listen for a request locally. Upon receiving the request, popen the executable in that separate process. That way the Deadline/Python is simply sending a socket request out and closing.
Pretty roundabout plan that creates more dependencies. I’d like to avoid doing this if possible.
Let me know if I can provide you with any more details. Thank you!
Interesting - Deadline’s not going to be a helper here. Plan B is the most Deadline compatible method, but I think Deadline’s just the wrong tool for starting an application that needs to stay running without occupying a Worker.
You could run multiple Workers on the machine and have one of the Workers manage starting and stopping that application. But at that point it might be simpler to shove it into a service and let Deadline/PDG trigger the start and stop of that service.
Very good! Plan B is being implemented and will work fine. Thank you for confirming this functionality is out of the scope of Deadline. I appreciate the support provided. Cheers!
1 Like