[SOLVED] [THREADING] Any advice on how best to multi-thread a set of method calls within a Post-Job Task?

Deadline
1

Hi there,

I’m currently looking to execute a series of file deletions on temporary job data using python’s stock threading module.

When I’m enumerating the threads and telling it to join threads I’ve created it’s also detecting the ‘dummy’ threads and telling it to not join them but I’m still getting deadline process deadlock. I’ve capped the max threads to 3 or 4 etc.

The script outwith Deadline works nicely but keen to integrate this into our pipe to help speed up things further.

Is there any additional considerations when executing such threading setups?

I’m testing using Windows on 10.0.21.5 and re-queuing a job and marking as complete in Deadline Monitor to trigger this in my tests.

Cheers

Jamie

2

I discovered that mainThread, which my local tests was referring to, was not the post-job task’s parent but rather the thread that launched the Deadline client processes.

If I take that into account and identify what the post-job’s parent thread is, in this case Dummy-1, and be careful to not join those threads into my grouped thread launches we were then able to execute the file/folder deletion processes in more efficient manner.

Cheers

Jamie

2 Likes