Plugin which uses multiprocessing

Deadline
1

Hey,

is it possible to use pythons multiprocessing module, to start processes next to the main process?
I need to watch a log file to set the progress for a plugin, but it looks like that Deadline ignores my process.
When I run the function itself, it works fine and updates the progress in Deadline. But when I run the function an a process nothing happens (It should print into the log and update the plugin progress).

def SetProgress (Job, progressAmount, currentProgress, arg01, arg02):
    # [...]
    Job.SetProgress(currentProgress)
    Job.SetStatusMessage('Plugin bla blub: %s'%round(currentProgress)+"%")
    # [...]

# [...]
        # [...]
        from multiprocessing import Process
        Job = self
        p = Process(target=SetProgress , args=(Job, progressAmount, currentProgress, arg01, arg02,) )
        p.start()
        Logger(" Process: %s"%str(p)) # This log print works fine with the result: "0: PYTHON:  Process: <Process(Process-1, started)>"
        while not result:
            result = self.MainPluginProcess()
            # [...]
        p.join()
        # [...]

SetProgress is a function and not inside the plugin class. I was testing the multiprocess with dummy files and it worked fine.

Would be great if any one could help me with this.

Thanks

2

Hello,

I think we might have a bit better understanding of what you are doing, and the context of those choices, if you could send over the plugin you are using. If you aren’t comfortable sharing it in public, you could send it to support@thinkboxsoftware.com and we can look it over. The base request should be possible, we’d just like to see where code is going and how it’s being done so we can either make suggestions for your code, or make fixes to ours if needed. Thanks.

3

Hi Dwight,

thanks for the quick reply.
I don’t think that it would help to send the plugin file to you guys, because it’s using a lot a classes and function from our pipeline.
Th plugin itself is just executing the different operations and tools we are using.

I was trying to implement the multiprocess in our pipeline tool, as well inside the plugin itself.
Just to proof if Deadline needs multiprocesses executed in the main process.

I also tried to simplify the multiprocess with just print statements (Deadlines LogInfo and pythons sys.stdout.write) inside the plugin, but with no results.

If you find out a solution or have a fix for working with multiprocesses inside a plugin, please let me know.

Thanks

4

Hello,

I am going to try to have one of our devs look over this, and while I am not sure when this might be, we’ll let you know if we find out anything.

5

Thank you