Autoreque frames that hangs.

Deadline
1

Hi!

We are struggeling with when rendering with mantra in Deadline 8. This is a mantra issue and not Deadline related. The render hangs on 0% and renders forever. If we reque the frame it works on the same node and starts instantly.

So what i was hoping is that Deadline could auto reque jobs that useses more than x minutes on 0% of a frame.

Is this possible with deadline using python?

2

Not quite yet. There is an option to have Deadline re-queue a task if the program hasn’t written output for the last X seconds. You can find that API setting here:

docs.thinkboxsoftware.com/produc … dc7fbf6c84

You can use that in “[repo]/plugins/Mantra/Mantra.py” like so:

    def InitializeProcess(self):
        self.SingleFramesOnly=True
        self.StdoutHandling=True
        self.PopupHandling=True
        self.SetUpdateTimeout(45 * 60) # Wait 45 minutes for output and requeue if none is seen
3

Thanks for the fast reply! :slight_smile:

Will see if this helps. But i guess if the frame already exists this wont work.

4

It does not look at the output image, but the stdout of the application:

5

Yes! We noticed that now :slight_smile:

Doesnt seem to work atm but we will try some more.

6

Does this look right?
Mantra.rar (2.24 KB)

7

Yup. Looks good to me.

8

Hmm strange. Nodes that isnt outputting any stdout info for 10mins wont fail/reque.

Is there anything that needs to be enabled in the reposetory setting for this to work?

9

Not that I’m aware of. Testing this over here right now.

10

Hmm. No luck on my side here so far. I’ll ask around to see if there’s some magic I’m unaware of.

11

Ah, it looks like my problems were because I forgot it needs to be in the InitializeProcess function. I didn’t follow my own advice from earlier :slight_smile:

What you’re looking for to make sure the setting was applied correctly is this:

2017-02-21 16:18:49:  0: INFO: Update timeout has been set to 2700 seconds

I have to head out in a bit, so I’ll put my interval down to five minutes and see if I can get some results here before I have to run out the door.

12

Well, it’s been ten minutes and it hasn’t timed out. I’ll raise this with the development team and see what we can do.

13

Thanks for looking into this!

No luck here either so i realy hope you figure it out.
This would solve alot of problems we are currently having on our renderfarm.

14

Well, the good news is that it’s been added to the current development sprint. I’m hoping we’ll get it done in under two weeks. I’ll update here when I see that it’s resolved.

15

Thank you! :smiley:

16

Any news/progress on this?

17

The underlying API function was fixed and released yesterday in 8.0.17.0:
docs.thinkboxsoftware.com/produc … provements

18

Fantastic!

19

Thanks so much for the fix!

This works perfectly now!

20

Cool. Thanks for the feedback.