Houdini/Redshift log parsing update frequency

I’ve modified the default Houdini submitter to support task progress percentage for Redshift. This works great on multiple frame packets as I’m using the completed frame number to calculate the percentage…

I’m trying to add single frame progress percentage based on the “x/xxx block completed” that Redshift reports in the log.

My issue is that Deadline does not seem to update the log in anywhere near realtime. Rendering the same job from the command line updates the log every line that gets spit out, Deadline seems to update it in large chunks, or after a certain number of lines build up in the buffer or something, I’m not quite sure… But I can set a render off that takes 15 minutes and Deadline won’t update the log one single time after the frame starts rendering, then when it finishes it will spit out all the saved up lines at once.

I’m trying to understand why and see if there is any way to increase this frequency, so the progress can be monitored with more fidelity.

1 Like

@smbell Thank you for your modification of the houdini plugin. It makes a big difference in usability.

Could we get this man some support?

Hello!

Assuming that that progress reporting is going out to STDOUT what you’re describing should be working. Where are you seeing this buffering behavior? Is it when you connect to the Worker log through the Monitor? Or is the progress bar not updating as it should be?

There can be a delay when updating progress as the Worker only reports back to the Database at an interval somewhat set by the number of Workers connected to the farm. And the Monitor only updates every couple of minutes (which can also get adjusted based on the number of Workers on the farm). I’ve seen that eat about 5 minutes, but 15 minutes going by makes me think that maybe those progress lines aren’t being captured by the plugin at all.

Could you share the work you’ve done? I got dug into a different progress reporting issue about a month ago so I’ve learned a couple of things not to do!

I can’t speak for @smbell, but here’s his plug in:

https://www.redshift3d.com/forums/viewthread/24300/#204587

Hi, sorry for the delay, I wasn’t getting the emails for this thread for some reason. The buffering I’m seeing is when I am connected to the worker log and viewing the output from Redshift. I don’t have the subframe progress implemented in the current version of my scripts, but when I tried it a while ago, the progress bar would not update until the same time as the worker log. Here is a link to the current scripts for Redshift.

https://drive.google.com/file/d/1ehtSlFERiZ_ytbKgB8t3YK7sAElGnLlH/view?usp=sharing

If you have Redshift/Houdini there, you can set off a render and compare the output in the worker log vs the output from a command line render.

I can implement the subframe progress and try it again, but this was the behavior I have had in the past so I didn’t move forward with it.

Edit: If I get some time in the next few days, I will set it up as I had it before and we can see how it goes this time.

Privacy | Site terms | Cookie preferences