How to tell the Worker to keep the application open if it's going to render a frame from the same job

Deadline
1

Hello,

I have discovered a problem with the Worker in how it renders our Houdini jobs…

It’s involving load times…

Basically we have a render with a heavy scene and it takes a long time for the Worker to open the scene before it starts rendering… Once it finishes the task it’s rendering the Worker then closes the scene, Deadline then instructs the same Worker to pick up another task from the same job and the whole process repeats… i.e it opens up the scene again from scratch then starts rendering…

Why can’t the Worker just keep the scene open…?

I have discovered that if we do batch rendering (i.e 3 frames per task) then the Worker keeps the scene file open for the duration of this task and effortlessly renders 3 tasks in record time!

See the results of my tests here:

image (1)

As you can see, the total time of the render went from 46minutes (1 frame per task) to only 15minutes! (3 frames per task)

I just told the artists at my facility to start using multi frames renders (I suggested 5 frames per task) but they pointed out a bit of a flaw in doing that… Yes, the render is quicker (when you consider the clock runtime of the job), but each task will take longer as the Worker will be doing 5 frames instead of just one per task… So if you have frames which take 1 hour, then that Worker will be in-use for 5hours… This become a problem if there is a higher priority job that is launched which needs that node ASAP. Ps we do not have interruptible jobs, and do not want to enable that.

TLDR: I was wondering… as a better solution to doing batch renders with multiple frames per task… is it possible to put in a setting that tells the Worker to keep the application open if it knows it’s going to render a frame from the same job as its next task?

Thanks :slight_smile:

2

The “keep application open” approach only works with applications that are capable of doing it, and we have implemented the integration plugins accordingly. We call this “Batch Mode” in the documentation, and you can find more about it here:

The list of applications that support Batch Mode is

3ds Max, AutoCAD, Cinema 4D, Fusion, Lightwave, Maya, MicroStation, modo, Nuke, Softimage

It also depends on the integration plugin type - for example the 3dsmax plugin supports it, but 3dscmd does not. The former is an “advanced plugin” which launches a managed process and talks to the application via a socket created by a dedicated C++ plugin, the latter is simple command line rendering.

As you can see, Houdini is not on the list. It could be done, but would require a full rewriting of the plugin. I will log it as a Wish.

1 Like
3

Hello,

Thank you for the information.

Cheers for plonking it on the to-do list. It would be amazing if you could implement this. It would be a game changer for us and would literally save days of render time.

Cheers,
Shaun.