Maya slow rendering issue

Deadline
1

I’m having a problem with slow render times when rendering Maya scenes through Deadline, and I was wondering if you could help me figure out what’s going on.

Here’s my setup: My repository and Deadline monitor are both running on my main machine (quad core 3Ghz, 8GB RAM, over 1TB of free disk space), the same machine that also stores my Maya projects (on a network shared drive). I have 5 render computers (also quad cores, 4GB RAM, over 300GB free disk space) running the Deadline slave. All machines are running 64bit Vista SP1, have been defragged, rebooted, etc.

What I’m doing is rendering about 600 frames of an animation that includes a Maya fluid effects 3d fluid container. The fluid is fully cached into the projects directory on the network shared drive. My render quality settings are average (nothing fancy: 320x240, high quality, no multi-pixel filtering or motion blur or shadowing, shading quality of 3 on the fluid).

If I batch render from within Maya on my main machine alone (not using Deadline), each frame takes about 2-3 minutes to render, no matter which frame it is or how far along in the animation the frame is. So a total render time for 600 frames comes out to a bit less than 1800 minutes.

When I use Deadline, the render times are significantly higher. I’m rendering with 5 slaves, each slave getting tasks in 10 frame chunks. At the beginning (frame 1), the renders go quickly (25 to 30 minutes per 10 frames, or 2-3 minutes per frame, so comparable to the batch render from within Maya on my main machine). However, as the job progresses, it starts taking longer and longer (and longer) to render each 10 frame chunk. By frame 300-309, it’s taking 324 minutes to render those 10 frames - at 32 minutes per frame, that’s a ten-fold increase from back at frame 1. The actual increase in render times is exponential, so the more frames I have to render, the longer and longer it takes to the point that we’re talking 2 hours per frame and worse. Yikes.

I should point out that this only happens with renders involving Maya fluids. While the cache files for the fluid aren’t trivial (3MB per frame), I can’t imagine having to move 3MB over the network should create such huge delays. Is Deadline maybe trying to read all the cache files up to the current frame each time it renders a frame? (In other words, at frame 100, is it trying to read all the cache files from frame 1 to 100, then for frame 101 it’s trying to read all the cache files from frame 1 to 101?) Watching the network traffic on each render machine makes me think it’s got something to do with that (the render machines are pulling down tons of data from the network during these renders, and the CPUs on the render machines aren’t at 100% for a majority of the time).

Also, I’ve already tried setting the “Enable local rendering” option under “Configure Plugins…” for both MayaCmd and MayaBatch. And I’ve tried running these jobs with “Use Mayabatch Plugin” (under “Advanced Options” of “Submit Maya Job to Deadline”) both on and off. Neither made any tangible difference.

Again, this doesn’t happen with any of my other Maya scenes that don’t have fluid containers (and large caches). Those render in quick, consistent times.

Thanks for any help. (And cool South Park avatar.)

2

Have you tried rendering from the command line using Render.exe to see if you experience the same issue? If you’re using the MayaCmd plugin (ie: MayaBatch option is turned off), the command line arguments that Deadline passes to Render.exe are printed to the slave log, so you could use those to replicate the same options from a command prompt. You can still render in chunks of 10 frames, and it would be interesting to see if the later frames take longer than then earlier frames. The MayaCmd plugin is essentially performing command line rendering, so if you’re experiencing the slowdown using this plugin, I wouldn’t be surprised if manually rendering from the command line produces the same results. If this is the case, it might be a Maya issue, or just a necessary evil of doing fluids through Maya.

I am slightly surprised though that the MayaBatch plugin also produces the same results, since that plugin keeps the scene loaded in memory between tasks (and thus I would expect the cache to remain intact). Note that Deadline doesn’t explicitly control any internal cache - this is all left up to Maya to control. It could be possible that Maya behaves differently in Workstation mode (which I’m assuming is where you’re running your batch render from).

A possible way to get around the slowdown could be to render the fluid job as one large task (set the chunksize to 600 in this case). Of course this isn’t ideal, because you don’t get the benefit of multiple machines working on the job simultaneously, but it would be interesting to see if you still experience the slowdown in the later frames when it’s all rendered in one go.

Cheers,

  • Ryan