AWS Thinkbox Discussion Forums

Deadline Crashing on FumeFX/FusionWorks

We recently started rendering render elements for FumeFX on the farm and noticed that deadline is continually crashing and throwing up errors. I’ve attached the error report as well as the error that appears in dealdine.

\

Anytime these four render elements get added it produces the errors and crashes. We have tried a slew of different things.

We are using 3ds Max 2016 all updates and service packs.
Deadline Monitor 9.0
Deadline Client Version: 9.0.3.0 Release
FranticX Client Version: 2.4.0.0 Release
Repository Version: 9.0.3.0
Integration Version: 9.0.3.0

Any thoughts or help would be much appreciated. We are using the latest version of Vray 3.60.02
renderElements.PNG

I also tried rendering each element individually and each one cause the same crash.

So, “SEH Exception: Access Violation” means that there was an Access Violation crash within Max or one of its plugins. That could be a C/C++ bug or the machine running out of memory (checked, it was only 3% used, 120GB+ were free).

I’d try running that FumeFX scene on “psdc-rn061” if you can.

This error is occurring on every render node on the farm which is a few hundred. What do you think it could be?

Did you try opening the scene on psdc-rn061 itself to see if it renders? You can also try to isolate the render from Deadline to see what happens when you submit from command line.

Isolate from Deadline: docs.thinkboxsoftware.com/produ … oting.html

Ok so here is the situation now. I submitted two renders using the Deadline Monitor

  1. Submitted with 3ds Command and it rendered perfectly fine

  2. Submitted with 3dsmax and it crashes with the same errors as reported before.

What do you suggest I do next?

When i open the max file manually on that render node it crashed max until i changed the viewport settings to use Software instead of nitrous. I then was able to open the file manually on the render node and render it successfully. However it still crashes when submitting through deadline using 3dsmax whereas 3ds Command does render without crashing.

As far as getting the render to work, I’d start trying to simplify the scene if you can. Lightening (the plugin that normally controls Max) is pretty stable, so I’m not expecting the crash to be there. Removing more plugins (if any) fromt he scene is going to be helpful here, but maybe just passing this one edge case through 3dsMax Cmd is the better use of time.

If you’re curious though, I think we can find out at least what library that crash happened in. Windows tracks crash reports in its event viewer and will show what library the fault occurred in:

I’m expecting that we’ll see the library it’s died in. In my case it was “wlansvc.dll” I believe. You can save the whole event and sent it here.

Couple of things I would like to add here…

  1. If possible, please upgrade to latest SP of Deadline 9 (9.0.12.0 ~its free for you!). Even better, upgrade to DL 10.0.11.1.

  2. I should note, V-Ray 3.60 support wasn’t tested and added until 9.0.8.6.

  3. We fixed some concurrency issues in DL 9.0.4.0 for 3ds Max. ~Maybe related.

  4. Are you using the latest version of FumeFX?

  5. Once I know the exact version of FumeFX being used, I’ll ry to reproduce. However, a really simple test scene file would be appreciated for an old version of 3ds Max ideally. If it comes down to something weird in FumeFX where it just prefers to run in software viewport mode, then we could look at adding the viewport override when we launch 3ds Max at render time and override the 3dsmax.ini setting (and expose it to our 3dsmax plugin):

knowledge.autodesk.com/support/ … C-htm.html

knowledge.autodesk.com/support/ … E-htm.html

You could prove this works now by editing your “3dsmax.py” file around line: 1191 (in DL 10.0.10.4):

parameters += " -p \"" + pluginIni + "\" -q"

and add the “-v” option, followed by whichever driver you want to force loaded.

Personally, I’d like to get to the root problem here though by testing a scene file.

An interesting thought when it comes to your previous comment about: “it works when I run it in software mode”, can be seen in the 3ds Max offical docs:

Nitrous Software
Selects the Nitrous graphics driver and viewport system.
The Nitrous Software driver doesn’t require hardware support. It can be slower than the hardware supported versions, but does support multithreading and multiple CPUs.

I wonder if the above “software” mode makes FumeFX happy and other viewport modes are more crashy. Also, on the render-nodes that it crashes on, I guess they do not have a graphics card installed…so…what “driver” version are they currently running anyway? Software, right?

Finally, I should note that due to the powerful Lightning plugin we use, to drive 3dsMax, its no surprise to me that it might have an issue in our “3dsmax” plugin but ‘just’ seems to work in our “3dsCmd” plugin. Our Lightning dll approach is more powerful, but like any highly strung supercar, sometimes asks for more attention. If your short for time, you could just use our “3dsCmd” plugin and monitor/in-app submitter and be done with it.

Great information Mike. Below is in response to your message…

  • We are currently running FumeFX 4.0.6 Commercial.
  • You are correct the render nodes do not have video cards. 3ds Max still appear to launch with Nitrous as the default mode.

Attached is the file causing errors. I noticed if i open a 3ds Max file and do File New and then add the fumeFX elements it renders. However if i use our startup file, which is what this file is, it errors on the farm. So it appears the culprit is something in this file…?

Not sure what it would be. There is only a spline and a mesh.

A 3dsMax startup file is essentially a scene file, so it could contain state set, scene set, scene state info, metadata hidden animation tracks, hidden 3rd party plugin param data. If in doubt, I would destroy and re-create your “startup” file from a known good benchmark.

I see no attachment. I would need a reproducible scene file to try and debug this further if required. I will investigate what versions of FumeFX we have here as well. Can you also tell me what version of 3dsMax this scene file will open in. Thanks!

I’ve attached the error file. Let me know if it shows up for you.

Otherwise you can download it from my dropbox.
dropbox.com/s/3xpei2ac4vzwh … e.max?dl=0

The file was originally created from 3ds Max 2016 after doing a Max Reset.
What’s odd though is we have been using this file for over 2 years now without any issues. However we recently started using fumefx and for whatever reason when we add those render elements to this file it crashes on the farm but not locally.

I’ve been playing with this for a while now and can’t reproduce any issue here. Can you provide a few Deadline log reports so I can everything that is going on here? Your not using concurrency when rendering, right? FumeFX hates more than 1 instance of 3dsMax running on a Deadline Slave.

We are only one a single instance. I can possibly share some logs with you directly. I would not be able to post them directly the forums.

OK. You can send zipped up data securely via creating a private support ticket:
support.thinkboxsoftware.com/

Privacy | Site terms | Cookie preferences