AWS Thinkbox Discussion Forums

Hanging Forever: Sandbox process exited unexpectedly while waiting for response from Event

Hi,

We’ve been having some weird issues with Mantra over the last few months, specifically when we’re rendering crowds but I’m not sure if that’s the issue or not. Basically the slaves hangs, it doesn’t complete or fail it just hangs. If you requeue the task it will then go through fine.

error.jpg

We can render the job locally in Houdini (not an ifd in Mantra) and it will render the entire timeline without an error. So i’m not sure if it’s a Mantra or Deadline issue. I’ve included the last bit of the slave log (it doesn’t save this when I fail the job manually) and it looks as if something has crashed?

Any ideas on what might be causing it would be great.

Thanks

Nick

Oh, that one is definitely us. The sandbox process it’s referring to is new to 8.0 and now runs all the Python code (they’re more disposable instead of the whole Slave).

9.0 SP 5 and 6 had a number of fixes to those sorts of problems, so if you’ve got the time to do a minor upgrade, it’ll likely help out here. From the changelogs:

  • Fixed a bug that prevented Slave Events in the Slave’s Sandbox process from properly generating error reports when the event throws an error.
  • Improved how the Deadline applications deal with Event Sandbox crashes/lockups.
  • Reports will now be generated when the Event Sandbox crashes and/or stops responding.

Update: I also hijacked the title so Google and users find this guy easier

Hi Edwin,

I’ll try the update. I’ve just been waiting for a project to finish before making any system changes.

Thanks

Nick

We’ve installed the latest update and we’re still getting the hanging, although I’ve not let it go as long so I’m not sure if we would still get the Sandbox crash. We’re using slightly out of date versions of Houdini (15.0.313 and 16.0.504.20) and it may be that is causing the issue but we can’t update until the projects finished so we’ll keep an eye on it and see if that helps.

Thanks

Nick

Is it the same error? The sandbox is definitely not supposed to die at all so if there’s some reliable way for it to go down (short of running out of memory) we should try and fix it. I certainly don’t see the sandbox dying in many of my remote session support calls, so maybe it’s something within the Houdini plugin that’s causing the issues.

The Sandbox didn’t go down, but the render itself hung so I’m assuming it’s a Houdini error. I only let it go for 20 mins before I re-queued the job.

Thanks

Nick

Noted. Let me know if you find.

Fun thing I noticed this morning, that render hung for days:

2017-08-04 20:47:56: 0: STDOUT: [20:47:56] Creating geometry (/obj/crowdsource:female_sunglasses:17)
2017-08-06 10:37:40: Info Thread - An error occurred while updating the slave's info: Sandbox process exited unexpectedly while waiting for response from Event. (Deadline.Plugins.PluginException)

I’m fairly impressed by the sandbox now…

Hehe, yeah. It was over the weekend with no timeout set. The frames are actually 2-15 mins normally! :unamused:

Privacy | Site terms | Cookie preferences