Houdini Sim ec2 Segmentation Fault

[oliverxyz]

Hi all,
I bought some Houdini Engine UBL to simulate fluids on an ec2 instance.

-I changed this base AMI:
Deadline Worker Base Image Linux 2 10.1.1.1 with Houdini 17.5.360 and Redshift 2.6.48

-removed redshift, because it asks for a license:
sudo rm -rf /usr/redshift

-renamed the houdini.env, so all redshift stuff gets ignored:
mv houdini.env houdini.env.old

This works alright, but there is a ~75% chance, the sim won’t start and throw a Segmentation Fault immediately:

2019-11-15 14:26:17: 0: STDOUT: ALF_PROGRESS 0% 2019-11-15 14:26:18: 0: STDOUT: No OpenCL platform has the specified device type (HOUDINI_OCL_DEVICETYPE): GPU. Falling back to built-in CPU OpenCL driver. 2019-11-15 14:26:18: 0: STDOUT: 8395: Fatal error: Segmentation fault (sent by pid 0) 2019-11-15 14:26:18: 0: STDOUT: -- TRACEBACK BEGIN -- 2019-11-15 14:26:18: 0: STDOUT: Traceback from hython 17.5.360 (Compiled on linux-x86_64-gcc6.3): 2019-11-15 14:26:18: 0: STDOUT: stackTrace() <libHoudiniUT.so> 2019-11-15 14:26:18: 0: STDOUT: signalCallback(UTsignalHandlerArg) <libHoudiniUT.so> 2019-11-15 14:26:18: 0: STDOUT: UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t*, void*) const <libHoudiniUT.so> 2019-11-15 14:26:18: 0: STDOUT: UT_Signal::processSignal(int, siginfo_t*, void*) <libHoudiniUT.so> 2019-11-15 14:26:18: 0: STDOUT: __funlockfile <libpthread.so.0> 2019-11-15 14:26:18: 0: STDOUT: -- TRACEBACK END -- 2019-11-15 14:26:19: 0: INFO: Process exit code: 139

This happens on different machine types. Most powerful ones I tried were c5.18xlarge.

This might be more of a houdini issue, but maybe someone can point me in some direction.
Thanks!

[anthonygelatka]

Is it trying to use a GPU? and is it a GPU instance?
If you’re modifying it take a look at the Nvidia driver and card in use (nvidia-smi) and whether there is a houdini specific override for it

nvidia-smi

HOUDINI_OCL_DEVICETYPE=CPU

[oliverxyz]

Hey @anthonygelatka, thanks for the help.
It is not a GPU instance.

I added
HOUDINI_OCL_DEVICETYPE=CPU
to houdini.env and it gets rid of the No OpenCL Warning. The Segmentation Fault remains the same though.

I added the whole LOG to a pastebin here

[Epigraph]

Bump…I’m having the same issue here while trying to render on a custom AMI (H18 + Vray). Does anybody found a workaround for this ? See the worker log below:

2020-06-22 19:46:24:  0: STDOUT: [2020/Jun/22|19:46:24] VFH |    Info| * V-Ray 4.30.03
2020-06-22 19:46:25:  0: STDOUT: Warnings were generated during load.
2020-06-22 19:46:25:  0: STDOUT: Error loading: /mnt/Data/P617804c1fb5864457089ac8352017473/Project/3D/Houdini/Exterior_v23.hip
2020-06-22 19:46:25:  0: STDOUT: Warning:     "/out/deadline1" using incomplete asset definition (full definition not found).
2020-06-22 19:46:25:  0: STDOUT:              The following node types are using incomplete asset definitions:
2020-06-22 19:46:25:  0: STDOUT:     Driver/deadline
2020-06-22 19:46:25:  0: STDOUT: ROP type: vray_renderer
2020-06-22 19:46:25:  0: STDOUT: Rendering frame 300 to 336
2020-06-22 19:46:25:  0: STDOUT: [2020/Jun/22|19:46:25] VFH |    Info|   V-Ray: Loading plugins from "/home/ec2-user/vray_adv_43003_houdini18.0.460/vfh_home/dso/../../appsdk/bin/plugins/libvray_*.so"
2020-06-22 19:46:25:  0: STDOUT: [2020/Jun/22|19:46:25] VFH |    Info|   V-Ray: 90 plugin(s) loaded successfully
2020-06-22 19:46:25:  0: STDOUT: [2020/Jun/22|19:46:25] VFH |    Info|   V-Ray: Finished loading plugins.
2020-06-22 19:46:25:  0: STDOUT: [2020/Jun/22|19:46:25] VFH |    Info| * Processing scene for frame 300.000...
2020-06-22 19:46:44:  0: STDOUT: 6525: Fatal error: Segmentation fault (sent by pid 0)
2020-06-22 19:46:44:  0: STDOUT: -- TRACEBACK BEGIN --
2020-06-22 19:46:44:  0: STDOUT: Traceback from hython 18.0.460 (Compiled on linux-x86_64-gcc6.3):
2020-06-22 19:46:44:  0: STDOUT: stackTrace() <libHoudiniUT.so>
2020-06-22 19:46:44:  0: STDOUT: signalCallback(UTsignalHandlerArg) <libHoudiniUT.so>
2020-06-22 19:46:44:  0: STDOUT: UT_Signal::UT_ComboSignalHandler::operator()(int, siginfo_t*, void*) const <libHoudiniUT.so>
2020-06-22 19:46:44:  0: STDOUT: UT_Signal::processSignal(int, siginfo_t*, void*) <libHoudiniUT.so>
2020-06-22 19:46:44:  0: STDOUT: __funlockfile <libpthread.so.0>
2020-06-22 19:46:44:  0: STDOUT: -- TRACEBACK END --

[JakeS]

Hello,
We are having a similar issue in some HtoA renders. Has anyone found a fix for this?
Thanks,
-Jake

[tom_voll]

It would be great if anyone had a solution to post here.
Thanks!