I’m seeing peak ram usage being way, way off on my vray renders. For instance, I have one running right now that reports its peak RAM usage as 11.046 MB, but when I look at the actual process running on the machine it’s using 42.46 GB. It’s been running with that memory for about 3 hours, so it’s not like it’s a quick spike that could have happened between samples. Has anyone seen this happen, or have a guess as to what could be happening here? Maybe it’s not monitoring spawned processes by default? When we run the command vray it spawns a vray.bin process that does the actual render, so maybe that’s it…?
That’s my guess too. I think we just monitor the process that the path to “Render Executable” creates. Probably any forking or other process chain stuff starts isn’t accounted for.
I’ll log it and see if we can’t walk the process tree to get more accurate results.
Can you maybe give me a process status dump (ps aux | grep vray
) when that’s happening so I can grab some easy proof.
So, the devs have assured me that we are walking the process tree. So now we’ll need to see what your calls to ps
dig up. We’re actually getting the info from the /proc tree, so we may do some digging in there.
Here’s an example:
ps -elf | grep vray
0 S thungqu+ 3561 3530 0 80 0 - 4156 wait Aug11 ? 00:00:00 /bin/bash /usr/autodesk/maya/vray/bin/vray -scenefile=…
0 S thungqu+ 3562 3561 99 80 0 - 9384404 futex_ Aug11 ? 7-07:27:55 /usr/autodesk/maya2015-x64/vray/bin/vray.bin -portNumber=20207 -scenefile=…
Top shows process 3562 using about 30G of memory, but Deadline shows peak usage at 8.416M. Looks like /proc gives me the correct info:
echo 0 $(awk ‘/Rss/ {print “+”, $2}’ /proc/3562/smaps) | bc
35301672
Let me know if there’s anything else I can try to help you debug. Thanks!
Actually, I have a couple of test slaves running 7.2, and they seem to report memory usage correctly on the same processes that our 7.0 slaves get bogus data for. I’m guessing that’s a result of this line in the 7.1 release notes:
“The slaves now report memory usage for a task more reliably on Linux.”
Sorry, I probably should have updated to the most recent version prior to submitting a bug report. You can consider this issue solved.
Thanks for the update, awilson. There’s always a silver lining: Your report gives us independent confirmation that the memory usage reporting fix actually works.