Ah, yes, someone else had the same problem trying to render exrs, and the bug will be fixed for the next release. I’ve attached a patch that should resolve the problem for now. First, make a backup copy of \your\repository\plugins\Modo\Modo.py, then unzip the attached file to \your\repository\Modo\Mody.py. Note that this patch uses the default process priority of below-normal, so if you want to continue testing with normal priority, you’ll have to make that change again.
Note that this process priority setting would affect all slaves, so there would be nothing extra you have to do to push this out to the other machines. I’m going to contact Luxology about this to see if they have any ideas.
Last night I ran some further tests on a variant of the file I sent you. Frame rendering with modo’s DR system (7 nodes) in about 21min. Via Deadline, tiled to 8 pieces, it rendered some tiles to completion in times ranging from 12min to 1:10hr, but the last piece was still kicking away after 2 hours and I killed it. The CPU on that machine was hovering around 30% for most of that time.
I think I’ll wait to see if Lux has any info for you instead of throwing time at it fruitlessly for now, but if you have any other suggestions or solutions I can give it a try.
I ran one more test with the updated script and that seems to have resolved the .exr /.tga problem. However, the same time issue exists.
I ran the file directly in modo, using 4 machines as well as the workstation and it took about 20min. I submitted to Deadline and sent it just to 4 i7 machines (others are in use on jobs atm) and tiled the render 4X. 2 parts were done in about 50min and 1 hour, the third took 1:30 and the last one I finally just killed after 2:30hr and no joy.
Watching the task manager on that machine, the CPU hovered around 30% but the RAM slowly filled up and then the page file slowly crept up to around 11GB. It was still steadily climbing when it died. I have noticed that happen with modo slaves before - steadily growing page files - but that was using the modo DR system. It does not usually happen when rendering locally as far as I know. Anyway, something was stalling and it was just cranking up the memory, but rendering it locally only used about 3-4GB of RAM, so the scene certainly doesn’t need that much memory/swap file. Something is maybe amiss on the modo end? If so then I’m sunk - using Deadline was my last hope to get around Lux’s own network render problems
Thanks for confirming that the updated script resolved the exr problem. We’re just waiting on a response from Luxology regarding the render time issue. When we hear something back, we’ll let you know!
np. I have also been in touch with Luxology via their forum - Brad has replied that they are looking into it so hopefully an answer will be forthcoming one way or the other.
Does this render time inconsistency ever occur for the first tile of a job that a slave picks up, or does it only tend to happen on subsequent tiles for the same job on the same slave? If it’s the latter, try submitting a new job in the suspended state. Then right-click on the job in the Monitor and select Modify Properties. Under the Advanced tab there is a “Reload Plugin Between Tasks” option. Enable it, click OK to save the changes, then resume the job and see if the problem still occurs.
This option would force Modo to shutdown and restart between tiles, which may help the render time and memory problems.
It does occur for the first tiles - I have 7 nodes and tend to send out 8 tiles on these tests, assuming one node will do two. More than one bog down and all show the CPU fluctuations. Unless I"m doing something else wrong I don’t think that is it.
However, that restart function will definitely help with the memory creep problem.
Hey Ryan -
Just heard that someone else had the new version of Deadline running tile renders with modo. Did you guys manage to sort out the CPU usage issue with Luxology for the new release?
No changes have been made on our end. Other than a few minor tweaks, Modo support in 4.0 is pretty much the same as it was in 3.x. Here are the Modo specific changes in Deadline 4.0:
* Fixed issues with submission from within Modo.
* Added support for Modo 401.
* Fixed bug where specifying an exr to override output would cause output to be saved as tga.
I noticed on Luxology’s website that Modo 401 SP3 is available, so maybe this update helps address the problem you’re running into?
Thanks Ryan. I am not sure if Sp3 made a difference, although the changelog made no mention of anything network/rendering specific. I take it you did not hear anything back from Luxology yourself? In any case, I think that if it’s not confirmed to be supported I will likely just hold off. I am hesitant to spend another few days installing the demo and messing around for nothing. I’ve done too much of that for modo
Yeah, we didn’t get a concrete answer from Luxology. Did you actually talk to someone that’s had success with tile rendering with Modo and Deadline, or did you hear it through the grapevine? I was just wondering if it’s possible to ask them if they’ve noticed a similar problem.
As far as we’re concerned, Deadline supports tile rendering for Modo. I don’t think we were ever able to reproduce this fluctuation problem on our end (it’s been a while, so I gave this thread a re-read just to be sure), and we have heard success stories from a few other users.
From re-reading the thread, one thing I don’t think I asked you was if you ran into this problem with every Modo file you created. For example, does a very simple Modo file produce these problems? If not, maybe the issue is related to a particular feature or plugin that perhaps not everyone is using. You could start small and slowly add things until you see the issue, and that might help pinpoint the source.
Someone did post on the Lux forum that they had been using Deadline 4.0 and with some help from you guys it was running smoothly. I have just posted there to ask him specifically about these issues I was having. I am not aware of anyone else who has tried, as most users seem to be doing animation/frame rendering instead of tile rendering. I’ll let you know what they say.
Regarding the problem, it happened with all kinds of files. I made simple scenes and used scenes that rendered fine on just my workstation. It was occurring on all the machines I have tried, so it happens on a variety of hardware configs and both Windows XP64 and Vista Ultimate 64. I don’t really use any special modo plugins or scripts.
I wasn’t really sure if you guys reproduced the problem there, I don’t think you posted about it - just that you had gotten in touch with Luxology so I took that to mean you had seen it. Anyway, there’s only so much time and work any of us can justify for making modo run properly. I’ll probably just shelve it again till I hear some confirmation.
I am having the same problem as listed in this thread.
When I send a split frame render from Modo 401 to deadline; it spends time tessellating the scene (I can see this by the memory increase in task manager on each of the nodes). Then, as expected the increase flattens out, which indicates tessellation has finished. I went away for a bit and checked all of the nodes a bit later on, and though there was nearly full memory use; the processor percentage (on all nodes) was ticking over at between 2/3% and 8%.
The scene is a relatively complex scene, with a fair amount of displacements, and a reasonably high irradiance setting. However, I have had the same render running on a single node and it renders more quickly, and the processors are running at 100% all the time.
It’d be great to shed some light on this, do I need to set up different configs (or one global config?) for the Modo command line on the nodes? If so, how is this done.
I have pointed the plugin config locations on all of the nodes to the Modo command line executable, and removed the string to the standard executable as well; so I don’t think that’s the cause of the problem.
That’s interesting that you only notice the CPU usage issue during tile rendering, as we were under the impression this could happen for both tile jobs and normal jobs. That helps narrow things down a bit.
That being said, we’re not doing anything fancy for a tile render. We just set the region settings for that particular tile using these Modo commands:
We’ve been running some tests, we think we have narrowed it down a bit more.
It doesn’t seem to be tile rendering based, it seems to be a memory/scene complexity issue. Simple scenes render fine, both on standard and tile based renders.
There is a preference setting for geometry cache in the Modo config file (MODO401.CFG). If this setting is too low then Modo will exhibit these slowdowns on the cpu. We need to be able to set this value on the command line version of Modo.
Where does deadline find the config file for Modo? Is there a way of setting a single shared config file for all nodes to access within deadline?
Deadline currently doesn’t control the config file, so I’m assuming that Modo is just reading the config file from the default location. I’m not familiar with this Modo config file, so I’m not sure where this default location is (I’ll have to read up on that). If controlling the settings in the Modo cfg file is the the way to go though, I’m sure it’s something we can add to Deadline. Even better, if there is a Modo command that sets this value in a running instance of Modo, we could just do that instead of modifying the file directly.
I think there is a command for pointing towards a config file. I’ve done that before with Lightwave, which has similar requirements for access to a config file (preferably a shared config and plugins directory on the server). I’ll dig into the Modo documentation and see if I can find anything. Thanks.
Btw, the Modo config is usually here on XP/2000 systems. C:\Documents and Settings\Administrator\Application Data\Luxology\
I found a command that sets the Geometry Cache Size. For example, this command sets it to 1 GB:
pref.value render.cacheSize 1073741824
This would allow us to expose this option in the submitter and set it before rendering. This way, we don’t have to worry about overwriting any local config files.
The Modo Scripting and Commands.pdf states as follows:
"Console mode uses its own config file. For modo 401 on Windows, this is modo_cl401.cfg, while on OS X this
would be com.luxology.modo_cl401.
The -config switch tells modo to use a different path for the user configuration file. This can be used by render
nodes to load a specific config, or used in conjunction with -license to run modo off of a memory stick without
requiring anything to be written to the local machine.
The path provided can either point to an actual config file, or to a directory containing a config file with the
standard user config filename. A list of standard config filenames for different versions of modo can be found in
Appendix K. The specified config will used in place of the default, which means that not only will initial settings
be loaded from this config, but settings will also be saved to this file on quit.
Note that on OS X, all paths must be absolute; relative paths, including the home (~) path, are not supported"
Strangely the modo_cl401.cfg cannot be found on any of our systems or render nodes so I’m not sure what it should contain.
Providing a means of changing this in the submitter would certainly be handy although might cause issues for those people with a mixed RAM farm (the geo cache should on;y be a percentage of your available RAM). Being able to use a per node config would be ideal and is apparently intended to work this way.
A colleague has asked about this on the Luxology forum so perhaps it might be worth waiting for a response from Lux.
