AWS Thinkbox Discussion Forums

XMesh MY Reader

I glad to hear that ! Since the lighters have a bit of work to do with the cache, light linking, shader tweak… they navigate a lot in the scene and the “frame selection” (f) feature of Maya is of great help to them, and from the first feedbacks I had they find it quite frustrating that it doesn’t work as usual when dealing with XMesh data. So the sooner the better :slight_smile:

Anyway, nice reactivity :slight_smile:
thanks for your time !

Hi, me again !

I’m having some severe performance issues I didn’t anticipated.
In a small scene I have 180 sequenceXMesh nodes each one pointing to a xmesh cache of a few hundreds polygons maximum (cumulated geometry: around 100k quads) and no special attributes. When reopening this scene, Maya freezes for about 3min and my network activity jumps to a steady 50 Mbits/s input traffic. I switched all loaders to display mode “Bounding box” hoping it would prevent this heavy loading time, but the result is the same.

Multiplying this network activity by the loading time gives us a total traffic volume (50 Mbits/s * 180s = 9000 Mbits = 1.125 Go) roughly equal to the total size of all frames of every object cumulated (around 1Go). Like if Maya was loading the full sequence (about 1000 frames) for each Xmesh node even though I just want to read the current frame.

This might be due to some configuration issue on my side. Indeed I am not sure about the option “Enable Playback Graph”:
Whether I let it unchecked or I check it and plug the maya time into “inPlaybackGraph”, the result appears to be the same: it shows the right frame after an incredibly long loading time.

Is its the expected behavior ? What is the influence of the “Loading Mode option” on file access and performance ?

Thanks for your help !

[edit]: If I unplug the inTime and change the “inPlaybackGraph” value by hand it seems to load the wanted frame faster. But if I plug an animation curve for example it never updates. Some “attribute affects” bug maybe ? I am working with Maya 2016 SP2 and XMesh 1.3.3.58675 on windows 7 by the way :slight_smile:

Yes, “Bounding Box” is supposed to improve loading times, because it reads the bounding box information directly from the .xmesh file, and it avoids opening any other data files.

I don’t think that XMesh itself should generate such traffic. When the scene is first opened, it will retrieve a list of files in the xmesh sequence directory, to determine which frames are available. Then it will load the data files for the current frame. It will not load data for any other frame.

Perhaps you are running some sort of anti-virus or file syncing software, that’s reading the entire file contents when we get the list of files in a directory?

Could you please send us a scene file that reproduces this problem? You can post your file on this forum, or you can send it to us by using our ticket system.

I submitted a report for the “inPlaybackGraph” update issue.

Concerning the performance issue I think I’m starting to understand where the issue comes from.
Since my XMesh cache is on a network share, and I have many objects over many frames, opening a maya file that contains 180 sequenceXMesh over 1000 frame means accessing to at least 360.180 files (lowest estimation: 1 face file + 1000 vertex files + 1000 xmesh files per sequenceXMesh) “at once”, even if no file is loaded, simply testing existence of each one of them is a tremendous amount of network traffic !

Does it seems consistent ?
If so could it be possible to activate some kind of “manifest mode” that would prevent this pre-loading and simply trust the data contained in a descriptive file ?

If no this mean that every machine on the render farm, before rendering one single frame, has to test access over 360k files and doing so flood the file server with file requests, which make it nearly unusable :confused:

Thanks a lot !

That does not sound right.

If you have the XMesh Loaders set to Bounding Box mode, opening the Maya scene should access exactly 180 .XMesh XML files, read the bounding box min and max values, and be done with it. However, reading small files over a network optimized for large files (some companies do that to make Nuke operate less sluggish than usual) can be relatively slow. Note that when Paul said “XMesh will retrieve a list of files in the xmesh sequence directory”, he meant the equivalent of a dir command to get the file NAMES in the folder, not opening each file.

If you have the XMesh Loaders set to Vertices %, opening the Maya scene should access 360 files - the .XMesh XML file of each sequence for the current frame only, and the .XMdat binary file for the vertex list.

If you have the XMesh Loaders set to Faces %, opening the Maya scene should access 540 files - the .XMesh XML files, the Vertex .XMdat files, and the Faces .XMdat files of the current frame.

If you have the XMesh Loaders set to display the Mesh, then the number of files accessed over the network will depend on the number of channels saved. But in general, it should only read the current frame, so if you have, say, 4 .XMdat files (Vertices, Faces, UVs, Normals), then it would have to access 180x5=900 files on opening the scene, and on any time change. Note that the loading of XMdat files is performed in parallel on multiple threads per loader, but each loader will process sequentially (I assume, don’t know enough about Maya’s internal workings), so it should be reading and unzipping 4 data files in parallel at any given time.

That being said, there could be something in the way Maya loads the scene that causes the XMesh Loaders to actually attempt to evaluate all frames as you described. That would be unexpected, but given the overall funkiness of Maya, we have to keep the possibility in mind. :smiling_imp:

Thanks! I received your scene file.

You’ll need to keep inTime connected to time1.outTime, even if you’re using a playback graph.

You could fix the updates in your scene file by going into the the Node Editor by re-connecting time1’s Out Time to xmesh_cube_1’s In Time, or by entering the following command in MEL:

connectAttr time1.outTime xmesh_cube_1.inTime

My bad for the “inTime”, it seemed redundant so I disconnected it. I was mentioning it just in case this was some sort of bug, if it is the expected behaviour all is good !

On the performance side I’m still struggling. I used “API Monitor” to watch the calls made by maya when the scene in question is opened, and indeed I catch about 300k fileQuery function call.

I ran this simple test:

[code]import os
import time
folder = “//data-server/project/test/xmesh/character1/v001” # I used only one cache for the test, a production scene contains easily a dozen of them
t0 = time.time()
files = os.listdir(folder)
for f in files:
os.path.isfile(folder + r"\" + f)
print len(files), time.time() - t0

34097 42.0470001698[/code]

This is a naive approach and Python is way less performant than C++, but its a good way to grasp the problems we are facing.
Having an unnumbered .xmesh file acting as manifest (a meta .xmesh file)could remove the need to do this listing every time a sequenceXmesh node is created (every time the scene is open either by an artist or a render engine each sequenceXmesh has to pull the full list of files). This is not a small feature but it would greatly improve the versatility of XMesh (and save my life :p) !

I’ll keep digging :slight_smile:

Have a nice weekend.

Hi,

I managed to mitigate the issue by exporting each object in its own folder, which will do for now.

I stumbled upon another issue, due to Autodesk rather than XMesh but you still might be interested !

Since Maya 2016 there are several DG evaluation mode available: legacy, Serial, Parallel (+GPU). When activating Parallel the playback of XMesh sequences is a bit faster, which is nice ! However the export command doesn’t like that !

Let’s say an animator export a XMesh sequence of a character being blown by an explosion. The FX guy loads this XMesh sequence and deforms it (for example to add shock waves running through it’s face). All works well.
But once the FX guy tries to export the deformed mesh for the lighting guys, XMesh Saver stops after 2 frames and maya get stuck in what seems to be an infinite loop (maya is running 100% on one core and its memory footprint stays the same, I let it run half an hour without any changes). This issue does not arise when dealing with native Maya nodes.

I now force the Evaluation mode to classic before any XMesh is exported so we could say the situation under control but I wanted to let you know :slight_smile:

Cheers.
Alexis

Good catch! Thank you for letting us know.

Thank you for telling us about this! Unfortunately I am having some trouble reproducing the problem. Do you have a scene file that consistently reproduces this problem? If so, would it be possible for you to please send it to us?

Hi,

Sorry for the delay, busy week !

While trying to setup a scene to show the issue concerning parallel execution mode, I found out that the issue was coming from one of our own node, used to control time offset and time scale for all connected sequenceXMeshNode ! Glad it was “that simple” :unamused:

But I’m having a new issue now ! We can’t seem to get a motionVector AOV out of Arnold.
Is there a way to get the velocity property back in maya after loading ? As an extra attribute on my mesh for exemple.
I guess Xmesh loader can access it somehow since the “velocity offset” loading mode works well.

Thx again :smiley:

Alright, thank you for letting us know!

We copy the velocity information into a color set named “velocityPV”. Does that work for you? If not, please let me know, and I will look into this further.

Perfect !

With a aiUserDataColor (Arnold) node we can access the velocityPV channel and feed it to any shader or AOV. (It only works in “Velocity offset” or “Velocity offset Subframe” Loading Mode thou).

Thanks :slight_smile:

Great! Thank you for letting us know.

Back with a new question ! (The velocity thing is taken care of, for anyone reading this there is just a last subtlety, the velocity exported by Xmesh is in worldspace so to be usable out of the box in a compositing package you need to convert it to camera space).

My question today is about the loading mode “Single Frame”. I have a scene with about 3k sequenceXMesh readers set to “Single Frame” but it still takes ages when scrubbing through the scene.

Using the profiler we can see that every sequenceXMesh node is still evaluated when time changes. Is it the expected behaviour ? Could it be optimized somehow on my side (appart from reducing the number of reader, I’m trying to educate the artists to do so) ? From the few maya nodes I’ve written I know we can easily know which input triggered the “Compute” function and if irrelevant simply exit before doing any work. Do you have such a test in place ?

Thanks a lot !

I think you should be able to avoid the extra computations by disconnecting the sequenceXMesh’s inTime attribute.

It would be nice to handle this better on our end. I’ll add that to our wish list.

Simply removing the inTime connection did the trick.

That’s definitely better, thanks guys !

Hi,

Happy new year to you !

New year, new issues !

We are on a project with objects moving and rotating really fast, so we can’t rely on interpolation between frames any more. So we bake our geometry using subframes.

The thing is setting the read mode to “Interpolate” or “Interpolate, subframe” produce the same result, as if the sequenceXMesh node wasn’t able to find the subframe. In my example, I baked the character with a substep of 0.1, which should give us plenty of subframes to interpolate from.

On the other hand setting read mode to “Single Frame” and selecting by hand a subframe gives me the expected result, which confirms that the export is working fine but lead me to think there might be an issue with the way “Interpolate, subframe” is working in Maya.

Repro case:

Create a cube. Animate its rotation Y (frame 1=0, frame 10=720). Bake it in worldspace, samples per frame 5 for example.
Import it. Let’s go to frame 5.60

Velocity: is quite close to the original motion. (value by default)
Velocity Subframe: is way off !!! I was expecting something way more precise than simple Velocity
Frame interpolation: is really off also, but given this is a fast motion and a simple linear interpolation, this is expected
Frame interpolation Subframe: exactly the same as Frame interpolation. I was definitely not expecting this :confused:

Did I miss something ?

Thank you for your report! I noticed a few issues while I was testing this. Please check your private messages.

I replied !

There is another issue (or deliberate behaviour), if you are not in one of the Velocity reading mode, the velocity channel isn’t loaded.

This could be nice when you want to bake a precise animation and still be allow FX guys to use the velocity to emit things or feed the velocity to a shader later on !

Thx:)

Thank you for your suggestion! I added it to our wish list.

Privacy | Site terms | Cookie preferences