Hello,
sending cloud jobs to deadline and the ifd seems to get broken while pathmapping as I get Mantra complaining about some binary data.
2022-11-16 12:41:19: 0: STDOUT: [12:41:19] reading geometry from stdin
2022-11-16 12:41:20: 0: STDOUT: [12:41:20] /obj/SANDBANKS_RNDR/RNDR (6210296 primitives)
2022-11-16 12:41:20: 0: STDOUT: [12:41:20] reading geometry from stdin
2022-11-16 12:41:20: 0: STDOUT: [12:41:20] mantra: Error loading geometry /obj/RIVER_RNDR/RNDR from stdin
2022-11-16 12:41:20: 0: STDOUT: [12:41:20] mantra: JSON ERROR[770931563]: Invalid binary token [0x65] (near byte offset 770931563, line 1, column 770931564)
2022-11-16 12:41:20: 0: STDOUT: [12:41:20] mantra: Invalid binary token [0x72] (near byte offset 770931564, line 1, column 770931565)
2022-11-16 12:41:20: 0: STDOUT: [12:41:20] mantra: Invalid binary token [0x72] (near byte offset 770931565, line 1, column 770931566)
2022-11-16 12:41:20: 0: STDOUT: [12:41:20] mantra: Invalid binary token [0x61] (near byte offset 770931566, line 1, column 770931567)
Anyone have any clues here?
Just updating this thread.
I pathmapped my IFD file with gsar
Replaced on premise drivemount with the stack mount and forced the AWS Portal Asset Server to refresh that file.
I also don’t really get why Cloud rendering Windows to Windows does not just emulate the exact mount points. No need for pathmapping.
So to conclude, Deadline’s Pathmapping seems to break the Binary Mantra IFD file… 
are you using short path names? I find if I’m using things like F:\ to /mnt/share then it’ll get confused with “opdef:/” and replace names in files.
I try to avoid these short names and use multiple lines like F:\projectA to /mnt/share/projectA. I find this also causes issues with Arnold
2 Likes
Nope, I did indeed run into that problem long ago but not right now.
Paths are pretty unique and should definitely not clash.
Yep binary ifds break when path mapping is performed. In fact if you look at the source for Houdini plugin and the related modules, you will see comments about this.
The easiest way is to use ascii ifds.
Would look at gsar though, interesting!
Right, not saving inline would make the IFD file non binary.
But this then still won’t work since any references to alembics and other files will then live in the temporary ifd files that are not readable or pathmappable.
The most annoying thing for the AWS Portal Asset Server Implementation is that Deadline does not replicate the mountpoints in the Cloud.
For example, local mount:
G:/Projects
becomes:
E:\GProjects<stackname>
This then breaks any Mantra rendering where you try to be efficient with IFD generation using temporary geometry files and reading lots of geo as packed disk primitives (stored in the temporary geo files and non human readable → not path mappable).
So Deadline would theoretically only work if we saved out all geometry into the IFD itself or use gsar for the path mapping which from preliminary tests seems like it won’t break the binary file.
If the mountpoints can be replicated then deadline does not have to pathmap anything and the problem would go away.
1 Like
Hm, are you talking about two different things?
ascii IFDs do work, and referencing alembics on disk (with path mapping) is supposed to work… or do you mean geometry generated during the same render?
Yeah I’ve found out what doesnt work.
We save some of our ifd’s as bgeo.sc files so they don’t take up about 3 times the amount of space.
Those are binary and unreadable/unmappable by deadline so we are missing all alembic references.
Unfortunately Deadline’s own implementation of the Cloud render system kind of breaks things unnesesarily.
For example:
G:/Projects on Windows On Premise
will be mounted as
E:/GProjects<stackid> on the cloud
Which basically forces the need for remapping.
I can honestly not think of a reason why the windows render nodes in the cloud cannot just map the same mountpoint as the on premise.
Same for Linux, its possible to recreate the mountpoints exactly so no pathmappping is necesary.
That would be ideal since any files missed from the sync will be pulled from on premise.
But do you mean that those bgeo.sc files reference other files? If they do not, then it should be possible to just path map ascii ifds, which reference bgeos.
The bgeo.sc files seem to contain the references to alembics etc.
I’ll need to find the settings but the reason we are splitting out our ifd’s is because houdini can apply compression to parts of the ifd (the bgeo.sc shared files) to reduce filesize.
When you are dealing with quite massive scenes this can be the difference between 500MB per ifd (with a few shared .bgeo.sc that contain framerange info) or 5gb per ifd per frame.