Missing Xref in Xref problem

Hello ,
So i’ve got kind of inception here :slight_smile: The situation is as follows :
There is a scene , which has several xrefs.
In one of the XRefs there is another xref with some objects (usually that is scene xref)
When work on the file is finished - those secondary xrefs are being disabled (NOT removed). Occasionally folder or wherever that secondary XRef is can be renamed and that secondary XRef becomes missing , but as it is turned off - that does not matter.
So i have ticked on “Merge XRefs” and after submission i have error about missing XRef. That is that secondary DISABLED XRef which was moved.
I really need to have Merge XRefs checked on because sometimes i have missing buckets on spawners without it. If i uncheck it - everything renders fine. Is there an option where i can force ignoring Disabled xrefs when Merge Xrefs is checked on ?

My guess is that the error message is coming from 3ds Max when we try to reload the original scene. I am not sure we can do anything about it - we do not do anything in our code beside loading the backup copy of the original main .MAX file…

However, I seem to remember you were looking to disable the restoring of the scene due to an unknown system exception before - are you still running the original unmodified SMTD that restores the scene’s XRefs after submission?

I guess we could add an Option checkbox to the SMTD UI to NOT restore the scene after merging XRefs - this would probably solve both the system exception case and your current case, while allowing others to restore the scene the way it was…

Thing is that locally scene renders fine , because xref is disabled - 3dsmax is not reporting any critical error about it , when i try to merge or whatever i am doing.
And it is not error from Submitter itself - it sends job just fine - the error appears when job starts rendering.
When i open locally the file which was submitted to deadline repository (we have option to save file to repository) it opens without any error , and renders without error as well , but even if i send this new extracted from repository job (merged) with disabled missing xref inside , and checked merge xrefs and ignore missing xrefs errors - it still gives error about missing xref. I came to conclusion that “ignore missing xref error” checkbox is not working ?

Got this problem again - conditions are exactly the same… Any ideas on how to get around it ? It really looks like missing xref’s is just being ignored… Should i just make a script which will delete missing turned off xrefs in scene before the initial submission ? Can you please guide me the best way to do that ?
Thanks.

I am going to attempt to reproduce this issue here first thing on Monday. (Sorry for not having done this yet).

If “Ignore Missing XREF Errors” is checked, the slave should not care whether an XRef is disabled, its path missing etc.
If merging XRefs with nested XRefs causes an error when these errors are disallowed, then it would be a bug in Deadline or 3ds Max.
Also, SMTD might have to perform multiple iterations of merging to ensure no XRefs are left unmerged if nested deeper than one level.

Thanks a lot for your help, Bobo , will wait for the results . If it will be needed - i can make a video on how to reproduce this problem step by step.

Hi!

Attached is a modified version of SMTD which should handle XRefs slightly differently.
SMTD_MergeXRefs_20150914.zip (173 KB)
To install,

  • Go to your Repository.
  • Navigate to the \Submission\3dsmax\Main\ subfolder.
  • Back up the files “SubmitMaxToDeadline.ms” and “SubmitMaxToDeadline_Functions.ms” into a subfolder or a zip archive.
  • Unzip the attached modified versions and copy them into the folder to replace the old versions.
  • Start SMTD in 3ds Max and test

It turns out that when merging a Scene XRef that contains Object XRefs, the active Object XRefs get merged into the submission scene even if Merge Object XRefs is not checked - 3ds Max just does it implicitly. This is a bit unexpected, but probably not critical.
However, if an Object XRef is in a Scene XRef MAX file and is disabled there, it gets merged as a disabled Object XRef. This is a bit unexpected.
So I modified the code to remove disabled XRefs from the submission scene:

  • The original scene gets saved to a temp. file if there are any XRefs and merging was requested for either XRef type.
  • Then we merge the Scene XRefs, if requested and present (which potentially also merges active Object XRefs from the XRef-ed scenes as their sources, while leaving inactive ones as Object XRefs).
  • If a Scene XRef is disabled, it gets removed from the submission scene.
  • If requested, we merge the Object XRefs that are active, and delete the Object XRefs that are inactive.
  • Then we save the resulting scene for submission and send it to Deadline.
  • Finally we restore the original scene with the Scene and Object XRefs before the modifications.

In the old version, the handling of Object XRefs was before the merging of Scene XRefs, and the collection of Object XRefs was performed before the merging of the Scene XRefs, so it was easy to end up with a bunch of unmerged disabled Object XRefs.

Please test this modified version and let me know if it handles your scenes better than the old one…

Thanks a lot for that , Bobo , i will update that tomorrow first thing. Human factor always causes problems , we are always asking our artists to remove unused xrefs , but in about 80% of cases they forget to do so …

I had a problem with padding so i edited code of SMTD a bit to remove it completely, because even when i checked “Remove Padding” - it was a dash still remaining , but when i used empty delimiter - i had problem with Object ID channel files , because in our workflow they end with 0001-0005…
So i’ve noticed that there are changes in Padding handling also , quite a bit of code removed - how that will affect this thing ?
Thanks !

I would suggest downloading the modified SMTD again tomorrow before you start testing.
I will update it further today - I just started testing it again and did not like the result :frowning:

I did not work on the padding changes, I will have to ask the Deadline developers about that.

Hah , ok no worries and no hurry !
Oh i though you are as a Maxscript guru working solely on this.
Anyway , give a shout when this will happen and i will go on to testing !

The SMTD scripts I posted are the 7.2 Beta version. I totally missed the fact you might not be running 7.2 builds yet.
That would explain the huge differences you see in the code.

There are a few people touching the SMTD scripts, depending on the feature. I wrote the original version and maintained it for about 5 years, but the last 5 years it has been a collaborative effort.

Here is the latest iteration of my changes, please test this one:
SMTD_MergeXRefs_20150915.zip (174 KB)

What’s new:

  • Fixed the removal of disabled XRefs which was not working right in the previous iteration, possibly thanks to 3ds Max peculiarities.
  • Added detailed logging of every XRef merging or removal operation so it is easy to see what got merged and what got removed
  • Added better error handling if some of these operations fails or a file is missing during the merging.
  • Added a counter of non-fatal errors/warnings issued during the submission, with a final line after a successful submission telling you whether there is anything in the log to pay attention for.

Sorry for disappearing , we have here some stuff changing , domain users security and etc. I will check this script and will get back with any results. Thanks !

So i’ve updated the script , tested it on couple of jobs and for now looks like problem is solved !
Also looks like the problem with Delimiter and padding is solved as well , because i did not change anything in new script - there are no additional symbols with “Remove Filename Padding” checked on !
Thanks a lot.
Now a bit more patience while dynamic VRay distributed slave will work :nerd:

Thanks for the feedback!

The updated script contains all changes and fixes panned for Deadline 7.2, so it will soon be the official version.

Bobo , i don’t want to disappoint you but i’ve got exactly the same error again , it stops on line 3048 - fetchMaxFile.
Everything is exactly the same, though this time i do not have xrefs in xrefs , missing or anything. I’ve got 2 xrefs , they are enabled and not missing. I’ve checked them all and they do not have any files missing or any x-refs records…

In Log the last line i have is “Restoring Original Scene From Backup File…”
Tested it 4 times - it gives error every time , not anything like random or etc.

Once again, if 3ds Max fails at fetchMaxFile(), it is a problem of 3ds Max and not SMTD. We are simply saving and reloading the scene using 3ds Max features. What happens if you Hold and Fetch manually the same scene through the Edit menu? Does it restore without an error?

Yeah i understand that this is 3dsmax error , and yes - it throws out when i try manually hold - fetch. Just thought that there may be a workaround ?
Quite interesting that this happens on some of the scenes. Though i do not see any special about this one really.

So i’ve got one more thing here. I’ve got a scene sent on deadline , with merge xref turned on and Ignore missing xrefs turned on as well. But for some reason there are disabled xrefs in main file which is on repository already , and when scene is being loaded to render on a slave - it throws me error that there is file missing (which is disabled x-ref record of a file which was moved or renamed).
Looks like new script works on some scenes , but some of them are still with the same issue for some reason… If that is something about 3dsmax bug again - i will make a script which will be fired on scene submission to check and delete all disabled xrefs upon sending if that is possible.

It would be very useful to take a look at the SMTD Log from such a submission!

When an XRefs record is disabled, SMTD now has code to remove it from the submitted file, and there is a log line for the cases where it succeeded or failed to remove it. I would like to see if the deleting actually works.

However, due to an Autodesk bug, in order to remove an XRef record, that record must be enabled first. So SMTD enables the record, tries to remove it, and if it fails, it prints a log line that it failed, but it actually leaves it enabled, which sounds like a bad idea to me (and is totally my fault). I wonder if in your case the call to objXRefMgr.RemoveRecordFromScene() fails, but then the XRef record goes to Deadline as enabled…

You could fix that case by locating the following code block in “SubmitMaxToDeadline_Functions.ms”, and modifying it slightly:

else --if disabled, remove the record to avoid surprises at render time ( theRecord.enabled = true --have to enable it before removing?! WTF Autodesk? local result = objXRefMgr.RemoveRecordFromScene theRecord --remove disabled record if result == true then ( try(SMTD_MainRollout.Ui_report ("+Removed Disabled XRef Object Record "+ i as string + " ["+ theSourceFileName+"]") )catch() ) else ( try(SMTD_MainRollout.Ui_report ("--FAILED to Remove Disabled XRef Object Record "+ i as string + " ["+ theSourceFileName+"]") )catch() SMTDSettings.nonFatalErrorsCount+=1 theRecord.enabled = false --> NEW LINE - removing failed, disable it again! ) )