We’ve got a 9 blade server render farm running Win7 and Deadline 6. We have constant issues with jobs submitted from 3DS Max 2014 being stuck on ‘Starting Job’ and then just hanging there for up to 30 minutes before stalling. The 3D guys keep resubmitting until eventually it goes through but it takes a lot of jockeying and time wasting. Apparently they had this problem back in the day when they used Deadline 5 as well. Thought I’d come here to see if anyone could shed any light.
From the slave log there are two entries that standout:
2015-08-14 12:17:10: 0: An exception occurred: Error in StartJob: 3dsmax: Invalid max file: “C:/Users/Spinifex/AppData/Local/Thinkbox/Deadline6/slave/RENDERPC02B/jobsData/55cd4ef1c968dc2a505595c8/Vampire_Sh020_v2_011.max”
2015-08-14 12:17:10: \slave\RENDERPC02B\plugins\55cd4ef1c968dc2a505595c8\deadlineStartupMax2014.max
Exception Details
IOException – The process cannot access the file ‘\emu\Deadline6\plugins\3dsmax\3dsmax.dlinit’ because it is being used by another process.
Exception.Data: ( )
Exception.TargetSite: Void WinIOError(Int32, System.String)
Exception.Source: mscorlib
Exception.StackTrace:
at Deadline.Slaves.SlaveSchedulerThread.SyncPluginFiles()
at Deadline.Slaves.SlaveSchedulerThread.RenderTasks()
at Deadline.Slaves.SlaveSchedulerThread.ThreadMain()
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
This slave log is indicating that this particular Max scene file is invalid whilst trying to be opened on the machine that generated this issue. I would check a few things here.
Does this particular slave have exactly the same ‘Product Update’ / ‘Service Pack’ version of 3dsMax 2014 installed on it? (You may well have installed it on all your machines, but it’s worth double-checking here). Ensure the Machine you submit the job from has exactly the same version as the machines which it tries to render on.
Is this Max file corrupt/broken in some way? Does it (a) open on your machine ok? & (b) does it open up on the test slave machine ok?
On the slave in question, via Task Manager, is/was there multiple copies of 3dsMax running when this error occured? Can you restart the machine to ‘clean’ it up and try again?
Were you running with “Concurrent Tasks” set to higher than 1? Were you running with multiple Deadline Slaves on 1 machine? This error:
“The process cannot access the file ‘\emu\Deadline6\plugins\3dsmax\3dsmax.dlinit’ because it is being used by another process” is indicating that possible more than 1 instance of 3dsMax was running on the slave in question when things went bad for you. However, this could be a false-positive, as this error could mean other things. Let’s see what #1 to #3 suggestions bring up first.
Are you on annual support? I ask, as many major improvements have been made in Deadline v7.1 to help resolve these kind of 3dsMax issues if they ever come up.
Final thing to check. If all of the above, really isn’t solving the issue for you, try going (remote desktop/physical) to a machine which seems ‘stuck’ and via Task Manager, if a process is running called something like: “WSCommCntr*.exe”, then right-click, kill it and see if that essentially releases the machine and it continues to render ok? If that is the issue, then you need v7.1 to automate a fix for this situation.
It seemed to be moreso in deadline6, though I have re-imaged the whole farm and moved to deadline7 and it has improved (more like 1% stuck rather than 5%)
But I haven’t taken the time to work out the issue yet, I don’t think it was specific machines but even that I am not certain of yet.
Using auto timeout on the job avoids them getting stuck forever but its just a temp fix.
@Simon - Can you also run through and answer the questions that I posted for the original op. Many subtle improvements were made in v7.0, v7.1 and v7.2 (beta) to help narrow this stuff down, which explains the 5% in Deadline v6.x. I’m pretty confident with the logs out of v7.1 or later, I can get pretty close to telling you what the issue is, combined with the questions being answered here…if you wanna try and remove that 1%?
Thanks for the suggestions Mike. My 3D guys have worked their way through the list of your suggestions but have had no luck so far in solving it. I’m not sure what our license agreement is but we’ll look into moving to DL7 when we can. We’ll start by moving to Max 2016 once we can upgrade our VRAY licenses and look at moving to DL7 then.
I found that a certain machine was always having this issue, and after logging into it max would just freeze on start up. So I’ve re-imaged the machine. That might be the end of that issue…