I have two Slaves and one Client and am running 3dsMax 2013. (Hotfix 5 has been applied to all). I (think) that I have everything installed properly, but am having some issues:
-
In the Deadline Monitor on my client, I can see the two slaves, but one of them keeps stalling, but occasionally comes back on line. But for the most part remains stalled.
update - This issue seems to have stabilized somewhat after re-starting the one Slave. BUT, any time I pick “Check for Stalled Slaves” from the SuperUser menu, it will cause that Slave to stall out for a few minutes after which point I can refresh the slave list to see it as “Idle” again. The message I get from the Slave window on the other slave says “Slave 'RN-02” has stalled because it has not updated its state in 3.008 hrs." I notices that it still stalls for a few seconds and quite often.
-
I successfully sent a job to Deadline from the Client, but the Task Progress never changes and just says “Active”. Furthermore, there’s an error (stalled slave), plus the slave that IS working, shows the “License-Free Mode” window as “Not Responding” even though the Manager says that the Status is “Idle”.
update - Finally got this working once both slaves are on-line.
-
Should V-Ray Distributed Rendering be unchecked in the Render Setup Settings?
update - Yes. V-Ray DR must be unchecked otherwise you get a warning…
-
Do I need to be running V-Ray Spawner services on the slaves if Deadline does not use V-Ray DR? (I’m not running it at present)
Still don’t know the answer to this one.
-
Does it matter if V-Ray Frame Buffer is active or not?
update. This needs to be “off” otherwise you get a warning. I’ve tried the tiles method but am disappointed in that even though an Alpha channel is present in each of the tiles, it gets stripped out in the final composited file.
If I can get the bugs worked out with these two Slaves, I have another one or two to add, but don’t want to spend money on licenses until I make sure that it will suit my needs.
Thanks!
Dave
Hi Dave,
-
Check to see if the date/times between your slave and repository machines are in sync. It is a known issue in v5 where slaves can be incorrectly reported as stalled when the date/times are off. If you happen to be running Pulse, check that machine as well.
-
Sounds like this issue has been resolved, but correct me if I’m wrong!
-
Yes, as you’ve discovered.
-
No, the spawner only needs to be running if you want to use DR. Since you are submitting with DR disabled, you do not need the spawner running anywhere.
-
What output format are you writing to? It could be a bug (or limitation with ImageMagick) with a specific format.
I should mention it IS possible to do Vray DR in Deadline using a bit of a hybrid approach. For this example, let’s say you have 6 nodes. You could run the Deadline slave on two of your nodes (we’ll call them dl-01 and dl-02). Then you would run the VRay spawner on your 4 other nodes (we’ll call them vr-01, vr-02, vr-03, vr-04). You would configure VRay on dl-01 to use vr-01 and vr-02 for DR jobs, and dl-02 would use vr-03 and vr-04. When you submit the job to Deadline, you would turn on the VRay Distributed Rendering option.
What you are doing here is offloading the master from your workstations to one of the deadline slaves. The slave that picks it up will use it’s dedicated spawners to do a DR render. With the example above, you could have 2 DR jobs going at the same time. If you only need to one DR job at a time, then you could just have one node running the Deadline slave, and the rest running the spawner.
I hope that makes sense!
Let me know if you have any questions about this hybrid setup.
Cheers,
Thanks for the response! The hybrid approach is exactly what I want to be able to do. My work is always with single-frame high res images.
Could you step me through a setup specific for what I have?
Basically, I have three render nodes: RN-01, RN-02, RN-03 and my V-Ray workstation (WS). If I understand you right, I set it up like this:
RN-01 runs DL Slave
RN-02 and RN-03 Run Spawner
WS would have VR DR enabled, pointing to RN-02 and RN-03
Jobs would be started by the Deadline menu item and sent to DL Slave which in turn would use RN-02 and RN-03.
Is that correct? If so, I can’t seem to make it work!? Maybe i need to have DL Slave running on my workstation instead, and RN-01 would be a third instance of Spawner?
Thanks again!
The description you have there sounds correct, so we’ll have to do some digging to find out why it isn’t working. To start, I guess it wouldn’t hurt to test with the slave on your workstation and have the spawner running on your 3 nodes. If this also doesn’t work, then we know it’s a more general setup issue.
Also, how is it not working? Does the frame fail to render, or does it render but the spawner nodes don’t join in the render?
Finally, can you post a log from the job? You can find it in the Deadline Monitor by right-clicking on the job and selecting Job Reports -> View Log Reports. We can take a look to see if anything stands out.
I know other clients have got this setup to work, so it’s definitely doable!
Cheers,
I have Spawner set up on three nodes, and DL Slave set up on my WS. V-Ray DR is set for the three nodes.
The issue I’m having is that when I send a job to DL, I get three errors:
Failed to find the file [RegionManipulator.ms] in Dealine Repository. (error code: 1040)
Failed to find the file [TileManipulator.ms] in Dealine Repository. (error code: 1042)
Failed to find the file [SubmitMaxTodeadline_Functions] in Dealine Repository. (error code: 1004)
The files are all there though. In fact, if I “Explore Repository Root” via super user in DL Manager, it points right to the repository on the network. I did have it working yesterday when I was playing around, but it’s not working now. I tried reinstalling the client, but nothing. Also, tried running the Macro from 3ds, but again, nuttin.
So that’s were I’m at. Errors when trying to send a job to DL. Since I can’t get a job to actually go to DL, I can’t send a Log Report. Sorry.
Thanks
Dave
This is all with Max 2013 right? I’ve seen those missing file errors in Max 2012 because it didn’t have all of its updates applied:
usa.autodesk.com/adsk/servlet/ps … eID=123112
I should also mention that Max 2013 requires product update 2 to be installed to work with Deadline, but that only affects the rendering side of things, not the submission.
If we can’t get this figured out today, perhaps we can set up a gotomeeting on Monday so I can see your desktop and try to figure out what the problem is.
Yes, it’s 2013 Design. I was having a similar issue but ran the latest product “Update 5” and that seemed to have fixed it. But now it’s broken again so I don’t know what to try. If you’d like to take a stab at fixing it via GoToMeeting, that would be fine, but there’s no urgency since I figured out a way to get by sending VR DR jobs to Backburner. Still though, Deadline is a fine piece of software and blows BB away (IMO).
Dave
Weird… have you tried things like restarting 3dsmax or restarting your workstation to see if that helps?
Also, if you open a command prompt, and run “deadlinecommand.exe -getrepositoryroot”, what’s the output? The 3dsmax submitter uses this command to get the repository path, and then build up the paths to those scripts it’s complaining about. If this path is incorrect, or you get an error that deadlinecommand.exe isn’t a valid application, that could explain the problem.
If neither of these things lead anywhere, I think we’ll have to do the gotomeeting on monday.
I ran “getrepositoryroot” and it returned the correct path (Y:) which is a dedicated share that I set up on the server. Still, when I tried to send something to Deadline, I would get the same errors. So I re-booted and re-ran getrepositoryroot. Same thing, it found it. Then I tried sending another job to Deadline. Same errors… In all cases, the SMTD 5.2 window eventually does come up once you click through the errors.
For fun, I decided to run the Sanity Check anyway. This time I noticed “Workstation Mode must be enabled to use VRay Distributed Rendering”. Hmmm. I hadn’t seen that before. So I assume that it is the “Force Workstation Mode” checkbox on the Render tab? Anyhow, I ticked it and then closed the SMTD dialog box wanting to make sure I had the DR servers set right. I then re-launched Deadline, and NO ERRORS!?. Hmm… Very strange. I then ran a job (with slave running on my WS and Spawner on my three nodes) and it seemed to work fine. The one thing I noticed is that it opened another full instance of 3ds and a VR Frame Buffer. Is that normal that DL needs to run another full instance, even if the Slave is running on a node?
Thanks
Dave
Hi Dave,
That is strange that you had to jump through so many hoops to get this to work, but I’m glad to hear it worked in the end.
Sorry that I didn’t mention this before, but yes, it is necessary for Max to run in workstation mode when it’s acting as the master in a DR job. It just completely slipped my mind.
Cheers,
At least I’m on the right track! But the need for DL Slave to run a licensed instance of 3ds 2013 (to achieve V-Ray DR) is cost prohibitive at this point
Dave
Hmm, maybe there is another way this could work. You may or may not have noticed, but there is also a 3dscmd plugin for Deadline:
thinkboxsoftware.com/deadline-5-3dscommand/
The difference between this plugin and the 3dsmax plugin you are currently using is that the 3dsCmd plugin just uses the 3dsmaxcmd.exe command line executable. It’s not as fully featured as the 3dsmax plugin, but because it just wraps the command line renderer, it can do anything the command line renderer can do.
I’m curious to know if it would work to submit your VRay DR job to the 3dsCmd plugin instead of the 3dsmax one. I would expect it to, since it works with backburner. If it DOES work, then we’ll need to do some digging on our end to figure out why our 3dsmax plugin must start Max in workstation mode to get the same results.
If you’re willing to give this a try, we would love to know the results. Now that you’ve had experience with setting up the 3dsmax submitter, it should be fairly simply to set yourself up to use the 3dsCmd submitter as well (note that both can be installed at the same time, and won’t conflict with each other).
Cheers,
I installed the CMD plugin and gave that a whirl. The job shows up in the Monitor, and the Slave sees it, but beyond that, nothing happens (3ds on the slave or the 2 DR nodes does not activate in any way).
I’ve attached the log file from the slave, if that helps…
deadlineslave_Streetwisexp64(Streetwisexp64)-2012-09-17-0007.log (272 KB)
Ah, it looks like you just need to configure the 3dsCmd plugin, which you can do from the Tools menu in the Monitor while in super user mode. Just select Tools -> Configure Plugins, and then select 3dsCmd from the list on the left. In the 2013 executable setting, you just need to change the path to include C:\Program Files\Autodesk\3ds Max Design 2013\3dsmaxcmd.exe. The defaults don’t have the Design path, which is why it’s failing.
Cheers,
(might be an idea to include the design paths in the default shipped version)
I configured the correct path statement and tried sending a job to the CMD plugin, but it just throws the Slave into a loop. Not sure what’s going on, but it’s clear that it’s not working. Possibly because under that plugin, there’s no option for “Force Workstation Mode”? Bummer.
deadlineslave_Rn-01(Rn-01)-2012-09-18-0000.log (33.7 KB)
Actually, it looks like we’re making progress! The render is now failing because of Deadline’s built-in popup detection. We just need to ignore that popup to prevent this error from occurring.
I’ve attached a patch for the 3dsCmd plugin so that it ignores this popup. To install, go to \your\repository\plugins\3dsCmd and make a backup copy of the original 3dsCmd.py file. Then unzip the attached file to the same folder. Submit a new VRay DR job and if it continues to loop, send us an error report and we’ll take a look.
Note that when the slave is looping like this, errors are being generated for the job. You can view these errors in the Deadline Monitor by right-clicking on the job and selecting Job Reports -> View Error Reports.
Cheers,
Yes, making progress! Your patch seemed to do the trick for the loop issue. I can actually watch the VR FB do it’s thing on the slave machine. But the saga continues. Probably just a simple configuration error at this point. As you can see by the attached file, the buckets that are rendering from the Slave machine seem to have a gamma issue (too bright). The buckets from the other two nodes render fine. The other issue is that my Alpha and RenderID elements seem to have been ignored. In fact, looking at the image almost looks like something has been applied and then flattened (there are no shadows on the background). Weird stuff. It is working with BB, so I don’t know what up with that?
Do I need to check “Enable Local Rendering” or “Gamma Correction” when submitting?
Weird! I would have figured anything run through 3dsmaxcmd.exe would produce the same results as backburner. Can you post the render log from this job? I just want to check the command line arguments to see if we’re passing anything to 3dsmaxcmd.exe that might explain this. You can find the log for the job by right-clicking on it in the Monitor and selecting Job Reports -> View Log Reports.
Local rendering shouldn’t make a difference here. After I take a look at the, we can then try to figure out which settings to tweak.
Cheers,
I re-ran it this am just to make sure I had the right log file. See attached.
Thanks!
deadlineslave_Rn-01(Rn-01)-2012-09-19-0000.log (14.4 KB)