Starter for MIXED Farms.

So basically we have a company that has a PC CG side, all 3D Max with about 32 render blades and a Mac side that is After Effects and some NUKE. Deadline has been up and running for the 3D Max side for a while and they are getting along swimmingly, the problem is I am trying to get the mac side to be able to use some of their Slaves, as well as be able to setup some of the Mac computers, about 20+ computers as Mac Slaves, then made into a mac group, to be used with Deadline as well.

I have been doing trial and error for a while, but I wonder if their is a primer or a FAQ to get started.

I have successfully installed Deadline 4.1 sp1 on the Mac and I am able to see the Depository on the PC side. I have been trying some simple renders and working through the bugs of the file system differences between MAC and PC.

My goal is to be able to render After Effects, NUKE, Quicktime, maybe even SHAKE on the MAC side and render NUKE on the PC side.

If anyone has any place to start, then I would appreciate it, but if they don’t I am going to use this thread to ask specific questions as I begin to figure Deadline out.

Which OS do you have the repository installed on? If it’s Windows, then permissions shouldn’t be an issue on the Mac side, but if it’s *nix, there may be some additional work that needs to be done.

Once you know that permissions won’t be a problem, you can check out this documentation:
thinkboxsoftware.com/deadlin … splatform/

Path Mappings will allow you to render cross-platform AE and Nuke jobs. Just refer to the cross-platform sections of their respective docs for more info:
thinkboxsoftware.com/deadlin … iderations
thinkboxsoftware.com/deadlin … iderations

Hope that helps get you started. Just let us know if you have any questions or concerns!

Cheers,

  • Ryan

Thanks RYAN.

So the SLAVES I am trying to access are all PC running windows. We are only PC(WINDOWS XP/7), and MAC OS X 10.6.7.

First set of problems. So I am testing having the MAC side, using NUKE submit to the RENDER SLAVES running (XP and 7). I finally was able to get the Slaves to open the render program and start to render, but having trouble with the volume writing.

ERROR.

0: STDOUT: Writing /volumes/blah/blah
0: STDOUT: Write1: Can’t write, no such directory

So, in my “Configure Repository Options” I have settings in my “Mapped Path Setup (For Mixed Farms)” that say.

Path to Replace Windows Path Linux Path Mac Path
\network02\ \network02\ /volumes/
/volumes/ \network02\ /volumes/

So it looks like it can find the files fine, even though my scripts are on MAC and see /volumes/blah/blah/blah… but it doesn’t seem to be able to write anything using the same naming convention. Like it see’s the files, but can’t write the files.

Another thing.

if I am switching just the volume mount location, say \network02\blah\blah and only saying change \network02\ to /volumes/
You will eseentially get files that look like this inside of nuke.

\network02\blah/blah/bah/blah

or visa versa

/volumes/blah\blah\blah

Do I have to do the absolute directory for each and every sub directory? How can this work?

The mount location should be enough, so you won’t need to do every subdirectory. For the path separators, Nuke is very good and figuring them out, but Deadline will always explicitly set them to “/” when performing a path swap (which is what Nuke tends to use by default anyways).

Would you be willing to post a log report from a job that produced this error? If privacy is a concern, you could just email it to support (at) thinkboxsoftware (dot) com. You can find the log by right-clicking on the job in the Monitor and selecting Job Reports → View Log Reports. The logs will contain a bit more info than the error report, because the error reports only show the last few lines of the Nuke stdout. Also, a screenshot of your current path mappings would be helpful too. Finally, the actual Nuke script itself would be great, because we can look at the paths in it and try to figure out why they might not be getting swapped properly.

Cheers,

  • Ryan

Hey Ryan, thanks for the information. I think we are kinda of getting the hang of it, I’ll email you directly with any errors or logs. Hey also, I am trying to find a way to automatically have Quicktimes made from DPX/EXR sequences and I am not getting anywhere. Do you know if there is a way to write a script that automatically does this? Do you know of a way to make a “Submit to Quicktime” job automatically start after a sequence is made?

Is there a way to add Compressor to the Plugins, so you can have “Submit to Compressor?”

Thank You.

Glad to hear you’re making progress!

For automatically generating quicktimes, there are a few options. One is to write a custom submission script (or edit one of our existing ones) to submit a dependent QT job alongside the original.

A more general approach though is to create an Event Plugin for Deadline to do this whenever a job completes (Event Plugins were introduced in Deadline 5). We actually have an example of this in our documentation:
thinkboxsoftware.com/deadlin … on_Example

I just re-read your original post though and noticed you are still running Deadline 4.1, so for now, the only option really is to have a custom job submitter.

Cheers,

  • Ryan

Hey Ryan, we are told by IT that upgrade to v5 of deadline will take a while and we have to wait until we have a small amount of downtime between jobs, but while we are using v4, I would love to know if you have anything to help with making quicktimes. I noticed that for 3dMAX submissions there is an option to make a quicktime, is there anyway to add that option to the NUKE and AFTER EFFECTS submission scripts?

Also, I have been getting people to use Deadline for compositing for both After Effects and NUKE. I have been getting some After Effects people asking me WHY Deadline is better than using watch folders, since that is what they know and are a little slow to take on new things. Do you have a few one liners as to speed and stuff, as to why Deadline is better than watch folders?

Also, I remember a program called Nucleo Pro for speeding up After Effects renders. It would open a render for each processor. Is there any way or any benefits of having Deadline open multiple render nodes per operation, so if I looks at one of the render slaves it would have 8 instances of a program rendering insted of 1. Is there any benefit there…

The 3dsmax QT support is deeply integrated into the 3dsmax submitter, so unfortunately there is no easy way to port that code to the AE and Nuke submitters. The least invasive way to do what you’re looking for in Deadline 4.1 is to write a generic post job script that grabs the output information from the job that just rendered and then submit a QT job to make a movie from that output. You would place this script on the network somewhere, and then edit the Nuke and AE submitters to add the following line to the job info file they create (this is the file that defines the frame range, priority, pool, plugin, etc):

PostJobScript=\\path\to\qt\script.py

Now we don’t have a pre-written post job script to do this, but one can be pieced together using the post job script example here:
thinkboxsoftware.com/deadlin … ob_Scripts

…and the QT event plugin script I referred to earlier. The Post Job Script example shows how to get the output file paths, and the QT event plugin shows how to submit a QT job to Deadline once you have those paths.

I don’t have any experience with AE’s watch folder system, so I took a look at the documentation:
help.adobe.com/en_US/aftereffect … 79a2a.html

From what I’ve read, it seems like a pretty bare-bones system. There likely isn’t any difference in terms of rendering speed (both systems use multiple machines to work render frames from sequences simultaneously), but there are definitely benefits to using Deadline from an artist’s perspective:

  • The submission interface seems more elegant, and allows you to control additional options like memory management and multiprocess rendering.
  • Monitor application allows one to control and modify jobs after they have been submitted.
  • Render logs and error reports are easily accessible from the Monitor UI.
  • Deadline can notify you via email when you job is complete or when it has failed (so you don’t have to keep refreshing a web page).

There are many other benefits as well (priority systems, having all render jobs controlled by one render manager, etc), but I tried to focus on the Artist experience. ** Note that I’m making an assumption that the watch folder system doesn’t have these features, and I’m basing that assumption on the documentation I linked to above.

In the Deadline submitter, there is an option called Concurrent Tasks. You can increase this number to 8, and then each node will render up to 8 tasks at a time (using 8 instances of AE). By default, a node will only dequeue as many tasks as it has CPUs, so if you set it to 8, you don’t have to worry about your 4 core machines picking up more than 4 tasks.

As far as I know, AE renders a frame using a single processor. If you enable multiprocessing, it only works if you are telling AE to render a group of frames, as AE will use one processor per frame. So as long as the node has the RAM, there is a benefit to using concurrent tasks in Deadline. Just make sure to disable multiprocess rendering when you do. :slight_smile:

Hope I’ve been helpful!

  • Ryan

Thanks Ryan, I will look into how to setup .py scripting…

Also, I wonder if you can decode this error I am getting from NUKE 6.2v4 for MAC. This particular render pool is Mac also.


=======================================================
Error Message
=======================================================
ApplicationName='which', CommandLine='top', CurrentDirectory=''

=======================================================
Slave Log
=======================================================
icationName='which', CommandLine='top', CurrentDirectory='' (System.ComponentModel.Win32Exception)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Exception Details
Win32Exception -- ApplicationName='which', CommandLine='top', CurrentDirectory=''
Win32Exception.NativeErrorCode: 14
ExternalException.ErrorCode: -2147467259 (mono-io-layer-error (-2147467259))
Exception.Source: System
Exception.TargetSite: Boolean Start_noshell(System.Diagnostics.ProcessStartInfo, System.Diagnostics.Process)
Exception.Data: ( )
  Exception.StackTrace: 
  at System.Diagnostics.Process.Start_noshell (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) [0x00000] in <filename unknown>:0 
  at System.Diagnostics.Process.Start_common (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) [0x00000] in <filename unknown>:0 
  at System.Diagnostics.Process.Start () [0x00000] in <filename unknown>:0 
  at (wrapper remoting-invoke-with-check) System.Diagnostics.Process:Start ()
  at FranticX.IO.Path2.GetApplicationPath (System.String applicationName) [0x00000] in <filename unknown>:0 
  at FranticX.Management.ComputerSystem.get_UsedPhysicalMemory () [0x00000] in <filename unknown>:0 
  at FranticX.Diagnostics.MachineMonitor.SetInitialValues () [0x00000] in <filename unknown>:0 
  at FranticX.Diagnostics.MachineMonitor.Start () [0x00000] in <filename unknown>:0 
  at Deadline.Slaves.SlaveRenderThread.ThreadMain () [0x00000] in <filename unknown>:0 
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
 Scheduler Thread - Synchronizing job files
 Scheduler Thread - Synchronization time for job files: 153.512 ms
Constructor: Nuke
 Scheduler Thread - Render Thread 0 threw an error: 
 Scheduler Thread - ApplicationName='which', CommandLine='top', CurrentDirectory=''
 Scheduler Thread - Unexpected Error Occured: ApplicationName='which', CommandLine='top', CurrentDirectory='' (System.ComponentModel.Win32Exception)


=======================================================
Error Type
=======================================================
Win32Exception

=======================================================
Error Stack Trace
=======================================================
  at System.Diagnostics.Process.Start_noshell (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) [0x00000] in <filename unknown>:0 
  at System.Diagnostics.Process.Start_common (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) [0x00000] in <filename unknown>:0 
  at System.Diagnostics.Process.Start () [0x00000] in <filename unknown>:0 
  at (wrapper remoting-invoke-with-check) System.Diagnostics.Process:Start ()
  at FranticX.IO.Path2.GetApplicationPath (System.String applicationName) [0x00000] in <filename unknown>:0 
  at FranticX.Management.ComputerSystem.get_UsedPhysicalMemory () [0x00000] in <filename unknown>:0 
  at FranticX.Diagnostics.MachineMonitor.SetInitialValues () [0x00000] in <filename unknown>:0 
  at FranticX.Diagnostics.MachineMonitor.Start () [0x00000] in <filename unknown>:0 
  at Deadline.Slaves.SlaveRenderThread.ThreadMain () [0x00000] in <filename unknown>:0 

I can’t figure out the error, eventually the frames render, but it does seem like a lag a big somewhere…

This is actually a known issue in Deadline 4.x (it’s a general Deadline issue and isn’t specific to Nuke). The issue has been fixed in Deadline 5.0 though, so hopefully this doesn’t impact the renders for your current jobs too badly. I apologize for the inconvenience though.

Cheers,

  • Ryan

It seems like everything eventually renders, but I can tell its slowing it down a lot when dishing out these errors. I have another set of errors from After Effects renders, are these the same errors in After Effects that I am getting in NUKE?

=======================================================
Error Message
=======================================================
ApplicationName='which', CommandLine='top', CurrentDirectory=''

=======================================================
Slave Log
=======================================================
ad 0 threw an error: 
 Scheduler Thread - Exception during render: An error occurred in RenderTasks(): ApplicationName='/Applications/Adobe After Effects CS5.5/aerender', CommandLine=' -project "/Users/kiwi/Deadline/slave/jobsData/ENS_Egg_Sh06_v23.aep" -comp "02_Main" -s 75 -e 99 -output "/Volumes/ENS_Good_Egg-2703/Footage/AE_Edit/Shot06/ENS_Egg_Sh06_v23/ENS_Egg_Sh06_v23_[#####].dpx"', CurrentDirectory='/Applications/Adobe After Effects CS5.5'
  at Deadline.Plugins.ScriptPlugin.RenderTasks (Int32 startFrame, Int32 endFrame, System.String& outMessage) [0x00000] in <filename unknown>:0 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Exception Details
RenderPluginException -- Exception during render: An error occurred in RenderTasks(): ApplicationName='/Applications/Adobe After Effects CS5.5/aerender', CommandLine=' -project "/Users/kiwi/Deadline/slave/jobsData/ENS_Egg_Sh06_v23.aep" -comp "02_Main" -s 75 -e 99 -output "/Volumes/ENS_Good_Egg-2703/Footage/AE_Edit/Shot06/ENS_Egg_Sh06_v23/ENS_Egg_Sh06_v23_[#####].dpx"', CurrentDirectory='/Applications/Adobe After Effects CS5.5'
  at Deadline.Plugins.ScriptPlugin.RenderTasks (Int32 startFrame, Int32 endFrame, System.String& outMessage) [0x00000] in <filename unknown>:0
RenderPluginException.Cause: JobError (2)
Exception.Source: deadline
Exception.TargetSite: Void RenderTask(Int32, Int32)
Exception.Data: ( )
  Exception.StackTrace: 
  at Deadline.Plugins.Plugin.RenderTask (Int32 startFrame, Int32 endFrame) [0x00000] in <filename unknown>:0 
  at Deadline.Slaves.SlaveRenderThread.RenderCurrentTask () [0x00000] in <filename unknown>:0 
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Constructor: AfterEffects
 Scheduler Thread - Render Thread 0 threw an error: 
 Scheduler Thread - ApplicationName='which', CommandLine='top', CurrentDirectory=''
 Scheduler Thread - Unexpected Error Occured: ApplicationName='which', CommandLine='top', CurrentDirectory='' (System.ComponentModel.Win32Exception)


=======================================================
Error Type
=======================================================
Win32Exception

=======================================================
Error Stack Trace
=======================================================
  at System.Diagnostics.Process.Start_noshell (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) [0x00000] in <filename unknown>:0 
  at System.Diagnostics.Process.Start_common (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) [0x00000] in <filename unknown>:0 
  at System.Diagnostics.Process.Start () [0x00000] in <filename unknown>:0 
  at (wrapper remoting-invoke-with-check) System.Diagnostics.Process:Start ()
  at FranticX.IO.Path2.GetApplicationPath (System.String applicationName) [0x00000] in <filename unknown>:0 
  at FranticX.Management.ComputerSystem.get_UsedPhysicalMemory () [0x00000] in <filename unknown>:0 
  at FranticX.Diagnostics.MachineMonitor.SetInitialValues () [0x00000] in <filename unknown>:0 
  at FranticX.Diagnostics.MachineMonitor.Start () [0x00000] in <filename unknown>:0 
  at Deadline.Slaves.SlaveRenderThread.ThreadMain () [0x00000] in <filename unknown>:0 

If we upgrade to v5 of deadline, will these errors go away?

Yup, it’s the exact same error, and it has been fixed in 5.0.

Hey Ryan,

We have a very similar renderfarm setup to jj_joseph, is there a way for Deadline Slaves to detect the Operating System and then re-adjust its own pool/group automatically?

For example, if the machine is restarted to Windows 7 it will automatically group itself as “3dsMax”, while if we flip it back to Mac mode, it automatically group itself as “AfterEffects”

We are using Deadline 5

Thanks!
Roy

Hi Roy,

Currently, no. We want to look at adding Pools/Groups to Deadline’s AutoConfigure feature, but that’s still a little ways away.

The slaves create themselves based on the name of the machine they’re rendering on. Is it possible to give the machine a different host name based on the OS it’s running as? If it was, that could be a way to workaround this.

Cheers,

  • Ryan

Thanks Ryan, assigning different host name based on the OS works!

Hey Ryan we are starting to get some problems now with cross platform rendering… Its weird because it worked before, but I am not sure why it is not working. I am getting 0: STDOUT: [17:59.56] ERROR: Read4: errors because the media can’t be found on the PC render nodes as sent from MAC, but we have all the “Mapped Path Setup” configured properly and it worked for a while but not now.

How does deadline know what is a PC render node and what is a MAC and what is LINUX? Where do I establish that? in the group/pool/slave settings? If I can force Deadline to see these PC render nodes as PC, then I know the /volume/ will be swapped properly.

Thank You.

Here is another error…

[code]Slave Log

Nuke6.2v4/plugins/FilterErode.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/Multiply.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/Reformat.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/Transform.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/Dilate.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/SplineWarp.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/DiskCache.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/Keymix.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/Vectorfield.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/Text.dll
0: STDOUT: Loading C:/Program Files/Nuke6.2v4/plugins/dpxWriter.dll
0: STDOUT: [ 9:51.34] ERROR: Read4: /volumes/blah-blah-blah/
[/code]

So it shows that its starting a PC program, but on the last line.

0: STDOUT: [ 9:51.34] ERROR: Read4: /volumes/blah-blah-blah/

Its obvioulsy trying to open a MAC volume, if its loading a PC program wouldn’t it know how to properly swap the directory to \pc-volume\ as setup by our Mapped Path Setup for mixed farms?

The only reason why Deadline wouldn’t be swapping a path is that the Path Mapping settings aren’t resulting in a “match”. A screen shot of your current Path Mapping settings, as well as the full render log from the job would be helpful. You can find the log by right-clicking on the job and selecting Job Reports -> View Log Reports.

If this stuff contains sensitive data, please feel free to email them directly to us at: support (at) thinkboxsoftware (dot) com.

We can compare the screen shot with the log to see if anything stands out.

Thanks!

  • Ryan

Hey Ryan, Email sent…

Hmm, we haven’t received anything yet. Maybe try using the web interface to the ticket system:
support.thinkboxsoftware.com/

Sorry for the inconvenience.