Setting up Deadline to save output renders to a local directory and then collecting

Deadline
1

Hi there everyone,

so, I finally jumped on the bandwagon with all the Cloud computing fuss and I’m having such a hard time setting everything up properly, I’m on the edge of giving up.

But, here’s a major culprit I’m dealing with: transfer speeds. I just can’t have my slaves sourcing my files from my local machines, compute the renders and then saving the files over the network back to my local machine.

So, I’m thinking of doing this:

  • Having each and every render slave render and save the files locally to a temporary destination
  • then I’ll have to run WinRAR (or 7Zip or whatever) on each slave, after it finishes rendering, and having it collect all the render outputs
  • then I’ll have each slave upload these packages to an FTP server of mine (fast connection)
  • then shutdown.

All of this has to happen on Spot instances on Amazon.

Right now I’m fighting so much crap with the cloud machines (Deadline not pinging them, not being able to remotely control them, latency, not being able to verify my credentials, etc…)

However, my question is, I remember reading in the manuals some long time ago about having slaves render and save locally and then being able to cllect the files later. Is this functionality supported by Deadline, natively? Or do I have to use the event system, for example?

Or, is there a better alternative to this?

Thank you in advance, cheers…

2

Hi loocas,

The most common approach is to set up cloud storage. Prior to rendering, assets needed in the cloud would be synchronized up to the cloud storage. Then the cloud Slaves would read assets from, and write outputs to, the cloud storage. This uses the path remapping feature of Deadline. And finally the results would be synchronized back down locally. It’s a good idea to render proxy output for review before pulling down full resolution images, since data egress has a cost.

There is an example synchronization script on our public GitHub repository. We have some cloud documentation in progress that may be helpful. If you send me a private message with your e-mail address I can send you an advance copy.

3

Ok, thanks for the tip, I’ll see what I can do.

The worst thing is, I can seem to get Deadline connect to my Amazon account and thus managing my instances is a bit harder.

4

Hi Lukas,

This page describes the settings needed to connect Deadline to AWS: docs.thinkboxsoftware.com/produc … mazon.html

5

Thank you!