AWS Thinkbox Discussion Forums

Consolidation/Management of multiple repositories

Not sure if there is a way to manage this better, but viewing a Deadline Repo at another site over a “Long Fat Network” is exceptionally slow and painful.

With the cloud stuff coming into play will there be improvements aimed at having a single consolidated Deadline farm that multiple sites on a very large/fat network can use? We currently have multiple sites each with their own Repo, it isn’t exactly ideal since if a given site isn’t busy their nodes sit idle.

I would think that the cloud stuff (while cool and very forward thinking) isn’t something that is useful to the larger shops since most of their production networks probably aren’t touching the internet, or won’t be if the MPAA/HBO/etc will plan on giving them work. I would be willing to bet that more and more companies will have to “certify” with MPAA regs at some point too, causing further problems with this workflow.

Thoughts?

Connecting to a remote repository with Deadline 6 will be much better than it was in Deadline 5. When we were giving our demo at Siggraph, we were actually connected to a repository on EC2 with a slow internet connection, and the response time of the Monitor felt like it was a local repository. Of course, as you’ve mentioned, using a single farm for multiple offices introduces its own set of problems.

Chris discussed some ideas regarding Cloud rendering here:
viewtopic.php?f=156&t=8227
viewtopic.php?f=156&t=8228

Some of this would apply to a single-repository setup as well, since many of the problems are the same (ie: asset synchronization).

Yeah I read the first one from Chris but hadn’t seen the second, Asset sync can’t be that big of a deal right? Seems like it would be more time consuming than anything else, most of the apps have files that can be scrubbed through for scene data and the like; it’s just a matter of building something (for each app) that would scan the scene file and collect up everything that it needed to move/copy no?

Asset sync may or may not be tricky, depending on a number of factors. The first part is identifying assets. For some apps it’s easy, for others it’s not. 3D apps tend to have a mechanism to enumarate external references, but it’s still possible to trip these up. Assets referenced via hard-coded paths in a script, or perhaps built dynamically through RegEx, would not be recognized, for example. So special case handling will be needed for these.

Next is the synching of the assets. Synching many gigabytes can potentailly take longer than the render, so some thought needs to go into the process. Some of the assets may already exist at the destination while others may not or may have been versioned up. Assuming sync is resolved, it’s important for the asset paths to be consistent or to be altered in the scene file. Again, the degree of difficulty of repathing is dependent on the app.

Short answer: The concept is straight forward, but the implementation is frought with edge cases.

Privacy | Site terms | Cookie preferences