Cloud Render - how do i get to the images?

[twuelfing]

This may be a stupid question,

how do i get to the images that the 2 test jobs rendered?

thanks for the help.

[eosiowy]

Hey Troy,

Definitely not a stupid question, more of a documentation failing on our part.
The output for the 2 demo jobs that are included in the Cloud Wizard can be found at //<repository_ip>/Data.
You can also find it under the Metadata tab on the GCE Console. There should be a cloudwizard tag. It will have an asset portion that matches the path above.

Thanks,
Eric

[twuelfing]

I am probably doing something silly, but when i browse to the repository the image below is what i see, with no folder called data.

am i looking in the wrong spot?

[Coulter]

Hey Troy,

It looks like you are accessing a local machine rather than the server VM on the cloud. What is the IP address of the virtual machine that has a name like cw-[networkname]-repository (where [networkname] is the name you gave your virtual network) in your Google cloud account?

[twuelfing]

I have figured some of this out, but have not been able to find the actual files.

I have found both of these locations, but its not clear how i can go about downloading the work the render nodes did.
The output for the 2 demo jobs that are included in the Cloud Wizard can be found at //<repository_ip>/Data.
You can also find it under the Metadata tab on the GCE Console. There should be a cloudwizard tag. It will have an asset portion that matches the path above.

next question:
the wizard setup the repository on a cloud machine, when this exits beta and we go to put a cloud rendering solution into production will I be able to use my existing repository that is local to our studio? If not how do you see the integration into existing pipelines going?

we are very interested in having a cloud capacity in our back pocket, but not using it under normal circumstances, only when we can bill an expedite fee to our clients.

Ideally for us it would be able to accommodate the following, i know some of this is a tall order, some it probably already does, and some of it is probably silly ideas.

1. use our current repository so all the studios jobs are tracked and scheduled in one spot
2. let us create a cloud “pool” or something similar that users can submit to that will access the cloud capacity and will be very clear when it does so as there are hard costs associated with it.
   a. user access control to who can submit a job to the cloud pool
3. An easy way to set limits for budgets
   a. when the budget is exhausted the cloud job shuts down and it only renders on our local hardware
4. A clear way to visualize expenditures so everyone can see the costs associated with the rendering and monitor costs
5. easily integrate with our rendering tools, in our case max and vray.
6. not require extensive IT knowledge to make it work or maintain it, we are a small studio and its hard to devote someone to managing this full time. Its ok if its a bit complex to get it going, but it needs to be easy to use for artists.
7. saves or moves the files created to the folder the artist wanted them saved in on our network
8. perhaps a prediction tool where we could send our job to our farm to render a few frames, we could then get a report back showing some type of cost and time estimate, or a graph showing costs vs time based on how many nodes would be spun up to render the job and then give the artist a slider to choose how many nodes to create to balance the cost and delivery time based on the prediction.

keep up the great work, this is looking very promising!

[Coulter]

I have figured some of this out, but have not been able to find the actual files.

I have found both of these locations, but its not clear how i can go about downloading the work the render nodes did.
The output for the 2 demo jobs that are included in the Cloud Wizard can be found at //<repository_ip>/Data.
You can also find it under the Metadata tab on the GCE Console. There should be a cloudwizard tag. It will have an asset portion that matches the path above.

I should preface by saying the Cloud Wizard, in its current form, is meant to demonstrate a basic cloud installation and get users over the initial hurdles. It’s obviously not at a level where it creates a full production pipeline, but rest assured we hear you and we have many ideas for how Cloud Wizard can make it both easier and faster for our customers to extend their pipelines to the cloud. This is just the beginning.

So back to your question. With the VPN connection, if you browse to the render output location on the repository file server you should see the rendered files. Obviously you could copy and paste them to a folder on your local computer if you just want to review them. For production use, there are a number of strategies that might be used.

A common approach is to use a robocopy or FTP script to synchronize the output down to a local location at regular intervals. Another approach would be to write a right-click transfer script or a job completed event plugin. We recognize that each of these approaches takes some technical knowledge and effort, so we are working on some ideas to make this easier. Stay tuned!

next question:
the wizard setup the repository on a cloud machine, when this exits beta and we go to put a cloud rendering solution into production will I be able to use my existing repository that is local to our studio? If not how do you see the integration into existing pipelines going?

The first release version of Cloud Wizard will be essentially the same as the beta, although we hope to do a little more work on the storage configuration and also widen the set of demo job types, as time permits.

For customers who prefer to submit jobs to a local queue, a good approach would be to then use the job transfer script to selectively transfer jobs to the cloud repository. It’s also possible to configure the cloud Slaves to work directly with a local repository, but considerations need to be made for the internet bandwidth and asset transfer. Again, we have some ideas for this and hope to begin discussing them with our beta testers in the near future.

we are very interested in having a cloud capacity in our back pocket, but not using it under normal circumstances, only when we can bill an expedite fee to our clients.

This is one of the key use cases that we are supporting. Cloud is great for overflow, so we want to make it really easy for our customers to turn cloud compute on and off like a spigot when needed.

Ideally for us it would be able to accommodate the following, i know some of this is a tall order, some it probably already does, and some of it is probably silly ideas.

1. use our current repository so all the studios jobs are tracked and scheduled in one spot

As I mentioned, there are some bandwidth considerations that come into play when using a single repository. That said, we understand the desire for centralized tracking.

2. let us create a cloud “pool” or something similar that users can submit to that will access the cloud capacity and will be very clear when it does so as there are hard costs associated with it.
   a. user access control to who can submit a job to the cloud pool

The control of which jobs are eligible for cloud rendering is handled through Groups and configured through the Group Mapping feature in the Cloud Provider settings. I like the idea of adding access control for submitting cloud jobs.

3. An easy way to set limits for budgets
   a. when the budget is exhausted the cloud job shuts down and it only renders on our local hardware
4. A clear way to visualize expenditures so everyone can see the costs associated with the rendering and monitor costs

Right on. These are on the wishlist!

4. easily integrate with our rendering tools, in our case max and vray.
5. not require extensive IT knowledge to make it work or maintain it, we are a small studio and its hard to devote someone to managing this full time. Its ok if its a bit complex to get it going, but it needs to be easy to use for artists.
6. saves or moves the files created to the folder the artist wanted them saved in on our network

These are very much in line with the purpose of the Cloud Wizard: Make cloud super easy to manage and consume.

7. perhaps a prediction tool where we could send our job to our farm to render a few frames, we could then get a report back showing some type of cost and time estimate, or a graph showing costs vs time based on how many nodes would be spun up to render the job and then give the artist a slider to choose how many nodes to create to balance the cost and delivery time based on the prediction.

This one is tricky. A typical render queue is a chaotic system that is naturally averse to prediction. That said, there are things that can be done to help customers estimate costs and plan resources. Again, we hear you.

keep up the great work, this is looking very promising!

Thanks for the kind words and great feedback. Keep it coming!

[twuelfing]

thanks so much for the feedback,
we are eternally grateful to have such a fantastic vendor for our rendering scheduling, you guys are the best.
i will keep posting questions and comments as we experiment more, some will likely be silly hopefully a few will be useful.

thanks again!

[Timor]

Hey Troy,

I would suggest you look into bittorrent sync for getting the files back to you (and maybe also the assets into the cloud).

We pretty much replaced FTP with bitsync, as it is so much faster, better and more secure.

Cheers

Timor