Asset Crawler for Balancer

[anon65281487]

Hi,
Is there any asset crawler (i assume python) script for using with vmx and cloud?

Best regards,
Daniel

[Coulter]

Hi Daniel,

It appears the Asset Crawler is not shipping with the builds. We’ll get that corrected for future builds. In the mean time I’ve attached it below. Here are Jon’s original instructions, with some minor edits/updates:

Attached is a copy of the server-side asset checker script we wrote for VMX.

This script basically just listens on a port (default is 53544, you can change this by editing the script) for requests from the Balancer. The Balancer will send it asset paths, and the script checks if they are accessible from its current location (returning true or false). The idea is to determine whether or not required Assets are available on the Cloud before spinning up VMs for a Job, to just proceed to fail on missing assets over and over.

There are a couple things to note:
For this to be useful at all, you need to be specifying required assets for Jobs at submission time.
You also need to make sure whatever machine you run this script on will be seeing files the same way Cloud Slaves in that region will be.

The Balancer obviously also needs to be able to talk to the machine running this script; that able to resolve its hostname/IP, and allowing the port through firewalls (again, 53544 by default). These settings are both configured in the Balancer tab of the Cloud Region setup.

It’s just a python script, so serverside you just need to run “python </path/to/script.py>”, assuming Python has already been set up.

AssetCrawler_Server.zip (3.05 KB)

[anon65281487]

Thanks James.