I was wondering if it’s possible to, outside of deadline, send a query to the Mongo DB to get specific info on a node (based on node ip) ?
I did find the a summary of some sort when connecting to the server ip with port 28017.
But here are only job ID’s used and I cannot get the job name from it.
Ideally i’m looking for this information to access outside of deadline:
job name
job plugin (max/realflow etc)
task progress
uptime
I did find the last 2 as they are in the summary, but wasn’t able to get the first 2 in ‘plain’ data.
is there any way to access that information?
the reason is that I have a small sized renderfarm and created some desktop tools that show me the state of all the nodes (cpu/ram/temp etc).
Ideally I would like to include this Deadline information in the node overview.
Here is an example to connect to the deadline DB and query slaves by hostname, ip address, user or mac address
You can change the Constants in the Deadline class to match your server name and port
from pymongo import Connection
# MongoDB classes
class MongoDb(object):
""" Base class for MongoDB database connection """
SERVER = 'localhost'
PORT = 27017
NETWORK_TIMEOUT = 20
DATABASE = 'test'
def __init__(self):
super(MongoDb, self).__init__()
self.connection = None
self.db = None
def connect(self):
""" Connect to the Mongo db server """
if self.connection is not None:
return
self.connection = Connection(self.SERVER, self.PORT, network_timeout=self.NETWORK_TIMEOUT)
def getDb(self, name):
""" Set the current db to the provided db name. Returns the db """
self.connect()
self.db = self.connection[name]
return self.db
class Deadline(MongoDb):
""" Deadline mongo database connection """
SERVER = 'deadline6'
PORT = 27017
DATABASE = 'deadlinedb'
def __init__(self):
super(Deadline, self).__init__()
self.getDb(self.DATABASE)
def findSlave(self, ip=None, host=None, user=None, mac=None):
query = {}
if ip: query['IP'] = ip
if host: query['Host'] = host
if user: query['User'] = user
if mac: query['MAC'] = mac
return self.db['SlaveInfo'].find(query)
def test():
def printResult(rslt):
print 'Found %d slaves' % rslt.count()
print [s for s in rslt]
print '-'*100
deadlinedb = Deadline()
print 'Find all slave:'
printResult(deadlinedb.findSlave())
print 'Query by IP:'
printResult(deadlinedb.findSlave(ip="192.8.200.166"))
print 'Query by host name:'
printResult(deadlinedb.findSlave(host="3D36"))
print 'Query by user:'
printResult(deadlinedb.findSlave(user="sberger"))
print 'Query by Mac address:'
printResult(deadlinedb.findSlave(mac="00:E0:81:75:DF:25"))
if __name__ == '__main__':
test()
To be honest… I had hoped it could be done with adding some text to the URL as I have 0.1 programming experience
And have been looking at the installation page… but have no clue on what to install where and how to proceed?
With the desktop tool (rainmeter) I can only access a webpage and use a webparser to extract data from it.
So it should be something that’s always running in bthe background and easy to access.
Does that mean that for each ip I need to make a separate “page” that is running all the time?
I had hope to be able to do something like this:
<ServerIP>:28017/GetInfo:<nodeip>
where will be the running Deadline server
and being fed by the rainmeter tool as it’s node specific.
Just to get an idea of the context, attached a screenshot of my desktop with for each node a widget
Where in the unfolded state I want to display the deadline job.
Hi,
I think you would in this context be better of using our RESTful API (in Deadline v6.1) to query the DB via Pulse. You just need to run the Pulse application, enable the WebService in Repo Options (reserve a namespace if running Pulse on Windows) and you can query it like this:
(msdn.microsoft.com/en-us/library/ms733768.aspx)
(more info here: thinkboxsoftware.com/deadlin … ctionality)
See the ‘user manual’ docs for all the functions available for you, which can be found in the same link you were sent by our Sales team to download Deadline v6.1.
Perfect!! Works like a charm thank you for that option.
Is it correct it only updates the values every 10 to 12 seconds or is that something set within Pulse?
The seconds are not that important for my purpose, was just curious.
I have a feeling that Pulse in v6.1 still caches it’s information every 10 seconds from the DB (I need to double-check with a colleague) and this value isn’t exposed to users. Is this going to be an issue for you guys? Is working with data that is up to about 10-15 seconds old an issue? (I assume you are only reading and not needing to write any info back to the DB?)
Cheers,
Mike
For me not at all at this moment.I only noticed it due to the rendertime making jumps of 10-12 seconds.
The tool I’m making now uses OpenHardwareMonitor to get data about cpu speeds, loads, temperature etc from the nodes.
I thought that maybe a future version could get all this info via deadline (thus no need to install monitoring app on each node)
but deadline doesn’t monitor cpu-temp nor gives the actual cpu speed (only system registered cpu speed).
Therefor my nodes are registered in deadline as running on 3.5ghz while they are safely overclocked to 4.0 ghz.
so I only need it to monitor the jobs etc… a small delay isn’t a problem there.
Could it be that if Pulse gets 10-20 requests for Node information (8080/api/slaves?Name=NodeName&Data=info) in the same second it get’s overloaded?
As I notice when I refresh my application and all requests are send at once there is no information returning.
Not sure though if it’s due to the application or the pulse response, so I figured to ask both ways
the pulse server is running on a AMD x2 dual 5200+ 2.7Ghz with 4GB ram.
It’s mainly running pulse/DL server and usually is somewhere between 50-75% cpu use (so there is capacity for cpu spikes)
Can I ask the OS and version you run on the pulse machine? That would be a good start down that path. There is some chance the OS might have an issue with that, but Pulse itself should be fine as long as it is set to a decent amount of connections in Pulse settings.
Please, please note that I’m not a 100% sure that it’s Pulse causing this.
I’ve posted the similar question on the software’s forum (rainmeter)
It basically works the first time when I start up the program, but when I refresh all Node elements it just doesn’t receive any info anymore.
The first time when the program starts the build-up of all the elements is slower, when it’s refreshed it reloads all items quicker so I thought that maybe it could be overload related.
If I just request a page in the browser it seems to work, but I can only request 1 page at a time
In that program there are 2 webparser-elements :
1 - which reaches each node individually to get info about the system state (using the openhardware plugin that sends all data realtime to a local webpage)
2 - a deadline related webparser that requests information about 10 different nodes from the single deadline server machine.
The first webparser always works, the second usually only the first time when loading… sometimes the second too, but more often it doesn’t
again, I am not sure it’s deadline related. there is some other stuff (related to the deadline webparser) that does weird things within the software
I mainly put it here to see if there is a known issue with multiple reloads to exclude this option.
I wouldn’t want you to spend too much time on it researching it as I haven’t heard back from rainmeter on this topic.
The functions that Mike provided should return fresh results each time they are called. While it seems unlikely, it’s possible that the browser itself is caching the results, since the URL does not change between calls. I’m not sure how your code is structured, but if you’re using javascript to perform updates, something like
location.reload(true)
should make the browser reload the results and not use the browser cache (that’s what the ‘true’ does). Another thing to try is to add some debugging print statements (or just output somewhere in the html itself) the raw results of the function calls. That may help narrow down where the problem is occurring.
I actually thought to present it on the forum for people to be used once it’s fully tuned.
It’s all based on the open-source software Rainmeter and I think it can be useful for small farms (it will only support up to 11 dails by default > 9 nodes, 1 workstation, 1 server)
I now have it running on a 7inch touch-monitor (only the circles) on the side (always visible) and the fullgraph-version on the desktop.
Really gives a good instant impression of what the farm is doing.
Think it will lose it’s purpose when you have 100 of these things on a monitor thou
I do plan to make 1 dail for the average/summary of the whole farm.
Could imagine that to be used for different Deadline pools/groups.
Now it does depend on the having the Open Hardware Monitor installed on each node.
But when the first version is optimized I think it might be good to make a deadline-only version (OHM isn’t too stable either when the render is at full force)
Via deadline it will have a bit less information available, but in return it will be less hassle to get it running and is 100% stable.
I notice when a slave is doing a job the value here is “1”
When it’s idle it’s “2” and when offline it’s “3”
I’m guessing this is no coincidence, but I seem to be unable to find this in the documents and being able to doublecheck / see what other stat’s there are.
can anyone tell me more about this?
Looks like the list of possible status values might be missing from the docs (or maybe I was looking in the wrong place too), so I just peeked under the hood:
It appears 5, 6, and 7 referred to status values that are now handled in an entirely different way, such as identifying that a slave is New (5). New slaves are now treated as Unknown (0) and go through a registration process.
thank you for that option.
