I work at a pretty large company and constantly have to provide usage metrics to justify farm expansion. The stats reporting collects a lot of great information but I have a few issues to comment on.
-
if an artist suspends a job it is ignored by the stats even though it consumed resources.
-
If an a job is manually marked as completed it is ignored by the stats
-
it would be nice if there were some total utilization metrics, not necessarily by processor usage percentage, but available render hours vs hours spent rendering. (right now we calculate this in an excel sheet) if the deadline system goes to a database this would get even easier.
-
We are calculating average frame render time, average job render time, total number of jobs, total number of frames, average ram usage … there are the metrics that are useful to us (along with utilization). it would be nice if deadline could generate a report, however once we see a database version of all this i think we can create our own querys and reports.
-
I would love a “farm monitor view” that allows me to monitor the overall farm status. (we put the monitor up on a big 55 inch LCD in our studio so we can see whats happening, but the current views are very task centric)
a graph maybe of the total utilization over time updating at a modifiable interval.
total jobs pending, total jobs running, time until farm is idle estimate (i know this one is a little pie in the sky as you cant estimate render time for a job that has not started)
perhaps a list of the nodes with the current ram and proc usage as a percentage of available ( I realize that people with a large number of nodes may have trouble fitting it all, but it could be a multi-column list)
-
this one isnt for the stats or “farm monitor view” but when we look at jobs our artists frequently only care when someone elses job will finish. I would be sweet to be able to have a job view that showed just a list of jobs in the order that they will currently be executed and then next to each job the ones that are running have a count down until finished and a clock time that shows what time of day they will complete. and the next job in the queue would then have a count down until start. This would help with planning and prioritizing.
OK i know this is a lot, but I wanted to provide some feedback about where we see some shortcomings in the data presentation.
thanks
Thanks for the feedback. I should note that it is our intention to start storing statistics in a Database, and we’re hoping to implement this for the next major release of Deadline. We will definitely take some of your suggestions under consideration, whether it be for the initial DB stats feature, or for future development on this feature.
-
Currently, our jobs stats system only keeps track of completed jobs. So stats from suspended jobs, failed jobs, or jobs deleted mid-render will never make it to the stats system. Once we have a DB storing this info, it will likely allow us to track more information.
-
Which version of Deadline are you running? The Deadline 4.1 release fixed this problem.
3 & 4. With the DB solution, you will definitely be able to write your own queries and pull this information out.
-
We already have a feature like this on our wish list, and we would love to implement this sometime soon. However, we don’t have an ETA for it at the moment.
-
You could use the Job Monitor application to monitor specific jobs:
software.primefocusworld.com/sof … onitor.php
We can look at showing more information in the Job Monitor, like estimated remaining render time. We’ll add that to the wish list.
However, calculating expected start time is surprisingly difficult, since job order is different for different slaves (based on pool, group, machine limit, etc). We would have to figure out the scheduling order for every queued and active job for every individual slave, and then figure out some sort of average to guesstimate when the job might start. It’s really not something that can be done reliably.
Cheers,
Hi,
we are using 5.1 and had enjoyed the dynamics and speed retrieving statistics from database,
however, is it possible to modify what attributes to track for statistics gathering in the configuration so we can store custom/more sets of data into the database?
eg. in the tools> configure repository options> statistics gathering there’s option ( in an added tab) to tick on/off attributes under each table of " Job", “repository”,“slave” to track/store/ report by deadline
ex:
in “Slave” it currently stores CPU, RAM, and names, it would be great to also store pool assignment, group assignment.
We think this would be a great feature to many deadline customers to supply them with more customization to data gathering;
and having a more complete access to utilization data can enable admins to engage more variety of perspectives with an organization.
It currently isn’t possible to modify what gets stored in the database. We would have to explicitly add those attributes and then add new columns to the db tables. I’m curious how you would use assigned pools and groups if they were available. I guess you could figure out how well they were utilized in their current assignments?
I was thinking that it would be great to have database that has the same access to all information we can get from deadline Monitor,
This would add database enhancement to our monitoring of resources and utilization on the farm.