Ways to pull farm statistics from DeadlineCommand

[eamsler]

We got a request in on the ticket system, and I thought I’d share the answer here because there are quite a few different ways to pull stats out of DeadlineCommand. The ask was this:

I’ve been tasked with finding out of there’s any graceful way of exporting the farm statistics automatically as a monthly report.

We added something to DeadlineCommand a little while ago called “FarmReports”. This is going to have a bit more private data in it, so I’ll just post the paramters you can pass in:

FarmReports
   Allows you to view farm reports. If a report name is specified, that report
   will instead be automatically be exported to the given file.
      [Report Name] The name of an existing report
      [Start Date] The start date, in yyyy-MM-dd format
      [End Date] The end date, in yyyy-MM-dd format
      [Export Mode] CSV or TSV
      [Export File Name] The file to export to

​​
​​Another option is ‘GetFarmStatistics’:

​​c:\Program Files\Thinkbox\Deadline10\bin>deadlinecommand GetFarmStatistics
Queued Jobs= 0
Rendering Jobs= 0
Suspended Jobs= 17
Pending Jobs= 1
Completed Jobs= 7
Failed Jobs= 4

Machines Starting Jobs: 0
Rendering Machines: 0
Idle Machines: 0
Stalled Machines: 0
OfflineMachines: 8
Disabled Machines: 0

​​There is also the job stats export (this allows some filtering by pool, group, and plugin). Giving empty quotes for the last three will remove those filters.

c:\Program Files\Thinkbox\Deadline10\bin>deadlinecommand JobStatistics 2018-01 2018-08 "[poolname]" "[groupname]" "[pluginname]"
[5a6f518e652b55297ca9fabd]
JobName=Ping Test
JobId=5a6f518e652b55297ca9fabd
User=amsler
Comment=
Department=
SubmitDateTime=2018/01/29 10:53:34
StartDateTime=2018/01/29 10:53:55
FinishDateTime=2018/01/29 11:01:50
Pool=none
Group=none
Priority=51
FrameList=1-10,20,11-19,21-100
TaskCount=4
FrameCount=100
Plugin=CommandLine
PluginDataFileName=
PluginDataFileSize=0
MachineLimit=0
ErrorCount=0
AverageConcurrentTasks=1
ClockRenderTime=00d 00h 07m 55s
TotalTaskTime=00d 00h 00m 21s
AverageTaskTime=00d 00h 00m 05s
MedianTaskTime=00d 00h 00m 05s
TotalTaskStartupTime=00d 00h 00m 00s
AverageTaskStartupTime=00d 00h 00m 00s
MedianTaskStartupTime=00d 00h 00m 00s
TotalTaskRenderTime=00d 00h 00m 19s
AverageTaskRenderTime=00d 00h 00m 04s
MedianTaskRenderTime=00d 00h 00m 04s
TotalNormalizedTaskRenderTime=00d 00h 00m 19s
AverageNormalizedTaskRenderTime=00d 00h 00m 04s
MedianNormalizedTaskRenderTime=00d 00h 00m 04s
AverageFrameTime=00d 00h 00m 00s
MedianFrameTime=00d 00h 00m 01s
AverageFrameStartupTime=00d 00h 00m 00s
MedianFrameStartupTime=00d 00h 00m 00s
AverageFrameRenderTime=00d 00h 00m 00s
MedianFrameRenderTime=00d 00h 00m 01s
AverageNormalizedFrameRenderTime=00d 00h 00m 00s
MedianNormalizedFrameRenderTime=00d 00h 00m 01s
WastedErrorTime=00d 00h 00m 00s
WastedRequeueTime=00d 00h 00m 00s
TotalImageFileSize=0 Bytes
AverageImageFileSize=0 Bytes
MedianImageFileSize=0 Bytes
PeakRamUsage=3.379 MB
AverageRamUsage=2.344 MB
MedianRamUsage=2.531 MB
PeakSwapUsage=46.773 KB
AverageSwapUsage=33.827 KB
MedianSwapUsage=35.080 KB
PeakCpuUsage=0%
AverageCpuUsage=0%
MedianCpuUsage=0%
UsedCpuClocks=0 Bytes
TotalCpuClocks=0 Bytes​​​​

[Rob1]

Brilliant! Great addition to the deadlinecommand.
Is there a way to specify a custom report when doing an export? I’ve archived a report to a .json but when I specify the report (which is all data, no graphs) I get told:

Error: Report not found, or report does not support exporting: C:\ProgramFiles\Thinkbox\Deadline10\bin\report.json

Also, when I add a custom report to the GUI and close it, the report disappears next time I open it? Am I missing a step?

[eamsler]

The custom report is the right way to do this.

Are you able to reliably have the report disappear? I’ve had this happen occasionally but I don’t know the steps to reproduce.

Can you throw some examples of how you’re running the report as well? It doesn’t support you providing a path to the JSON file as far as I know.

[Rob1]

Ah, that makes more sense, I’ve just been archiving the reports and trying to run from there.
I’ve got the reports to stick around now, apparently clicking the OK button is quite important!

[Panupat]

Can we specify the job we want the statistic for with job ID?

Or, on another thought, can we access this report from an event? Our sup wants to have statistic easily access with each shots, so my plan is to gather this with Job Finished event and reformat the information he needs into a .log or .json file. Stats such as average render time.

[eamsler]

Yeah, an example of grabbing stats after the job finishes already exists here.

You’d just have to modify it to fit your needs.

[Panupat]

Thank you. I’ll check it out.