Get the total nodes used to render a job (re:jobstat.py)

Deadline
1

Hi, I’ve been looking at the jobstat.py as a start to write some additional info after the job is completed. I can get these two but not the last one.

a) Number of tasks – i can get this from stats.Taskcount
b) Total hours of rendertime – i can get this from the stats.averagetaskrendertime and convert to hour
c) Total nodes used by the job — pls help how can I get this. Several tasks in a job could be rendered in the same node; so it should just calc how may unique nodes were used to render the job.

Thanks in advance!

2

I think the basic plan would be to grab the Slaves from the reports, then create a set of them to remove duplicates, then count them.

Here’s the basic jist, but I haven’t tested this. Let me know if you need help getting it working:

from sets import Set

...

report_collection = RepositoryUtils.GetJobReports(string_job_id)
reports = list(report_collection.GetAllReports(report_collection))

slaves = Set()
# This could be done with a Python one-liner. I'm just a noob :D
for report in reports:
    slaves.add(report.ReportSlaveName)

slave_count = len(slaves)
3

An alternative apporach is to start by calling Deadline.Scripting.RepositoryUtils.GetJobTasks. This will return a TaskCollection. Get a list of normal tasks with the TaskCollectionTasks property. For each Task there are an number of details that can be retrieved, such as TaskSlaveName.

It might be worth trying both approaches and seeing which is faster.

4

Edwin, unfortunately i could not get the code to run as i am totally not a programmer. pls help with a working script, thanks again.

5

No problem sir! Also, I liked James approach more than mine since it should show only the machines which successfully rendered and should in most cases be faster (at least a smaller or equal number of tasks will exist versus job reports).

You can copy and paste either “tasks_way” or “reports_way” into your script and call it.

from Deadline.Plugins import *
from Deadline.Scripting import *

from sets import Set


def __main__():
    job_id = "5851875be9faf18185fa9334"
    slave_count = reports_way(job_id)
    print("Number of slaves: {}".format(slave_count))
    slave_count = tasks_way(job_id)
    print("Number of slaves: {}".format(slave_count))


def tasks_way(string_job_id):
    job = RepositoryUtils.GetJob(string_job_id, True)
    tasks = RepositoryUtils.GetJobTasks(job, True)
    tasks = list(tasks.TaskCollectionAllTasks)

    print("Found {} tasks.".format(len(tasks)))

    slaves = Set()
    for task in tasks:
        slaves.add(task.TaskSlaveName)

    return len(slaves)


def reports_way(string_job_id):
    '''
    	Return the number of Slaves which worked on this job. Note that this
    	will include *all* Slaves, including those which may have thrown errors
    '''
    report_collection = RepositoryUtils.GetJobReports(string_job_id)
    reports = list(report_collection.GetAllReports())

    print("Found {} reports.".format(len(reports)))

    slaves = Set()
    # This could be done with a Python one-liner. I'm just a noob :D
    for report in reports:
        slaves.add(report.ReportSlaveName)

    return len(slaves)

If you want to run it straight, just send it through deadlinecommand on any of the three platforms:

Mobile-029:Resources edwin.amsler$ ./deadlinecommand executescript /tmp/slaves.py
6

SUperb. Thank you Edwin. it works well. Thx again

7

Welcome sir! I really needed an excuse to practice the task/report lookup code again, so it was definitely worth it on my end too. :slight_smile: