Log for entire job?

Berkyjay · October 24, 2016, 5:39pm

Is there a way to get a log for an entire job? Not a log for each individual task, but a log for the entire job. Like something that shows up in the task window?

dwallbridge · October 24, 2016, 7:09pm

Hello James,

Wow, I can honestly not recall a single time we have been asked for this, likely due to the fact that this would make a massive log file. Can you give us some more information on the need for something like this, instead of the individual job reports we have now? I am told that this is something that could be scripted using an archived job, but there is no functionality inside Deadline to do this at all

Berkyjay · October 24, 2016, 7:39pm

Hey Dwight,

so basically what I’m looking for would be a simple rundown of the details one would see in the task window put into a txt file.

Kind of like:

EDIT: The formatting didn’t come out like I had hoped.

I basically need to be able to show how a job performed on our farm on a macro level. To illustrate, we are currently testing out new hardware and I need to send Dell some logs from our test renders so they can compare to their own diagnostics. A handful of individual task logs doesn’t really help them in this.

I was hoping that there was a feature that I missed that I could get this from. But if there is a script that can help me get this that would be great as well.

Thanks!

panze · October 25, 2016, 6:55am

I think you can just copy most of that straight from jobs tasks view in monitor? Just select everything and right click --> copy.

eamsler · October 25, 2016, 2:37pm

My usual MO is to save all those logs to a folder, then run Unix commands on them.

To shove all logs into one (not very helpful yet):

cat * > all.txt

Then look for lines with particular text:

grep "Slave Name: " all.txt

Because all of the text will be in the right order, you can do some crafty things with regular expressions: (note the Slave name is on the bottom of each set of stats because of its location in the log)

Mobile-029:Testing2 edwin.amsler$ cat * | egrep "CPU Usage|Slave Name"
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric2
CPU Usage: 24%
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric2
CPU Usage: 24%
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric3
CPU Usage: 50%
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric3
CPU Usage: 50%
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric3
CPU Usage: 50%

Berkyjay · October 25, 2016, 4:46pm

You’re totally right, thanks for the tip!

Berkyjay · October 25, 2016, 4:47pm

eamsler:

My usual MO is to save all those logs to a folder, then run Unix commands on them.

To shove all logs into one (not very helpful yet):
cat * > all.txt
Then look for lines with particular text:
grep "Slave Name: " all.txt
Because all of the text will be in the right order, you can do some crafty things with regular expressions: (note the Slave name is on the bottom of each set of stats because of its location in the log)
Mobile-029:Testing2 edwin.amsler$ cat * | egrep "CPU Usage|Slave Name"
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric2
CPU Usage: 24%
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric2
CPU Usage: 24%
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric3
CPU Usage: 50%
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric3
CPU Usage: 50%
Average CPU Usage: 0%
Peak CPU Usage: 0%
Slave Name: Mobile-029-eric3
CPU Usage: 50%

If only I was using a Unix OS.

But I think Panze gave me the best solution.

eamsler · October 26, 2016, 2:21pm

I think their solution is probably best too.