AWS Thinkbox Discussion Forums

pending/housecleaning logs stopped working

Odd behavior, but the logs stopped being generated for both the pending & housecleaning processes 2 days ago:

[root@deadline02 Deadline6]# ls -al /var/log/Thinkbox/Deadline6/pendingjobscan-deadline02-2014-09-*
-rw-r–r-- 1 root root 81568248 Sep 18 23:59 /var/log/Thinkbox/Deadline6/pendingjobscan-deadline02-2014-09-18.log
-rw-r–r-- 1 root root 243170862 Sep 19 23:59 /var/log/Thinkbox/Deadline6/pendingjobscan-deadline02-2014-09-19.log
-rw-r–r-- 1 root root 167447823 Sep 20 15:38 /var/log/Thinkbox/Deadline6/pendingjobscan-deadline02-2014-09-20.log

[root@deadline02 Deadline6]# ls -al /var/log/Thinkbox/Deadline6/housecleaning-deadline02-2014-09-*
-rw-r–r-- 1 root root 2651673 Sep 18 23:58 /var/log/Thinkbox/Deadline6/housecleaning-deadline02-2014-09-18.log
-rw-r–r-- 1 root root 4796931 Sep 19 23:59 /var/log/Thinkbox/Deadline6/housecleaning-deadline02-2014-09-19.log
-rw-r–r-- 1 root root 2155248 Sep 20 15:36 /var/log/Thinkbox/Deadline6/housecleaning-deadline02-2014-09-20.log

Deadline pulse has been running since the 18th, no stop (ram usage is quite high… 7gigs):

[root@deadline02 Deadline6]# ps aux | grep deadlinepulse
root 20727 9.2 15.2 7180596 5028400 pts/3 Sl Sep18 514:10 mono-sgen --gc=sgen --runtime=v4.0 /opt/Thinkbox/Deadline6/bin/deadlinepulse.exe -nogui

Its still spawning new houecleaning and pending scan processes on queue, but no logs are made…:

[root@deadline02 Deadline6]# ps aux | grep deadline
root 5656 0.0 0.0 103244 844 pts/1 S+ 11:00 0:00 grep deadline
root 17496 0.2 0.1 1312804 38164 pts/3 Sl Sep18 13:48 /opt/mono-2.10.9/bin/mono /opt/Thinkbox/Deadline6/bin/deadlinelauncher.exe -pulse -nogui -upgraded
root 20727 9.2 15.3 7205172 5052792 pts/3 Sl Sep18 514:12 mono-sgen --gc=sgen --runtime=v4.0 /opt/Thinkbox/Deadline6/bin/deadlinepulse.exe -nogui
root 26467 38.8 0.4 678632 136444 pts/3 Dl 10:59 0:14 /opt/mono-2.10.9/bin/mono /opt/Thinkbox/Deadline6/bin/deadlinecommand.exe -DoPendingJobScan True
root 30180 82.5 2.5 1383808 843184 pts/3 Rl 11:00 0:20 /opt/mono-2.10.9/bin/mono /opt/Thinkbox/Deadline6/bin/deadlinecommand.exe -DoHouseCleaning 10 True

The processes are doing stuff:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30180 root 20 0 1409m 887m 9972 R 100.2 2.8 1:21.24 deadlinecommand
25983 root 20 0 793m 125m 11m R 55.4 0.4 0:03.80 deadlinecommand

In 6.2.1 beta 2, you need to enable separate logs in the Housecleaning section of the repository options. I think in the build that Jon gave you, it was just enabled by default.

Cheers,
Ryan

Yes, we have those settings set. The logs worked for 3 days, and then stopped working without any change.

To clarify, the logs are NOT being put in the main pulse log neither. THey are going nowhere, if they are working at all somewhere…

Well. Ill be damned. They are going to the main pulse log again.

I think what happened is that the deadline.dll i got to be able to run the job report housecleaning separately rolled back the defaults in the repo settings…
I was running it on my workstation, and was tweaking repo options here and there.

Privacy | Site terms | Cookie preferences