Automatic Job Cleanup doesn't work

Brad_the_Beast · May 15, 2015, 8:02pm

Our configuration is set to delete jobs after two days. It has not been working so I have been deleting jobs manually. The checkbox is checked, two is entered into the field, and the mode is set to delete.

dwallbridge · May 15, 2015, 8:34pm

Hello Brad,

Can you do me a favor and turn on slave verbose logging in the application logging section of repo options, then wait until there are jobs that should be removed, and then trigger house cleaning, and when it’s done send over a slave (or pulse if it’s running) log for us to look at? I want to see what the house cleaning is seeing in that regard. As well, can you screenshot the cleanup tab in job settings so we can confirm what should be happening? Thanks!

Brad_the_Beast · June 8, 2015, 7:14pm

2015-06-08 12:04:22: 2015-06-08 12:04:22: 2015-06-08 12:04:26: 2015-06-08 12:04:27: 2015-06-08 12:04:27: 2015-06-08 12:04:27: 2015-06-08 12:04:27: 2015-06-08 12:04:27: 2015-06-08 12:04:27: 2015-06-08 12:04:27: 2015-06-08 12:04:27: 2015-06-08 12:04:27: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:37: 2015-06-08 12:12:38: 2015-06-08 12:12:38: 2015-06-08 12:12:38: 2015-06-08 12:12:38: 2015-06-08 12:12:39: 2015-06-08 12:12:40: 2015-06-08 12:12:40: 2015-06-08 12:12:40: 2015-06-08 12:12:42: 2015-06-08 12:12:43: 2015-06-08 12:12:43: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: 2015-06-08 12:12:44: Deadline Monitor 7.0 [v7.0.2.3 R (24b5c0a7f)]
Time to initialize: 342.000 ms
Auto Configuration: No auto configuration for Repository Path could be detected, using local configuration
Auto Configuration: Picking configuration based on: 00-23945.local / 20.8.37.192
Auto Configuration: No auto configuration could be detected, using local configuration
Time to connect to Repository: 506.000 ms
Time to check user account: 4.000 ms
Time to purge old logs and temp files: 37.000 ms
Time to synchronize plugin icons: 81.000 ms
Time to initialize main window: 292.000 ms
Main Window shown
Time to show main window: 64.000 ms
Performing house cleaning
Performing Job Cleanup Scan…
Job Cleanup Scan - Loading completed jobs
Job Cleanup Scan - Loaded 7 completed jobs in 6.754 ms
Job Cleanup Scan - Scanning completed jobs
Job Cleanup Scan - Deleted 0 and archived 0 completed jobs in 4.114 ms
Job Cleanup Scan - Done.
Purging Unsubmitted Jobs
Unsubmitted Job Scan - Loading unsubmitted jobs
Unsubmitted Job Scan - Loaded 0 unsubmitted jobs in 1.083 ms
Unsubmitted Job Scan - Done.
Purging Deleted Jobs
Deleted Job Scan - Loading deleted jobs
Deleted Job Scan - Loaded 4 deleted jobs in 1.558 ms
Deleted Job Scan - Purged 0 deleted jobs in 104.000 μs
Deleted Job Scan - Done.
Purging Old Job Auxiliary Files
Auxiliary File Scan - Scanning for auxiliary directories
Auxiliary File Scan - Found 56 auxiliary directories in 17.177 ms
Auxiliary File Scan - Loading job IDs
Auxiliary File Scan - Loaded 12 job IDs in 3.551 ms
Auxiliary File Scan - Purging job auxiliary files ‘556cc6efde237404df17740a’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556ce0a37a7a6d0a633450f8’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556ce2247a7a6d0ae285a0bc’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556ce2747a7a6d0b1a260bb6’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556dd3247a7a6d021e26bb9a’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556df5347a7a6d10b248eaa9’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e10e67a7a6d12faf06e2a’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e12ff7a7a6d13572b3362’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e1314de237403cb46c136’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e13317a7a6d138aba9f12’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e13a47a7a6d13c69b3011’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e17927a7a6d14411fc5f3’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e1df7de2374044539c98a’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556f5ae2de237404257ac253’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556f6d52970b130641bae2d0’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘557082cf3e13f35fcbea8563’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556cc6efde237404df17740a’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556ce0a37a7a6d0a633450f8’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556ce2247a7a6d0ae285a0bc’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556ce2747a7a6d0b1a260bb6’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556dd3247a7a6d021e26bb9a’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556df5347a7a6d10b248eaa9’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e10e67a7a6d12faf06e2a’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e12ff7a7a6d13572b3362’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e1314de237403cb46c136’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e13317a7a6d138aba9f12’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e13a47a7a6d13c69b3011’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e17927a7a6d14411fc5f3’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556e1df7de2374044539c98a’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556f5ae2de237404257ac253’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘556f6d52970b130641bae2d0’ because the job no longer exists
Auxiliary File Scan - Purging job auxiliary files ‘557082cf3e13f35fcbea8563’ because the job no longer exists
Auxiliary File Scan - Purged 32 auxiliary folders in 105.392 ms
Auxiliary File Scan - Done.
Purging Old Job Reports
Job Report Scan - Loading job report collections
Job Report Scan - Found 27 report collections in 6.496 ms
Job Report Scan - Loading job IDs
Job Report Scan - Loaded 12 job IDs in 1.051 ms
Job Report Scan - Purging job reports for ‘556cc6efde237404df17740a’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556df5347a7a6d10b248eaa9’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556e10e67a7a6d12faf06e2a’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556e1314de237403cb46c136’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556e1df7de2374044539c98a’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556f5ae2de237404257ac253’ because the job no longer exists
Job Report Scan - Purging job reports for ‘557082cf3e13f35fcbea8563’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556ce0a37a7a6d0a633450f8’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556ce2247a7a6d0ae285a0bc’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556ce2747a7a6d0b1a260bb6’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556dd3247a7a6d021e26bb9a’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556e12ff7a7a6d13572b3362’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556e13317a7a6d138aba9f12’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556e13a47a7a6d13c69b3011’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556e17927a7a6d14411fc5f3’ because the job no longer exists
Job Report Scan - Purging job reports for ‘556f6d52970b130641bae2d0’ because the job no longer exists
Job Report Scan - Purged 16 report collections in 6.853 s
Job Report Scan - Purging old job report files
Job Report Scan - Purged 0 report files in 1.769 ms
Job Report Scan - Done.
Purging Obsolete Slaves
Obsolete Slave Scan - Skipping because it is disabled in the Repository Options
Purging Old Slave Reports
Slave Report Scan - Loading slave report collections
Slave Report Scan - Found 2 report collections in 2.779 ms
Slave Report Scan - Loading slave IDs
Slave Report Scan - Loaded 2 slave IDs in 1.037 ms
Slave Report Scan - Purged 0 report collections in 16.000 μs
Slave Report Scan - Done.
Purging Old Limits
Old Limit Scan - Loading machine limits
Old Limit Scan - Found 0 machine limits in 686.000 μs
Old Limit Scan - Done.
Purging Temporary Repository Files
Temporary File Scan - Scanning for ‘connectReadWriteTest’ files
Temporary File Scan - Deleted 0 temporary files in 19.007 ms
Temporary File Scan - Done.
Purging Old Statistics
Old Statistics - Skipping job statistics because the option to purge them is disabled in the Repository Options
Old Statistics - Purging slave statistics that are older than Feb 08/15 12:12:44
Old Statistics - Purged old slave statistics in 1.870 ms
Old Statistics - Purging repository statistics that are older than Feb 08/15 12:12:44
Old Statistics - Purged old repository statistics in 3.679 ms
Purging Deleted Document Stubs From Database
Deleted Document Stubs - Deleting stubs that are older than 3 days
Deleted Document Stubs - Deleted 0 stubs in 906.000 μs
Triggering House Cleaning Events

Brad_the_Beast · June 8, 2015, 9:03pm

Attached screenshot.

eamsler · June 8, 2015, 9:48pm

It looks like I got the same thing here.

Steps to reproduce:

Set some job to have an old date (I needed to use a MongoQ query):

db.Jobs.update({ _id: "552fb0b2a9600f22cc604d18"},{$set: {"DateComp": ISODate("2014-01-01T01:01:01.000Z")}});
// confirm it worked
db.Jobs.find({_id: "552fb0b2a9600f22cc604d18"}).pretty();

Set times as Brad has
Run house cleaning through the Monitor. I got this at the job HC step:

2015-06-08 16:45:32: Performing Job Cleanup Scan... 2015-06-08 16:45:32: Job Cleanup Scan - Loading completed jobs 2015-06-08 16:45:32: Job Cleanup Scan - Loaded 3 completed jobs in 20.001 ms 2015-06-08 16:45:32: Job Cleanup Scan - Scanning completed jobs 2015-06-08 16:45:32: Job Cleanup Scan - Deleted 0 and archived 0 completed jobs in 40.002 ms 2015-06-08 16:45:32: Job Cleanup Scan - Done.

Internal bug report ticket created.

eamsler · June 9, 2015, 9:42pm

Ryan’s fixed this. It should be in Deadline 7.2 when it enters beta (it’s going to be next week or the week after).

Brad, are you folks in the beta program? If not I can send you the forms or you can e-mail beta@thinkboxosftware.com.