I periodically have an issue where my repo says it’s House Cleaning but I cannot find a trace of it anywhere and it ‘lasts’ for about 2 hours - this is a massive problem for us as it prevents job submission as we have a process tied into regular House Cleaning that submits a ‘prequel’ job to the farm before the rest of the users submission can appear , any pointers on where else i can look? I checked repo/db/logs/system logs and all workers already.
Could you provide more details about the configuration repository options available for housekeeping tasks?
All good - speaking to Justin (thanks!) in a support ticket, and will copy the notes here if anyone else has this issue. Please note you need a running RCS or Pulse to handle the House Cleaning here, make sure the Workers are not permitted to run the process.
If you are on Linux, you may find the deadline_mongo file here as an example depending where you installed it:
/opt/Thinkbox/DeadlineDatabase10/mongo/application/bin
Justin notes here! 
We’re going to restrict where the process can be run, and keep it from trying to do too much and possibly timing out before it gets to your events.
To do that, in the Monitor under Tools → Configure Repository Options → House Cleaning → House Cleaning set the following:
- Leave the ‘House Cleaning Interval’ at the default 60 seconds for now.
- Disable ‘Allow Workers to Perform House Cleaning if Pulse is not Running’
- Enable ‘Run House Cleaning in a Separate Process’
- Enable ‘Write House Cleaning Output to Separate Log File’
- This will give us a log for just the House Cleaning process on the Pulse / Remote Connection Server machine and simplify chasing this issue.
- Leave the ‘House Cleaning Process Timeout’ at the default 1800 seconds for now.
- Enable all the options in the ‘House Cleaning Maximums Per Session’ section and leave them at their defaults.
- This way if the House Cleaning process is spending all its time trying to clear out a newly deleted pile of jobs it’ll limit how much it does per pass. Everything that doesn’t get cleaned will be cleared out in following passes.
If after all that you’re not getting a House Cleaning log on the machine it could be the database lock to track if House Cleaning is running isn’t getting released properly.
To do that you’ll need a MongoDB console connection. Use the steps here from MongoDB or use the deadline_mongo.bat file in your DeadlineDatabase10\mongo\bin directory.
Once starting the mongo console run:
use deadline10db
The house cleaning lock can be inspected with:
db.DeadlineSettings.find({"_id":"lock_entries"}).pretty();
If this includes any entries like "InProgress" : true we’ll try clearing this out to force House-cleaning to run. If there’s no "InProgress" : true then there’s something else going on, don’t bother with the rest of these steps.
To clear out this entry in the collection run:
db.DeadlineSettings.deleteOne({"_id":"lock_entries"});
Should get a response like:
{ "acknowledged" : true, "deletedCount" : 1 }
1 Like