We use what are basically crons to do some farm housekeeping, health monitoring, etc. I’m wondering what the immediate reactions are to the idea of adding event hooks for Deadline’s various house cleaning operations, so that custom code could piggy-back on the scheduling of these operations.
It also seems like an addition like this would be well paired with some facility for pre-filtering event listeners (to prevent unnecessary loading of listener modules).
Anyway, this is sort of a crazy thought, and I don’t know if you guys already have plans to expand the options for running periodic tasks in Deadline, but I thought it might be worth bringing up.
Thanks for the info Mike. I guess because there are a lot of “house cleaning” operations that occur at different intervals, I was kind of thinking of a more granular system where some contextual information could be passed to the event-handling code. Some of the operations that come to mind immediately are:
Purging deleted jobs
Automatic job cleanup based on job age
Power management checks
Repository clean-up (which may encompass the first two)
Pending job scan
There may be other things that I couldn’t find settings for in the repository options. However, in the case of things like purging deleted jobs for example, it seems like it would be nice to get an event immediately before this occurs that passes the (deleted but still present) Job object along, to allow event handlers to perform cleanup operations that might hinge on the job’s plugin info data.
We could add an OnJobPurged event trigger. The OnJobDeleted event would trigger when the job is deleted, and the OnJobPurged event would trigger when the job is actually removed from the database.
The OnJobDeleted event is already triggered by this. Probably don’t need another trigger?
Can you give an example of what types of events you’re looking for here? In Deadline 7, we already have 5 slave-based events:
OnSlaveStarted: Slave is starting up
OnSlaveStopped: Slave is shutting down
OnSlaveStartingJob: Slave is starting a job
OnSlaveRendering: Slave is rendering a job
OnSlaveIdle: Slave is idle
Since power management is basically starting and stopping slaves, maybe these would suffice? Or are you looking for something more specific to power management?
This is the one that Mike was referring to. It fires at the end of every housecleaning interval.
We already have the trigger that fires when a pending job is released. Can you maybe give more specifics here as well? Not sure we need a general OnPendingJobScan trigger, since the OnHousecleaning trigger already fires every interval.
Sorry, I was just giving examples of possible house-cleaning operations, but it sounds like most of those would overlap with existing functionality. The OnJobPurged event is probably the only one I would definitely be interested in, provided it was fired before the job’s data was removed from the database. Honestly, from looking at the repository configuration, it’s tough to know exactly what “house-cleaning” encompasses, or how frequently it is run.
Cool. I’ve added OnJobPurged to the todo list, and we should be able to make fire before the purge takes place.
If you do a manual housecleaning from the tools menu in the monitor, you can check the console to see everything that housecleaning is looking at. When Pulse or the Slaves perform it, the odds of each operation are randomized a bit (1/10).
