AWS Thinkbox Discussion Forums

Feature Request: Include EndJob output in log

For plugins using the AdvancedPlugin interface, the output of any registered EndJobCallbacks isn’t included in the job report. It does show up in the slave’s console output, but seems like it may be running after the task log is finalized. Is this by design? It would be nice to be able to log things from an end job callback and be able to log output from them.

Thanks,

-Nathan

Hey Nathan,

This is by design. The issue is that when a slave gets a new task, EndJob is called as part of a pre-rendering step if the new task belongs to a different job. EndJob is also called if a slave doesn’t find any new tasks. So when the slave is finalizing the log for the current task, it doesn’t know if EndJob will be called.

Basically, what I’m saying is that it would take some significant refactoring to get the EndJob log to be included with the final task log, so it’s probably something we want to avoid changing at this time.

Cheers,

  • Ryan

Hmm, I see. This is one of the things I’ve been making some notes on to share with you guys, but one of the things I have in mind as a wishlist item is a more generic and flexible callback system. Right now, the way it’s set up with the differences between the simple and advanced plugins is somewhat confusing, especially since they’re still technically the same class.

Without getting into all of that completely right now, I picture the base DeadlinePlugin class being an abstract, non-functional type, with DeadlineSimplePlugin and DeadlineAdvancedPlugin deriving from it. The base class would then implement a set of standard callbacks that would exist and be called for all jobs in a consistent way:

JobStart
TaskStart
CheckLogLine
TaskDone
JobDone
JobFail (?)
JobArchive
JobDelete

On top of this, each subclass could also define its own more specific set of callback slots (if necessary). Taking this one step further would be the ability to register arbitrary callback slots by a name or pattern, and then fire them at any point in the plugin’s execution. I haven’t put too much thought into this side of it yet, but I’ll keep thinking it through and adding to my notes.

Now, I imagine anything like what I’ve listed above would take a significant amount of work, but If you’ve got any thoughts on any of this, I would love to hear them.

And to follow up on the original topic, I guess my next question would be, is there a callback slot that can be used as a cleanup handler, both for the job and for each task, without using a try/finally in the RenderTasks slot or something similar?

Thanks,

-Nathan

Our Event Plugin system is meant for the more general stuff:
thinkboxsoftware.com/deadlin … eventssdk/

That covers most of the events you’ve listed, except for the TaskStart (which we would like to add in the near future) and the CheckLogLine (which I’m not so sure about).

EndJob only gets called at the end of the job. There is PostRenderTask for simple plugins, but that only gets called when the task succeeds. If you want guaranteed cleanup for each task, then you’ll wand to use an Advanced plugin and wrap the contents of RenderTasks with a try/finally block.

Cheers,

  • Ryan

Thanks for the heads-up on the event plugins… I think those are new since the last time I used Deadline in production, so I’ll have to familiarize myself with them.

One impression I get is that, once you’ve created an event plugin, all its methods will be called for every event of every job regardless of job type or settings (unless SuppressEvents is True, in which case none are executed). This seems like it creates the unfortunate side effect of forcing code branching within an event plugin to implement behavioral differences per job class or job settings. Am I correct in this assumption?

Now, this obviously isn’t a deal-breaker, but it would be nice to have a more explicit way of registering or excluding specific events or event plugins from any given job or job class. I imagine what I’ll end up doing for now is defining class-specific event behavior on the job class itself, and then just calling the appropriate methods from within the event plugin without branching. In the meantime, I’ll add a more flexible registration and exclusion scheme to my wish-list, but again, the current system looks to be perfectly usable.

Thanks again,

-Nathan

Yes, but there are ways to deal with this like you’ve suggested. It’s a trade off, because sometimes it’s nice to have the same event fire for all types of jobs. However, I also like the idea of being able to explicitly tell a job which event(s) it should trigger (the same way you tell a job which plugin to render with). Maybe there could be a setting for the event to control whether it fires for all jobs, or just ones that explicitly specify it.

Privacy | Site terms | Cookie preferences