Hi, I’m making an advanced plugin, which calls its own RenderTasks, handles subprocesses, etc.
The issue is that EndJob is called when each Task ends, and not the whole Job.
Apparently each Task has its own plugin instance, which prevents me from collecting the complete results from the job, e.g. in an instance variable.
Is there an easy way around this, short of communicating through a separate channel or the file system?
Bump, this is still a problem.
Also, EndJob log is not visible anywhere in the job logs. Is there a solution to this? I’m guessing it’s because it’s running in its own sandbox process.
In an advance plugin, the EndJob function is fired when the Worker is stopping work on the Job. So in this case it’s expected that it fires for each task.
Collect your output within the RenderTasks function instead, and only rely on EndJob for cleanup and shutdown.
The reason that EndJob doesn’t show up in the render log is that it fires after the render thread has been terminated, it does show up in the Worker log however. But that’s not as easy to grab while you’re troubleshooting.
Hey Justin, thanks, this is exactly what I was going for.
So what is the way to do something at the end of the entire job - count the number of completed tasks in EndJob and act if all of them have been? Could you guide me in which dl modules I should use for that?
Another problem we have with EndJob is that it times out - and sometimes operations performed there can be slow. This makes it unreliable to use EndJob for its purpose. How to solve this?
Long-term I think it makes sense to record the EndJob log as a separate task log for the job (e.g. the way event plugins are recorded).
I’d use an event plugin that uses OnJobFinished or a post-job script to run something after a job has been completed.
These will get their own logs in the task like you’re looking for as well.
Awesome thanks, was already looking at these options, but it’s great that you confirmed.
post-job script it is.
I have a further question about this - is there a recommended way to collect data (python objects) from all completed tasks? Currently it seems the primary method would be for each task to pickle to a file in the job’s auxiliary directory, which then would get processed in the PostJobScript.
Any other options?
More basically - the post job script is called with a plugin instance as its argument, is this the last plugin that executed a task?