Hello! I’m in the process of implementing a few python scripts that will be run as jobs. a few will take quite a while… I can monitor the logs locally but would prefer to have them show up in the worker logs.
Currently the console output is only populated once the job is complete.
Is there any documentation on how to ensure the output from the python script shows up in the worker logs as the jobs progresses?
Also, is there a function I can call to update the Progress bar of the job?
Thanks!
If you open the code for the Python plugin, you will see it contains an StdOut handler with a regex “.Progress: (\d+)%.” that will match lines containing “Progress: 33%”, “Progress: 48%” etc. Just calculate the completion percentage in your script and output it with a print statement.
You did not specify whether python will be running in a subprocess.
If yes, then follow josephEA’s advice and capture the subprocess output.
If not, then there is the universal ClientUtils.LogText, and plugins have self.LogInfo, self.LogError, etc.
Thank you both @mois.moshev @josephEA. I’m running it using the Python plugin that ships with Deadline so sending 'Progress: 20%" to stdout works great!
Looking at the stdout handler in the python plugin, do you know how to get live updates in the worker log when running a python script? If I connect to the worker itself I can see the stdout output of my log just fine, but if I look at the log at the Task itself, the log isn’t populated until the job is done.
I’m sure I’m missing something simple but I’m guessing I need to add something to the python plugin itself?
As far as I know, the report/log is not sent from the worker to the repository until the job finishes (or fails).
You can monitor the progress in the individual task’s progress column in the Tasks panel, or overall job progress in the progress column in the Jobs panel. This information is sent every time the plugin detects a progress update, and the monitor will update shortly there after (roughly every 10 seconds by default I think).
Besides that, if you want live updates you’d need to connect to the worker in question and monitor the live stdout.
As far as I know, the report/log is not sent from the worker to the repository until the job finishes (or fails).
Oh, crap, this piece is NOT documented anywhere in Deadline docs, thanks for point that out!
Logs are stored in the application logs locally on the machine. As the name suggest “Job Report/Task Report” is only available on the completion of Task of the job.
I can create an internal ticket to add that to the docs.