Progress update for command line job

Deadline
1

Hey,

I have some command line jobs that I need progress update. Based on docs I figured out that all it needs is printing Progress: #% to STDOUT but it doesn’t seem to do anything.

Example from the job log:
2018-09-05 05:58:57: 0: STDOUT: Progress: 11%
2018-09-05 05:58:57: 0: STDOUT: Progress: 12%
2018-09-05 05:58:57: 0: STDOUT: Progress: 13%
2018-09-05 05:58:57: 0: STDOUT: Progress: 14%

From my understanding this should update the job progress, but it doesn’t. I’m still on Deadline 9.0.2. Am I doing something wrong or it’s an bug ?

Best,
Tomasz

2

Maybe it’s some kind of encoding issue ?

The command line job actually starts up python with script and some additional arguments. It’s simple ftp uploader, and the code that I use for printing looks like this:

print(“Progress: {}%”.format(progress))

3

Looks like that progress feature landed in 8.1.5, and I think your encoding and format are good here.

Considering the Slave updates its progress every 7 seconds, maybe it’s going too fast? Looking at your output it looks like it’s just flying through that work. Looking at the Slave UI directly might show it updating faster.

4

Huh, good point… it looks like it prints everything all at once at the very end of the job process…

2018-09-04 23:24:58: 0: INFO: Invoking: Run Process
2018-09-04 23:36:14: 0: STDOUT: Progress: 0%
2018-09-04 23:36:14: 0: STDOUT: Progress: 1%
2018-09-04 23:36:14: 0: STDOUT: Progress: 2%
2018-09-04 23:36:14: 0: STDOUT: Progress: 3%
2018-09-04 23:36:14: 0: STDOUT: Progress: 4%
…
2018-09-04 23:36:19: 0: STDOUT: Progress: 98%
2018-09-04 23:36:19: 0: STDOUT: Progress: 99%
2018-09-04 23:36:19: 0: STDOUT: Progress: 100%
2018-09-04 23:36:19: 0: INFO: Process returned: 0
2018-09-04 23:36:19: 0: Done executing plugin command of type ‘Render Task’
2018-09-04 23:36:19: 0: Render time for frame(s): 11.360 m

When I run the script locally I see it’s being printed in real time over that 12 minutes or so… But when it runs on the slave it’s different. Is there some in-between process that catches the output ?

Best,
Tomasz

5

If it’s python you’ll need to flush your standard output stream more often, or pass it the ‘-u’ flag. Good details over here on Stack Overflow.

6

I’m having this same issue.
I tried using python -u and sys.stdout.flush()
Here’s the output once the task finishes:
2019-06-25 21:29:44: 0: STDOUT: Progress: 1%
2019-06-25 21:29:45: 0: STDOUT: Progress: 2%
2019-06-25 21:29:46: 0: STDOUT: Progress: 3%
2019-06-25 21:29:47: 0: STDOUT: Progress: 5%
2019-06-25 21:29:48: 0: STDOUT: Progress: 6%
2019-06-25 21:29:49: 0: STDOUT: Progress: 7%
2019-06-25 21:29:50: 0: STDOUT: Progress: 8%
2019-06-25 21:29:51: 0: STDOUT: Progress: 10%

Other than that, I can’t see the progress.

7

I am still having this problem.

8

Did anyone get this to work? It doesn’t seem to be a python buffer issue for me. The progress gets outputted to the log correctly over the course of 7-10 minutes but the task progress bar still doesn’t update.

9

I just got this working in Powershell, which probably doesn’t super help, but one thing I found was that the percent needed to be an integer for the progress bar to update:

2022-02-28 14:11:19:  0: STDOUT: Progress: 7.14%
2022-02-28 14:11:22:  0: STDOUT: Progress: 14.29%
2022-02-28 14:11:25:  0: STDOUT: Progress: 21.43%
2022-02-28 14:11:28:  0: STDOUT: Progress: 28.57%
2022-02-28 14:11:31:  0: STDOUT: Progress: 35.71%
2022-02-28 14:11:34:  0: STDOUT: Progress: 42.86%
2022-02-28 14:11:37:  0: STDOUT: Progress: 50%
2022-02-28 14:11:40:  0: STDOUT: Progress: 57.14%
2022-02-28 14:11:44:  0: STDOUT: Progress: 64.29%
2022-02-28 14:11:46:  0: STDOUT: Progress: 71.43%
2022-02-28 14:11:49:  0: STDOUT: Progress: 78.57%
2022-02-28 14:11:53:  0: STDOUT: Progress: 85.71%
2022-02-28 14:11:56:  0: STDOUT: Progress: 92.86%
2022-02-28 14:11:59:  0: STDOUT: Progress: 100%

Only updated at 50%, while

2022-02-28 14:43:55:  0: STDOUT: Progress: 7%
2022-02-28 14:44:00:  0: STDOUT: Progress: 13%
2022-02-28 14:44:05:  0: STDOUT: Progress: 20%
2022-02-28 14:44:10:  0: STDOUT: Progress: 27%
2022-02-28 14:44:15:  0: STDOUT: Progress: 33%
2022-02-28 14:44:20:  0: STDOUT: Progress: 40%
2022-02-28 14:44:25:  0: STDOUT: Progress: 47%
2022-02-28 14:44:30:  0: STDOUT: Progress: 53%
2022-02-28 14:44:35:  0: STDOUT: Progress: 60%
2022-02-28 14:44:40:  0: STDOUT: Progress: 67%
2022-02-28 14:44:45:  0: STDOUT: Progress: 73%
2022-02-28 14:44:50:  0: STDOUT: Progress: 80%
2022-02-28 14:44:55:  0: STDOUT: Progress: 87%
2022-02-28 14:45:00:  0: STDOUT: Progress: 93%
2022-02-28 14:45:05:  0: STDOUT: Progress: 100%

Updated at 33% and 93%.

10

I’m using integers and it’s still not working. strange that it works for powershell. i’m using python on linux

11

Attached a screenshot as a reference. Still not working for me.

Screenshot 2022-03-11 135217
Screenshot 2022-03-11 1352171562×320 89.8 KB

12

I’ve spent at least a day on this problem and I’m beginning to think the failure to write progress is a bug. I’m using deadline 10.1.20.3

I wrote a quick python script that prints progress:

import time, sys
num = 10
for _ in range(num):
out=‘Progress: %s%%’%((_+1)*10)
sys.stdout.write(out+’\n’)
sys.stdout.flush()
time.sleep(5)

Then submit this python script as a job to the deadline:
deadlinecommand.exe -SubmitCommandLineJob -executable python.exe -arguments c:/path/to/script/progress.py

The script runs fine but gives no progess feedback. Am I doing something wrong?

If command line progress really does work can someone from Thinkbox Software please show me with a working example like the one above?

13

I think I found my own answer. I just need to submit the job as a managed process by using “execute in shell”. The give away was the line in the CommandLine.py:

StdOut/Err will NOT be captured here as unmanaged process.

14

This makes sense. At least you will not get the output until the process completes and the caller prints its whole output.

The best solution that I found is to use RPC and send messages from the subprocess, which the plugin then prints with self.LogInfo.
The other way would be to connect the subprocess’s stdout to the current process’s when calling Popen.