Job continuing long after task is complete - STDOUT overload

Hi Guys,

I am new to deadline scripting, and I am trying to write a simple script plugin that calls a simple python script that calls some simple commandlines. Setting up the plugin is all good. All it does is merge together some Irriadance data into a single giant IRAD Map. If I execute the python script straight from the command prompt, my example job will take about 20 seconds to stitch together a few files and save out the final IRAD Map. But my problem is what happens when I execute the plugin. For simplicty we can simulate the problem using the command script job plugin…

If I execute the same python script by submitting a command script job, then the files take a little longer to write out, but then even when all files have been written out there is still huge amounts of STDOUT data being written out into the slave. This data continues being written out for ages and ages, as in the job can still be writing it out to the slave an hour later! This is causing the job to stay as active, and not complete, even though all the files have successfully been written out. I need the job to complete, so another job (the final render job) which is dependent on this IRAD map, can start.

I have uploaded a small video that demonstrates the problem occuring, it also talks through the example and tries to explain my problem clearly:

3dframework.com/Resources/Pr … Dilema.zip

I have tried suppressing the cast majority of the output by using the lines like:

       	 	# Set the stdout handlers.
       	 	self.AddStdoutHandler( "Adding samples...*.*", self.HandleStdoutSupress ) 
	
	def HandleStdoutSupress(self):
		self.SuppressThisLine()

This helps a little, but the same job that takes 20 seconds from the command line, takes 4minutes 49seconds with the “Adding samples” STDOUT suppressed (Adding samples … makes us 99% of the STDOUT). I need to get it much faster than this!! The final IRAD for this example is 9mb, but we deal with camera paths where the IRAD is 500mb, so this solution wont scale at all :frowning:

Is this an STDOUT problem that is slowing? How can I control it better? Can I direct that STDOUT somewhere else, and log my own progress reports instead? If you could point me towards examples of how I can control the STDOUT and the executable, then I would be really grateful!

Thanks for your time everyone.

Cheers,

Rich

Hey Rich,

Thanks for reporting this. It definitely sounds like an output buffer issue, as Deadline is trying to catch up on the vast amount output that is written. Would it be possible for you to post the python script you’re using to weld the IRAD maps, along with a zip file containing those vrmap files in your test? We could use that to reproduce the problem here, and figure out if there is a workaround that you can add to your plugin to solve it for now. We could also see if there is something we can do in our code to improve performance as well.

Thanks!

  • Ryan

Actually, don’t worry about posting the stuff I asked for previously. We have a test executable that reproduces the problem, so we’ll see what we can figure out.

Cheers,

  • Ryan

Hi Ryan,

Thanks so much for looking at this. With all the IRAD parts, it turned out to be quite a big bundle so I have uploaded it to my ftp and emailed a link through to:

software@primefocusworld.com

I hope this is ok. It is just a little example setup with the python script, Imapview.exe, and all the IRAD parts. I have talked through it in my email, and annotated the script file. If you unzip the main folder to your C Drive then everything should link up from there…

Let me know if there is a problem with anything. A work around/solution would be so so welcome, as it would help us so much with our projects and teaching.

Thanks for your time,

Rich

Whoops,

Didnt refresh the forum!

Cool that you can reproduce it. My example is there too if it is of any use.

Thanks again!

Rich

The good news is that we were able to fix the stdout buffer problem. The bad news is that the fix needed to be implemented in our code, so you won’t be able to use the fix until the next release (which is scheduled for the end of November). I’m sure that’s not the result you wanted to hear, but we really appreciate you bringing this up so that we could get it fixed!

Cheers,

  • Ryan

Hi Ryan,

That is shame, I was hoping there was going to be a quick fix, that I had overlooked. Not to worry though, it is great that it has beem caught now, and that the next release isnt too far off.

I really appreciate you looking into it, and I am looking forward to developing more plugins to smooth our workflow. You guys really have set things up so the potential is huge.

Really enjoying it.

Cheers,

Rich