AWS Thinkbox Discussion Forums

DEADLINE takes long time to start the job, stuck at certain phase for about 10 min

scene

a small note, so dev’s could dive into the files

  • there are 2 files inside, v3, 4 -

i wanna to clean up the scene from garbage nodes, so i started deleting them and i ended up with v4 scene, with only 2 nodes, geo node and 1 light node

so i have submited to deadline v4 scene and notice that render is working fine??? surprise :slight_smile: so i have uploaded the original the v3 scene the one that ‘does not’ work too…

by the way if possible check these versions as well
https://www.redshift3d.com/forums/viewthread/24300/#228110
guy put here modifications on scripts from deadline which inlc;udes nice working prgoess bar ins tatus as well.
so wwhen fixing thiss stuck ip now could take a look and implement his upgrade as well please :slight_smile:

2 Likes

Hey mirkoj, and everyone else!

seems like there has been some progress! really appreciate everyone’s effort, especially yours @bobo.

sorry I am new to the deadline, so please bear with you.

  1. i have downloaded the file hrender_dl_10_0_26.zip, unzipped it
  2. i copied and overwrote the hrender_dl file on C:\DeadlineRepository10\plugins\Houdini
  3. submitted the scene but I am still having the same lag I had previously.

did I do something wrong?

anybody else have the problem fixed using this method?

Thanks!

second this as well, i have installed this as a plugin as well, and it really makes the whole experience better (or more like as it should be i assume, coming from other render managers).

would be awesome to just have it in the next deadline update.

Thanks!

i am finding this as well.

as in it feels like its not a simple ON or OFF issue, but incremental (or relative to file size) issue.

I have posted in detail above, but my “original” file was 64MB, deadline will take about 20+ min to launch HOUDINI.

i removed nodes to get the file down to about 30MB, deadline will take about 10min

and when i bring it down to about 10MB, deadline will take about 3 min to launch.

the filesize to load time does not seem “linear” but definitely has some kind of relationship.

Hello everyone,

I believe I have figured out what the issue is for this based on the test scene that bazuka provided. When we are performing pathmapping on the houdini scene files we are currently calling hou.fileReferences which appears to be causing every DOP (and maybe other node types) to repeatedly cook out their results.

I have suppressed this cooking in the attached script, if any of you would be able to test it with your scenes and confirm if it fixes it for you that would be much appreciated. If it fails to fix it, then any reproduction scenes that could be used to deduce what else is failing would also be appreciated.

In order to test this fix please make a backup of /plugins/HoudiniBatch then replace the exisiting hrender_dl.py (34.3 KB) with the attached version.

Thanks,
Grant Bartel

2 Likes

Thanks Grant.

first, can i get you to see if I am doing this right,

  1. download the hrender_dl.py (34.3kb) from your post
  2. copy and overwrite the same file name in “C:\DeadlineRepository10\plugins\Houdini” (windows, default).
  3. submit the scene to deadline and see.

i have done above method, but I am still getting the long delay.

also, I have posted a detailed post above, but.

in attempt to somehow workaround this issue for production, I have striped my file down significantly (down from 60mb to 30mb).

and all this file has, is bunch of nodes with REDSHIFT proxies (and some keyframes) and lot of ROP nodes (about 30-40).

there is no DOP, no solver or simulation involved, just straight forward redshift PROXY and few lights.

the REDSHIFT proxies are somewhat heavy (ranging from 100mb to 20mb, about 10 of those per scene or less), so there could be small loadtime, but should not be as significant as 5-10min.

Perhaps this problem is one of those same symptom, but different cause, because i have tried using other methods mentioned above (going back to older version submitter) posted by BOBO, did not seem to solve my issue.

perhaps i have applied the solution wrong.

this is causing quite a significant issue with production on my end, so really need to find a solution soon.

thus, i will try my best to prepare a “NON” production scene when i can.

however, the tricky thing is, again, as mentioned on previous posts, when i keep deleting nodes, the “delay” seems to get shorter. for example, when i was working with my 60mb scene, there was about 20 min lag, however, when i trimmed down the file, i am getting about 5 min lag.

anyway, any other suggestions would be very appreciated, or test results.

also if i am applying the fix accurately with the hrender_dl.py file.

Thanks!

@christian_kim Sounds like you have applied the fix correctly, and you have a different problem. Esp. if the “revert back to 10.0.26” trick did not affect your scene. We were debugging with the particular example posted by @bazuka, a problem that was referenced in a number of other forum posts. You might be right that your particular problem is caused by something else.
I am not sure what the best way to address it would be - is there any way you could produce a reasonably sized test scene that demonstrates the problem in Deadline but does not have the slowdown outside of Deadline? If yes, I would create a separate issue internally for the team to look into.

Thank you for your prompt response Bobo.

i am actually in process of preparing a scene for testing and had a finding.

as mentioned previously, my current scene (30mb) consists of all REDSHIFT proxies.
however, since I can’t send the proxy files for confidentiality reasons, I just made a copy of the HOUDINI file alone. so basically all links to the PROXY files are broken, and its just a 30mb Houdini file.

i tried testing DEADLINE with this file, and there is still that lag, so it seems its not “loading” lag, but something else.

anyway, I will proceed with preparing the file, but is there any way I can send the file to you directly?

because from what I found previously, once I start deleting nodes and lighten the file, the lag do decrease, so wanted to send this file as is to get to the problem directly.

thanks!

I don’t think we have a standard way for getting data from customers - most use some service like Dropbox (it has a 2GB Free account option) or Google Drive. You can email me a download link at bobo AT amazon DOT com. Or you can DM me on this forum.

Hi Bobo, just sent you an email to “bobo at amazon” with a link to the file.

i thought it will be best not to temper with the file too much, so the file is same as the production file i am having problem with, however, its missing all the REDSHIFT proxy files.

but from my testing, even with no PROXY files, the problem persisted.

i am on DEADLINE 10.1
Deadline Client Version: 10.1.7.1 Release (f5e24d0e2)

let me know if there is anything else i can assist you with

I received it, and download it. Will pass it on to the engineers.

Thank you!

hey BOBO, any update on this matter?

would be awesome to know the problem can be reproduced on your end as well?

Yes, sorry, I was away last Friday.

Our engineers saw the same issue even without having all the proxies, but could not find a way to speed it up. It is definitely a separate problem to the “nodes cooking themselves to death” issue solved by the update posted in this thread.

intresting, well let me know if you find more details.

this problem has been with me from 18.0 to 18.5 (and through updated REDSHIFT version as well).

would be great to figure it out.

Hi BOBO, just following up to see if there has been any progress in the matter.

hey Bobo, was wondering if there has been any update on this matter?

i am jumping back into full production soon, and the delay is still persistent and is really causing severe production issues on our end.

Thanks!

Hello, time to bring this old thread back from the dead I guess.
I’m on H19.5.435 at the moment, latest redshift as well as latest deadline at this point it is 10.2.0.10

And same old problem is still here. I did not try to implement any of those changes mentioned so far, but will try and see if they work with this latest version as well…

Honestly a bit disappointing to see that it is so hard to implement things that seems to be working, this and also that other script that fixes % of rendering display for Houdini that guy was offering for over a year now. Any plans to implement any fixes and upgrades or Deadline is moving to be like end of development?

Just to udpate that latest hrender_dl.py is not working with latest deadline. Issue is still very much hear. Still same old… when i start houdini command promt and use script it jst loads scene, rather fast and off to rendering. Here seems like it is laoding scene forever… 30min in this case

wondering if there is really no one else still having this same issue? what is everyone else using to render Houdini tasks?

Privacy | Site terms | Cookie preferences