Yes, I added the frame parsing and padding functions from another plugin to HuskStandalone.py (looks like you did the same). From what I can tell, the Deadline API parses frames in the string by looking for any number at the end of the scene filename. This causes issues with non-sequence USD files like “USD_Render_v01.usd”, as it looks for “USD_Render_v02.usd” for frame 2. I might make a different regex solution but I haven’t gotten around to it, it’s odd that it’s being treated like this to begin with so I may have done something wrong. I had assumed it would only detect numbers between periods, with no non-numerical values allowed.
3 Likes
also hitting this issue, but just with Arnold and not Karma. anything it doesn’t like in override option gives an empty output path, but any 4 digit number (.0000.) gets converted.
Has anyone run across errors with huskstandalone plugin where the husk process seems to hang indefinitely? It appears from the log that it’s triggered by either a time out or canceled task command that maybe is not being handled correctly?
2024-04-11 04:37:07: 0: STDOUT: ALF_PROGRESS 18%
2024-04-11 04:37:20: 0: STDOUT: ALF_PROGRESS 19%
2024-04-11 04:55:11: 0: Task timeout is 6539 seconds (Auto Task Timeout)
2024-04-11 04:59:11: 0: Task timeout is 6652 seconds (Auto Task Timeout)
2024-04-11 06:39:02: Scheduler Thread - Task "42_1043-1043" has been requeued, because ChangedStatus. Cancelling task...
2024-04-11 06:39:02: 0: RenderThread CancelCurrentTask called, will transition from state None to None
2024-04-11 06:39:02: 0: In the process of canceling current task: ignoring exception thrown by PluginLoader
2024-04-11 06:39:02: 0: Unloading plugin: HuskStandalone
2024-04-11 06:39:02: 0: Executing plugin command of type 'Cancel Task'
2024-04-11 06:39:02: 0: Done executing plugin command of type 'Cancel Task'
2024-04-11 06:39:12: 0: Error in EndJob: Cancelled or timed out.
2024-04-11 06:39:14: Sending kill command to process tree with root process 'deadlinesandbox.exe' with process id 196326
2024-04-11 06:39:16: 0: System.AggregateException: One or more errors occurred. (Unable to write data to the transport connection: Broken pipe.)
2024-04-11 06:39:16: 0: ---> System.IO.IOException: Unable to write data to the transport connection: Broken pipe.
2024-04-11 06:39:16: 0: ---> System.Net.Sockets.SocketException (32): Broken pipe
2024-04-11 06:39:16: 0: at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
2024-04-11 06:39:16: 0: at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.SendAsyncForNetworkStream(Socket socket, CancellationToken cancellationToken)
2024-04-11 06:39:16: 0: at System.Net.Sockets.NetworkStream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
2024-04-11 06:39:16: 0: at Deadline.Net.DeadlineMessageUtils.SendDeadlineMessage(NetworkStream networkStream, DeadlineMessage deadlineMessage, CancellationToken cancellationToken)
2024-04-11 06:39:16: 0: at Deadline.Net.DeadlineMessageUtils.SendDeadlineMessage(NetworkStream networkStream, DeadlineMessage deadlineMessage)
2024-04-11 06:39:16: 0: at Deadline.Slaves.CommandListener.c()
2024-04-11 06:39:16: 0: at Deadline.Slaves.CommandListener.a()
2024-04-11 06:39:16: 0: at Deadline.Slaves.CommandListener..ctor(Int32 commandPort, Boolean isRunningAsService)
2024-04-11 06:39:16: 0: at DeadlinePython.DeadlineSandboxApp.a(CommandLineParser d)
2024-04-11 06:39:16: 0: at DeadlinePython.DeadlineSandboxAppStartup.Main(String[] args)
2024-04-11 06:39:16: 0: --- End of stack trace from previous location ---
2024-04-11 06:39:16: 0: --- End of inner exception stack trace ---
2024-04-11 06:39:16: 0: --- End of inner exception stack trace ---
2024-04-11 06:39:16: 0: at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
2024-04-11 06:39:16: 0: at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
2024-04-11 06:39:16: 0: at System.Threading.Tasks.Task.Wait()
2024-04-11 06:39:16: 0: at Deadline.Net.DeadlineMessageUtils.SendDeadlineMessage(NetworkStream networkStream, DeadlineMessage deadlineMessage, CancellationToken cancellationToken)
2024-04-11 06:39:16: 0: at Deadline.Net.DeadlineMessageUtils.SendDeadlineMessage(NetworkStream networkStream, DeadlineMessage deadlineMessage)
2024-04-11 06:39:16: 0: at Deadline.Slaves.CommandListener.c()
2024-04-11 06:39:16: 0: at Deadline.Slaves.CommandListener.a()
2024-04-11 06:39:16: 0: at Deadline.Slaves.CommandListener..ctor(Int32 commandPort, Boolean isRunningAsService)
2024-04-11 06:39:16: 0: at DeadlinePython.DeadlineSandboxApp.a(CommandLineParser d)
2024-04-11 06:39:16: 0: at DeadlinePython.DeadlineSandboxAppStartup.Main(String[] args)
2024-04-11 06:40:02: 0: WARNING - Waiting for unresponsive Render thread "Waiting for cancel for 00:01:00.4403066". Current plugin is "HuskStandalone" and job "6616be53696d0120285ea290". Cannot proceed until it exits.
2024-04-11 06:41:02: 0: WARNING - Waiting for unresponsive Render thread "Waiting for cancel for 00:02:00.4487961". Current plugin is "HuskStandalone" and job "6616be53696d0120285ea290". Cannot proceed until it exits.
Do you have the auto timeout option enabled? I’m not sure if the 100mins is for the total job or pause between updates (the former i think). You could up the verbosity (only goes to -V 9 not 11!)
@anthonygelatka
Thanks for your reply.
We don’t have auto timeout enabled on those jobs. I’ll up the verbosity and hope that we can get a bit more info regarding this hang.
I’m less concerned about the task cancel, than what happens because of the cancel - the husk process hanging, and entering the never ending message log:
“WARNING - Waiting for unresponsive Render thread “Waiting for cancel for 00:01:00.4403066”. Current plugin is “HuskStandalone” and job “6616be53696d0120285ea290”. Cannot proceed until it exits”
I noticed that the plugin does not use the argument “–fast-exit 0”. I’ve seen a few threads suggesting that it’s probably a good idea to use this? Not sure it would have any affect on a canceled render process, but maybe it does? Anyone have experience with this?
@Justin_B Hi there, not sure why there is not answer from @eamsler but maybe you can give us some information?
you can try add --fast-exit 0 parameter to husk cmd. But if your husk hangs in the middle of render it won’t help
1 Like
Hey there, thanks so much for all the help so far. I’ve gotten up to this point myself but cant seem to figure out the frame number thing. Would you have some examples for how you set up that proper frame parsing/padding stuff?
I’ve been chasing in Edwin’s stead, I don’t have any news I can share at the moment. Don’t be afraid to keep checking in, it’s appreciated!
1 Like
I always have frame lengths of 4 or more, and versioning is always less than or equal to three digits, so this gets around the versioning issue previously mentioned. Just a band-aid though.
currPadding = FrameUtils.GetFrameStringFromFilename( usdFile )
paddingSize = len( currPadding )
if paddingSize >= 4:
newPadding = StringUtils.ToZeroPaddedString( self.GetStartFrame(), paddingSize, False )
usdFile = FrameUtils.SubstituteFrameNumber( usdFile, newPadding )
1 Like
A better solution would be to require a dot before the frame placeholder, or something else to unequivocally define what’s what.
2 Likes
I must have missed this update, thanks, got me rendering crosss platform today
@eamsler @Justin_B How are the updates / secret roadmap for the Husk plugin coming along? Some thorough testing going on at AWS HQ?
With the release of H20.5 on the horizon and another 6 week passed I am wondering if there has been some conclusion?
2 Likes
There was never an official Houdini 20.0 release, nor python 3.10. It’s possible to create a 20.5 entry in the param file, and you’ll need to copy the py310libs folder to py311libs.
1 Like
I am painfully aware of that! my question was actually more towards if we can create a github repro so everyone who is willing to add to that can do so and it is in one place for everyone to access.
3 Likes
Did someone from the official side say that we can’t do a github? Is it better to go ahead and just do this then remove it if is a problem? does the whole repo need to go on git or just this plugin? and should it be forked from the original husk plugin?
I wonder if the slow releases are leading into an opensource release of Deadline, and it’s just the legal aspects of this which take so long, either that or deadline is going to be replaced by deadlineCloud and some AWS only system?
1 Like
No, it was not said but it was also not denied. I am actually at the point where I would go that route as there is clearly no intention from AWS to give any answer on that topic nor does it seem they will support solaris within deadline (10). I am all in for Deadline Cloud if it means there is some proper support again and we can use our local renderfarm. I was not a big fan of deadline turning free because it was so clear what would happen.
1 Like
Thanks adlabac,
after some changes, it works for me too. Because of License, i Export the USD, and then put the husk job on Deadline. But,Deadline-ROP , under “HUSK USD EXPORT” if i set “Export USD locally” , then press “submit to Deadline” , Houdini wants to Render (not usd export) the USD-RENDER_ROP. If i skip, the “husk” job gets on Deadline . How i can prevent, that Houdini wants to render?
Not at all. Probably you are using USD Render rop then you must untick the “Render command”
or use USD ROP instead
3 Likes