SetJobFrameRange corrupts my renderjob

[jho]

Hi,
I have a strange behaviour of “SetJobFrameRange”. In a pre render script I simply run the following:

RepositoryUtils.SetJobFrameRange(job, “1-6048”, 4)

We have a repository server which is located on our site. But it is used in two regions. When I render in our region the command runs fine. If I run the command on a machine in the other region, the commands corrupts the job.
The task view does not start with the pre render script and task 0, 1, 2 etc. It starts with task 55, 56 and so on. The job cannot be suspended. I can only delete it.

Can it be that SetJobFrameRange is too slow and the server cancel the task in the middle of the process and therefore corrupts the job? the job is not marked as corrupted.

The less tasks I have the higher is the change that the SetJobFrameRange command is working as expected. I pinged the server already and it looks fine. 20ms in region 1, and 30ms in region 2.

Can I somehow tell the server to wait a little more? Probably somewhere in the performance settings of the repository. Timeout settings in the job did not help here.

What I also saw in the Deadline Monitor how it updates the task in the view. It updates not immeadiately but in chunks. So I only see e.g. Task 80 to 120. In the end, the view is updated completey and everything looks like it should. I think I need to raise a timeout value in the repository somewhere.

Resubmit job works e.g. and is my workaround currently. But not a nice one. Maybe someone knows something. It is really strange.

Deadline version: 10.1.9.2

Error Message from the log window:
2023-01-20 11:49:02: Scheduler Thread - Task “-2_0” has been requeued, because DeletedTask. Cancelling task…
2023-01-20 11:49:02: 0: RenderThread CancelCurrentTask called, will transition from state None to None
2023-01-20 11:49:02: 0: In the process of canceling current task: ignoring exception thrown by PluginLoader
2023-01-20 11:49:02: 0: Executing plugin command of type ‘Cancel Task’
2023-01-20 11:49:02: 0: Done executing plugin command of type ‘Cancel Task’

Here you see that the first task is 803.

Thanks!

[zainali]

Hello @jho

I think you are right about how the function call and job is behaving on a network, it makes sense why the job is being corrupted. I have not seen anyone using this function call in pre render script. What are you trying to achieve after sending a job and then changing the Frame Range?

@jho

Can I somehow tell the server to wait a little more?

This is not configurable feature yet.

@jho

What I also saw in the Deadline Monitor how it updates the task in the view. It updates not immeadiately but in chunks. So I only see e.g. Task 80 to 120. In the end, the view is updated completey and everything looks like it should. I think I need to raise a timeout value in the repository somewhere.

I believe this is because an operation is performed on a network and the way Monitor’s Task view works. It is updating Database at backend. Is the machine having this issue connected directly to the Repo/DB or via the RCS? You can try switching connection but the update is happening in the DB level it will still need time to update in Monitor.

[jho]

Hello @zainali,
thanks for the detailied answer. We have a data file which stores all render information for a job. It is an inhouse tool. I need to read out that data but it is too slow to do that before submission. Therefore it is done in the prerender script. The data contains the new frame range as every Deadline Frame represents a different frame range. E.g.: 1-10; 20-100 etc.

So I need to use SetFrameRange and the end of the pre render script.

As a solution we would use resubmit job as it seems to be more stable and is not altering an existing job.

[jho]

I just checked our connection to the repository. The delay is 30ms for the “slow” machines and for the side which is working, it is <1ms. So yeah, probably the connection is too slow here.

[zainali]

Hello @jho

If you need to change the frame range I think you might be able to do that on submission of the job. Like it will not be done at the time of render instead it will done at the time when the job is submitted.

Look into creating an event plugin based on event “OnJobSubmitted” you can follow examples from our github scripts. Like “ForcePoolChoice” is applying pools after job is submitted.

[jho]

I just saw that I forgot to mention that this data file is created in the pre render script as it takes too much time (3min) to do that during the submission. I will stick with the resubmission as it seems the most stable solution.