Hi,
We’ve been experimenting with applying limits to the jobs to reduce the network load as render nodes switch between jobs. I setup a simple limit (see image below) and applied it to the 3DS Max plugin as per the instructions in the your blog post. This seemed to be working and was limiting the machines that loaded the job, unfortunately it also caused multiple concurrent tasks to run for each new slave. We have a concurrent task limit of either 1 or 2 on all our slaves and 3DS Max jobs are set automatically to 2 concurrent tasks. I’m assuming there is some kind of weird bug going on here… Any thoughts what might be happening?
Thanks
Nick
The limit of “10” will ensure no more than 10 Concurrent Tasks are running on a Slave at once. So, you running 2 x Concurrent Tasks, that limit of 10 was never reached.
EDIT: Ignore, Usage Level as per OP is “Task” based.
I thought the limit of 10 would prevent more than 10 tasks being spawned at any one time and have no effect on the number of concurrent tasks?
Nick
After re-reading your post a few times now and also discussing it with another developer here, we are both still a little confused by what you are actually wanting to achieve here as a behaviour. I’m sure it is possible, just need to understand the actual intent here. Could you possibly re-describe the situation again, perhaps with a working example? What exact version of Deadline are you using? (Perhaps there is an actual bug here). How many Slaves are you running per machine?
Hi Mike,
We’re currently using version 8.0.9.1. We have some very heavy data jobs currently on the render farm, when multiple machines pick these up at the same time our fileserver takes a massive hit and I’m trying to prevent this from locking up the server. So I followed the instructions in the blog post below for limiting data access:
deadline.thinkboxsoftware.com/fe … -resources
The idea being that I want to limit the renderfarm so that only 10 machines can load a 3DS Max job at any time, but when they hit 1% they will release the limit and other machines can then pick up the job. A staggered start basically. I don’t want this to affect how many concurrent tasks are running, which is mixed. All our 3DS Max jobs are submitted with 2 concurrent tasks, but the slaves themselves either have a limit of 1 or 2 concurrent task (newer and older machines).
To me it looks like a bug. The slaves just seemed to be creating more and more concurrent task each time a new machine picked up the job.
Thanks
Nick
Are you saying the Slaves are individually exceeding their concurrent task limit (either individual, or the one imposed by the job)? If that’s the case then it is indeed a bug. Can you clarify how many tasks each of the Slaves were dequeuing? Were the total number of tasks still capped by the overall limit of 10?
I have been unable to reproduce this behaviour internally; we have fixed a similar issue recently, which, so it might be that it will be fixed in the upcoming 8.0 release (8.0.11.x, for posterity).
I find this unlikely, however, since the fix in question was relating mostly to using Limits for usage-based Third-Party licensing. Would you be able to provide us with full Slave logs from some of the slaves that are exceeding their concurrent task limits in this manner? It would be helpful if these were logs with Verbose Logging turned on as well, which can be toggled located at ‘Tools’ → ‘Configure Repository Options’ → ‘Application Data’ → ‘Slave Verbose Logging’ (in Super User mode).
Hi Jon,
Yes the machines are individually exceeding their concurrent task limit, I’m not sure if they made it to 10 as I stopped it before it got that far. I’ll turn it on again and see if I can get you some extra information…
Nick
