I am getting slow startups on slaves. What is the best was to troubleshoot this? Is there some sort of verbose open up or log to review so I can see where is it slow?
I can see the bottom of the slave splash screen it says “Initializing” and subsequently it takes about 2-3 minutes to open up the slave.
Any additional hints/tips/tricks would be appreciated.
The Monitor logs do show something, but not the Slave so much.
Once it comes up, there’s an option in the UI under help to dump text stats. They’ll show something like this:
Group Name Count Average Time Total Time
Scheduling Dequeue Tasks 14 0.00171429 0.024
Slave Report Slave Info 2 9.1465 18.293
Slave Pending Job Scan 1 0.079 0.079
Slave Repository Repair 1 0.071 0.071
Slave House Cleaning 1 0.07 0.07
------------------ ---------------------------------------- ------------ ------------ ------------
GROUP TOTAL 5 2.34163 18.513
I’m wondering if one of those operations is taking awhile.
Also, can you post the top 20 lines in your Slave’s log file?
Thanks for that I have managed to troubleshoot that to narrow down the issue
So the slave opening times are;
fast - 3secs,
slow - 1min 45sec
The slow timing is consistently the same which suggests to me there is some sort of timeout that it is hitting?
Clients are using an internal DNS server to resolve hostnames via DNS suffix, I wonder if there is some sort of conflict, or the DNS suffix is being added the slaves startup? Either way perhaps the 1min 45sec timeout is a hint?
At this stage I can fix it by one of these solutions;
manually enter IP config + settings DNS to 8.8.8.8
manually enter IP config + leave DNS as Internal + remove DNS suffix
I don’t have any immediate suggestions beyond what Edwin as already advised, but I am curious about this issue. Could you share your C:\ProgramData\Thinkbox\Deadline7\deadline.ini file ? If it has sensitive info, just use xx.xx.xx for IPs and change server names. Or attach it to a support ticket and copy me on the ticket at jamescoulter (at) thinkboxsoftware.com .
Also, you mentioned “manually enter IP config”. Could you be more specific as to what you are changing?
In terms of manually enter IP config, I take the settings (excluding DNS suffix) that the network adapter has picked up via a DHCP lease and manual enter them into the adapters IPV4 settings. ie. windows.microsoft.com/en-nz/wind … =windows-7
To be honest, I’m not sure what to start checking here…
One thing we sometimes see with this kind of problem is that the name resolution sometimes goes to an IPv6 address. If you’re using IP address in your Deadline.ini file as it looks like you are, there shouldn’t be much issue here.
We’ve also noticed slowdowns when trying to connect to the database, so try swapping those around to IP addresses. You’ll find that at “[repo]/settings/dbConnect.xml”.
It looks like you’re mostly using direct IPs in your deadline.ini, so that shouldn’t be a source of slowdowns. The dbConnect.xml file is another place to look. Barring that, you would probably need to get out Wireshark and do some low level network traffic analysis to see what is polling.