Modifying pools/groups on a single rendernode can take up to 5 minutes for some reason
After saturdays failure, this issue caused it to be practically impossible to requeue failed jobs, as simply marking resuming a job would take upwards to 10 minutes (‘normally’, its just around 4-5 minutes, which is still hardcore). We ended up having to restrict network access on the mongodb machine to a single box and then do everything from there -> resuming was instant.
I think this is because the slaves are pulling a lot of data constantly. Not sure how much information they actually require doing their regular tasks, but it seems like they (when they access the database) go upwards to 5-150Mb for several seconds.