Hi,
We’ve deployed a tool to simplify the spot fleet json configuration file, it is available on Deadline SFMT (trackit.io). You can also find a quick introduction to the tool via the following medium article SFMT: Simplifying Spot Fleet Configuration for Deadline Render Farms | by Maram | TrackIt | May, 2024 | Medium.
We would be happy to get some suggestions/feedback from you guys!
Thanks!
Thanks Thierry for posting this here on the forums, I hope this will be helpful for the Spot Event Plugin users.
Thanks for the tool. Good timing as i have been struggling a bit with spot requests recently.
We are unable to get our target capacity in one request and have to create multiple requests (of the same instance pool type. this really only seems to have with the 192vcpu instance types) in order for us to get the amount of nodes we actually need. ie we want 20 nodes and we have a target capacity of 20, but we only ever seem to get 3 at a time. So we have to create 7 separate fleet requests in order to get our 20 spots launched. It might be because we are using the deadline spot utility config tool to create our spot request configs and i think it is limited and does not support targetcapacity weighting parameter. And we see warnings in our spot request on the console saying target capacity is not divisible by target capacity weight. But am yet to solve it.
Some questions:
Thanks,
Richard
@Thierry_Delran
So it seems we must have a launch template and we must have an override for it in your tool?
However in the override section when i search for the instance type i want, it is not in your list to pick. I am looking for some 192vcpu instance types. ie c6a.48xlarge or c7i.metal-48xl. Others as well but don’t see them included in the drop down list you have
Thanks,
Richard
Hi Richard,
Thank you very much for your feedback.
1&2: We don’t plan to integrate specifying instance configuration outside of launch template, so yes, you need a launch template for each of your fleets.
3: You can copy the resulting json file within deadline 10 spot event (Tools> super user mode > Tools > Configure Events… > Spot > Spot Fleet Request Configurations) also related medium article should be updated with that info soon.
4: You are right, we now only add the request type, but we also need the tag you provided, it will be updated and mandatory.
Regarding the instance list, we’ll get that updated. I guess those were added after we did setup validation. That is expected, we don’t have a good way to update the validation list now.
Regarding your specific issue with large instances:
The way weight works now is that your TargetCapacity is a score that the spot fleet request wants to fulfill. If you don’t provide a score, every instances will have a default weight score of 1. If you do provide a score, spot fleet request will try to fulfill it capacity with the biggest weight score first. You can also use allocation strategy like capacityOptimizedPrioritized so the spot fleet request will look for the instance types with lowest priority first.
Also, do you use subnets on every AZs available on your region? It will allow the spot fleet request to look for those large instance types you’re looking for, the problem could be that there are not enough instance types available as spot instances on the AZs you have selected now.
We will add a weight score for each instance type selected next for each instance type fields.
Thanks Thierry for the response and I am glad my feedback was useful.
Appreciate you helping on the issue. I think once i bypass trying to use the Spot Request Configuration plugin tool from Deadline and just try pasting resulting configs from your tool and or the aws console, then weight info might pass.
@karpreet any chance your team can update the spot request configuration tool to allow for the newer spot fleet request elements as well as update your documentation on spot fleet requests so it aligns with the latest views aws console experience/options.
Thanks,
Richard
We can take a look, though we can’t say anything about the roadmap.