When launching instances using a Spot Fleet, EC2 gives you the option to assign weights to your instances using ‘Instance Weighting’. This essentially defines the importance of each instance type and the weight it holds, while attempting to maintain a certain target capacity. The weights need to be assigned by the customers, and Spot Fleet carries out the rest of the process, based on your specification.
Target capacity can be defined in terms of Instance Memory (RAM), vCPU units, number of instances, storage or a combination of factors.
Instance weight is simply the capacity units that each instance contributes to your target capacity or the number of units each instance type represents in the target capacity. Ideally, the weights are defined in the same metric or units as the target capacity.
Instance Price per hour is either the On-Demand price or the Spot Price or a combination of both of each instance type.
Unit price per hour is the cost metric used in Instance Weighting, which is the price per unit of the instance, as dependent on the weights. It is calculated by dividing the instance price per hour of each instance type by the weight.
Number of instances to launch is calculated for each instance type by dividing the target capacity by the weight of that particular instance type. If the answer is an integer, it is rounded up to the next whole number so that the minimum target capacity is always met.
When Spot Fleet is launching the instances, it will pick the instances based on your launch specifications. If your allocation strategy is ‘lowestprice’, it will pick the instance pool with the lowest per-unit price. If it is ‘diversity’, it will pick instances from all the instance pools, even if it exceeds the target capacity.
Tip: While defining target capacity, it is good to keep in mind the rounding-off method that Spot Fleet uses. If the target capacity is defined in such a way that when divided by the weight, the resulting integer is rounded off to a higher number, Spot Fleet launches more instances than you need, which costs you more.
The AWS Example
The primary source you would approach in order to understand Instance Weighting is the AWS docs. The AWS docs explain how the basics of instance weighting work, but it can get slightly confusing while understanding the process of assigning weights to the instances. Here is what the AWS Docs have cited as an example:
“Consider a Spot Fleet request with the following configuration:
This example can leave you confused due to its vagueness in terms of how AWS has defined the target capacity and arrived at the weights. AWS gives customers the authority to assign weights to the instances using custom metrics, but these numbers - 6 and 5 just don’t add up! The units of the weights are not in RAM, vCPU units, storage or even number of instances. It is possible that AWS has used a hidden metric or values to assign these weights in their example, but considering the fact that we value instances ourselves while submitting a Spot Fleet request, we ought to know how it's done.
We have listed out a few ways, if not all, in which you can assign weights that’ll enable you to make the most out of your spot instances.
How to assign weights?
RAM is one of the most critical factors when running instances. For spot workflows, the application can be RAM intensive or vCPU centric. In the cases of RAM intensive applications, using RAM as a basis for weights becomes a recommended practice. Whenever an application runs out of RAM, the server becomes instantly slow; this indicates how pivotal it becomes to select the instances with the right RAM capacity. One of the approaches to assigning weights is using the RAM to define the target capacity and individual instance contribution.
Let’s take an example.
Assume we decide that our target capacity is 32GB (RAM), based on our internal requirements. While launching a spot fleet request, we deploy certain launch configurations, defining the instance type and price.
As we have defined the target capacity in RAM, the weight of each instance will be the amount of RAM that it contributes to the target capacity. In this case, the weights will be:
The Spot Fleet does the heavy lifting while deciding which instances to launch from which pool; and this depends on your allocation strategy. If it is ‘lowestprice’, the Fleet will choose the instances from either pool. It will launch 4 m5.large instances or 2 m5.xlarge instances; to meet the target capacity of 32GB RAM.
Similarly, if it is ‘diversity’, it will choose from both the pools. The combination here could be 1 m5.xlarge instance (16GB) and 2 m5.large instance (16GB), resulting in the total of 32GB.
Similar to RAM, vCPU is another vital factor that impacts the selection and usage of instances by an organization. For some applications, vCPU might be more pressing than RAM and can end up being a pressure point. This metric can be used as a basis to define the target capacity and the weights.
Using the same example as above, assume the target capacity is 16 vCPU units.
The weights would be equal to the vCPU units of each instance type, assigned as the following:
In this case, the low-cost strategy will instruct the Fleet to select instances either one of the pools, as they have the same per-unit price. So the Fleet will launch either 8 m5.large instances or 4 m5.xlarge instances.
Various combinations of the 2 instance pools can be launched by the Spot Fleet if the strategy is focused on diversification.
Another method to define target capacity and weights is based on a combination of multiple specifications, like core, RAM and storage. This is done by deciding a base RAM and vCPU specification for the application. This figure serves as the base while assigning weights. The total target capacity can be a value ‘x’ times the base, which would be the decided value for the company’s applications.
Taking an example, let’s assume the organization needs a minimum of 8GB RAM and 2 vCPUs, and decides that 16 times this specification is what will suit the organization. The target capacity is 16.
The launch configurations are as follows:
The m4.large instance is used as a base as the organization needs a minimum of 8GB and 2 vCPU; hence this is assigned a weight of 1. The second instance type has specifications which are double the m4.large; hence it is assigned a weight of 2 and so forth.
The weights will be as follows:
If the specification is low cost, the Fleet will launch either one of the specifications, as the unit price is equal. If the focus is on diversity, the Fleet will launch 1 m4.4xlarge, 1 m4.2xlarge, 1 m4.xlarge and 2 m4.large.
Instance Weighting prioritizes the type of instance and the value they add to your overall target capacity, but it becomes essential to weight the instances in an accurate manner. The ambiguity of the AWS docs while explaining the weighting process can be a hindrance that slows down your process or results in inefficiency. We hope that these listed out ways assist you in calculating weights in the right way!