Instance Weighting For Spot Instances

When launching instances using a Spot Fleet, EC2 gives you the option to assign weights to your instances using ‘Instance Weighting’. This essentially defines the importance of each instance type and the weight it holds, while attempting to maintain a certain target capacity. The weights need to be assigned by the customers, and Spot Fleet carries out the rest of the process, based on your specification. 


Basic Concepts:

Target capacity can be defined in terms of Instance Memory (RAM), vCPU units, number of instances, storage or a combination of factors. 

Instance weight is simply the capacity units that each instance contributes to your target capacity or the number of units each instance type represents in the target capacity. Ideally, the weights are defined in the same metric or units as the target capacity. 

Instance Price per hour is either the On-Demand price or the Spot Price or a combination of both of each instance type.

Unit price per hour is the cost metric used in Instance Weighting, which is the price per unit of the instance, as dependent on the weights. It is calculated by dividing the instance price per hour of each instance type by the weight.

Number of instances to launch is calculated for each instance type by dividing the target capacity by the weight of that particular instance type. If the answer is an integer, it is rounded up to the next whole number so that the minimum target capacity is always met. 

When Spot Fleet is launching the instances, it will pick the instances based on your launch specifications. If your allocation strategy is ‘lowestprice’, it will pick the instance pool with the lowest per-unit price. If it is ‘diversity’, it will pick instances from all the instance pools, even if it exceeds the target capacity. 


Tip: While defining target capacity, it is good to keep in mind the rounding-off method that Spot Fleet uses. If the target capacity is defined in such a way that when divided by the weight, the resulting integer is rounded off to a higher number, Spot Fleet launches more instances than you need, which costs you more. 


The AWS Example

The primary source you would approach in order to understand Instance Weighting is the AWS docs. The AWS docs explain how the basics of instance weighting work, but it can get slightly confusing while understanding the process of assigning weights to the instances. Here is what the AWS Docs have cited as an example:

“Consider a Spot Fleet request with the following configuration:

  • A target capacity of 24
  • A launch specification with an instance type r3.2xlarge and a weight of 6
  • A launch specification with an instance type c3.xlarge and a weight of 5”

This example can leave you confused due to its vagueness in terms of how AWS has defined the target capacity and arrived at the weights. AWS gives customers the authority to assign weights to the instances using custom metrics, but these numbers - 6 and 5 just don’t add up! The units of the weights are not in RAM, vCPU units, storage or even number of instances. It is possible that AWS has used a hidden metric or values to assign these weights in their example, but considering the fact that we value instances ourselves while submitting a Spot Fleet request, we ought to know how it's done. 

We have listed out a few ways, if not all, in which you can assign weights that’ll enable you to make the most out of your spot instances. 


How to assign weights?

  1. Based on RAM 

RAM is one of the most critical factors when running instances. For spot workflows, the application can be RAM intensive or vCPU centric. In the cases of RAM intensive applications, using RAM as a basis for weights becomes a recommended practice. Whenever an application runs out of RAM, the server becomes instantly slow; this indicates how pivotal it becomes to select the instances with the right RAM capacity. One of the approaches to assigning weights is using the RAM to define the target capacity and individual instance contribution. 

Let’s take an example.

Assume we decide that our target capacity is 32GB (RAM), based on our internal requirements. While launching a spot fleet request, we deploy certain launch configurations, defining the instance type and price. 


As we have defined the target capacity in RAM, the weight of each instance will be the amount of RAM that it contributes to the target capacity. In this case, the weights will be:


The Spot Fleet does the heavy lifting while deciding which instances to launch from which pool; and this depends on your allocation strategy. If it is ‘lowestprice’, the Fleet will choose the instances from either pool. It will launch 4 m5.large instances or 2 m5.xlarge instances; to meet the target capacity of 32GB RAM. 

Similarly, if it is ‘diversity’, it will choose from both the pools. The combination here could be 1 m5.xlarge instance (16GB) and 2 m5.large instance (16GB), resulting in the total of 32GB.


  1. Based on vCPU units


Similar to RAM, vCPU is another vital factor that impacts the selection and usage of instances by an organization. For some applications, vCPU might be more pressing than RAM and can end up being a pressure point. This metric can be used as a basis to define the target capacity and the weights.

Using the same example as above, assume the target capacity is 16 vCPU units.

The weights would be equal to the vCPU units of each instance type, assigned as the following:


In this case, the low-cost strategy will instruct the Fleet to select instances either one of the pools, as they have the same per-unit price. So the Fleet will launch either 8 m5.large instances or 4 m5.xlarge instances. 

Various combinations of the 2 instance pools can be launched by the Spot Fleet if the strategy is focused on diversification. 


  1. Based on multiple specifications

Another method to define target capacity and weights is based on a combination of multiple specifications, like core, RAM and storage. This is done by deciding a base RAM and vCPU specification for the application. This figure serves as the base while assigning weights. The total target capacity can be a value ‘x’ times the base, which would be the decided value for the company’s applications. 

Taking an example, let’s assume the organization needs a minimum of 8GB RAM and 2 vCPUs, and decides that 16 times this specification is what will suit the organization. The target capacity is 16.

The launch configurations are as follows:

The m4.large instance is used as a base as the organization needs a minimum of 8GB and 2 vCPU; hence this is assigned a weight of 1. The second instance type has specifications which are double the m4.large; hence it is assigned a weight of 2 and so forth. 

The weights will be as follows:


If the specification is low cost, the Fleet will launch either one of the specifications, as the unit price is equal. If the focus is on diversity, the Fleet will launch 1 m4.4xlarge, 1 m4.2xlarge, 1 m4.xlarge and 2 m4.large. 


Conclusion

Instance Weighting prioritizes the type of instance and the value they add to your overall target capacity, but it becomes essential to weight the instances in an accurate manner. The ambiguity of the AWS docs while explaining the weighting process can be a hindrance that slows down your process or results in inefficiency. We hope that these listed out ways assist you in calculating weights in the right way!





Instance Weighting For Spot Instances

Smart Scheduling at your fingertips

Go from simple to smart, real-time AWS resource scheduling to save cost and increase team productivity.

Learn More
More Posts

You Might Also Like

Cloud Computing
20 Cloud Influencers You Should Be Following in 2020
It’s important to follow the right individuals so that you remain on the loop and always find yourself learning things that you were unaware of. These thought leaders and influencers can only be the avenues by which you meet other interesting technologists.
September 23, 2020
Hrishikesh
Cloud Computing
Everything You Need To Know About Kubernetes Scheduler
When creating a Kubernetes cluster, scheduling the pod to an available node is an important component of the process. This component works under specific rules and technicalities that I’d like to explore in this article...
September 23, 2020
Hrishikesh
Cloud Automation
New In: No-code cloud management workflows for Azure, VMware & Private Cloud (in addition to AWS)
At TotalCloud, we’ve been enabling workflow-based cloud management for AWS to make it intuitive, accelerated, and no-code. Instead of programming cloud management use cases or depending on siloed solutions, we built out a platform that gives you building blocks to assemble any cloud management solution. 
September 4, 2020
Sayonee
Cloud Computing
List of Essential Kubernetes Tools
Kubernetes is a Container-as-a-Service with tons of unique tools to choose from. External tools play a role in integrating with different systems or maintaining control over the clusters you deploy. Manual health checks and troubleshooting is not ideal to keep a system in full health.This list of tools will provide ample support to your containers and have enough configuration to leave management flexible...
August 12, 2020
Hrishikesh
AWS Use Case Files
TotalCloud Inventory Actions: Giving a new meaning to Cloud Inventory
Learn how the TotalCloud Inventory Dashboard can become equivalent to your cloud provider’s SDK. Carry out any action on any discovered resource with Inventory Actions.
July 30, 2020
Sayonee
AWS Tips & Tricks
AWS Tutorial: Create an AWS Instance Scheduler with Terraform
Terraform is a popular IaaS tool used by many to create, update, and maintain their AWS architecture. If you use Terraform to provision your AWS architecture, you won’t be disappointed with our new AWS tutorial video.We provide you with the means to set up your own instance scheduler from Terraform...
July 20, 2020
Hrishikesh