Automation Hacks On Aws Every Devops & Cloudops Must Know

Public cloud, like AWS, which offers a plethora of services brings an extremely dynamic environment to an IT setup. Manually monitoring continuous changes happening beneath several cloud resources is a herculean task for any DevOps and CloudOps engineer. On the other hand, CloudOps, as well as DevOps processes of today, still need to catch up with the time-to-market expectations, with manual processes taking the life out of the engineers.For instance, tasks such as manually creating cloud backups, monitoring resources, managing hundreds of instances of different types with different roles are very challenging. However, with context-aware automation on AWS, it’s just a matter of few steps and one-time scripting.

Here are few intelligent automation hacks on AWS you as a DevOps or CloudOps can live by:

1. Custom Auto-Scaling

AWS allows for auto-scaling using CloudFormation templates, Amazon Machine Images (AMIs), Puppet scripts, and other configuration management tools such as Chef. It requires a significant investment in time to create an automated, self-healing architecture, which replaces failed instances with new instances that scale out itself with little or no human intervention. Primarily, the effectiveness of any environment’s auto-scaling process will purely depend on the skills of cloud automation engineers.

And to reach the point of true auto-scaling, a cloud engineer needs to work with JSON in CloudFormation, install Puppet master, attach to those cloud resources the node requires to operate them (like Elastic IPs, network interfaces, or additional block storage), and then integrate deploy process to auto-scaling. For instance, updating new EC2 instances added to auto-scaling groups automatically. Post deployment, AMIs can be used to spin up exact replicas of instances, create a virtual machine within EC2, and then serve as the basic unit of deployment.

Even though this needs some effort, it is one of the few intelligent automation hacks on AWS every DevOps or CloudOps can live by.

2. Automating CRON and Other Jobs/Tasks

Every AWS resource costs money. Hence, optimal use of resources is pivotal in keeping a check on TCO. For instance, your company might have paid to run several RDS instances round the clock. Our workflow editor also recently underwent a huge engine upgrade that makes this task even simpler. There’s a higher chance that few RDS instances might go unused by developers in a non-production environment for a long period of time. Yet, your company is billed for the entire duration. Even though you terminate them, you will still be on the hook for provisioned storage, manual snapshots, and automated backup storage. And when you stop an instance, it will be paused only for seven days. So, after seven days, you will once again begin to accrue fees.

This is just the scenario for RDS. How about other resources running on your AWS inventory?

All resources residing in your AWS inventory cost money.

So, we’ve jotted down a quick cheat sheet of several jobs you can create, which help a DevOps and CloudOps to be on the bleeding edge of cloud automation on AWS:

For EC2 AMIs:

  • De-register Old EC2 AMIs
  • Create EC2 AMIs based on instance tags
  • Copy EC2 AMIs based on instance tags across regions
  • Copy EC2 AMIs based on EC2 Instance tags across accounts
  • Copy EC2 AMIs based on EC2 AMIs tags across regions

For EBS Snapshots:

  • Delete Old EBS Snapshots jobs
  • Copy EBS snapshot based on EBS volume tags across regions
  • Create EBS snapshot based on EC2 instance tags & IDs
  • Copy EBS snapshot based on EC2 instance tags across regions & accounts

For RDS Snapshots:

  • Copy RDS snapshot based on DB Instance tags across regions

For Redshift:

  • Create Redshift snapshots based on Redshift cluster tags

Route 53 hosted zone:

  • Create backups for hosted zones

3. Customizing and Automating Monitoring of Alerts

Creating alarms, such as CloudWatch Alarms, plays a vital role in AWS cloud optimization. For example, AWS EC2 instances that use less than 10% daily average CPU on at least five of the previous 14 days are marked as low CPU utilization EC2 instances. Even the network IOs that are less than 5MB are marked as low utilization. These instances do not require higher configurations. In such scenarios, automate the task of changing the type of such instances to have a lesser configuration. By doing so, you will aid in drastically reducing the overall cost of AWS cloud infrastructure.

And with a plethora of such alarms being created, day in, day out, for ITOps, monitoring them leads to alerts fatigue. Of course there are innumerable monitoring tools available in the market that help prioritize alerts, however it’s taxing as the business scales on the cloud. The best way forward is to automate these IT alerts and intelligently monitor them.

The Wrap: Context-Aware Automation on AWS is the Way Forward

All your AWS cloud management and optimization efforts, either made in silos or with an integrated approach, such as AWS cost management, AWS governance, AWS cloud security and compliance, AWS cloud operations automation, and DevOps practices, need to be regularly monitored, nurtured, and evolved from time to time. More so, the DevOps practices have to adapt to the current cloud-driven world. Context-aware AWS automation is one such weapon that can help you win the battle.

Of course, there are several AWS services at your disposal to engineer a fully automated, continuous integration and delivery system, and help you be at the bleeding edge of DevOps practices and context-aware automation. It is, however, easier said than done.

To this end, TotalCloud   can help. Our Workflow Editor allows script-less automation through simple workflows. These workflows can help you achieve almost any AWS use-case, including tasks like monitoring of alerts and alarms. Get access to complete flexibility and customization through the TotalCloud platform. You can get started with creating workflows right now!

Automation Hacks On Aws Every Devops & Cloudops Must Know

Smart Scheduling at your fingertips

Go from simple to smart, real-time AWS resource scheduling to save cost and increase team productivity.

Learn More
More Posts

You Might Also Like

Cloud Computing
How To Migrate To Azure Faster?
Migrating from on premise data centers to a cloud provider is always considered a difficult endeavor. From the cost, to the planning and resource allocation, plenty of preliminary work is gone to setting up a cloud infrastructure. Which is why, Microsoft Azure’s new program stands to benefit many organizations still on the fence about migrating to the cloud.
July 21, 2021
Cloud Computing
Everything You Need To Know About Kubernetes Scheduler
When creating a Kubernetes cluster, scheduling the pod to an available node is an important component of the process. This component works under specific rules and technicalities that I’d like to explore in this article...
September 23, 2020
Cloud Computing
20 Cloud Influencers You Should Be Following in 2020
It’s important to follow the right individuals so that you remain on the loop and always find yourself learning things that you were unaware of. These thought leaders and influencers can only be the avenues by which you meet other interesting technologists.
September 23, 2020
Cloud Automation
New In: No-code cloud management workflows for Azure, VMware & Private Cloud (in addition to AWS)
At TotalCloud, we’ve been enabling workflow-based cloud management for AWS to make it intuitive, accelerated, and no-code. Instead of programming cloud management use cases or depending on siloed solutions, we built out a platform that gives you building blocks to assemble any cloud management solution. 
September 4, 2020
Cloud Computing
List of Essential Kubernetes Tools
Kubernetes is a Container-as-a-Service with tons of unique tools to choose from. External tools play a role in integrating with different systems or maintaining control over the clusters you deploy. Manual health checks and troubleshooting is not ideal to keep a system in full health.This list of tools will provide ample support to your containers and have enough configuration to leave management flexible...
August 12, 2020
AWS Use Case Files
TotalCloud Inventory Actions: Giving a new meaning to Cloud Inventory
Learn how the TotalCloud Inventory Dashboard can become equivalent to your cloud provider’s SDK. Carry out any action on any discovered resource with Inventory Actions.
July 30, 2020