Automation Hacks On Aws Every Devops & Cloudops Must Know

Public cloud, like AWS, which offers a plethora of services brings an extremely dynamic environment to an IT setup. Manually monitoring continuous changes happening beneath several cloud resources is a herculean task for any DevOps and CloudOps engineer. On the other hand, CloudOps, as well as DevOps processes of today, still need to catch up with the time-to-market expectations, with manual processes taking the life out of the engineers.For instance, tasks such as manually creating cloud backups, monitoring resources, managing hundreds of instances of different types with different roles are very challenging. However, with context-aware automation on AWS, it’s just a matter of few steps and one-time scripting.

Here are few intelligent automation hacks on AWS you as a DevOps or CloudOps can live by:

1. Custom Auto-Scaling

AWS allows for auto-scaling using CloudFormation templates, Amazon Machine Images (AMIs), Puppet scripts, and other configuration management tools such as Chef. It requires a significant investment in time to create an automated, self-healing architecture, which replaces failed instances with new instances that scale out itself with little or no human intervention. Primarily, the effectiveness of any environment’s auto-scaling process will purely depend on the skills of cloud automation engineers.

And to reach the point of true auto-scaling, a cloud engineer needs to work with JSON in CloudFormation, install Puppet master, attach to those cloud resources the node requires to operate them (like Elastic IPs, network interfaces, or additional block storage), and then integrate deploy process to auto-scaling. For instance, updating new EC2 instances added to auto-scaling groups automatically. Post deployment, AMIs can be used to spin up exact replicas of instances, create a virtual machine within EC2, and then serve as the basic unit of deployment.

Even though this needs some effort, it is one of the few intelligent automation hacks on AWS every DevOps or CloudOps can live by.

2. Automating CRON and Other Jobs/Tasks

Every AWS resource costs money. Hence, optimal use of resources is pivotal in keeping a check on TCO. For instance, your company might have paid to run several RDS instances round the clock. Our workflow editor also recently underwent a huge engine upgrade that makes this task even simpler. There’s a higher chance that few RDS instances might go unused by developers in a non-production environment for a long period of time. Yet, your company is billed for the entire duration. Even though you terminate them, you will still be on the hook for provisioned storage, manual snapshots, and automated backup storage. And when you stop an instance, it will be paused only for seven days. So, after seven days, you will once again begin to accrue fees.

This is just the scenario for RDS. How about other resources running on your AWS inventory?

All resources residing in your AWS inventory cost money.

So, we’ve jotted down a quick cheat sheet of several jobs you can create, which help a DevOps and CloudOps to be on the bleeding edge of cloud automation on AWS:

For EC2 AMIs:

  • De-register Old EC2 AMIs
  • Create EC2 AMIs based on instance tags
  • Copy EC2 AMIs based on instance tags across regions
  • Copy EC2 AMIs based on EC2 Instance tags across accounts
  • Copy EC2 AMIs based on EC2 AMIs tags across regions

For EBS Snapshots:

  • Delete Old EBS Snapshots jobs
  • Copy EBS snapshot based on EBS volume tags across regions
  • Create EBS snapshot based on EC2 instance tags & IDs
  • Copy EBS snapshot based on EC2 instance tags across regions & accounts

For RDS Snapshots:

  • Copy RDS snapshot based on DB Instance tags across regions

For Redshift:

  • Create Redshift snapshots based on Redshift cluster tags

Route 53 hosted zone:

  • Create backups for hosted zones

3. Customizing and Automating Monitoring of Alerts

Creating alarms, such as CloudWatch Alarms, plays a vital role in AWS cloud optimization. For example, AWS EC2 instances that use less than 10% daily average CPU on at least five of the previous 14 days are marked as low CPU utilization EC2 instances. Even the network IOs that are less than 5MB are marked as low utilization. These instances do not require higher configurations. In such scenarios, automate the task of changing the type of such instances to have a lesser configuration. By doing so, you will aid in drastically reducing the overall cost of AWS cloud infrastructure.

And with a plethora of such alarms being created, day in, day out, for ITOps, monitoring them leads to alerts fatigue. Of course there are innumerable monitoring tools available in the market that help prioritize alerts, however it’s taxing as the business scales on the cloud. The best way forward is to automate these IT alerts and intelligently monitor them.

The Wrap: Context-Aware Automation on AWS is the Way Forward

All your AWS cloud management and optimization efforts, either made in silos or with an integrated approach, such as AWS cost management, AWS governance, AWS cloud security and compliance, AWS cloud operations automation, and DevOps practices, need to be regularly monitored, nurtured, and evolved from time to time. More so, the DevOps practices have to adapt to the current cloud-driven world. Context-aware AWS automation is one such weapon that can help you win the battle.

Of course, there are several AWS services at your disposal to engineer a fully automated, continuous integration and delivery system, and help you be at the bleeding edge of DevOps practices and context-aware automation. It is, however, easier said than done.

To this end, TotalCloud   can help. Our Workflow Editor allows script-less automation through simple workflows. These workflows can help you achieve almost any AWS use-case, including tasks like monitoring of alerts and alarms. Get access to complete flexibility and customization through the TotalCloud platform. You can get started with creating workflows right now!

Automation Hacks On Aws Every Devops & Cloudops Must Know

Smart Scheduling at your fingertips

Go from simple to smart, real-time AWS resource scheduling to save cost and increase team productivity.

Learn More
More Posts

You Might Also Like

AWS Tips & Tricks
How to Schedule RDS Instances with an AWS Lambda function?
Save your costs with scheduled operation of your database. Start and stop the RDS instances manually with python Lambda functions.
February 25, 2020
Introducing the TotalCloud Smart Scheduler
Announcing the launch of the TotalCloud Resource Scheduler! Take complete advantage of AWS's 'pay for what you consume' model by putting a power control for every AWS resource that you use.
January 14, 2020
AWS Use Case Files
Increase EBS Volume Size In Aws
Learn how a simple workflow can auto-remediate and increase the EBS volume size when disk utilization goes beyond 90%
December 2, 2019
AWS Use Case Files
Reporting Untagged Resources
Learn how simple workflow can be set up to generate a report of untagged resources. This enables you to save costs, monitor resources & improve performance
October 24, 2019
AWS Use Case Files
Aws EC2 Instance CPU Utilization Report
Learn how a single workflow can be used to generate a CPU Utilization Report for EC2 Instances and ensure you aren't over or under utilising them.
October 24, 2019
AWS Use Case Files
Reporting Security Groups - Tcp Port 22 (Ssh) Public Ip Access
Learn how to use a simple workflow to generate a report of Security Groups with unrestricted Port 22 access from Public IP
October 24, 2019