Automation Hacks On Aws Every Devops & Cloudops Must Know

Public cloud, like AWS, which offers a plethora of services brings an extremely dynamic environment to an IT setup. Manually monitoring continuous changes happening beneath several cloud resources is a herculean task for any DevOps and CloudOps engineer. On the other hand, CloudOps, as well as DevOps processes of today, still need to catch up with the time-to-market expectations, with manual processes taking the life out of the engineers.For instance, tasks such as manually creating cloud backups, monitoring resources, managing hundreds of instances of different types with different roles are very challenging. However, with context-aware automation on AWS, it’s just a matter of few steps and one-time scripting.

Here are few intelligent automation hacks on AWS you as a DevOps or CloudOps can live by:

1. Custom Auto-Scaling

AWS allows for auto-scaling using CloudFormation templates, Amazon Machine Images (AMIs), Puppet scripts, and other configuration management tools such as Chef. It requires a significant investment in time to create an automated, self-healing architecture, which replaces failed instances with new instances that scale out itself with little or no human intervention. Primarily, the effectiveness of any environment’s auto-scaling process will purely depend on the skills of cloud automation engineers.

And to reach the point of true auto-scaling, a cloud engineer needs to work with JSON in CloudFormation, install Puppet master, attach to those cloud resources the node requires to operate them (like Elastic IPs, network interfaces, or additional block storage), and then integrate deploy process to auto-scaling. For instance, updating new EC2 instances added to auto-scaling groups automatically. Post deployment, AMIs can be used to spin up exact replicas of instances, create a virtual machine within EC2, and then serve as the basic unit of deployment.

Even though this needs some effort, it is one of the few intelligent automation hacks on AWS every DevOps or CloudOps can live by.

2. Automating CRON and Other Jobs/Tasks

Every AWS resource costs money. Hence, optimal use of resources is pivotal in keeping a check on TCO. For instance, your company might have paid to run several RDS instances round the clock. Our workflow editor also recently underwent a huge engine upgrade that makes this task even simpler. There’s a higher chance that few RDS instances might go unused by developers in a non-production environment for a long period of time. Yet, your company is billed for the entire duration. Even though you terminate them, you will still be on the hook for provisioned storage, manual snapshots, and automated backup storage. And when you stop an instance, it will be paused only for seven days. So, after seven days, you will once again begin to accrue fees.

This is just the scenario for RDS. How about other resources running on your AWS inventory?

All resources residing in your AWS inventory cost money.

So, we’ve jotted down a quick cheat sheet of several jobs you can create, which help a DevOps and CloudOps to be on the bleeding edge of cloud automation on AWS:

For EC2 AMIs:

  • De-register Old EC2 AMIs
  • Create EC2 AMIs based on instance tags
  • Copy EC2 AMIs based on instance tags across regions
  • Copy EC2 AMIs based on EC2 Instance tags across accounts
  • Copy EC2 AMIs based on EC2 AMIs tags across regions

For EBS Snapshots:

  • Delete Old EBS Snapshots jobs
  • Copy EBS snapshot based on EBS volume tags across regions
  • Create EBS snapshot based on EC2 instance tags & IDs
  • Copy EBS snapshot based on EC2 instance tags across regions & accounts

For RDS Snapshots:

  • Copy RDS snapshot based on DB Instance tags across regions

For Redshift:

  • Create Redshift snapshots based on Redshift cluster tags

Route 53 hosted zone:

  • Create backups for hosted zones

3. Customizing and Automating Monitoring of Alerts

Creating alarms, such as CloudWatch Alarms, plays a vital role in AWS cloud optimization. For example, AWS EC2 instances that use less than 10% daily average CPU on at least five of the previous 14 days are marked as low CPU utilization EC2 instances. Even the network IOs that are less than 5MB are marked as low utilization. These instances do not require higher configurations. In such scenarios, automate the task of changing the type of such instances to have a lesser configuration. By doing so, you will aid in drastically reducing the overall cost of AWS cloud infrastructure.

And with a plethora of such alarms being created, day in, day out, for ITOps, monitoring them leads to alerts fatigue. Of course there are innumerable monitoring tools available in the market that help prioritize alerts, however it’s taxing as the business scales on the cloud. The best way forward is to automate these IT alerts and intelligently monitor them.

The Wrap: Context-Aware Automation on AWS is the Way Forward

All your AWS cloud management and optimization efforts, either made in silos or with an integrated approach, such as AWS cost management, AWS governance, AWS cloud security and compliance, AWS cloud operations automation, and DevOps practices, need to be regularly monitored, nurtured, and evolved from time to time. More so, the DevOps practices have to adapt to the current cloud-driven world. Context-aware AWS automation is one such weapon that can help you win the battle.

Of course, there are several AWS services at your disposal to engineer a fully automated, continuous integration and delivery system, and help you be at the bleeding edge of DevOps practices and context-aware automation. It is, however, easier said than done.

To this end, TotalCloud   can help. Our Workflow Editor allows script-less automation through simple workflows. These workflows can help you achieve almost any AWS use-case, including tasks like monitoring of alerts and alarms. Get access to complete flexibility and customization through the TotalCloud platform. You can get started with creating workflows right now!

Automation Hacks On Aws Every Devops & Cloudops Must Know

Smart Scheduling at your fingertips

Go from simple to smart, real-time AWS resource scheduling to save cost and increase team productivity.

Learn More
More Posts

You Might Also Like

AWS Use Case Files
Launch EC2 Instances with CloudFormation
CloudFormation is the gateway to Infrastructure-as-code for AWS users. Learn how you can deploy Cloudformation templates through Totalcloud workflows and increase your customization.
June 25, 2020
Hrishikesh
AWS Use Case Files
JIRA Triggered Cloud Management
What if cloud management were as easy as raising a JIRA ticket? Almost every DevOps team uses JIRA as a standard means of issue tracking & task management. It’s a given that it would be a seamless process if you could also integrate your cloud processes with it.
June 16, 2020
Hrishikesh
AWS Use Case Files
Totalcloud Launches New Temporary Rightsizing Feature
You can't always shut down your EC2 machine outside of business hours since some machines are needed up for longer periods. Totalcloud's new downgrade feature lets you optimize your costs by letting you downgrade your machines in a fixed schedule.
June 8, 2020
Hrishikesh
AWS Use Case Files
S3 Cost Saving: Archiving Compressed S3 Data into Glacier
We've devised a new workflow to cut your archiving costs. Simplify the storage, compression, and transfer of your S3 data into S3 Glacier with 1 workflow and 8 nodes.
June 8, 2020
Hrishikesh
AWS Use Case Files
Creating a 3-tier Application With Totalcloud’s Code-Free Workflows
As part of a new request by a customer, we've developed a workflow to deploy 3-tier applications much faster. Utilising merely 3 workflows to achieve a result that would have you scripting and troubleshooting for hours. This post gives you an idea of how this workflow functions, the services being used, and how you can benefit from it.
June 2, 2020
Hrishikesh
AWS Tips & Tricks
Componentized Cloud Management: The way ahead for Cloud Automation
When something gets complex, our primary approach is to break it down — even cloud management. If you’re a part of a growing company that uses the cloud, you can see your infrastructure becoming more…
May 29, 2020
Sayonee