Launch EC2 Instances with CloudFormation

CloudFormation is the gateway to Infrastructure-as-code for AWS users. Plenty of AWS services are supported by CF and therefore you get a good set of templates to build an infrastructure on. This is greatly beneficial to anyone who wants to set up a reproducible staging environment or test environment. It gives you the ability to tear down and rebuild infrastructures based on their activity. The templates are available to be adopted or authored in JSON/YAML format. 


It has seen an increased adoption among cloud users, because of its ease-of-use. In a sense, CloudFormation takes the responsibility of handling the many contingencies or parameters while setting up your infrastructure. Setting up test environments and relegating less active services to occasional deployments are a popular use case with CloudFormation. You can easily build and destroy what you’ve made, then you can take parts of it to build something else.


When you launch a CloudFormation template, the following stack is launched and configured:


  • Custom VPC
  • Public subnet in that VPC
  • Route table
  • Internet Gateway
  • Network ACL Rules
  • Security Group Rules
  • EC2 instances


With every new AWS service being given its own CloudFormation support, there’s a lot of flexibility required in how you’d want to use it. Currently, this flexibility is made possible with the addition of Lambda-backed custom resources. You write a Lambda function to implement custom actions during the template Launch. However, what’s the point of having a template if you have to keep coding to experiment? 


The Lambda-backed resources are also a way to mend some of the shortcomings of CloudFormation. The AWS services presentation page may tell you that CloudFormation can help set up a complete cloud infrastructure, but that’s not entirely true.

Some resources are still not compatible to be authored in a template.
For example:

  • You cannot define Cognito UserPool settings for third party Identity Providers 
  • Routing tables cannot be routed to a Transit Gateway
  • SSH keys cannot be provisioned for an EC2 launch

 

Totalcloud has devised a means to improve the flexibility and customization of your CloudFormation templates. We use workflows to discard the need for the aforementioned Lambda resources. Workflows will be able to solve the incompatible resources of CloudFormation without any work. 


Why launch CF from TotalCloud?

TotalCloud integrates with CloudFormation to give you the flexibility of actions, in addition to automating your CF template launch. In a basic sense, you can schedule a certain time, at which the CF will automatically deploy - but we’ve seen customization being a key element, especially in large-scale environments, where you want to integrate with other DevOps tools, automate sequential actions & reuse parts of your CF code to minimize coding. In order to make it hassle-free, we’ve used a workflow that can easily be edited to suit your needs. Here are some of the customizable use cases our users have adopted. 



User approved Launches


Usually launching new resources or an entire infrastructure needs approvals & an organization process to be followed - especially when you’re located across geographies & have multiple teams, it becomes difficult. With a workflow, you can enhance your CF launch by adding a ‘user approval’ condition, so that a certain user/team can give approval before the launch is automated. 


You can even make it exclusive to certain members of a team or department to regulate access control To go further, you could even restrict access based on specific templates. For example, The template for security groups can be restricted to certain members while others can launch the rest.


JIRA Triggered launches


You can trigger your CF script from any external tool - we’ve seen JIRA to be the most popular, where a user can create a staging environment by just raising a JIRA ticket.  By simply adding the EC2 tags to the description of the ticket, the workflow can launch the moment the ticket is raised.


As you know, when launching templates for your infrastructure, you have the option to pick templates besides the mandatory “resources” template. With this feature, you could raise JIRA tickets that trigger add-ons to the infrastructure with new templates of different services.



Integrate the automation with Jenkins


Once you’ve set up your environment, you’ll want to build applications or deploy a certain software on it. You can integrate it with Jenkins (or any other build system) to automate your CI/CD pipeline, as a sequential process. Automate software installations, patches, and updates into your infrastructure with the workflow. 


Prefix/Suffix events with the launch


Add events before or after the launch to supplement your CloudFormation launch with other management activities. You could set up a service deployment before the staging environment is set up. You could launch a part of the template for a separate deployment once your current environment is disabled. 


Modularize the template to reuse specific parts of a template (Security group, Apache server, etc)


When CF is written to launch a certain infrastructure, more often than not, some parts of the code are reusable. You shouldn’t have to rewrite that common piece of code. Breaking it down into components allows you to directly reuse a certain component. 


For example, in this particular use case - the CF can be split into 2 templates - one that takes care of your network and the other that deploys the actual databases & servers. The Network - which includes the VPC, Subnets, Route Tables, etc, is usually defined by a certain standardized policy on how it needs to be done. So modularizing it will allow teams to distribute access control or reuse specific parts for different environments.





Process



Step 1- Decide the trigger

The trigger could be any condition you want. As stated earlier, raising a JIRA ticket is a popular trigger. You could also make it time-based,  alarm-based or have it triggered by an application.


Step 2- Creating the CloudFormation Stack

This node will contain all the permissions and configurations of your EC2 launch template. You can add custom values to it to add to the template as well.


Conclusion

Many other use cases can be translated similarly to our workflows. It’s a matter of combining the resources accurately. In a similar manner, you can create custom workflows to achieve any cloud management use case, irrespective of how complex it is.

Launch EC2 Instances with CloudFormation

Smart Scheduling at your fingertips

Go from simple to smart, real-time AWS resource scheduling to save cost and increase team productivity.

Learn More
More Posts

You Might Also Like

AWS Use Case Files
JIRA Triggered Cloud Management
What if cloud management were as easy as raising a JIRA ticket? Almost every DevOps team uses JIRA as a standard means of issue tracking & task management. It’s a given that it would be a seamless process if you could also integrate your cloud processes with it.
June 16, 2020
Hrishikesh
AWS Use Case Files
Totalcloud Launches New Temporary Rightsizing Feature
You can't always shut down your EC2 machine outside of business hours since some machines are needed up for longer periods. Totalcloud's new downgrade feature lets you optimize your costs by letting you downgrade your machines in a fixed schedule.
June 8, 2020
Hrishikesh
AWS Use Case Files
S3 Cost Saving: Archiving Compressed S3 Data into Glacier
We've devised a new workflow to cut your archiving costs. Simplify the storage, compression, and transfer of your S3 data into S3 Glacier with 1 workflow and 8 nodes.
June 8, 2020
Hrishikesh
AWS Use Case Files
Creating a 3-tier Application With Totalcloud’s Code-Free Workflows
As part of a new request by a customer, we've developed a workflow to deploy 3-tier applications much faster. Utilising merely 3 workflows to achieve a result that would have you scripting and troubleshooting for hours. This post gives you an idea of how this workflow functions, the services being used, and how you can benefit from it.
June 2, 2020
Hrishikesh
AWS Tips & Tricks
Componentized Cloud Management: The way ahead for Cloud Automation
When something gets complex, our primary approach is to break it down — even cloud management. If you’re a part of a growing company that uses the cloud, you can see your infrastructure becoming more…
May 29, 2020
Sayonee
AWS Tips & Tricks
Cost Optimization with AWS Serverless Resource Scheduling
You must be aware of EC2 scheduling and its benefits on cost optimization. However, scheduling doesn't need to stop at just EC2 or RDS. There are plenty of other AWS serverless resources that can be scheduled to save costs. While the traditional way might be tedious, Totalcloud provides a safe alternative.
May 28, 2020
Hrishikesh