Launch EC2 Instances With CloudFormation

CloudFormation is the gateway to Infrastructure-as-code for AWS users. Plenty of AWS services are supported by CF and therefore you get a good set of templates to build an infrastructure on. This is greatly beneficial to anyone who wants to set up a reproducible staging environment or test environment. It gives you the ability to tear down and rebuild infrastructures based on their activity. The templates are available to be adopted or authored in JSON/YAML format.

It has seen an increased adoption among cloud users, because of its ease-of-use. In a sense, CloudFormation takes the responsibility of handling the many contingencies or parameters while setting up your infrastructure. Setting up test environments and relegating less active services to occasional deployments are a popular use case with CloudFormation. You can easily build and destroy what you’ve made, then you can take parts of it to build something else.

When you launch a CloudFormation template, the following stack is launched and configured:

• Custom VPC
• Public subnet in that VPC
• Route table
• Internet Gateway
• Network ACL Rules
• Security Group Rules
• EC2 instances

With every new AWS service being given its own CloudFormation support, there’s a lot of flexibility required in how you’d want to use it. Currently, this flexibility is made possible with the addition of Lambda-backed custom resources. You write a Lambda function to implement custom actions during the template Launch. However, what’s the point of having a template if you have to keep coding to experiment?

The Lambda-backed resources are also a way to mend some of the shortcomings of CloudFormation. The AWS services presentation page may tell you that CloudFormation can help set up a complete cloud infrastructure, but that’s not entirely true.

Some resources are still not compatible to be authored in a template.
For example:

• You cannot define Cognito UserPool settings for third party Identity Providers
• Routing tables cannot be routed to a Transit Gateway
• SSH keys cannot be provisioned for an EC2 launch

Totalcloud has devised a means to improve the flexibility and customization of your CloudFormation templates. We use workflows to discard the need for the aforementioned Lambda resources. Workflows will be able to solve the incompatible resources of CloudFormation without any work.

Why launch CF from TotalCloud?

TotalCloud integrates with CloudFormation to give you the flexibility of actions, in addition to automating your CF template launch. In a basic sense, you can schedule a certain time, at which the CF will automatically deploy — but we’ve seen customization being a key element, especially in large-scale environments, where you want to integrate with other DevOps tools, automate sequential actions & reuse parts of your CF code to minimize coding. In order to make it hassle-free, we’ve used a workflow that can easily be edited to suit your needs. Here are some of the customizable use cases our users have adopted.

User approved Launches

Usually launching new resources or an entire infrastructure needs approvals & an organization process to be followed — especially when you’re located across geographies & have multiple teams, it becomes difficult. With a workflow, you can enhance your CF launch by adding a ‘user approval’ condition, so that a certain user/team can give approval before the launch is automated.

You can even make it exclusive to certain members of a team or department to regulate access control To go further, you could even restrict access based on specific templates. For example, The template for security groups can be restricted to certain members while others can launch the rest.

JIRA Triggered launches

You can trigger your CF script from any external tool — we’ve seen JIRA to be the most popular, where a user can create a staging environment by just raising a JIRA ticket. By simply adding the EC2 tags to the description of the ticket, the workflow can launch the moment the ticket is raised.

As you know, when launching templates for your infrastructure, you have the option to pick templates besides the mandatory “resources” template. With this feature, you could raise JIRA tickets that trigger add-ons to the infrastructure with new templates of different services.

Integrate the automation with Jenkins

Once you’ve set up your environment, you’ll want to build applications or deploy a certain software on it. You can integrate it with Jenkins (or any other build system) to automate your CI/CD pipeline, as a sequential process. Automate software installations, patches, and updates into your infrastructure with the workflow.

Prefix/Suffix events with the launch

Add events before or after the launch to supplement your CloudFormation launch with other management activities. You could set up a service deployment before the staging environment is set up. You could launch a part of the template for a separate deployment once your current environment is disabled.

Modularize the template to reuse specific parts of a template (Security group, Apache server, etc)

When CF is written to launch a certain infrastructure, more often than not, some parts of the code are reusable. You shouldn’t have to rewrite that common piece of code. Breaking it down into components allows you to directly reuse a certain component.

For example, in this particular use case — the CF can be split into 2 templates — one that takes care of your network and the other that deploys the actual databases & servers. The Network — which includes the VPC, Subnets, Route Tables, etc, is usually defined by a certain standardized policy on how it needs to be done. So modularizing it will allow teams to distribute access control or reuse specific parts for different environments.

Process

Step 1- Decide the trigger

The trigger could be any condition you want. As stated earlier, raising a JIRA ticket is a popular trigger. You could also make it time-based, alarm-based or have it triggered by an application.

Step 2- Creating the CloudFormation Stack

This node will contain all the permissions and configurations of your EC2 launch template. You can add custom values to it to add to the template as well.

Conclusion

Many other use cases can be translated similarly to our workflows. It’s a matter of combining the resources accurately. In a similar manner, you can create custom workflows to achieve any cloud management use case, irrespective of how complex it is.