Stax

Create and manage CloudFormation stacks in AWS

Posted by Phil Cryer on July 8, 2015

When we were looking to build out an infrastructure in AWS (Amazon Web Services), we took some time to understand how the aws-cli (Universal Command Line Interface for Amazon Web Services) worked. With ideas from other projects found on GitHub, we started wrapping commands in simple BASH scripts, and pointed it to CloudFormation templates written in JSON. From there stax was born, and we have released it as open source under the BSD 3-clause license.

It’s easy to get started with stax to create and manage CloudFormation stacks (aka stax) in AWS (Amazon Web Services) and the project provides several CloudFormation templates, or you can use your own.

For launching new AWS instances with stax, you can use Linux (Debian GNU/Linux 7 and Ubuntu 14.04 are supported) and Apple OS X (tested on 10.10 and 10.9). When running with a default template, you will end up with a VPC setup like the following:

AWS Stax Diagram

NOTICE: As you can see, stax creates an AWS environment from scratch, so it needs to be run under an IAM user or role with permissions to create all AWS resources in your template, including potentially IAM roles. You should understand the risk associated with giving users these permissions in your AWS account before using stax.

CoreOS hosts run Docker containers on the Gateway and Service layer, while Amazon Linux instances serve as NAT devices to direct traffic for the internal hosts. Additionally, one of the biggest requests we had was to simplify the connectivity to nodes once the stax was created. For this we have the stax connect function, that will ssh through the bastion/ssh host, and log you on to either a host on the gateway layer, or the services layer.

The Gateway layer:

$ ./stax connect gateway
     _
    | | every cloud has a silicon lining
 ___| |_ __ ___  __
/ __| __/ _` \ \/ /
\__ \ || (_| |>  <
|___/\__\__,_/_/\_\  0.9

[ ---- ]  checking if stax build is complete
[ ---- ]  describe stax
[ NAME ]  vpc-stax-42179-unrepulsed
[ ---- ]    querying aws
[ ---- ]    query complete
[ ---- ]    see /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.json for details
[ ---- ]    stack vpc-stax-42179-unrepulsed build complete
[ ---- ]  getting public IP (EIP)
[ ---- ]    public IP (EIP): 52.6.248.224
[ ---- ]    writing to /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.bastion
[ ---- ]  get gateway and service IPs
[ ---- ]    querying aws for gateway hosts
[ ---- ]    getting service leaders
[ ---- ]    querying aws for service hosts
[ ---- ]  creating ssh_config
[ ---- ]    writing /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.ssh_config
[ ---- ]    created /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.ssh_config
[ ---- ]  connecting to stax: gateway
Last login: Tue Apr 28 18:57:07 2015 from 10.183.1.208
CoreOS stable (633.1.0)
core@ip-10-183-0-131 ~ $

So you’re logged into a gateway CoreOS box, and from the current /etc/motd you can see what version of CoreOS you are running.

Now we’ll login to the Service layer:

$ ./stax connect service
     _
    | | all glory to the hypnotoad!
 ___| |_ __ ___  __
/ __| __/ _` \ \/ /
\__ \ || (_| |>  <
|___/\__\__,_/_/\_\  0.9

[ ---- ]  checking if stax build is complete
[ ---- ]  describe stax
[ NAME ]  vpc-stax-42179-unrepulsed
[ ---- ]    querying aws
[ ---- ]    query complete
[ ---- ]    see /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.json for details
[ ---- ]    stack vpc-stax-42179-unrepulsed build complete
[ ---- ]  connecting to stax: service
CoreOS stable (633.1.0)
core@ip-10-183-0-111 ~ $

Keeping in mind that this is only available when you’re in development, no external access is allowed to internal nodes directly without the bastion/ssh host, and shutting it down when running in production is highly recommended.

Lastly a quick run of stax without arguments will show you the usage, giving you an idea of all lf the tasks the tool is capable of:

$ ./stax
Usage: stax [OPTIONS] COMMAND

Options:
  -c,--config=CONFIG       Use file CONFIG rather than config/vpc-default.json
  -d,--debug               Turn on verbose messages
  -h,--help                Output this message
  -j,--jump=IP             SSH through host with IP address IP
  -m,--module=MOD          Use config/MOD.json and template/MOD.json
  -t,--template=TEMPLATE   Use file TEMPLATE rather than template/vpc-default.json
  -v,--version             Print name and version information
  -y,--yes                 Do not prompt for confirmation

If an argument is required for a long option, so to the short. Same for
optional arguments.

Commands:
  add                Add functionality to an existing VPC
  check              Run various tests against an existing stax
  connect [TARGET]   Connect to bastion|gateway|service in the VPC stax over SSH
  create             Create a new VPC stax in AWS
  describe           Describe the stax created from this host
  delete             Delete the existing VPC stax
  dockerip-update    Fetch docker IP addresses and update related files
  fleet              Run various fleetctl commands against the fleet cluster
  help               Output this message
  history            View history of recently created/deleted stax
  list               List all completely built and running stax
  rds PASSWORD       Create an RDS instance in the DB subnet
  rds-delete RDSIN   Delete RDS instance RDSIN
  remove ADD         Remove the previously added ADD
  services           List servers that are available to run across a stax
  start SERVICE      Start service SERVICE in the fleet cluster
  test               Automated test to exercise functionality of stax
  validate           Validate CloudFormation template

For more help, check the docs: https://github.com/MonsantoCo/stax

To try it out, just clone the repo and follow the documentation in the readme.

git clone https://github.com/MonsantoCo/stax
more stax/README.md

We hope that this tool will be useful to the community and look forward to feedback on what people think of it. We’re open to issues you might create or pull requests to make stax better.

posted on July 8, 2015 by
Phil Cryer