Infrastructure Repository - Code your infrastructure
Back to the Blog Page

Infrastructure Repository - Code your infrastructure

by Chris Roberts | Wednesday, April 29, 2015

Chef Workflow has become an increasingly popular topic lately, so it was refreshing to see Jon Cowie speaking at ChefConf about workflow design. One of the great takeaways from the talk was the idea of no right way. Rather than there being a single, prescribed right way, the right Chef workflow is the one that works best for a specific team. Heavy Water has, and does, make use of various workflows; however, we have found the infrastructure repository workflow to be the most effective with the greatest variety of teams.

The infrastructure repository

An infrastructure repository is a discrete collection of provisioning and configuration resources that comprise a complete infrastructure set.

An infrastructure repository is designed to describe a project’s infrastructure. This repository describes the entirety of the infrastructure, not just an isolated subset. This includes: managed application stacks, supporting services, and other resources. It is a broader scope than other workflows, but allows for easier management by reducing unintended or unexpected conflicts due to difficult or impossible cross-repository integrations. An infrastructure repository is a discrete collection of provisioning and configuration resources that comprise a complete infrastructure set.

The layout of the repository is fairly straight-forward:

.
|____Batali
|____CHANGELOG.md
|____Gemfile
|____README.md
|____cloudformation/
|____data_bags/
|____environments/
|____roles/
|____site-cookbooks/

The layout of the infrastructure repository consists of a handful of directories and files. Lets define a short description for each of the items within the repository:

  • Batali - Cookbook requirements of the infrastructure (using Batali)
  • CHANGELOG.md - Log of changes
  • Gemfile - Local bundle of development tools
  • README.md - Description of the repository
  • cloudformation/ - Infrastructure provisioning templating resources (using SparkleFormation)
  • data_bags/ - Chef data bags
  • environments/ - Chef environments
  • roles/ - Chef roles
  • site-cookbooks/ - Infrastructure specific Chef cookbooks

The Workflow

This post is the first in a series of articles that will show by examples how to adopt an infrastructure repository workflow. In this initial example exercise we will use an infrastructure repository to create and configure our infrastructure, which will initially only be comprised of a single Chef server.

This will be accomplished using:

Are you ready? Let’s get started!

Define your infrastructure

Local setup

The rest of this post is intended to allow you to follow along at home. It utilizes a pre-made infrastructure repository which targets AWS as the infrastructure provider. To start, clone the repository from GitHub:

$ git clone git://github.com/hw-labs/infrastructure-repository

These examples require an AWS account for building infrastructure resources. The scripts used require account information to be set in the following environment variables:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_REGION

The example repository is currently setup to support two AWS regions:

  • us-east-1
  • us-west-2

SIDE NOTE: Using the bash shell and want easier configuration and more visual feedback? See: https://github.com/hw-labs/infra-config for some useful helpers.

Installing the bundle

Now that you’ve cloned the repository and set environment variables, change to the new repository directory and install the bundle of gems. Please note that these examples are driven using a system Ruby and do not require the ChefDK.

$ cd infrastructure-repository
$ bundle install

Plant a seed

The repository provides an executable file located at ./bin/seed, which will perform all the initial steps required to customize the repository and setup a required S3 bucket. This would be a great time to open the script and have a look at what it is doing under the hood. When run, it performs the following tasks:

  • Create bucket on S3
  • Create Chef client PEM for local user
  • Create user SSH key
  • Generate knife configuration (.chef/knife.rb)
  • Generate sfn configuration (.sfn)
  • Create user data bag item
  • Install dependency cookbooks
  • Create asset of repository state
  • Push asset to S3 bucket

These tasks can be performed manually. This script just helps to speed this process along within the context of this example. To run the script, just call it directly:

$ ./bin/seed

Inspecting the results

After executing the ./bin/seed script, it is a good time to inspect the resources and files it generated:

  • A data bag item (data_bags/users/<NAME>.json), containing user information
  • A Chef client PEM (.chef/client.pem)
  • An SSH private key (.chef/id_rsa.infra)
  • A knife configuration file (.chef/knife.rb), which includes AWS configurations
  • A new sfn configuration file (.sfn)
  • A new batali.manifest file that defines all required cookbooks
  • A new cookbooks directory containing all required cookbooks
  • A new S3 bucket containing two files:
    • repository asset (stable.zip)
    • Chef validation key (validator.pem)

Build your infrastructure

Now that the ./bin/seed script has configured the local repository and generated the required configuration files, it’s time to start building infrastructure. Building the infrastructure is done using the sfn CLI tool. The sfn gem provides two methods of invocation:

  1. Direct command
    • sfn --help
  2. Knife plugin
    • knife sparkleformation --help

The direct sfn command will use the configuration defined within the .sfn file. The knife plugin will use configuration defined within the .chef/knife.rb file. Outside of the configuration source, both invocations are equivalent. The direct sfn command will have a slight performance benefit as it will not require loading knife which can be slow when it has to traverse a large gemset searching for plugins to enable. For the following examples the direct sfn command will be used.

Validate AWS API

A quick way to validate that we have the correct API configuration is to list currently existing stacks, if any, within the region selected for the account:

$ bundle exec sfn list

If an authentication error occurs, check that the previously defined environment variables have been exported and are correct.

Validate stack template

In this initial example, a new Chef Server will be created.

NOTE: In this example a Chef 11 Server will be created. The next article in this series will involve building a more complex set of resources, including a Chef 12 Server.

The template for the Chef Server is located at cloudformation/base_services/chef_server.rb. This is a SparkleFormation template that will be used to create the JSON template sent to the AWS CloudFormation (CFN) service. A quick scan of this file shows it will be creating an AWS ELB resource and an AWS AutoScaling group that will be attached to the ELB.

The template can be validated using the AWS CloudFormation API:

$ bundle exec sfn validate --file cloudformation/base_services/chef_server.rb

Create stack

Now everything is ready to create a new stack. To preview the generated JSON that will be sent to the AWS CFN API, a --print-only flag can be used:

$ bundle exec sfn create example-chef-server --file cloudformation/base_services/chef_server.rb --print-only

The entire JSON contents will be printed to the screen and no action will be taken. Now lets create the stack:

$ bundle exec sfn create example-chef-server --file cloudformation/base_services/chef_server.rb

This will send the stack creation request to the AWS CFN API. It is important to note that this is a single request to the API and not a series of commands executed locally. Once the stack create request has been accepted, the sfn command will automatically output events for the newly created stack. If the event output is interupted for some reason, it can be restarted using the following command:

$ bundle exec sfn events example-chef-server --poll

Events will continue to be displayed until the stack reaches a complete state (success or failure). Once the stack has completed building, the outputs of the stack are printed and the command exits.

Inspecting the Chef Server

The outputs provided from the stack creation will include a ChefServerUrl. This is the end point to use for accessing the newly created Chef Server. To reproduce those outputs, request a description of the stack:

$ bundle exec sfn describe example-chef-server

Copy the output value of the ChefServerUrl and export it as an environment variable KNIFE_CHEF_SERVER_URL:

$ export KNIFE_CHEF_SERVER_URL='<STACK_OUTPUT_VALUE>'

Now we can list nodes registered with the Chef Server:

$ bundle exec knife status --run-list

This should return a list with one node, the Chef Server itself, configured with role[chef_server] as its run list. It will also provide the public address of the EC2 instance. Copy that address and SSH to the instance:

$ ssh <IP_ADDRESS> -i .chef/id_rsa.infra

The instance was automatically configured with a user account based on the users data bag item generated by the ./bin/seed command. Since the Chef Server provisions and configures itself, users are automatically configured as they would be with any other compute instances.

Inspecting the stack

The sfn commands provides commands for inspecting the stack directly.

Description

$ bundle exec sfn describe example-chef-server

The describe command will list all the resources currently defined for the stack, any tags attached to the stack, and all outputs for the stack.

Events

$ bundle exec sfn events example-chef-server

The events command will return all events that have occurred the requested stack. If the stack is undergoing changes, the events can polled using the --poll flag.

Inspecting

$ bundle exec sfn inspect example-chef-server

The inspect command uses the underlying miasma library to display resource information. The default output is the data composing the stack model. This data model can be queried for more information. One common query is finding all EC2 instances within the requested stack. It is built into the inspect command via the --nodes flag:

$ bundle exec sfn inspect example-chef-server --nodes

This will output all EC2 instances defined within the stack, including EC2 instances that are attached to ASG resources within the stack.

The inspect command also allows a more free form inspection utilizing the modeling provided by the miasma library. We can start by looking at the resources of the stack:

$ bundle exec sfn inspect example-chef-server -a 'resources.all'

Within this list of resources, we can see a ChefServerLoadBalancer resource. It’s the fourth item in the list, so lets isolate that single resource:

$ bundle exec sfn inspect example-chef-server -a 'resources.all.at(3)'

The output now is only the load balancer resource. From this point, we can use miasma to expand the resource model to the actual load balancer model:

$ bundle exec sfn inspect example-chef-server -a 'resources.all.at(3).expand'

The output generated from this command is distinctly different. This output is the underlying data for the load balancer model itself and includes the state of the load balancer, the public addresses and the servers attached to the load balancer (the single chef server). So lets isolate the server instance:

$ bundle exec sfn inspect example-chef-server -a 'resources.all.at(3).expand.servers.first'

This output provides the ID of the EC2 instance attached to the load balancer. We can continue using the miasma library to expand the model and provide the instance of the server:

$ bundle exec sfn inspect example-chef-server -a 'resources.all.at(3).expand.servers.first.expand'

Which provides information about the EC2 instance (the Chef Server).

Destroy your infrastructure

Now that the example is complete, the stack can be destroyed:

$ bundle exec sfn destroy example-chef-server

It will confirm destruction prior to sending the request to the AWS CFN API. After the request has been received, it will automatically transition to event polling in the same manner as the create command.

Review

In this example exercise we used an infrastructure repository to create and configure our infrastructure. This was accomplished using:

In the next post, we will expand on our example by provisioning a more complex set of resources while providing a closer look at the SparkleFormation based descriptions of the infrastructure.