Infrastructure Graphs
Back to the Blog Page

Infrastructure Graphs

by Chris Roberts | Thursday, March 17, 2016

The process of conceptualizing and designing a modern infrastructure – made up of mostly virtual resources with complex inter-dependencies – can be an exceedingly difficult task. The intangible nature of virtual infrastructure resources further compounds this difficulty by removing physical references. To help solve the problem of generating visual representations of virtual infrastructures created by orchestration APIs, the SparkleFormation CLI has added a new graph command.

The number of resources required to compose a single logical unit of a modern infrastructure can be surprisingly large. Tracking dependencies between these resources can also be difficult, even when isolated to a single logical unit. The new SparkleFormation CLI graph command helps solve this problem by generating graph files in the dot format used by Graphviz. If Graphviz is installed locally, the SparkleFormation CLI can utilize the graphviz command to automatically generate infrastructure graph images.

Visualizing Resource Creation

To demonstrate how the graph command works, lets start with a simple example: a basic network. This SparkleFormation template builds a very basic VPC within AWS:

SparkleFormation.new(:network) do

  dynamic!(:ec2_vpc, :network) do
    properties do
      cidr_block '172.0.0.0/16'
      enable_dns_support true
      enable_dns_hostnames true
    end
  end

  dynamic!(:ec2_dhcp_options, :network) do
    properties do
      domain_name join!(region!, 'compute.internal')
      domain_name_servers ['AmazonProvidedDNS']
    end
  end

  dynamic!(:ec2_vpc_dhcp_options_association, :network) do
    properties do
      dhcp_options_id ref!(:network_ec2_dhcp_options)
      vpc_id ref!(:network_ec2_vpc)
    end
  end

  dynamic!(:ec2_internet_gateway, :network)

  dynamic!(:ec2_vpc_gateway_attachment, :network) do
    properties do
      internet_gateway_id ref!(:network_ec2_internet_gateway)
      vpc_id ref!(:network_ec2_vpc)
    end
  end

  dynamic!(:ec2_route_table, :network) do
    properties.vpc_id ref!(:network_ec2_vpc)
  end

  dynamic!(:ec2_route, :network_public) do
    properties do
      destination_cidr_block '0.0.0.0/0'
      gateway_id ref!(:network_ec2_internet_gateway)
      route_table_id ref!(:network_ec2_route_table)
    end
  end

  dynamic!(:ec2_subnet, :network) do
    properties do
      availability_zone select!(0, azs!)
      cidr_block '172.0.0.0/16'
      vpc_id ref!(:network_ec2_vpc)
    end
  end

  dynamic!(:ec2_subnet_route_table_association, :network) do
    properties do
      route_table_id ref!(:network_ec2_route_table)
      subnet_id ref!(:network_ec2_subnet)
    end
  end

  outputs do
    network_vpc_id.value ref!(:network_ec2_vpc)
    network_subnet_id.value ref!(:network_ec2_subnet)
    network_route_table.value ref!(:network_ec2_route_table)
  end

end

The network template provides us with a logical unit of our overall infrastructure. That logical unit is composed of multiple resources that are inter-dependent. These dependencies will affect the order in which resources are created within AWS as resources will not be created until dependent resources are available. The graph command in the SparkleFormation CLI will illustrate this creation order flow of resources based on their dependencies:

network-graph

To expand our infrastructure a bit, we may want to create a compute instance. So we create a computes template to provide us with an EC2 instance and a security group to allow access:

SparkleFormation.new(:computes) do

  parameters do
    network_vpc_id.type 'String'
    network_subnet_id.type 'String'
  end

  dynamic!(:ec2_security_group, :compute) do
    properties do
      group_description 'SSH Access'
      security_group_ingress do
        cidr_ip '0.0.0.0/0'
        from_port 22
        to_port 22
        ip_protocol 'tcp'
      end
      vpc_id ref!(:network_vpc_id)
    end
  end

  dynamic!(:ec2_instance, :compute) do
    properties do
      image_id 'ami-25c52345'
      image_type 't2.micro'
      network_interfaces array!(
        ->{
          device_index 0
          associate_public_ip_address 'true'
          subnet_id ref!(:network_subnet_id)
          group_set [ref!(:compute_ec2_security_group)]
        }
      )
    end
  end

end

The computes stack is pretty simplistic. A single security group resource and a single EC2 instance resource. Graphing this is mostly redundant as a mental visualization of the stack this template will generate is easy, but lets do it anyway:

Cross Stack Resource Dependencies

These templates were structured to be compatible with the SparkleFormation CLI nesting functionality. This functionality allows us to easily create complex stacks composed of multiple nested stacks. Any stacks created with the computes template has dependencies that cannot be displayed in the graph. This is due to the dependency on the VPC and subnet resources being provided via user input (parameters). If we instead nest these into a single template, we can see the direct resource dependencies that are required between the separate stacks. Now we can create a new template to nest our logical units:

SparkleFormation.new(:infra) do
  nest!(:network, :infra)
  nest!(:computes, :infra)
end

Now our graph shows not only the creation flow based on dependencies within each stack, but also how the creation of resources in other stacks can be affected.

infra-graph

What’s Ahead

The graph command currently only supports AWS CloudFormation compatible templates as it was the initial target of the proof of concept. Support for other providers is in progress and will be released soon.