Batali - Light weight cookbook dependency and constraint solver
Back to the Blog Page

Batali - Light weight cookbook dependency and constraint solver

by Chris Roberts | Tuesday, March 17, 2015

Batali is a Chef cookbook solver. It works in much the same way that Librarian or Berkshelf work, only differently. There is no specific workflow requirement to enable Batali to function as expected. If your workflow is based around a single application cookbook that is deployed, great! If your workflow is based around a classic infrastructure repository, well that’s great too. Batali is just looking to solve for the required cookbooks. That’s it!

Familiar behavior

Lets start by looking at how Batali can act in a similar manner as existing solvers. We’ll start by creating a Batali file within an empty directory and adding in some simple content:

Batali.define do
  source 'https://supermarket.chef.io'
  cookbook 'users', '1.6.0'
end

Now we can resolve the dependencies by running:

$ batali resolve

Which will provide us with the resulting output:

[Batali]: Loading sources... complete!
[Batali]: Performing single path resolution.
[Batali]: Resolving dependency constraints... complete!
[Batali]: Writing manifest... complete!
[Batali]: Ideal solution:
users <1.6.0>

A pretty simple example providing us with the expected result: the 1.6.0 version of the users cookbook. This will also generate a batali.manifest file that contains source information for cookbook installation. This manifest will be very small (since we only have a single cookbook) and looks like:

{
  "cookbook": [
    {
      "name": "users",
      "dependencies": [

      ],
      "version": "1.6.0",
      "source": {
        "type": "Batali::Source::Site",
        "url": "https://supermarket.chef.io/api/v1/cookbooks/users/versions/1.8.1/download",
        "version": "1.6.0"
      }
    }
  ]
}

Now we can install the cookbooks defined within the manifest:

$ batali install

which will provide some installation feedback:

[Batali]: Readying installation destination... complete!
[Batali]: Installing cookbooks... complete!

and we can check to see it has installed what was expected:

$ ls cookbooks/
users-1.6.0

$ cat cookbooks/users-1.6.0/metadata.rb
name             "users"
maintainer       "Opscode, Inc."
maintainer_email "cookbooks@opscode.com"
license          "Apache 2.0"
description      "Creates users from a databag search"
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          "1.6.0"
recipe           "users", "Empty recipe for including LWRPs"
recipe           "users::sysadmins", "Create and manage sysadmin group"

%w{ ubuntu debian redhat centos fedora freebsd}.each do |os|
  supports os
end

Non-site sources

Batali also supports non-site sources:

  • git - git repository location
  • path - local path location

The path option works just like other solvers:

Batali.define do
  cookbook 'my-cookbook', path: '../my-cookbook'
end

and the git option works in the Librarian style:

Batali.define do
  cookbook 'my-cookbook', git: 'https://github.com/example/my-cookbook', ref: 'feature/example'
end

where ref is optional and will default to master if not provided.

Unfamiliar behavior

Okay, now that we’ve had a quick overview showing that Batali is pretty much like other solvers, lets have a look at how Batali is not like other solvers. Batali is built on top of the Grimoire library (background info) which allows Batali to provide some features unlike existing solvers. Two of these key features are: infrastructure resolution and least impact updates. These features provide more control over cookbook related upgrades as well as general workflow. Lets take a closer look at these two things.

Least Impact Updates

A default behavior of Batali that is different than other solvers is its approach to cookbook updates. After an initial resolution, performing an update will result in a NO-OP. Batali employs a least impact approach to updates. That is, versions will not be updated unless explicitly requested, or required for resolution. This approach makes updating less likely to introduce unexpected issues by upgrading things outside of a specific scope. We can easily demonstrate this behavior using the example above. Lets update our Batali file:

Batali.define do
  source 'https://supermarket.chef.io'
  cookbook 'users'
end

Removing the constraint means the default constraint will be applied (> 0). As the default behavior of Batali is least impact and we already have an existing solution in our manifest file, resolving will now result in a NO-OP:

$ batali resolve
[Batali]: Loading sources... complete!
[Batali]: Performing single path resolution.
[Batali]: Resolving dependency constraints... complete!
[Batali]: Writing manifest... complete!
[Batali]: Ideal solution:
users <1.6.0>

We can force Batali to update the cookbook to the latest available release three ways:

  1. Provide the cookbook name to the resolve command
  2. Use the --no-least-impact flag
  3. Force an update based on new constraint

Cookbook names provided to the resolve command will be allowed to upgrade:

$ batali resolve users
[Batali]: Loading sources... complete!
[Batali]: Performing single path resolution.
[Batali]: Resolving dependency constraints... complete!
[Batali]: Writing manifest... complete!
[Batali]: Ideal solution:
users <1.6.0 -> 1.8.1>

Another option is to use the --no-least-impact flag. Since this is enabled by default, we have to explicitly disable it. This flag completely disables the least impact feature, allowing all cookbooks to updated to their latest allowed values:

$ batali resolve --no-least-impact
[Batali]: Loading sources... complete!
[Batali]: Performing single path resolution.
[Batali]: Resolving dependency constraints... complete!
[Batali]: Writing manifest... complete!
[Batali]: Ideal solution:
users <1.6.0 -> 1.8.1>

Finally, we can force an update by providing a new constraint value within the Batali file that is not statisfied by the value held within the manifest:

Batali.define do
  source 'https://supermarket.chef.io'
  cookbook 'users', '1.7.0'
end
$ batali resolve
[Batali]: Loading sources... complete!
[Batali]: Performing single path resolution.
[Batali]: Resolving dependency constraints... complete!
[Batali]: Writing manifest... complete!
[Batali]: Ideal solution:
users <1.6.0 -> 1.7.0>

Infrastructure Resolution

Batali supports an infrastructure repository workflow, which could be seen as a modified monolithic repository. Instead of vendoring cookbooks directly into the repository, the cookbook requirements can be held within the Batali file. This allows managing the valid versions of cookbooks for the entire infrastructure, which can be tracked via the batali.manifest file in source control, without the need for vendoring cookbooks directly into the repository. This feature is not something that has previously been available with either Librarian or Berkshelf. Instead of providing a single path solution for a given set of dependencies and constraints, Batali will collect all versions of given dependencies to fullfill the requirements within the defined constraints.

Cool! But, what does that mean?

Bundler style

If we look at Librarian and Berkshelf, they are using the same solving approach as Bundler. This approach is to solve for an application. Since an application will only have a single instance running, it only requires a single path of resolved dependencies. That’s great! Until it’s not.

Solve for infrastructure

Since the Bundler comparison is often used, lets create an analogy:

One of the most difficult problems with the infrastructure repository is that none of the existing solvers actually solve for the infrastructure. They simply simulate what would be solved on a given node with a given run list. The infrastructure may require 4 different versions of the users cookbook to be available based on environment or run list specific restrictions. Resolving that using Librarian or Berkshelf is not possible. It requires either added workflow complexity, or manually adjusting requirement files and running installs.

Ideally we want to be able to define our requirements and the given constraints of our infrastructure. With that, we should be able to populate a fresh Chef server, and have all required dependencies for the various run lists within our environment. Using our analogy above, we want to solve for the requirements of RubyGems so that Bundler can resolve all the applications.

Batali for Infrastructure!

Running with our previous simple example, lets imagine our infrastructure requires only the users cookbook, but depending on the environment our Chef node may be located, we will use versions between 1.6.0 and 1.7.x. We can define this constraint within our Batali file:

Batali.define do
  source 'https://supermarket.chef.io'
  cookbook 'users', '>= 1.6.0', '< 1.8.0'
end

The structure of our Batali file still remains the same. We are still providing a list of requirements, and their constraints. Now we are simply being more mindful of what constraints we want enforced as these are constraints for resolution across the infrastructure. To determine what cookbooks are required for the infrastructure, we just need to enable the behavior with the --infrastructure flag:

$ batali resolve --infrastructure
[Batali]: Loading sources... complete!
[Batali]: Performing infrastructure path resolution.
[Batali]: Writing infrastructure manifest file... complete!

Now we can inspect our manifest and see we have all valid versions within the given constraint:

$ cat batali.manifest
{
  "cookbook": [
    {
      "name": "users",
      "dependencies": [

      ],
      "version": "1.7.0",
      "source": {
        "type": "Batali::Source::Site",
        "url": "https://supermarket.chef.io/api/v1/cookbooks/users/versions/1.7.0/download",
        "version": "1.7.0"
      }
    },
    {
      "name": "users",
      "dependencies": [

      ],
      "version": "1.6.0",
      "source": {
        "type": "Batali::Source::Site",
        "url": "https://supermarket.chef.io/api/v1/cookbooks/users/versions/1.6.0/download",
        "version": "1.6.0"
      }
    }
  ]
}

Once we have our manifest, we want to push these to the Chef server. Since Batali is just handling the cookbooks locally, we first want to install them:

$ batali install
[Batali]: Readying installation destination... complete!
[Batali]: Installing cookbooks... complete!

$ ls cookbooks/
users-1.6.0  users-1.7.0

and then upload:

$ knife cookbook upload --all

NOTE: When Batali resolves all cookbooks for the infrastructure, it will discover conflicting dependencies that may exist within cookbook metadata and properly provide required versions.

State of Batali

Batali is currently in an alpha state, with feature implementations still being completed and polished. However, Batali is still in a very usable state. If you’d like to give it a spin, just install the gem:

$ gem install batali

and let us know how it works for you!