Configuration Rodeo

March 21, 2012

Open a Ruby project and there is a good chance its configuration will greet you from several directions. A Rakefile here, a Gemfile there, a few hidden files, a config/ directory, perhaps some task files tucked away elsewhere. Every tool brought a suitcase. Somehow they all ended up in the front hall.

The clutter is irritating, but the harder problem is knowing which file controls what. Where should I look? Which settings belong to the project? Which are mine? Why did changing this value have no effect?

A naming convention can help. It can also make a directory look tidier without answering any of those questions. Before choosing one, it helps to separate the decisions we’re trying to make.

Three questions hiding in one

Location: Does configuration belong in a root file, a subdirectory, a user directory, or several of those places?

Representation: Is it a data document, command-line options stored in a file, or executable Ruby? Moving a Ruby script into config/ doesn’t make it a data format. Calling a YAML file Configfile doesn’t change what a parser must understand.

Discovery and combination: How does the tool find the settings? Does it search parent directories? Does it load the first match or merge several? What wins when the same option appears twice?

These choices affect one another, but they aren’t interchangeable. Much of the configuration rodeo comes from treating them as if they were.

The familiar places

I have called the Makefile family “Crapfiles,” an affectionate description of what happens when every tool decides it deserves another one. The names do have a virtue: an established entry point is easy to recognize. A single Rakefile is hardly a catastrophe. A dozen unrelated root files invite a more serious discussion.

A dotfile has a different trade-off. It stays out of an ordinary Unix listing, but it remains visible in other interfaces and in listings that include hidden entries. The dot is an interface convention, not access control. Nor does it tell us whether the file is generated or maintained by a person. A frequently edited setting can live in a dotfile perfectly well if users know where to find it.

File extensions help communicate representation, though many editors also recognize established filenames. What matters is whether people and tools can identify the file reliably. There is no need to invent a universal rule from a personal preference about capitalization.

Give files different jobs

A project can use more than one convention sensibly:

project/
  Rakefile
  Gemfile
  config/
    application.yml
  tasks/
    release.rake

The root files are recognizable tool entry points. The configuration directory holds application settings. The task directory holds executable task definitions, loaded explicitly by the build entry point. This layout illustrates responsibilities; it doesn’t imply that every tool will discover every file shown automatically.

Generated caches and indexes have another job again. They may sit near a configuration file, but they aren’t user instructions. Their location, version-control policy, and ability to be regenerated should reflect that. Putting them all into one category because their names begin with a dot obscures the distinction.

User preferences also differ from project policy. On systems following the XDG Base Directory Specification, user configuration belongs under $XDG_CONFIG_HOME, whose default is ~/.config. The specification separately identifies cache, data, and state locations. It doesn’t prescribe where a repository’s Rakefile belongs. An individual preference and a team-wide setting need not share a home.

The point of a convention is that the next person can predict where to look. A directory named config is entirely capable of doing that; its character count tells us very little about its merits.

Explain how the final value was chosen

A tidy file tree can still produce mysterious behavior. Suppose a tool reads a user setting, a project setting, and a command-line option. If all three specify the output directory, which one wins?

One reasonable policy might be:

built-in defaults
  → user preferences
  → project configuration
  → explicit command-line options

Later sources override earlier ones in this example. Another tool may need a different policy, particularly where project rules must be enforced. Either way, document it. Also distinguish finding the project root from finding a configuration file: running the tool in a nested source directory shouldn’t accidentally select an unrelated configuration farther up the tree.

Merging needs rules of its own. Replacing a scalar value is straightforward. Does a new list replace the old list or append to it? Do mappings merge recursively? How do I remove an inherited value? A shared filename doesn’t answer these questions.

A useful tool can go further and show the effective configuration, the sources it loaded, and where a particular value came from. That is often more valuable than saving another entry in the root directory.

The case for sharing

My RC project explored a more ambitious approach: several Ruby tools sharing a configuration file, with named blocks and profiles. Its documented .rubyrc interface groups each tool’s configuration under a config block. For example, a coverage profile can hold the setup needed before a particular test tool runs:

profile :coverage do
  config :qed do
    require 'simplecov'
    SimpleCov.start do
      coverage_dir 'log/coverage'
    end
  end
end

This gives related tools a place to share values and lets a profile collect settings for a particular activity. RC also provides mechanisms for importing configuration. The appeal is coordination as much as a smaller file count.

But a shared loader has costs. Each participating tool needs an integration path. Loading the configuration for one tool should avoid triggering unrelated setup for another. Imported configuration introduces dependencies that need versions and understandable override rules.

There is also a representation choice to acknowledge: executable Ruby can compute values and call setup routines, but interpreting it means executing code. A data-oriented editor or a tool in another language can’t simply assume it can read every possible setting. That is the same trade-off we would face if each tool used its own Ruby file.

RC is an experiment in sharing configuration, not evidence that every tool ought to surrender its own entry point. Consolidation earns its keep when it reduces duplicated knowledge without making ordinary tasks harder to understand.

Bringing the herd home

For a tool author, the most helpful convention is a predictable contract: where the tool looks, what it accepts, what overrides what, and how users can inspect the result. Follow the ecosystem’s established locations where they fit, provide an explicit path option where appropriate, and keep generated state distinguishable from settings people maintain.

For a project, retain the entry points its tools require. Group supporting files by responsibility, and share configuration where the tools have a clear way to consume it. Renaming or moving an expected file merely because another name looks nicer can create more work than it saves.

I’d still like fewer suitcases in the front hall. But I would be happier knowing whose they are, what is in them, and which one I need to open.

← Selected writing · Older writing