It's a common pattern that developers in your repo are expected to run some command-line tools as part of interacting with the code. For example, maybe they need to run terraform plan
when working with Terraform files.
However, it's a non-stop hassle to get the right version of Terraform installed on everyone's machine. There's always someone asking in Slack "hey it seems like there's some unsupported syntax in this code can you help me" and after some debugging you realize they installed the tool three years ago and that version isn't supported by the current code.
Bazel is great for this! However, the pattern to set it up has always been fiddly. I recently had a client who needed a better answer, so here it is!
Fetching the tools
This post assumes the tools you need to run are statically-linked as an executable. If you're instead meant to gem install
or pip install
or npm install
the tool, then this pattern is too simple to work.
I recently contributed to a nice utility ruleset for Bazel that gives you a JSON-format lockfile for the tools you want to fetch on their various platforms. Continuing with the terraform
example, we could have a tools/multitool.lock.json
file containing:
{
"$schema": "https://raw.githubusercontent.com/theoremlp/rules_multitool/main/lockfile.schema.json",
"terraform": {
"binaries": [
{
"kind": "archive",
"url": "https://releases.hashicorp.com/terraform/1.7.5/terraform_1.7.5_darwin_arm64.zip",
"sha256": "99c4d4feafb0183af2f7fbe07beeea6f83e5f5a29ae29fee3168b6810e37ff98",
"os": "macos",
"cpu": "arm64",
"file": "terraform"
},
{
"kind": "archive",
"url": "https://releases.hashicorp.com/terraform/1.7.5/terraform_1.7.5_darwin_amd64.zip",
"sha256": "0eaf64e28f82e2defd06f7a6f3187d8cea03d5d9fcd2af54f549a6c32d6833f7",
"os": "macos",
"cpu": "x86_64",
"file": "terraform"
},
{
"kind": "archive",
"url": "https://releases.hashicorp.com/terraform/1.7.5/terraform_1.7.5_linux_amd64.zip",
"sha256": "3ff056b5e8259003f67fd0f0ed7229499cfb0b41f3ff55cc184088589994f7a5",
"os": "linux",
"cpu": "x86_64",
"file": "terraform"
},
{
"kind": "archive",
"url": "https://releases.hashicorp.com/terraform/1.7.5/terraform_1.7.5_linux_arm64.zip",
"sha256": "08631c385667dd28f03b3a3f77cb980393af4a2fcfc2236c148a678ad9150c8c",
"os": "linux",
"cpu": "arm64",
"file": "terraform"
}
]
}
}
NB: I'd love to work on some tooling to generate this file for an arbitrary list of released binaries. Please fund us!
Now we just need to point Bazel at this JSON lockfile to get the tools installed. Paste the MODULE.bazel
or WORKSPACE
snippet from the latest release: https://github.com/theoremlp/rules_multitool/releases
Making it nice for developers
The desired end-user experience shouldn't involve Bazel and should be as simple as running the tool in your terminal:
$ ./tools/terraform -version
Terraform v1.7.5
on linux_amd64
First, as always, is to find a workaround for a Bazel footgun: bazel run
cannot possibly behave correctly because it will change the working directory. (It's designed with the assumption that you'd only want to run programs you wrote yourself...) If the working directory is wrong, then we run ./tools/terraform plan
it can't found the Terraform files: Error: No configuration files
. Oops! Come visit https://github.com/bazelbuild/bazel/issues/3325 and give it a thumbs up.
You might see —-run_under
suggested for this purpose, with a simple value to change the working directory like —-run_under=”cd $PWD &&”
. However this is broken under Bazel as well, as it discards the analysis cache. Oops! Come visit https://github.com/bazelbuild/bazel/issues/10782 and ask for this to happen only when the value of run_under
looks like a label.
To work around these issues, we're forced to use a mix of bazel build
to fetch the tool into bazel-out
, bazel cquery
to know what path it's installed under, and bazel info
to convert that relative path. Sigh.
Create a helper script in tools/_multirun_under_cwd.sh
, make it executable and add this content:
#!/bin/sh
target="@multitool//tools/$(basename "$0")"
bazel 2>/dev/null build "$target" && exec $(bazel info execution_root)/$(bazel 2>/dev/null cquery --output=files "$target") "$@"
See an updated and more full-featured version of this script in our
aspect init
template repo: https://github.com/aspect-build/aspect-workflows-template/blob/main/%7B%7B%20.ProjectSnake%20%7D%7D/tools/_run_under_cwd.sh
This script assumes that the name of the program being run (basename "$0"
) is the name we gave in the JSON file earlier "terraform"
. So the final step is just a symlink:
cd tools; ln -s _multitool_run_under_cwd.sh terraform
Now the tool behaves the way users expect, running in whatever working directory they're in. Let's try under our /infrastructure
folder:
/infrastructure$ ../tools/terraform init
Initializing the backend...
Initializing modules...
Nice! Now you can repeat this for other tools you need to run, just by adding an entry to the JSON file, and the corresponding symlink in the tools
folder.
UPDATE May 2024: rules_multitool now has a dedicated cwd
target: github.com/theoremlp/rules_multitool/pull/29 However the pattern above is still useful if you fetched your tools in some other way.