A blog icon.

From Hack To Feature: The “tools.go” Pattern in Modern Go Development

The Go Tool Dependencies proposal is set to transition from a workaround to an officially supported feature, enhancing Go's handling of tool dependencies. This change promises a streamlined and reliable development workflow, aligning with modern software development needs.
The logo for Google's Go programming language.

Developers are constantly seeking ways to enhance their workflows and streamline the development process. One such method has gained popularity in the Go programming language: the tools.go pattern. It can be found in some of the most popular open source projects on Github like Kubernetes, Open Telemetry, and thousands more. Clearly, the tools.go pattern is well regarded and utilized but still has some challenges to overcome.

The tools.go Pattern and its Merits

The pattern leverages the tools.go and go.mod files to manage various tools required for the project. This provides a number of advantages like defining the toolchain as code, building the tools from source with simple commands, using Dependabot or Renovate to automatically manage toolchain versions, etc. The beauty of this pattern lies in its simplicity and effectiveness in keeping development dependencies separate from production code.

Parsing tools.go... With Bash?

Let's look at Open Telemetry implements the tools.go pattern to manage tools in the opentelemetry-collector-contrib repo:

Don't spend too much time trying to decipher this Makefile wizardry; We skipped a lot of lines. There are two important details to note. First, the tool names are extracted from tools.go by running grep and tr. Second, these tool names are just passed to the tools.go command. Wait, really? Isn’t that a hack use of mod.go?

The pattern suffers from a discoverability problem as well. It’s generally not well documented outside of specific implementations. You can find several blogs describing the pattern but not many and there’s no standard implementation. Currently, the best source of information is to study and copy an existing implementation.

At this point, you may be asking why these important and professional projects are using a method like this. Don't we have a better way? The reason is simple: It works and it works well. Just don't look at it or make changes!

And no, we don't have a better way. At least, not yet.

Enter the Go Proposal: A Solution in the Making

Thankfully, the Go community is addressing this problem. Two years ago, Conrad Irwin introduced a proposal that aims to address the challenges associated with with the tools.go pattern. This proposal, titled Adding tool dependencies to go.mod, outlines a comprehensive plan to enhance the tooling ecosystem in Go by embracing the tools.go pattern directly in the go command.

The proposal describes two significant changes. First, the go.mod file will allow tool directives to define tool dependencies. This basically moves the contents of tools.go into go.mod. A welcome change considering that currently, we never actually compile tools.go. Second, the go command will be extended to support a new tools metapackage which will supply the following new go subcommands:

Forget the Makefile implementation. These proposed commands are dramatically simpler and offer all the same benefits.

This also addresses an ugly bootstrapping issue. Currently, most projects utilizing the tools.go pattern rely on a Makefile to define install commands. But what if make isn't available? By moving this functionality directly into the go binary, the toolchain can be installed without needing a Makefile to record complex install commands. Perhaps we can soon enjoy some Makefile alternatives!

Progress and Development

Excitingly, the Go community has accepted the proposal and the features are being developed in Go 1.22! Considering that Go 1.22 is very near its release, we don’t expect to see Tool Dependencies released until Go 1.23 or later. Whenever it comes, this new feature mark a significant step forward in providing a seamless and standardized experience for managing tools and dependencies in Go projects.

As we eagerly await the integration of the Go Tool Dependencies proposal, the tools.go pattern is set to evolve from a clever hack to a fully-supported and standardized feature in the Go programming language. This enhancement promises to make the lives of developers easier, fostering a more efficient and reliable development process. By anticipating these upcoming changes, developers can look forward to a future where managing tools and dependencies in Go projects is smoother, more consistent, and tailored to the needs of modern software development.

Share This Article
Have a question or comment?
Contact uS

Related Posts

An image of a coding icon.
Navigating Your Path to the Ideal IDP: Backstage vs. Getport.io
DevOps Tools

This blog explores the vital role of internal development portals (IDPs) like Backstage and Getport.io in enhancing development workflows.

Jon Rudy
2/22/2024
The Github logo, which looks like the outline of a cartoon cat sitting down, surrounded by the words "automate everything".
GitHub API — Automate Everything!
DevOps Tools

This blog explores the diverse capabilities of the GitHub API beyond version control, detailing how it can enhance CICD environments. From cleanup tasks and data funneling to enforcing standardization and enabling self-service jobs, the API's versatility is showcased.

Devin Leaman
2/18/2024
An icon of a software developer in front of a desktop.
ServiceNow is NOT a Developer Platform (iDP)
DevOps Tools

Exploring the limitations of ServiceNow in developer environments, this blog contrasts its rigid, ticket-based system with more agile developer portals, emphasizing the need for collaboration, rapid iteration, and developer autonomy in software development.

Mitchell Phillips
2/14/2024
The Liatrio logo mark.
About Liatrio

Liatrio is a collaborative, end-to-end Enterprise Delivery Acceleration consulting firm that helps enterprises transform the way they work. We work as boots-on-the-ground change agents, helping our clients improve their development practices, react more quickly to market shifts, and get better at delivering value from conception to deployment.