During my day job I’ve been working on re-factoring some the internals of Juju to reverse the trend of a growing number of dependencies in our core packages.

In this post I’ll show how I used go list to help me in this task.

The basics

Let’s start with the basics of go list. The default invocation of go list returns the name of the import path that represents the directory you are currently in, or the package path you provide.

% cd $GOPATH/src/code.google.com/p/go.crypto/ssh
% go list
code.google.com/p/go.crypto/ssh
% go list github.com/juju/juju
github.com/juju/juju

By itself this doesn’t appear to be particularly notable, however this simple example belies the power of go list.

The secret of the -f flag

Tucked away at the top of the documentation for go help list is this short piece

The -f flag specifies an alternate format for the list, using the syntax of package template. The default output is equivalent to -f '{{.ImportPath}}'.

Put simply, -f allows you to execute a Go template which has access to the internal data structures of the go tool, same structures that are used when building, testing or getting code.

This example, using -f '{{ .ImportPath }}', is equivalent to the previous invocation and gives us a glimpse into the power of go list

% go list -f '{{ .ImportPath }}' github.com/juju/juju                          github.com/juju/juju

Going a step further with go list

The godoc for cmd/go lists the structures available to -f, so I won’t repeat them verbatim. Instead I’ll highlight some uses of go list.

Listing the test files that will be built

% go list -f '{{ .TestGoFiles }}' archive/tar
[reader_test.go tar_test.go writer_test.go]

Or the external test files of that package

% go list -f '{{ .XTestGoFiles }}' archive/tar
[example_test.go]

Or a summary for a set of packages

% go list -f '{{.Name}}: {{.Doc}}' code.google.com/p/go.net/ipv{4,6}
ipv4: Package ipv4 implements IP-level socket options for the Internet Protocol version 4.
ipv6: Package ipv6 implements IP-level socket options for the Internet Protocol version 6.

Conditional builds and build tags

A important part of writing Go programs is dealing with portability issues across various operating systems or processors. I’ve written about conditional builds before, so I’ll just show an example listing the files that will be compiled on various platforms

% env GOOS=darwin go list -f '{{ .GoFiles }}' github.com/pkg/term
[term.go term_bsd.go]
% env GOOS=linux go list -f '{{ .GoFiles }}' github.com/pkg/term
[term.go term_linux.go]

Who depends on what?

go list can show both the packages that your package directly depends, or its complete set of transitive dependencies.

% go list -f '{{ .Imports }}' github.com/davecheney/profile
[io/ioutil log os os/signal path/filepath runtime runtime/pprof]
% go list -f '{{ .Deps }}' github.com/davecheney/profile
[bufio bytes errors fmt io io/ioutil log math os os/signal path/filepath reflect runtime runtime/pprof sort strconv strings sync sync/atomic syscall text/tabwriter time unicode unicode/utf8 unsafe]

The first command lists only the packages that github.com/davecheney/profile depends on directly. This is the unique set of import statements in all Go files, adjusted for build constraints. The second command returns the set of direct and transitive dependency of github.com/davecheney/profile.

Fancy templating

The set of data structures available to the go list template is quite specialised, but don’t forget we have the whole power of the Go template package at our disposal.

In the previous examples dealing with slices of values, the default output format follows the fmt package. However it is probably more convenient for scripting applications to have one entry per line, which we can do easily as the go list template contains a function called join which delegates to strings.Join.

% go list -f '{{ join .Imports "\n" }}' github.com/davecheney/profile
io/ioutil
log
os
os/signal
path/filepath
runtime
runtime/pprof

Putting it together

With the task of trying to unpick the forest of dependencies in our core packages, I can use go list to define a helper like

deps() {
        go list -f '{{ join .Deps  "\n"}}' . | grep juju
}

The usage is as simple as navigating to a particular package in my $GOPATH and checking to see its complete dependency list

% deps
github.com/juju/cmd
github.com/juju/errgo
github.com/juju/errors
github.com/juju/gojsonpointer
github.com/juju/gojsonreference
github.com/juju/gojsonschema
github.com/juju/juju/api/agent
... // many more lines elided

Conclusion

This short post has barely scratched the surface of the flexibility that go list provides.

If your development, build, or CI tooling is using hand rolled scripts, grep, awk, etc, to introspect Go packages and their interdependencies, consider switching to go list.