With the release of go1.2rc3 last week I have now merged the autobench-next branch into master in the autobench repository.
Go 1.2 is not expected to bring performance improvements of the same magnitude of Go 1.1, but moderate improvements are expected due to improvements in code generation, the runtime, the garbage collector, and the standard library.
If you would like to contribute a benchmark result from your own machine the instructions for doing so are included the README file.
This post explains how the Go build process works using examples from Go’s standard library.
This article focuses on the gc toolchain. The gc toolchain takes its name for the Go compiler frontend, cmd/gc, and is mainly used to distinguish it from the gccgo toolchain. When people talk about the Go compilers, they are likely referring to the gc toolchain. This article won’t be focusing on the gccgo toolchain.
The gc toolchain is a direct descendant of the Plan 9 toolchain. The toolchain consists of a Go compiler, a C compiler, an assembler and a linker. Each of these tools are found in the src/cmd/ subdirectory of the Go source and consist of a frontend, which is shared across all implementations, and a backend which is specific to the processor architecture. The backends are distinguished by their letter, which is again a Plan 9 tradition. The commands are:
5g, 6g, and 8g are the compilers for .go files for arm, amd64 and 3865c, 6c, and 8c are the compilers for .c files for arm, amd64 and 3865a, 6a, and 8a are the assemblers for .s files for arm, and64 and 3865l, 6l, and 8l are the linkers for files produced by the commands above, again for arm, amd64 and 386.It should be noted that each of these commands can be compiled on any supported platform and this forms the basis of Go’s cross compilation abilities. You can read more about cross compilation in this article.
Building a Go package involves at least two steps, compiling the .go files then packing the results into an archive. In this example I’m going to use crypto/hmac as it is a small package, only one source and one test file. Using the -x option I’ve asked go build to print out every step as it executes them
% go build -x crypto/hmac WORK=/tmp/go-build249279931 mkdir -p $WORK/crypto/hmac/_obj/ mkdir -p $WORK/crypto/ cd /home/dfc/go/src/pkg/crypto/hmac /home/dfc/go/pkg/tool/linux_arm/5g -o $WORK/crypto/hmac/_obj/_go_.5 -p crypto/hmac -complete -D _/home/dfc/go/src/pkg/crypto/hmac -I $WORK ./hmac.go /home/dfc/go/pkg/tool/linux_arm/pack grcP $WORK $WORK/crypto/hmac.a $WORK/crypto/hmac/_obj/_go_.5
Stepping through each of these steps
WORK=/tmp/go-build249279931 mkdir -p $WORK/crypto/hmac/_obj/ mkdir -p $WORK/crypto/
go build creates a temporary directory, /tmp/go-build249279931 and populates it with some skeleton subdirectories to hold the results of the compilation. The second mkdir may be redundant, issue 6538 has been created to track this.
cd /home/dfc/go/src/pkg/crypto/hmac /home/dfc/go/pkg/tool/linux_arm/5g -o $WORK/crypto/hmac/_obj/_go_.5 -p crypto/hmac -complete -D _/home/dfc/go/src/pkg/crypto/hmac -I $WORK ./hmac.go
The go tool switches the the source directory of crypto/hmac and invokes the go compiler for this architecture, in this case 5g. In reality there is no cd, /home/dfc/go/src/pkg/crypto/hmac is the supplied as the exec.Command.Dir field when 5g is executed. This means the .go source files can be relative to their source directory, making the command line shorter.
The compiler produces a single temporary output file in $WORK/crypto/hmac/_obj/_go_.5 which will be used in the final step.
/home/dfc/go/pkg/tool/linux_arm/pack grcP $WORK $WORK/crypto/hmac.a $WORK/crypto/hmac/_obj/_go_.5
The final step is to pack the object file into an archive file, .a, which the linker and the compiler consume.
Because we invoked go build on a package, the result is discarded as $WORK is deleted after the build completes. If we invoke go install -x two additional lines appear in the output
mkdir -p /home/dfc/go/pkg/linux_arm/crypto/ cp $WORK/crypto/hmac.a /home/dfc/go/pkg/linux_arm/crypto/hmac.a
This demonstrates the difference between go build and install; build builds, install builds then installs the result to be used by other builds.
You may be wondering what the pack step in the previous example does. As the compiler and linker only accept a single file representing the contents of the package, if a package contains multiple object files, they must be packed into a single .a archive before they can be used.
A common example of a package producing more than one intermediary object file is cgo, but that is too complicated for this article, instead a simpler example is a package that contains some .s assembly files, like crypto/md5.
% go build -x crypto/md5 WORK=/tmp/go-build870993883 mkdir -p $WORK/crypto/md5/_obj/ mkdir -p $WORK/crypto/ cd /home/dfc/go/src/pkg/crypto/md5 /home/dfc/go/pkg/tool/linux_amd64/6g -o $WORK/crypto/md5/_obj/_go_.6 -p crypto/md5 -D _/home/dfc/go/src/pkg/crypto/md5 -I $WORK ./md5.go ./md5block_decl.go /home/dfc/go/pkg/tool/linux_amd64/6a -I $WORK/crypto/md5/_obj/ -o $WORK/crypto/md5/_obj/md5block_amd64.6 -D GOOS_linux -D GOARCH_amd64 ./md5block_amd64.s /home/dfc/go/pkg/tool/linux_amd64/pack grcP $WORK $WORK/crypto/md5.a $WORK/crypto/md5/_obj/_go_.6 $WORK/crypto/md5/_obj/md5block_amd64.6
In this example, executed on a linux/amd64 host, 6g is invoked to compile two .go files, md5.go and md5block_decl.go. The latter contains the forward declaration for the functions implemented in assembly.
6a is then invoked to assemble md5block_amd64.s. The logic for choosing which .s to compile is described in my previous article on conditional compilation.
Finally pack is invoked to pack the Go object file, _go_.6, and the assembly object file, md5block_amd64.6, into a single archive.
A Go command is a package who’s name is main. Main packages, or commands, are compiled just like other packages, but then undergo several additional steps to be linked into final executable. Let’s investigate this process with cmd/gofmt
% go build -x cmd/gofmt WORK=/tmp/go-build979246884 mkdir -p $WORK/cmd/gofmt/_obj/ mkdir -p $WORK/cmd/gofmt/_obj/exe/ cd /home/dfc/go/src/cmd/gofmt /home/dfc/go/pkg/tool/linux_amd64/6g -o $WORK/cmd/gofmt/_obj/_go_.6 -p cmd/gofmt -complete -D _/home/dfc/go/src/cmd/gofmt -I $WORK ./doc.go ./gofmt.go ./rewrite.go ./simplify.go /home/dfc/go/pkg/tool/linux_amd64/pack grcP $WORK $WORK/cmd/gofmt.a $WORK/cmd/gofmt/_obj/_go_.6 cd . /home/dfc/go/pkg/tool/linux_amd64/6l -o $WORK/cmd/gofmt/_obj/exe/a.out -L $WORK $WORK/cmd/gofmt.a cp $WORK/cmd/gofmt/_obj/exe/a.out gofmt
The first six lines should be familiar, main packages are compiled like any other Go package, they are even packed like any other package.
The difference is the penultimate line, which invokes the linker to produce a binary executable.
/home/dfc/go/pkg/tool/linux_amd64/6l -o $WORK/cmd/gofmt/_obj/exe/a.out -L $WORK $WORK/cmd/gofmt.a
The final line copies and renames the completed binary to its final name. If you had used go install the binary would be copied to $GOPATH/bin (or $GOBIN if set).
If you go far enough back in time, back before the go tool, back to the time of Makefiles, you can still find the core of the Go compilation process. This example is taken from the release.r60 documentation
$ cat >hello.go <<EOF
package main
import "fmt"
func main() {
fmt.Printf("hello, world\n")
}
EOF
$ 6g hello.go
$ 6l hello.6
$ ./6.out
hello, world
It’s all here, 6g compiling a .go file into a .6 object file, 6l linking the object file against the fmt (and runtime) packages to produce a binary, 6.out.
In this post we’ve talked about how go build works and touched on how go install differs in its treatment of the compilation result.
Now that you know how go build works, and how to investigate the build process with -x, try passing that flag to go test and observe the result.
Additionally, if you have gccgo installed on your system, you can pass -compiler gccgo to go build, and using -x investigate how Go code is built using this compiler.
When developing Go packages that rely on specific features of the underlying platform or processor it is often necessary to provide a specialised implementation.
Go does not have a preprocessor, a macro system, or a #define declaration to control the inclusion of platform specific code. Instead a system of tags and naming convention defined in the go/build package and supported by the go tool allows Go packages to customise themselves for the specific platform they are being compiled for.
This post explains how conditional compilation is implemented and show you how you can use it in your projects.
Before we can talk about conditional compilation, we need to learn a little bit about the go list command. go list gives you access to the internal data structures which power the build process.
go list takes the most of the same arguments as go build, test, and install but does not perform any compilation. Using the -f, format flag we can supply a snippet of text/template code which is executed in a context containing a go/build.Package structure.
Using the format flag, we can ask go list to tell us the names of the files that would be compiled.
% go list -f '{{.GoFiles}}' os/exec
[exec.go lp_unix.go]
In the example above I asked for the list of files in os/exec package that would be compiled on this linux/arm system. The result is two files, exec.go which contains the common code shared across all platforms, and lp_unix.go while contains an implementation of exec.LookPath for unix-like systems.
If I were to run the same command on a Windows system, the result would be
C:\go> go list -f '{{.GoFiles}}' os/exec
[exec.go lp_windows.go]
This short example demonstrates the two parts of the Go conditional compilation system, known as Build Constraints, which we will now explore in more detail.
The first method of conditional compilation is via an annotation in the source code, commonly known as a build tag.
Build tags are implemented as comments and should appear as close to the top of the file as possible.
When go build is asked to build a package it will analyse each source file in the package looking for build tags. These tags control whether go build will pass the file to the compiler.
A build tags follow these three rules
As an example, the build tag found at the top of a source file
// +build darwin freebsd netbsd openbsd
would constrain this file to only building on BSD systems that supported kqueue.
A file may have multiple build tags. The overall constraint is the logical AND of the individual constraints. For example
// +build linux darwin // +build 386
constrains the build to linux/386 or darwin/386 platforms only.
One thing that generally catches people out when they are first trying to make build tags work is this
// +build !linux package mypkg // wrong
In this example there is no newline separating the build tag and the package declaration. Because of this the build tag is associated with the package declaration as a comment describing the package and thus ignored.
// +build !linux package mypkg // correct
This is the correct form, a comment with a trailing newline stands alone and is not associated with any declaration and go vet will detect the missing newline.
% go vet mypkg mypkg.go:1: +build comment appears too late in file exit status 1
When this feature was added to go vet it detected several mistakes in the standard library and sub repos, so don’t feel bad if you get it wrong the first time.
For reference, here is a sample showing a licence preamble, a build tag, and a package declaration
% head headspin.go // Copyright 2013 Way out enterprises. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. // +build someos someotheros thirdos,!amd64 // Package headspin implements calculates numbers so large // they will make your head spin. package headspin
The second option for providing conditional compilation is the name of the source file itself. This scheme is simpler than build tags, and allows the go/build package to exclude files without having to process the file.
The naming convention is described in the documentation for the go/build package. Simply put, if your source file includes the suffix, _$GOOS.go then it will only be built on that platform. All other platforms will behave as if the file is not present. The same applies for _$GOARCH.go. The two can be combined as _$GOOS_$GOARCH.go, but not _$GOARCH_$GOOS.go.
Some examples of file suffixes are,
mypkg_freebsd_arm.go // only builds on freebsd/arm systems mypkg_plan9.go // only builds on plan9
Your source files still require a name, a suffix is not sufficient, for example
_linux.go _freebsd_386.go
will be ignored, even on linux or freebsd systems, because the go/build package ignores any file beginning with a period or an underscore.
Build tags and file suffixes overlap in fuctionality. For example, a file called mypkg_linux.go that contained the build tag // +build linux is redundant.
In general, when choosing between a build tag or a file suffix, you should choose a file suffix when there is an exact match between the platform or architecture and the file you want to include. eg,
mypkg_linux.go // only builds on linux systems mypkg_windows_amd64.go // only builds on windows 64bit platforms
Conversely if your file is applicable to more than one platform or architecture, or you need to exclude a specific platform, a build tag should be used. eg,
% grep '+build' $HOME/go/src/pkg/os/exec/lp_unix.go // +build darwin dragonfly freebsd linux netbsd openbsd
builds on all unix like platforms.
% grep '+build' $HOME/go/src/pkg/os/types_notwin.go // +build !windows
builds on all platforms except Windows.
While this post has focused only on Go source files, build tags and file suffixes can be used with any source file that the go tool can build. This includes .c and .s files. The Go standard library, specifically the runtime, syscall, os and net packages contain great examples, I recommend studying them.
Test files also support build tags and file suffixes and behave in the same manner as Go source files, conditionally including test cases on a per platform basis. Again the standard library contains many great examples.
Finally, while the title of this article talks about the go tool, the conditional compilation features are not limited to just that tool. You can build your own tools to consume and analyse Go code with the same file suffix and build tag semantics using the go/build package.
One of the stated goals of Go was to provide a language which could be built without Makefiles or other kinds of external configuration. This was realised with the release of Go 1, and the go tool which incorporated an earlier tool, goinstall, as the go get subcommand.
go get uses a cute convention of embedding the remote location of the package’s source in the import path of the package itself. In effect, the import path of the package tells go get where to find the package, allowing the dependency tree a package requires to be discovered and fetched automatically.
Compared to Go’s contemporaries, this solution has been phenomenally successful. Authors of Go code are incentivised to make there code go getable, in much the same way peer pressure drives them to go fmt their code. The installation instructions for most of the Go code you find on godoc and go-search has become, in essence, go get $REPO.
Despite the significant improvement over some other language’s built in dependency management solutions, many Go developers are disappointed when the discover the elegance of go get is a double edged sword.
The Go import declaration references an abstract path. If this abstract path represents to a remote DVCS repository, the declaration alone does not contain sufficient information to identify which particular revision should be fetched. In almost all cases go get defaults to head/tip/trunk/master, although there is a provision for fetching from a tag called go1.
The Go Authors recognize that this is an issue, and suggest that if you need this level of control over your dependencies you should consider using alternative tools, suggesting goven as a possible solution.
Over the past two years no fewer than 19 other tools have been announced, so there is clearly a need and a desire to solve the problem. That said, none of them have achieved significant mind share, let alone challenged go get for the title of default.
Here are three small examples from my work life where go get isn’t sufficient for our requirements as we deliver our rewrite of Juju in Go.
Juju isn’t just one project, but more than a dozen projects that we have written or rely on. Often landing a fix or feature in Juju means touching one of these upstream libraries and then making a corresponding change in Juju.
Sometimes these changes make it easy for other developers on the team to detect that they don’t have the correct version of a dependency as Juju stops compiling. Other times the effects can be more subtle, a bug fix to a library can cause no observed breakage so there is no signal to others on the team that they are compiling against the wrong version of the package.
To date, we’ve resorted to an email semaphore whenever someone fixes a bug a package, imploring everyone else to run go get -u. You can probably imagine how successful this is, and how much time is being spent chasing bugs that were already fixed.
As a maintenance programmer on Juju, I regularly switch between stable, trunk and feature branches. Each of those expects to be compiled against exactly the right version of its dependencies, but go get doesn’t provide any way of capturing this so I may reliably reproduce a build of Juju from the past.
As our product eventually ends up inside Debian based distributions we have to deliver a release artifact which relies only on other Debian packages in the archive. We currently do this with a blob of shell scripts and tags on repos that we control to produce a tarball that contains the complete $GOPATH of all the source for Juju and its dependencies for exactly this release.
While it gets the job done, our pragmatism is not winning us any favors with our Debian packaging overlords as our approach makes their security tin foil hats itch. We’ve been lucky so far, but will probably at some point have a security issue in a package that Juju depends on, and because that package has been copied into every release artifact we’ve delivered fixing it will be a very involved process.
Recently William Kennedy, Nathan Youngman and I started a Google Group in the hope of harnessing these disparate efforts of the many people who have thought, argued, hit their head against, and worked on this problem.
If you care about this issue, please consider joining the [go-pm] mailing list and contributing to the Goals document. I am particularly interested in capturing the requirements of various consumers of Go packages via user stories.
Did you know that Go 1.2 will ship with a built in test coverage tool ? The tool is integrated into go test and works similarly to the profiling tool, producing an output file which is interpreted by a second command.
If you have Go 1.2rc2 or tip installed, you can use this short shell function to automate the basic usage of the cover tool to produce a per function coverage breakdown.
cover () {
t=$(tempfile)
go test -coverprofile=$t $@ && go tool cover -func=$t && unlink $t
}
Usage is straight forward, just call cover with no argument for the package in your current working directory, or pass the name of a package.
% cover bytes ok bytes 7.770s coverage: 91.6% of statements bytes/buffer.go: Bytes 100.0% bytes/buffer.go: String 100.0% bytes/buffer.go: Len 100.0% bytes/buffer.go: Truncate 100.0% bytes/buffer.go: Reset 100.0% bytes/buffer.go: grow 100.0% bytes/buffer.go: Grow 100.0% bytes/buffer.go: Write 100.0% bytes/buffer.go: WriteString 100.0% bytes/buffer.go: ReadFrom 94.7% bytes/buffer.go: makeSlice 75.0% bytes/buffer.go: WriteTo 78.6% bytes/buffer.go: WriteByte 100.0% bytes/buffer.go: WriteRune 100.0% bytes/buffer.go: Read 100.0% bytes/buffer.go: Next 100.0% bytes/buffer.go: ReadByte 100.0% bytes/buffer.go: ReadRune 83.3% bytes/buffer.go: UnreadRune 85.7% bytes/buffer.go: UnreadByte 100.0% bytes/buffer.go: ReadBytes 100.0% bytes/buffer.go: readSlice 100.0% bytes/buffer.go: ReadString 100.0% bytes/buffer.go: NewBuffer 100.0% bytes/buffer.go: NewBufferString 100.0% bytes/bytes.go: equalPortable 100.0% bytes/bytes.go: explode 100.0% bytes/bytes.go: Count 100.0% bytes/bytes.go: Contains 0.0% bytes/bytes.go: Index 100.0% bytes/bytes.go: indexBytePortable 100.0% bytes/bytes.go: LastIndex 100.0% bytes/bytes.go: IndexRune 100.0% bytes/bytes.go: IndexAny 100.0% bytes/bytes.go: LastIndexAny 100.0% bytes/bytes.go: genSplit 100.0% bytes/bytes.go: SplitN 100.0% bytes/bytes.go: SplitAfterN 100.0% bytes/bytes.go: Split 100.0% bytes/bytes.go: SplitAfter 100.0% bytes/bytes.go: Fields 100.0% bytes/bytes.go: FieldsFunc 100.0% bytes/bytes.go: Join 100.0% bytes/bytes.go: HasPrefix 100.0% bytes/bytes.go: HasSuffix 100.0% bytes/bytes.go: Map 100.0% bytes/bytes.go: Repeat 100.0% bytes/bytes.go: ToUpper 100.0% bytes/bytes.go: ToLower 100.0% bytes/bytes.go: ToTitle 100.0% bytes/bytes.go: ToUpperSpecial 0.0% bytes/bytes.go: ToLowerSpecial 0.0% bytes/bytes.go: ToTitleSpecial 0.0% bytes/bytes.go: isSeparator 80.0% bytes/bytes.go: Title 100.0% bytes/bytes.go: TrimLeftFunc 100.0% bytes/bytes.go: TrimRightFunc 100.0% bytes/bytes.go: TrimFunc 100.0% bytes/bytes.go: TrimPrefix 100.0% bytes/bytes.go: TrimSuffix 100.0% bytes/bytes.go: IndexFunc 100.0% bytes/bytes.go: LastIndexFunc 100.0% bytes/bytes.go: indexFunc 100.0% bytes/bytes.go: lastIndexFunc 100.0% bytes/bytes.go: makeCutsetFunc 100.0% bytes/bytes.go: Trim 100.0% bytes/bytes.go: TrimLeft 100.0% bytes/bytes.go: TrimRight 100.0% bytes/bytes.go: TrimSpace 100.0% bytes/bytes.go: Runes 100.0% bytes/bytes.go: Replace 100.0% bytes/bytes.go: EqualFold 100.0% bytes/reader.go: Len 100.0% bytes/reader.go: Read 100.0% bytes/reader.go: ReadAt 100.0% bytes/reader.go: ReadByte 0.0% bytes/reader.go: UnreadByte 0.0% bytes/reader.go: ReadRune 0.0% bytes/reader.go: UnreadRune 0.0% bytes/reader.go: Seek 91.7% bytes/reader.go: WriteTo 83.3% bytes/reader.go: NewReader 100.0% total: (statements) 91.6%
Clearly I’m biased when it comes to the popularity of Go, so here is another data point.
[line_chart title=”#golang tweets per month” v_title=”tweets” width=”600px” height=”400px” scale_button=”true”]
[‘Month’, ‘Tweets’],
[ ‘2009-11’ , 60 ],
[ ‘2009-12’ , 31 ],
[ ‘2010-01’ , 14 ],
[ ‘2010-02’ , 36 ],
[ ‘2010-03’ , 56 ],
[ ‘2010-04’ , 57 ],
[ ‘2010-05’ , 62 ],
[ ‘2010-06’ , 81 ],
[ ‘2010-07’ , 149 ],
[ ‘2010-08’ , 106 ],
[ ‘2010-09’ , 225 ],
[ ‘2010-10’ , 139 ],
[ ‘2010-11’ , 219 ],
[ ‘2010-12’ , 102 ],
[ ‘2011-01’ , 173 ],
[ ‘2011-02’ , 204 ],
[ ‘2011-03’ , 258 ],
[ ‘2011-04’ , 251 ],
[ ‘2011-05’ , 694 ],
[ ‘2011-06’ , 557 ],
[ ‘2011-07’ , 393 ],
[ ‘2011-08’ , 444 ],
[ ‘2011-09’ , 401 ],
[ ‘2011-10’ , 456 ],
[ ‘2011-11’ , 385 ],
[ ‘2011-12’ , 369 ],
[ ‘2012-01’ , 344 ],
[ ‘2012-02’ , 558 ],
[ ‘2012-03’ , 877 ],
[ ‘2012-04’ , 508 ],
[ ‘2012-05’ , 450 ],
[ ‘2012-06’ , 656 ],
[ ‘2012-07’ , 782 ],
[ ‘2012-08’ , 785 ],
[ ‘2012-09’ , 1132 ],
[ ‘2012-10’ , 1052 ],
[ ‘2012-11’ , 773 ],
[ ‘2012-12’ , 888 ],
[ ‘2013-01’ , 970 ],
[ ‘2013-02’ , 1439 ],
[ ‘2013-03’ , 1478 ],
[ ‘2013-04’ , 1917 ],
[ ‘2013-05’ , 4714 ],
[ ‘2013-06’ , 5891 ],
[ ‘2013-07’ , 6599 ],
[ ‘2013-08’ , 6886 ]
[/line_chart]
Data courtesy of trendsmap
Go 1.2 is on target for a December release and the Go team have just cut their first release candidate.
You can find the draft (no twitterverse, Go 1.2 isn’t released yet) release notes for Go 1.2 online here.
I have updated my unofficial ARM tarball distributions page with prebuilt go1.2rc1 tarballs. You can find them by following the link in the main header of this page.
If you are interested in following the performance improvements in Go 1.2, you may be interested in my autobench project.
I have updated my unofficial ARM tarball distributions to Go version 1.1.2.
You can find them by following the link in the main header of this page.
It looks like Go 1.4 will remove support for Go packages containing C code (as described below, don’t confuse this with CGO), so enjoy it while it lasts.
This is a short post designed to illustrate how Go package authors can write package level functions in C and call them from Go code without using cgo. The code for this article is available on GitHub, https://github.com/davecheney/ccode.
Before we start, there are a few warnings that need to be spelled out
The first example is a simple function called True that always returns true.
void ·True(bool res) {
res = true;
FLUSH(&res);
}
Even with this simple example there is a lot going on, let’s start with the function signature. The signature of True is void ·True(bool res). The void return code is required as all C to Go interworking is done via arguments passed on the stack. The Interpunkt, the middle dot, ·, is part of the package naming system in Go. By proceeding the name of the function with · we are declaring this function is part of the current package. It is possible to define C functions in other packages, or to provide a package name before the interpunkt, but it gets complicated when your package is heavily namespaced and so is beyond the scope of this article.
The next part of the function signature is the return argument specified in the C declaration style. Both calling and return arguments are supplied as parameters to any C function you want to call from Go. We’ll see in a moment how to write the corresponding forward declaration.
Moving on to the body of the function, assigning true to res is fairly straight forward, but the final line, FLUSH(&res) needs some explanation. Because res is not used inside the body of the function a sufficiently aggressive compiler may optimise the assignment away. FLUSH is used to ensure the final value of res is written back to the stack.
To make the True function available to our Go code, we need to write a forward declaration. Without the forward declaration the function is invisible to the Go compiler. This is unrelated to the normal rules for making Go a symbol public or private via a capital letter.
A forward declaration for the True function looks like this
// True always returns true.
func True() bool
The forward declaration says that True takes no arguments and returns one boolean argument. As this is a normal Go function, you can attach a comment describing the function which will appear in godoc (comments on the function in C code will not appear in documentation).
That is all you need to do make True available to Go code in your package. It should be noted that while True is a public function, this was not required.
Extending from the previous example, let’s define a function called Max which returns the maximum of two ints.
void ·Max(intptr a, intptr b, intptr res) {
res = a > b ? a : b;
FLUSH(&res);
}
Max is similar to the previous function; the first two arguments are function arguments, the final is the return value. Using res for the name of the return argument is not required, but appears to be the convention used heavily throughout the standard library.
The type of a and b is intptr which is the C equivalent of Go’s platform dependant int type.
The forward declaration of Max is shown below. You can see the how function arguments and return values map between Go and C functions.
// Max returns the maximum of two integers.
func Max(a, b int) int
In the previous two examples we have passed values to functions and returned copies of the result via the stack. In Go, all arguments are passed by value, and calling to C functions is no different. For this final example we’ll write a function that increments a value by passing a pointer to that value.
void ·Inc(intptr* addr) {
*addr+=1;
USED(addr);
}
The Inc function takes the address of a intptr (a *int in Go terms), dereferences it, increments it by one, and stores the result at the address addr. The USED macro is similar in function to FLUSH and is used mainly to silence the compiler.
Looking at the forward declaration, we define the function to take a pointer to the int to be incremented.
// Inc increments the value of the integer add address p.
func Inc(p *int)
To demonstate using these C defined functions in Go code I’ve written a few tests which exercise the code. The code for the tests are here, and the results of running the tests are shown below.
% go test -v
github.com/davecheney/ccode
=== RUN TestTrue
--- PASS: TestTrue (0.00 seconds)
=== RUN TestMax
--- PASS: TestMax (0.00 seconds)
=== RUN TestInc
--- PASS: TestInc (0.00 seconds)
PASS
ok github.com/davecheney/ccode 0.005s
In this short article I’ve shown how you can write a Go package that includes functions written in C. While a quite niche use case, it may come in handy for someone and also lays important groundwork for writing packages containing functions in raw assembler.
Earlier this year I wrote a small harness to compare the relative performance of Go 1.0 and the then just released Go 1.1. You can read the posts about the Go 1.1 performance improvements: amd64, 386 and arm.
As the Go 1.2 cycle is entering feature freeze next week, I’ve taken the opportunity to create a new branch of autobench, autobench-next which tracks Go 1.1 vs tip.
Using autobench is very simple, clone the repository and run make to produce a benchmark on your machine.
% cd devel % git clone -b autobench-next https://github.com/davecheney/autobench.git % cd autobench % make
You can stay up to date with the update target
% make update % make
Contributions and benchmark results are always welcome. As the Go 1.2 cycle draws to a close I will merge this branch back into master replacing the older 1.0 vs 1.1 comparisons.