Tag Archives: cross compilation

Cross compilation with Go 1.5

Now that Go 1.5 is out, lots of gophers are excited to try the much improved cross compilation support. For some background on the changes to the cross compilation story you can read my previous post, or Rakyll’s excellent follow up piece.

I’ll assume that you are using the binary version of Go 1.5, as distributed from the Go website. If you are using Go 1.5 from your operating system’s distribution, or homebrew, the process will be the same, however paths may differ slightly.

How to cross compile

To cross compile a Go program using Go 1.5 the process is as follows:

  1. set GOOS and GOARCH to be the values for the target operating system and architecture.
  2. run go build -v YOURPACKAGE

If the compile is successful you’ll have a binary called YOURPACKAGE (possibly with a .exe extension if you’re targeting Windows) in your current working directory.

-o may be used to alter the name and destination of your binary, but remember that go build takes a value that is relative to your $GOPATH/src, not your working directory, so changing directories then executing the go build command is also an option.

Example

I prefer to combine the two steps outlined above into one, like this:

% env GOOS=linux GOARCH=arm go build -v github.com/constabulary/gb/cmd/gb
runtime
sync/atomic
...
github.com/constabulary/gb
github.com/constabulary/gb/cmd
github.com/constabulary/gb/cmd/gb
% file ./gb
./gb: ELF 32-bit LSB  executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped

and that’s all there is to it.

Using go build vs go install

When cross compiling, you should use go build, not go install. This is the one of the few cases where go build is preferable to go install.

The reason for this is go install always caches compiled packages, .a files, into the pkg/ directory that matches the root of the source code.

For example, if you are building $GOPATH/src/github.com/lib/pq, then the compiled package will be installed into $GOPATH/pkg/$GOOS_$GOARCH/github.com/lib/pq.a.

This logic also holds true for the standard library, which lives in /usr/local/go/src, so will be compiled to /usr/local/go/pkg/$GOOS_$GOARCH. This is a problem, because when cross compiling the go tool needs to rebuild the standard library for your target, but the binary distribution expects that /usr/local/go is not writeable.

Using go build rather that go install is the solution here, because go build builds, then throws away most of the result (rather than caching it for later), leaving you with the final binary in the current directory, which is most likely writeable by you.

Ugh, this is really slow!

In the procedure described above, cross compilation always rebuilds that standard library for the target every time. Depending on your workflow this is either not an issue, or a really big issue. If it’s the latter then I recommend you remove the binary distribution and build from source into a path that is writeable by you, then you’ll have the full gamut of go commands available to you.

Cross compilation support in gb is being actively developed and will not have this restriction.

What about GOARM?

The go tool chooses a reasonable value for GOARM by default. You should not change this unless you have a good reason.

But but, what about GOARM=7?

Sure, knock yourself out, but that means your program won’t run on all models of the Raspberry Pi. The difference between GOARM=6 (the default) and GOARM=7 is enabling a few more floating point registers, and a few more operations that allow floating point double values to be passed to and from the ARMv7 (VPFv3) floating point co processor more efficiently. IMO, with the current Go 1.5 arm compiler, it’s not worth the bother.

Cross compilation just got a whole lot better in Go 1.5

Introduction

Cross compilation is one of Go’s headline features. I’ve written about it a few times, and others have taken this work and built better tooling around it.

This morning Russ Cox committed this change which resolved the last issue in making cross compilation simpler and more accessible for all Gophers. When Go 1.5 ships in August any Go programmer will be able to cross compile their program without having to go through a fussy set up phase.

Background

In the current version of Go, (if you’re from the future, read Go 1.4 and earlier) before you could cross compile a Go program, you needed to go through a set up phase to enhance your Go installation with the bits necessary to build for other platforms.

This worked ok if you had built Go from source, but if you were using one of the binary distributions or something from your operating system (brew, apt, etc), then the process would turn into a maze of twisty passages, all alike.

This was necessary because for successful cross compilation you would need

  • compilers for the target platform, if they differed from your host platform, ie you’re on darwin/amd64 (6g) and you want to compile for linux/arm (5g).
  • a standard library for the target platform, which included some files generated at the point your Go distribution was built.

With the plan to translate the Go compiler into Go coming to fruition in the 1.5 release the first issue is now resolved. That just left the small amount of customisation to the runtime done at installation time, which has been whittled away as part of the compiler transition (most of the customisation was generation of include files for the C parts of the runtime), and as of this morning, removed.

Try it out

If you can’t wait til August, you can try this out today by building the development version of Go.

Disclaimer: the development branch is changing rapidly due to the re-factoring of the compilers. If you’re using Go in production, the release version, Go 1.4.x, is always recommended.

Prerequisites

As mentioned above to build the development version of Go, you also need Go 1.4 installed to bootstrap. Once Go 1.5 comes out this step will be unnecessary.

These steps are a simple procedure to do this which I would recommend following

  1. Uninstall any version of Go you have on your system, including any $PATH variables.
  2. Check out Go 1.4 and build it 
    % git clone https://go.googlesource.com/go $HOME/go1.4
% cd $HOME/go1.4/src
% git checkout release-branch.go1.4
% ./make.bash
  3. Check out the development branch and build it 
    % git clone https://go.googlesource.com/go $HOME/go
% cd $HOME/go/src
% env GOROOT_BOOTSTRAP=$HOME/go1.4 ./all.bash
  4. Add $HOME/go/bin to your $PATH

Build something

Notice: if you are reading this from a future where Go 1.5 has been released, you can skip the previous step.

Now you have the development version of Go 1.5 installed, cross compiling this simple program is trivial

package main

import "fmt"
import "runtime"

func main() {
        fmt.Printf("Hello %s/%s\n", runtime.GOOS, runtime.GOARCH)
}

Now build for darwin/386

% env GOOS=darwin GOARCH=386 go build hello.go
# scp to darwin host
$ ./hello
Hello darwin/386

Or build for linux/arm

% env GOOS=linux GOARCH=arm GOARM=7 go build hello.go
# scp to linux host
$ ./hello
Hello linux/arm

That’s it!

Cross compilation can’t get any simpler than that.

Technical mumbo-jumbo

So what is happening under the hood when we compile a program for a different platform ? The -v flag gives us a clue.

% go build -v hello.go
command-line-arguments

% env GOOS=linux GOARCH=arm go build -v hello.go                                  runtime
errors
sync/atomic
math
unicode/utf8
sync
io
syscall
time
strconv
reflect
os
fmt
command-line-arguments

Comparing the two examples above, the first build is using the standard library that was built as part of the Go installation. That is to say, all the packages that fmt and runtime depend on are already built into $GOROOT/pkg/linux_amd64.

In the second example, the go tool detects that all the dependencies of this program need to be built for the target plaform before hello.go can be compiled and builds them, all the way back to the runtime package.

I have not included the output here because it is very verbose, but you can look at all the steps that are being performed if you pass the -x flag to go build.

Out of scope

Cross compilation while linking to libraries via cgo is the holy grail for some. Sadly the changes mentioned above do not change the situation with respect to cgo. In fact you may still need to rebuild your Go installation in the traditional way to pass environment variables like CC_FOR_TARGET. Please try it out.

Conclusion

In case you can’t tell, I’m over the moon about this improvement. Cross compilation is Go’s ace in the hole and Go 1.5 will make it even better.

Please try it out and if you find issues please let us know on the GitHub issue tracker.

An introduction to cross compilation with Go 1.1

This post is a compliment to one I wrote in August of last year, updating it for Go 1.1. Since last year tools such as goxc have appeared which go a beyond a simple shell wrapper to provide a complete build and distribution solution.

Introduction

Go provides excellent support for producing binaries for foreign platforms without having to install Go on the target. This is extremely handy for testing packages that use build tags or where the target platform is not suitable for development.

Support for building a version of Go suitable for cross compilation is built into the Go build scripts; just set the GOOS, GOARCH, and possibly GOARM correctly and invoke ./make.bash in $GOROOT/src. Therefore, what follows is provided simply for convenience.

Getting started

1. Install Go from source. The instructions are well documented on the Go website, golang.org/doc/install/source. A summary for those familiar with the process follows.

% hg clone https://code.google.com/p/go
% cd go/src
% ./all.bash

2. Checkout the support scripts from Github, github.com/davecheney/golang-crosscompile

% git clone git://github.com/davecheney/golang-crosscompile.git
% source golang-crosscompile/crosscompile.bash

3. Build Go for all supported platforms

% go-crosscompile-build-all
go-crosscompile-build darwin/386
go-crosscompile-build darwin/amd64
go-crosscompile-build freebsd/386
go-crosscompile-build freebsd/amd64
go-crosscompile-build linux/386
go-crosscompile-build linux/amd64
go-crosscompile-build linux/arm
go-crosscompile-build windows/386
go-crosscompile-build windows/amd64

This will compile the Go runtime and standard library for each platform. You can see these packages if you look in go/pkg.

% ls -1 go/pkg 
darwin_386
darwin_amd64
freebsd_386
freebsd_amd64
linux_386
linux_amd64
linux_arm
obj
tool
windows_386
windows_amd64

Using your cross compilation environment

Sourcing crosscompile.bash provides a go-$GOOS-$GOARCH function for each platform, you can use these as you would the standard go tool. For example, to compile a program to run on linux/arm.

% cd $GOPATH/github.com/davecheney/gmx/gmxc
% go-linux-arm build 
% file ./gmxc 
./gmxc: ELF 32-bit LSB executable, ARM, version 1 (SYSV), 
statically linked, not stripped

This file is not executable on the host system (darwin/amd64), but will work on linux/arm.

Some caveats

Cross compiled binaries, not a cross compiled Go installation

This post describes how to produce an environment that will build Go programs for your target environment, it will not however build a Go environment for your target. For that, you must build Go directly on the target platform. For most platforms this means installing from source, or using a version of Go provided by your operating systems packaging system.
If you are using

No cgo in cross platform builds

It is currently not possible to produce a cgo enabled binary when cross compiling from one operating system to another. This is because packages that use cgo invoke the C compiler directly as part of the build process to compile their C code and produce the C to Go trampoline functions. At the moment the name of the C compiler is hard coded to gcc, which assumes the system default gcc compiler even if a cross compiler is installed.

In Go 1.1 this restriction was reinforced further by making CGO_ENABLED default to 0 (off) when any cross compilation was attempted.

GOARM flag needed for cross compiling to linux/arm.

Because some arm platforms lack a hardware floating point unit the GOARM value is used to tell the linker to use hardware or software floating point code. Depending on the specifics of the target machine you are building for, you may need to supply this environment value when building.

% GOARM=5 go-linux-arm build

As of e4b20018f797 you will at least get a nice error telling you which GOARM value to use.

$ ./gmxc 
runtime: this CPU has no floating point hardware, so it cannot 
run this GOARM=7 binary. Recompile using GOARM=5.

By default, Go assumes a hardware floating point unit if no GOARM value is supplied. You can read more about Go on linux/arm on the Go Language Community Wiki.