Monthly Archives: July 2013

An introduction to cross compilation with Go 1.1

This post is a compliment to one I wrote in August of last year, updating it for Go 1.1. Since last year tools such as goxc have appeared which go a beyond a simple shell wrapper to provide a complete build and distribution solution.

Introduction

Go provides excellent support for producing binaries for foreign platforms without having to install Go on the target. This is extremely handy for testing packages that use build tags or where the target platform is not suitable for development.

Support for building a version of Go suitable for cross compilation is built into the Go build scripts; just set the GOOS, GOARCH, and possibly GOARM correctly and invoke ./make.bash in $GOROOT/src. Therefore, what follows is provided simply for convenience.

Getting started

1. Install Go from source. The instructions are well documented on the Go website, golang.org/doc/install/source. A summary for those familiar with the process follows.

% hg clone https://code.google.com/p/go
% cd go/src
% ./all.bash

2. Checkout the support scripts from Github, github.com/davecheney/golang-crosscompile

% git clone git://github.com/davecheney/golang-crosscompile.git
% source golang-crosscompile/crosscompile.bash

3. Build Go for all supported platforms

% go-crosscompile-build-all
go-crosscompile-build darwin/386
go-crosscompile-build darwin/amd64
go-crosscompile-build freebsd/386
go-crosscompile-build freebsd/amd64
go-crosscompile-build linux/386
go-crosscompile-build linux/amd64
go-crosscompile-build linux/arm
go-crosscompile-build windows/386
go-crosscompile-build windows/amd64

This will compile the Go runtime and standard library for each platform. You can see these packages if you look in go/pkg.

% ls -1 go/pkg 
darwin_386
darwin_amd64
freebsd_386
freebsd_amd64
linux_386
linux_amd64
linux_arm
obj
tool
windows_386
windows_amd64

Using your cross compilation environment

Sourcing crosscompile.bash provides a go-$GOOS-$GOARCH function for each platform, you can use these as you would the standard go tool. For example, to compile a program to run on linux/arm.

% cd $GOPATH/github.com/davecheney/gmx/gmxc
% go-linux-arm build 
% file ./gmxc 
./gmxc: ELF 32-bit LSB executable, ARM, version 1 (SYSV), 
statically linked, not stripped

This file is not executable on the host system (darwin/amd64), but will work on linux/arm.

Some caveats

Cross compiled binaries, not a cross compiled Go installation

This post describes how to produce an environment that will build Go programs for your target environment, it will not however build a Go environment for your target. For that, you must build Go directly on the target platform. For most platforms this means installing from source, or using a version of Go provided by your operating systems packaging system.
If you are using

No cgo in cross platform builds

It is currently not possible to produce a cgo enabled binary when cross compiling from one operating system to another. This is because packages that use cgo invoke the C compiler directly as part of the build process to compile their C code and produce the C to Go trampoline functions. At the moment the name of the C compiler is hard coded to gcc, which assumes the system default gcc compiler even if a cross compiler is installed.

In Go 1.1 this restriction was reinforced further by making CGO_ENABLED default to 0 (off) when any cross compilation was attempted.

GOARM flag needed for cross compiling to linux/arm.

Because some arm platforms lack a hardware floating point unit the GOARM value is used to tell the linker to use hardware or software floating point code. Depending on the specifics of the target machine you are building for, you may need to supply this environment value when building.

% GOARM=5 go-linux-arm build

As of e4b20018f797 you will at least get a nice error telling you which GOARM value to use.

$ ./gmxc 
runtime: this CPU has no floating point hardware, so it cannot 
run this GOARM=7 binary. Recompile using GOARM=5.

By default, Go assumes a hardware floating point unit if no GOARM value is supplied. You can read more about Go on linux/arm on the Go Language Community Wiki.

Introducing profile, super simple profiling for Go programs

This package has been superseded. Please read this blog post for more details.

Introduction

The Go runtime has built in support for several types of profiling that can be used to inspect the performance of your programs. A common way to leverage this support is via the testing package, but if you want to profile a full application it is sometimes complicated to configure the various profiling mechanisms correctly.

I wrote profile to scratch my own itch and create a simple way to profile an existing Go program without having to restructure it as a benchmark.

Installation

profile is go getable so installation is a simple as

go get github.com/davecheney/profile

Usage

Enabling profiling in your application is as simple as one line at the top of your main function

import "github.com/davecheney/profile"

func main() {
        defer profile.Start(profile.CPUProfile).Stop()
        ...
}

Options

What to profile is controlled by the *profile.Config value passed to profile.Start. A nil *profile.Config is the same as choosing all the defaults. By default no profiles are enabled.

In this more complicated example a *profile.Config is constructed by hand which enables memory profiling, but disables the shutdown hook.

import "github.com/davecheney/profile"

func main() {
        cfg := profile.Config {
                MemProfile: true,
                NoShutdownHook: true, // do not hook SIGINT
        }
        // p.Stop() must be called before the program exits to  
        // ensure profiling information is written to disk.
        p := profile.Start(&cfg)
        ...
}

Several convenience variables are provided for cpu, memory, and block (contention) profiling.

For more complex options, consult the documentation on the profile.Config type. Enabling more than one profile may cause your results to be less reliable as profiling itself is not without overhead.

Example

To show profile in action, I modified cmd/godoc following the instructions in the first example.

% godoc -http=:8080
2013/07/07 15:29:11 profile: cpu profiling enabled, /tmp/profile002803/cpu.pprof

In another window I visited http://localhost:8080 a few times to have some profiling data to record, then stopped godoc.

^C2013/07/07 15:29:33 profile: caught interrupt, stopping profiles
% go tool pprof $(which godoc) /tmp/profile002803/cpu.pprof
Welcome to pprof!  For help, type 'help'.
(pprof) top10
Total: 15 samples
       2  13.3%  13.3%        2  13.3% go/scanner.(*Scanner).next
       2  13.3%  26.7%        2  13.3% path.Clean
       1   6.7%  33.3%        3  20.0% go/scanner.(*Scanner).Scan
       1   6.7%  40.0%        1   6.7% main.hasPathPrefix
       1   6.7%  46.7%        3  20.0% main.mountedFS.translate
       1   6.7%  53.3%        1   6.7% path.Dir
       1   6.7%  60.0%        1   6.7% path/filepath.(*lazybuf).append
       1   6.7%  66.7%        1   6.7% runtime.findfunc
       1   6.7%  73.3%        2  13.3% runtime.makeslice
       1   6.7%  80.0%        2  13.3% runtime.mallocgc

Note In the example above we’re passing the godoc binary and the profile produced by running that binary to go tool pprof. When profiling your own code, you must pass your binary, and its profile to go tool pprof otherwise the profile will not make sense.

Licence

profile is available under a BSD licence.