Benchmarking Go 1.2rc5 vs gccgo

I’ve been doing a lot of work with gccgo recently and with the upcoming release of Go 1.2 I’ve also been collecting benchmark results for that release.

Presented below, using a very unscientific method, are the results of comparing the go1 benchmark results for the two compilers.

gc-vs-gccgo

Buried among that see of red are a few telltale signs that the more capable gcc backed optimiser can eek out better arithmetic performance.

As another data point, here is the floats benchmark from my autobench suite, run under the same conditions as above.

floats

I want to be clear that these are very preliminary results, and, like all all micro benchmarks are subject to interpretation.

I also want to stress that I am not dismissing gccgo based on these results. As I understand it gccgo lacks a few key features, such as escape analysis, which is probably responsible for most of the performance loss when the amount of computation is dwarfed by memory bookkeeping.

gccgo is developed largely by one person, ian Taylor, and is a significant achievement. Recently he and Chris Manghane have been working on decoupling the gofrontend code from gcc so it can be reused with other compiler backends, LLVM being the most obvious.

If you are interested in contributing to Go, please don’t forget about gccgo, or even llgo as possible outlets for your energies. Go itself is a stronger language because we have at least four implementations of the specification. This helps keep the compiler writers honest and avoids the language being defined by default by its most popular implementation.

Using Juju to build gccgo

The port of Juju to Go is a project I’ve been involved in at Canonical for some time now. The power behind Juju is charms, which are part configuration management and part reliable workflow engine.

One non-conventional use of Juju is something I cooked up a while ago when traveling, a Juju charm that compiles gccgo. This charm can be used to compile gccgo on a powerful instance in your cloud rather than on your puny laptop without having to worry about finding all the various dependencies that a modern gcc build requires.

The gccgo charm encapsulates all the instructions in http://golang.org/doc/install/gccgo, all you need to do is deploy it and wait for the result.

Getting started

To get started using the gccgo charm, checkout my charms repository from GitHub.

% cd $HOME
% git clone https://github.com/davecheney/charms

Bootstrap a juju environment

Each Juju service (an instance of a charm) needs to be deployed into a running environment. I’ve bootstrapped an environment on Amazon AWS as they have a nice 8 core machine which will get the job done quickly.

% juju bootstrap -e ap-southeast-2

Deploying the gccgo charm

The next step is to deploy an instance of the gccgo charm from my local charm repository. By default Juju requests the equivalent of an m1.small so we use a deploy time constraint to request a machine with a larger number of cores. The gccgo charm automatically adjusts itself to use all CPUs on the target machine.

% juju deploy --constraints "cpu-cores=8" --repository $HOME/charms \
     local:raring/gccgo

Monitoring the status of the build

All the magic of the build phase takes place in the hooks/start hook, so the build will stay at installed until the build completes (or fails).

% juju status gccgo
environment: ap-southeast-2
machines:
  "1":
    agent-state: started
    agent-version: 1.15.0.1
    dns-name: ec2-54-253-4-102.ap-southeast-2.compute.amazonaws.com
    instance-id: i-22c92a1e
    instance-state: running
    series: raring
    hardware: arch=amd64 cpu-cores=8 cpu-power=2000 mem=7168M root-disk=8192M
services:
  gccgo:
    charm: local:raring/gccgo-12
    exposed: false
    units:
      gccgo/0:
        agent-state: installed
        agent-version: 1.15.0.1
        machine: "1"
        public-address: ec2-54-253-4-102.ap-southeast-2.compute.amazonaws.com

You can also monitor the output of the build process itself using the juju debug-log command.

Grabbing the results

The gccgo charm has a number of configuration variables you can use to tweak the build process if necessary. The gccgo charm produces a tarball as its final result once the service moves to started state.

% juju get gccgo
charm: gccgo
service: gccgo
settings:
  prefix:
    default: true
    description: gccgo build prefix
    type: string
    value: /opt/gccgo
  tarfile:
    default: true
    description: gccgo final tarball
    type: string
    value: /home/ubuntu/gccgo.tar.bz2
  work:
    default: true
    description: gccgo build directory
    type: string
    value: /home/ubuntu/work

Now we know the location of the tarball, we can use the juju scp command to fetch it.

juju scp gccgo/0:/home/ubuntu/gccgo.tar.bz2 /tmp

Cleaning up

8 core virtual machines don’t come cheap, don’t forget to destroy this environment (or at least destroy the service and remove the machine) once you’re done.

# destroy service and remove build machine
% juju destroy-service gccgo
% juju destroy-machine 1    # from the output of juju status above
# or destroy the environment
% juju destroy-environment -y