Tag Archives: go

Mikio Hara’s ipv4 package

Yesterday Mikio Hara committed a new package for ipv4 handling to the go.net repository. I wanted to recognise Mikio’s work for two reasons.

  1. The level of detail and control this package offers is, in my opinion, unmatched by any other language. If you’re using C or C++, then you have the raw power of setsockopt(2) and ioctl(2) available, but you also have no guide or safety rail when using them. Mikio’s package provides exquisite control over ipv4 minutia without sacrificing the safety that Go provides.
  2. The package is a fantastic example of using conditional compilation and embedding to build a very low level package that works across all the supported Go platforms. This package compiles cleanly on Linux, *BSD, OS X and Windows without resorting to the lowest common denominator or using a single ifdef. If you are looking for examples on how to structure your code using build tags, or looking for ways to work within the Go1 API, you should study this package.

Check it out for yourselves, go.pkgdoc.org/code.google.com/p/go.net/ipv4.

Installing Go on the Raspberry Pi

Introduction

The Raspberry Pi has captured the imagination of hackers and makers alike. While it certainly wasn’t the first ARM development board on the market, its bargin basement price tag and the charitable philosophy of its inventors has sparked a huge interest in this little ARM system. What could be more appropriate for a new generation of programmers than a modern, safe and efficient programming language for their projects, Google Go.

This post describes the steps for installing Google Go from source on the Raspberry Pi. At the time of this post, trunk contains many improvements for ARM processors, including full support for cgo, which are not available in Go 1.0.x. It is expected that these enhancements will be available when Go 1.1 ships next year. If you are reading this post in September 2013, then it’s likely you will want to use the version of Go shipping with your operating system distribution rather than these instructions.

Update May, 2013 If you want to save yourself some time, precompiled binary releases of Go 1.1 for linux/arm are available. Please see the Unofficial ARM tarballs link at the top of the page.

Setting up your Pi

Before you compile Go on your Pi you should follow these steps to ensure a successful compilation. Briefly summarised, they are:

  1. Install Raspbian
  2. Configure your memory split
  3. Add some swap

Install Raspbian

The downloads page on the Raspberry Pi website contains links to SD card images for various Linux distributions. This tutorial recommends the Raspbian wheezy flavour of Debian. Follow the instructions for creating an SD card image and continue to the next step once you have ssh’d into your Raspbian installation.

Configure your memory split

The Pi comes with 256mb of memory which is shared between the video subsystem and the main processor. Compiling and linking Go programs can consume over 100mb of ram so it is recommended that the memory split be adjusted in favor of the main processor, at least while working with Go code. The Raspbian distribution makes this very easy with the raspi-config utility 

% sudo raspi-config


Then reboot your system 

% sudo shutdown -r now

Add some swap

To run the full test suite you will need some swap. This can be accomplished a variety of ways, using an external USB hard drive, or swapping over NFS. I’ll describe how to setup a NFS swap partition as this is the configuration I am using to generate this tutorial. 

% sudo dd if=/dev/zero of=/import/nas/swap bs=1024 count=1048576 1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied, 136.045 s, 7.9 MB/s
% sudo losetup /dev/loop0 /import/nas/swap
% sudo mkswap /dev/loop0
Setting up swapspace version 1, size = 1048572 KiB no label, UUID=7ba9443d-c64c-416f-9931-39e3e2decf0f
% sudo swapon /dev/loop0
% free -m
             total used free shared buffers cached
Mem:           232   78  153      0       0     24
-/+ buffers/cache:   52  179
Swap:         1123   15 1108

Installing the prerequisites

Raspbian comes with almost all the tools you need to compile Go already installed, but to be sure you should install the following packages, described on the golang.org website. 

% sudo apt-get install -y mercurial gcc libc6-dev

Cloning the source

% hg clone -u default https://code.google.com/p/go $HOME/go warning: code.google.com certificate with fingerprint 9f:af:b9:ce:b5:10:97:c0:5d:16:90:11:63:78:fa:2f:37:f4:96:79 not verified (check hostfingerprints or web.cacerts config setting)
destination directory: go
requesting all changes
adding changesets
adding manifests
adding file changes
added 14430 changesets with 52478 changes to 7406 files (+5 heads) updating to branch default
3520 files updated, 0 files merged, 0 files removed, 0 files unresolved

Building Go

% cd $HOME/go/src % ./all.bash

If all goes well, after about 90 minutes you should see 

ALL TESTS PASSED

---
Installed Go for linux/arm in /home/dfc/go
Installed commands in /home/dfc/go/bin

If there was an error relating to out of memory, or you couldn’t configure an appropriate swap device, you can skip the test suite by executing 

% cd $HOME/go % ./make.bash

as an alternative to ./all.bash.

Adding the go command to your path

The go command should be added to your $PATH 

% export PATH=$PATH:$HOME/go/bin
% go version
go version devel +cfbcf8176d26 Tue Sep 25 17:06:39 2012 +1000

Now, Go and make something awesome.

Additional resources

How the Go language improves expressiveness without sacrificing runtime performance

This week there was a discussion on the golang-nuts mailing list about an idiomatic way to update a slice of structs. For example, consider this struct representing a set of counters. 

type E struct {
         A, B, C, D int
}

var e = make([]E, 1000)

Updating these counters may take the form 

for i := range e {
        e[i].A += 1
        e[i].B += 2
        e[i].C += 3
        e[i].D += 4]
}

Which is good idiomatic Go code. It’s pretty fast too 

BenchmarkManual   500000              4642 ns/op

However there is a problem with this example. Each access the ith element of e requires the compiler to insert an array bounds checks. You can avoid 3 of these checks by referencing the ith element once per iteration. 

for i := range e {
        v := &e[i]
        v.A += 1 
        v.B += 2
        v.C += 3
        v.D += 4
}

By reducing the number of subscript checks, the code now runs considerably faster. 

BenchmarkUnroll  1000000              2824 ns/op

If you are coding a tight loop, clearly this is the more efficient method, but it comes with a cost to readability, as well as a few gotchas. Someone else reading the code might be tempted to move the creation of v into the for declaration, or wonder why the address of e[i] is being taken. Both of these changes would cause an incorrect result. Obviously tests are there to catch this sort of thing, but I propose there is a better way to write this code, one that doesn’t sacrifice performance, and expresses the intent of the author more clearly. 

func (e *E) update(a, b, c, d int) {
        e.A += a
        e.B += b
        e.C += c
        e.D += d
} 

for i := range e {
        e[i].update(1, 2, 3, 4)
}

Because E is a named type, we can create an update() method on it. Because update is declared on with a receiver of *E, the compiler automatically inserts the (&e[i]).update() for us. Most importantly because of the simplicity of the update() method itself, the inliner can roll it up into the body of the calling loop, negating the method call cost. The result is very similar to the hand unrolled version. 

BenchmarkUpdate   500000              2996 ns/op

In conclusion, as Cliff Click observed, there are Lies, Damn Lies and Microbenchmarks. This post is an example of a least one of the three. My intent in writing was not to spark a language war about who can update a struct the fastest, but instead argue that Go lets you write your code in a more expressive manner without having to trade off performance.

You can find the source code for the benchmarks presented in this below. 

package b
 
import "testing"

// SIZE=1000 results (core i5 late 2011 mac mini, 10.7.3)
// % go test -v -run='XXX' -bench='.'
// PASS
// BenchmarkUpdate 500000 2996 ns/op
// BenchmarkManual 500000 4642 ns/op
// BenchmarkUnroll 1000000 2824 ns/op

type E struct {
        A, B, C, D int
}

func (e *E) update(a, b, c, d int) {
        e.A += a
        e.B += b
        e.C += c
        e.D += d
}

var SIZE = 1000 // needed to make a valid testable package

func TestNothing(t *testing.T) {}

func assert(e []E, b testing.B) {
        for _, v := range e {
                if v.A != b.N || v.B != b.N2 || v.C != b.N3 || v.D != b.N4 {
                        b.Errorf("Expected: %d, %d, %d, %d; actual: %d, %d, %d, %d",
                        b.N, b.N2, b.N3, b.N*4, v.A, v.B, v.C, v.D)
                }
        }
}

func BenchmarkUpdate(b *testing.B) {
        var e = make([]E, SIZE)
        for j := 0; j < b.N; j++ {
                for i := range e {
                        e[i].update(1, 2, 3, 4)
                }
        }
        b.StopTimer()
        assert(e, b)
}

func BenchmarkManual(b *testing.B) {
        var e = make([]E, SIZE)
        for j := 0; j < b.N; j++ {
                for i := range e {
                        e[i].A += 1
                        e[i].B += 2
                        e[i].C += 3
                        e[i].D += 4
                }
        }
        b.StopTimer()
        assert(e, b)
}

func BenchmarkUnroll(b *testing.B) {
        var e = make([]E, SIZE)
        for j := 0; j < b.N; j++ {
                for i := range e {
                        v := &e[i]
                        v.A += 1
                        v.B += 2
                        v.C += 3
                        v.D += 4
                }
        }
        b.StopTimer()
        assert(e, b)
}

Why Go gets exceptions right

How does Go get exceptions right? Why, by not having them in the first place.

First, a little history.

Before my time, there was C, and errors were your problem. This was generally okay, because if you owned an 70’s vintage mini computer, you probably had your share of problems. Because C was a single return language, things got a bit complicated when you wanted to know the result of a function that could sometimes go wrong. IO is a perfect example of this, or sockets, but there are also more pernicious cases like converting a string to its integer value. A few idioms grew to handle this problem. For example, if you had a function that would mess around with the contents of a struct, you could pass a pointer to it, and the return code would indicate if the fiddling was successful. There are other idioms, but I’m not a C programmer, and that isn’t the point of this article.

Next came C++, which looked at the error situation and tried to improve it. If you had a function which would do some work, it could return a value or it could throw an exception, which you were then responsible for catching and handling. Bam! Now C++ programmers can signal errors without having to conflate their single return value. Even better, exceptions can be handled anywhere in the call stack. If you don’t know how to handle that exception it’ll bubble up to someone who does. All the nastyness with errno and threads is solved. Achievement unlocked!

Sorta.

The downside of C++ exceptions is you can’t tell (without the source and the impetus to check) if any function you call may throw an exception. In addition to worrying about resource leaks and destructors, you have to worry about RAII and transactional semantics to ensure your methods are exception safe in case they are somewhere on the call stack when an exception is thrown. In solving one problem, C++ created another.

So the designers of Java sat down, stroked their beards and decided that the problem was not exceptions themselves, but the fact that they could be thrown without notice; hence Java has checked exceptions. You can’t throw an exception inside a method without annotating that method’s signature to indicate you may do so, and you can’t call a method that may throw an exception without wrapping it in code to handle the potential exception. Via the magic of compile time bondage and discipline the error problem is solved, right?

This is about the time I enter the story, the early millennium, circa Java 1.4. I agreed then, as I do now, that the Java way of checked exceptions was more civilised, safer, than the C++ way. I don’t think I was the only one. Because exceptions were now safe, developers started to explore their limits. There were coroutine systems built using exceptions, and at least one XML parsing library I know of used exceptions as a control flow technique. It’s commonplace for established Java webapps to disgorge screenfuls of exceptions, dutifully logged with their call stack, on startup. Java exceptions ceased to be exceptional at all, they became commonplace. They are used from everything from the benign to the catastrophic, differentiating between the severity of exceptions falls to the caller of the function.

If that wasn’t bad enough, not all Java exceptions are checked, subclasses of java.Error and java.RuntimeException are unchecked. You don’t need to declare them, just throw them. This probably started out as a good idea, null references and array subscript errors are now simple to implement in the runtime, but at the same time because every exception Java extends java.Exception any piece of code can catch it, even if it makes little sense to do so, leading to patterns like 

catch (e Exception) { // ignore }

So, Java mostly solved the C++ unchecked exception problem, and introduced a whole slew of its own. However I argue Java didn’t solve the actual problem, the problem that C++ didn’t solve either. The problem of how to signal to caller of your function that something went wrong.

Enter Go

Go solves the exception problem by not having exceptions. Instead Go allows functions to return an error type in addition to a result via its support for multiple return values. By declaring a return value of the interface type error you indicate to the caller that this method could go wrong. If a function returns a value and an error, then you can’t assume anything about the value until you’ve inspected the error. The only place that may be acceptable to ignore the value of error is when you don’t care about the other values returned.

Go does have a facility called panic, and if you squint hard enough, you might imagine that panic is the same as throw, but you’d be wrong. When you throw and exception you’re making it the caller’s problem 

throw new SomeoneElsesProblem();

For example in C++ you might throw an exception when you can’t convert from an enum to its string equivalent, or in Java when parsing a date from a string. In an internet connected world, where every input from a network must be considered hostile, is the failure to parse a string into a date really exceptional? Of course not.

When you panic in Go, you’re freaking out, it’s not someone elses problem, it’s game over man. 

panic("inconceivable")

panics are always fatal to your program. In panicing you never assume that your caller can solve the problem. Hence panic is only used in exceptional circumstances, ones where it is not possible for your code, or anyone integrating your code to continue.

The decision to not include exceptions in Go is an example of its simplicity and orthogonality. Using multiple return values and a simple convention, Go solves the problem of letting programmers know when things have gone wrong and reserves panic for the truly exceptional.