Yearly Archives: 2013

Curious Channels

Channels are a signature feature of the Go programming language. Channels provide a powerful way to reason about the flow of data from one goroutine to another without the use of locks or critical sections.

Today I want to talk about two important properties of channels that make them useful for controlling not just data flow within your program, but the flow of control as well.

A closed channel never blocks

The first property I want to talk about is a closed channel. Once a channel has been closed, you cannot send a value on this channel, but you can still receive from the channel.

package main

import "fmt"

func main() {
        ch := make(chan bool, 2)
        ch <- true
        ch <- true
        close(ch)

        for i := 0; i < cap(ch) +1 ; i++ {
                v, ok := <- ch
                fmt.Println(v, ok)
        }
}

In this example we create a channel with a buffer of two, fill the buffer, then close it.

true true
true true
false false

Running the program shows we retrieve the first two values we sent on the channel, then on our third attempt the channel gives us the values of false and false. The first false is the zero value for that channel’s type, which is false, as the channel is of type chan bool. The second indicates the open state of the channel, which is now false, indicating the channel is closed. The channel will continue to report these values infinitely. As an experiment, alter this example to receive from the channel 100 times.

Being able to detect if your channel is closed is a useful property, it is used in the range over channel idiom to exit the loop once a channel has been drained.

package main

import "fmt"

func main() {
        ch := make(chan bool, 2)
        ch <- true
        ch <- true
        close(ch)

        for v := range ch {
                fmt.Println(v) // called twice
        }
}

but really comes into its own when combined with select. Let’s start with this example

package main

import (
        "fmt"
        "sync"
        "time"
)

func main() {
        finish := make(chan bool)
        var done sync.WaitGroup
        done.Add(1)
        go func() {
                select {
                case <-time.After(1 * time.Hour):
                case <-finish:
                }
                done.Done()
        }()
        t0 := time.Now()
        finish <- true // send the close signal
        done.Wait()    // wait for the goroutine to stop
        fmt.Printf("Waited %v for goroutine to stop\n", time.Since(t0))
}

Running the program, on my system, gives a low wait duration, hence it is clear that the goroutine does not wait the full hour before calling done.Done()

Waited 129.607us for goroutine to stop

But there are a few problems with this program. The first is the finish channel is not buffered, so the send to finish may block if the receiver forgot to add finish to their select statement. You could solve that problem by wrapping the send in a select block to make it non blocking, or making the finish channel buffered. However what if you had many goroutines listening on the finish channel, you would need to track this and remember to send the correct number of times to the finish channel. This might get tricky if you aren’t in control of creating these goroutines; they may be being created in another part of your program, perhaps in response to incoming requests over the network.

A nice solution to this problem is to leverage the property that a closed channel is always ready to receive. Using this property we can rewrite the program, now including 100 goroutines, without having to keep track of the number of goroutines spawned, or correctly size the finish channel

package main

import (
        "fmt"
        "sync"
        "time"
)

func main() {
        const n = 100
        finish := make(chan bool)
        var done sync.WaitGroup
        for i := 0; i < n; i++ { 
                done.Add(1)
                go func() {
                        select {
                        case <-time.After(1 * time.Hour):
                        case <-finish:
                        }
                        done.Done()
                }()
        }
        t0 := time.Now()
        close(finish)    // closing finish makes it ready to receive
        done.Wait()      // wait for all goroutines to stop
        fmt.Printf("Waited %v for %d goroutines to stop\n", time.Since(t0), n)
}

On my system, this returns

Waited 231.385us for 100 goroutines to stop

So what is going on here? As soon as the finish channel is closed, it becomes ready to receive. As all the goroutines are waiting to receive either from their time.After channel, or finish, the select statement is now complete and the goroutines exits after calling done.Done() to deincrement the WaitGroup counter. This powerful idiom allows you to use a channel to send a signal to an unknown number of goroutines, without having to know anything about them, or worrying about deadlock.

Before moving on to the next topic, I want to mention a final simplification that is preferred by many Go programmers. If you look at the sample program above, you’ll note that we never send a value on the finish channel, and the receiver always discards any value received. Because of this it is quite common to see the program written like this:

package main

import (
        "fmt"
        "sync"
        "time"
)

func main() {
        finish := make(chan struct{})
        var done sync.WaitGroup
        done.Add(1)
        go func() {
                select {
                case <-time.After(1 * time.Hour):
                case <-finish:
                }
                done.Done()
        }()
        t0 := time.Now()
        close(finish)
        done.Wait()
        fmt.Printf("Waited %v for goroutine to stop\n", time.Since(t0))
}

As the behaviour of the close(finish) relies on signalling the close of the channel, not the value sent or received, declaring finish to be of type chan struct{} says that the channel contains no value; we’re only interested in its closed property.

A nil channel always blocks

The second property I want to talk about is polar opposite of the closed channel property. A nil channel; a channel value that has not been initalised, or has been set to nil will always block. For example

package main

func main() {
        var ch chan bool
        ch <- true // blocks forever
}

will deadlock as ch is nil and will never be ready to send. The same is true for receiving

package main

func main() {
        var ch chan bool
        <- ch // blocks forever
}

This might not seem important, but is a useful property when you want to use the closed channel idiom to wait for multiple channels to close. For example

// WaitMany waits for a and b to close.
func WaitMany(a, b chan bool) {
        var aclosed, bclosed bool
        for !aclosed || !bclosed {
                select {
                case <-a:
                        aclosed = true
                case <-b:
                        bclosed = true
                }
        }
}

WaitMany() looks like a good way to wait for channels a and b to close, but it has a problem. Let’s say that channel a is closed first, then it will always be ready to receive. Because bclosed is still false the program can enter an infinite loop, preventing the channel b from ever being closed.

A safe way to solve the problem is to leverage the blocking properties of a nil channel and rewrite the program like this

package main

import (
        "fmt"
        "time"
)

func WaitMany(a, b chan bool) {
        for a != nil || b != nil {
                select {
                case <-a:
                        a = nil 
                case <-b:
                        b = nil
                }
        }
}

func main() {
        a, b := make(chan bool), make(chan bool)
        t0 := time.Now()
        go func() {
                close(a)
                close(b)
        }()
        WaitMany(a, b)
        fmt.Printf("waited %v for WaitMany\n", time.Since(t0))
}

In the rewritten WaitMany() we nil the reference to a or b once they have received a value. When a nil channel is part of a select statement, it is effectively ignored, so niling a removes it from selection, leaving only b which blocks until it is closed, exiting the loop without spinning.

Running this on my system gives

waited 54.912us for WaitMany

In conclusion, the simple properties of closed and nil channels are powerful building blocks that can be used to create highly concurrent programs that are simple to reason about.

What is the zero value, and why is it useful?

Let’s start with the Go language spec on the zero value.

When memory is allocated to store a value, either through a declaration or a call of make or new, and no explicit initialization is provided, the memory is given a default initialization. Each element of such a value is set to the zero value for its type: false for booleans, 0 for integers, 0.0 for floats, "" for strings, and nil for pointers, functions, interfaces, slices, channels, and maps. This initialization is done recursively, so for instance each element of an array of structs will have its fields zeroed if no value is specified.

This property of always setting a value to a known default is important for safety and correctness of your program, but can also make your Go programs simpler and more compact. This is what Go programmers talk about when they say “give your structs a useful zero value”.

Here is an example using sync.Mutex, which is designed to be usable without explicit initialization. The sync.Mutex contains two unexported integer fields. Thanks to the zero value those fields will be set to will be set to 0 whenever a sync.Mutex is declared.

package main

import "sync"

type MyInt struct {
        mu sync.Mutex
        val int
}

func main() {
        var i MyInt

        // i.mu is usable without explicit initialisation.
        i.mu.Lock()      
        i.val++
        i.mu.Unlock()
}

Another example of a type with a useful zero value is bytes.Buffer. You can decare a bytes.Buffer and start Reading or Writeing without explicit initialisation. Note that io.Copy takes an io.Reader as its second argument so we need to pass a pointer to b.

package main

import "bytes"
import "io"
import "os"

func main() {
        var b bytes.Buffer
        b.Write([]byte("Hello world"))
        io.Copy(os.Stdout, &b)
}

A useful property of slices is their zero value is nil. This means you don’t need to explicitly make a slice, you can just declare it.

package main

import "fmt"
import "strings"

func main() {
        // s := make([]string, 0)
        // s := []string{}
        var s []string

        s = append(s, "Hello")
        s = append(s, "world")
        fmt.Println(strings.Join(s, " "))
}

Note: var s []string is similar to the two commented lines above it, but not identical. It is possible to detect the difference between a slice value that is nil and a slice value that has zero length. The following code will output false.

package main

import "fmt"
import "reflect"

func main() {
        var s1 = []string{}
        var s2 []string
        fmt.Println(reflect.DeepEqual(s1, s2))
}

A surprising, but useful, property of nil pointers is you can call methods on types that have a nil value. This can be used to provide default values simply.

package main

import "fmt"

type Config struct {
        path string
}

func (c *Config) Path() string {
        if c == nil {
                return "/usr/home"
        }
        return c.path
}

func main() {
        var c1 *Config
        var c2 = &Config{
                path: "/export",
        }
        fmt.Println(c1.Path(), c2.Path())
}

With thanks to Jan MerclDoug LandauerStefan Nilsson, and Roger Peppe from the wonderful Go+ community for their feedback and suggestions.

Using screen for lazy dot files

I have a lot of shell accounts; on my laptops and workstations, on my ARM build boxes, on remote servers, and so on. I don’t make a lot of customisations to my login shell as the lowest common denominator of OS X, FreeBSD and various Linux distros has trained me to live with what is on the host.

I do have two exceptions, bash and screen1. Handling my screen config is easy; just scp my .screenrc file to the new host. Handling the small number of changes to my bash setup is more involved as bash has several places it looks in (.bashrc, .bash_profile, sometimes just .profile) and those files may already exist on the host.

Recently I’ve been experimenting with the idea of using screen to handle this customisation, which reduces the amount of configuration data copied to a new host to a single new file. Here is a sample .screenrc from an ARM FreeBSD build box.

startup_message off
vbell off

# Window list at the bottom.
hardstatus alwayslastline "%{wk}%-w%{Gk}[%n %t]%{wk}%+w%=%{Ck}%M%d %c%{-} %{=r} ${USER}@%H"

# who needs .bashrc ?
shell bash
setenv PS1 "\[\e]0;\u@\h: \w\a\]\h(\w) % "
setenv GOROOT /u/go                           
setenv GOPATH $HOME
# yup, screen can expand shell vars
setenv PATH $PATH:$GOROOT/bin:$GOPATH/bin 

autodetach on
term xterm-color
termcapinfo xterm ti@:te@

Combined with ssh $HOST -t -- screen -R -D, this makes setting up a new machine very simple.


1. Note to haters. I know that alternatives like zsh and tmux exist, but neither are installed by default on any mainstream distro, so until they are, I don’t care. At any rate, these suggestions probably apply equally well to your chosen shell and screen multiplexer.

Go, the language for emulators

So, I hear you like emulators. It turns out that Go is a great language for writing retro-computing emulators. Here are the ones that I have tried so far:

trs80 by Lawrence Kesteloot

I really liked this one because it avoids the quagmire of OpenGL or SDL dependencies and runs in your web browser. I had a little trouble getting it going so if you run into problems remember to execute the trs80 command in the source directory itself. If you’ve used go get github.com/lkesteloot/trs80 then it will be $GOPATH/src/github.com/lkesteloot/trs80.

trs80

GoSpeccy by Andrea Fazzi

GoSpeccy was the first emulator written in Go that I am aware of, Andrea has been quietly hacking away well before Go hit 1.0. I’ve even been able to get GoSpeccy running on a Raspberry Pi, X forwarded back to my laptop. Here is a screenshot running the Fire104b intro by Andrew Gerrand

GoSpeccy on a Raspberry Pi

Fergulator by Scott Ferguson

Like GoSpeccy, Fergulator shows the power of Go as a language for writing complex emulators, and the power of go get to handle packages with complex dependencies. Here are the two commands that took me from having no NES emulation on my laptop, to full NES emulation on my laptop.

lucky(~) sudo apt-get install libsdl1.2-dev libsdl-gfx1.2-dev libsdl-image1.2-dev libglew1.6-dev libxrandr-dev
lucky(~) % go get github.com/scottferg/Fergulator
Fergulator

sms by Andrea Fazzi

What’s this? Another emulator for Andrea Fazzi ? Why, yes it is. Again, super easy to install with go get -v github.com/remogatto/sms. Sadly there are no sample roms included with sms due to copyright restrictions, so no screenshot. Update: Andrea has included an open source ROM so we can have a screenshot.

sms

Update: Several Gophers from the wonderful Go+ community commented that there are still more emulators that I haven’t mentioned.