Tag Archives: error handling

Constant Time

This essay is a derived from my dotGo 2019 presentation about my favourite feature in Go.


Many years ago Rob Pike remarked,

“Numbers are just numbers, you’ll never see 0x80ULL in a .go source file”.

—Rob Pike, The Go Programming Language

Beyond this pithy observation lies the fascinating world of Go’s constants. Something that is perhaps taken for granted because, as Rob noted, is Go numbers–constants–just work.
In this post I intend to show you a few things that perhaps you didn’t know about Go’s const keyword.

What’s so great about constants?

To kick things off, why are constants good? Three things spring to mind:

  • Immutability. Constants are one of the few ways we have in Go to express immutability to the compiler.
  • Clarity. Constants give us a way to extract magic numbers from our code, giving them names and semantic meaning.
  • Performance. The ability to express to the compiler that something will not change is key as it unlocks optimisations such as constant folding, constant propagation, branch and dead code elimination.

But these are generic use cases for constants, they apply to any language. Let’s talk about some of the properties of Go’s constants.

A Challenge

To introduce the power of Go’s constants let’s try a little challenge: declare a constant whose value is the number of bits in the natural machine word.

We can’t use unsafe.SizeOf as it is not a constant expression. We could use a build tag and laboriously record the natural word size of each Go platform, or we could do something like this:

const uintSize = 32 << (^uint(0) >> 32 & 1)

There are many versions of this expression in Go codebases. They all work roughly the same way. If we’re on a 64 bit platform then the exclusive or of the number zero–all zero bits–is a number with all bits set, sixty four of them to be exact.

1111111111111111111111111111111111111111111111111111111111111111

If we shift that value thirty two bits to the right, we get another value with thirty two ones in it.

0000000000000000000000000000000011111111111111111111111111111111

Anding that with a number with one bit in the final position give us, the same thing, 1,

0000000000000000000000000000000011111111111111111111111111111111 & 1 = 1

Finally we shift the number thirty two one place to the right, giving us 641.

32 << 1 = 64

This expression is an example of a constant expression. All of these operations happen at compile time and the result of the expression is itself a constant. If you look in the in runtime package, in particular the garbage collector, you’ll see how constant expressions are used to set up complex invariants based on the word size of the machine the code is compiled on.

So, this is a neat party trick, but most compilers will do this kind of constant folding at compile time for you. Let’s step it up a notch.

Constants are values

In Go, constants are values and each value has a type. In Go, user defined types can declare their own methods. Thus, a constant value can have a method set. If you’re surprised by this, let me show you an example that you probably use every day.

const timeout = 500 * time.Millisecond
fmt.Println("The timeout is", timeout) // 500ms

In the example the untyped literal constant 500 is multiplied by time.Millisecond, itself a constant of type time.Duration. The rule for assignments in Go are, unless otherwise declared, the type on the left hand side of the assignment operator is inferred from the type on the right.500 is an untyped constant so it is converted to a time.Duration then multiplied with the constant time.Millisecond.

Thus timeout is a constant of type time.Duration which holds the value 500000000.
Why then does fmt.Println print 500ms, not 500000000?

The answer is time.Duration has a String method. Thus any time.Duration value, even a constant, knows how to pretty print itself.

Now we know that constant values are typed, and because types can declare methods, we can derive that constant values can fulfil interfaces. In fact we just saw an example of this. fmt.Println doesn’t assert that a value has a String method, it asserts the value implements the Stringer interface.

Let’s talk a little about how we can use this property to make our Go code better, and to do that I’m going to take a brief digression into the Singleton pattern.

Singletons

I’m generally not a fan of the singleton pattern, in Go or any language. Singletons complicate testing and create unnecessary coupling between packages. I feel the singleton pattern is often used not to create a singular instance of a thing, but instead to create a place to coordinate registration. net/http.DefaultServeMux is a good example of this pattern.

package http

// DefaultServeMux is the default ServeMux used by Serve.
var DefaultServeMux = &defaultServeMux

var defaultServeMux ServeMux

There is nothing singular about http.defaultServerMux, nothing prevents you from creating another ServeMux. In fact the http package provides a helper that will create as many ServeMux‘s as you want.

// NewServeMux allocates and returns a new ServeMux.
func NewServeMux() *ServeMux { return new(ServeMux) }

http.DefaultServeMux is not a singleton. Never the less there is a case for things which are truely singletons because they can only represent a single thing. A good example of this are the file descriptors of a process; 0, 1, and 2 which represent stdin, stdout, and stderr respectively.

It doesn’t matter what names you give them, 1 is always stdout, and there can only ever be one file descriptor 1. Thus these two operations are identical:

fmt.Fprintf(os.Stdout, "Hello dotGo\n")
syscall.Write(1, []byte("Hello dotGo\n"))

So let’s look at how the os package defines Stdin, Stdout, and Stderr:

package os

var (
        Stdin  = NewFile(uintptr(syscall.Stdin), "/dev/stdin")
        Stdout = NewFile(uintptr(syscall.Stdout), "/dev/stdout")
        Stderr = NewFile(uintptr(syscall.Stderr), "/dev/stderr")
)

There are a few problems with this declaration. Firstly their type is *os.File not the respective io.Reader or io.Writer interfaces. People have long complained that this makes replacing them with alternatives problematic. However the notion of replacing these variables is precisely the point of this digression. Can you safely change the value of os.Stdout once your program is running without causing a data race?

I argue that, in the general case, you cannot. In general, if something is unsafe to do, as programmers we shouldn’t let our users think that it is safe, lest they begin to depend on that behaviour.

Could we change the definition of os.Stdout and friends so that they retain the observable behaviour of reading and writing, but remain immutable? It turns out, we can do this easily with constants.

type readfd int

func (r readfd) Read(buf []byte) (int, error) {
       return syscall.Read(int(r), buf)
}

type writefd int

func (w writefd) Write(buf []byte) (int, error) {
        return syscall.Write(int(w), buf)
}

const (
        Stdin  = readfd(0)
        Stdout = writefd(1)
        Stderr = writefd(2)
)

func main() {
        fmt.Fprintf(Stdout, "Hello world")
}

In fact this change causes only one compilation failure in the standard library.2

Sentinel error values

Another case of things which look like constants but really aren’t, are sentinel error values. io.EOF, sql.ErrNoRows, crypto/x509.ErrUnsupportedAlgorithm, and so on are all examples of sentinel error values. They all fall into a category of expected errors, and because they are expected, you’re expected to check for them.

To compare the error you have with the one you were expecting, you need to import the package that defines that error. Because, by definition, sentinel errors are exported public variables, any code that imports, for example, the io package could change the value of io.EOF.

package nelson

import "io"

func init() {
        io.EOF = nil // haha!
}

I’ll say that again. If I know the name of io.EOF I can import the package that declares it, which I must if I want to compare it to my error, and thus I could change io.EOF‘s value. Historically convention and a bit of dumb luck discourages people from writing code that does this, but technically there is nothing to prevent you from doing so.

Replacing io.EOF is probably going to be detected almost immediately. But replacing a less frequently used sentinel error may cause some interesting side effects:

package innocent

import "crypto/rsa"

func init() {
        rsa.ErrVerification = nil // 🤔
}

If you were hoping the race detector will spot this subterfuge, I suggest you talk to the folks writing testing frameworks who replace os.Stdout without it triggering the race detector.

Fungibility

I want to digress for a moment to talk about the most important property of constants. Constants aren’t just immutable, its not enough that we cannot overwrite their declaration,
Constants are fungible. This is a tremendously important property that doesn’t get nearly enough attention.

Fungible means identical. Money is a great example of fungibility. If you were to lend me 10 bucks, and I later pay you back, the fact that you gave me a 10 dollar note and I returned to you 10 one dollar bills, with respect to its operation as a financial instrument, is irrelevant. Things which are fungible are by definition equal and equality is a powerful property we can leverage for our programs.

var myEOF = errors.New("EOF") // io/io.go line 38
fmt.Println(myEOF == io.EOF)  // false

Putting aside the effect of malicious actors in your code base the key design challenge with sentinel errors is they behave like singletons, not constants. Even if we follow the exact procedure used by the io package to create our own EOF value, myEOF and io.EOF are not equal. myEOF and io.EOF are not fungible, they cannot be interchanged. Programs can spot the difference.

When you combine the lack of immutability, the lack of fungibility, the lack of equality, you have a set of weird behaviours stemming from the fact that sentinel error values in Go are not constant expressions. But what if they were?

Constant errors

Ideally a sentinel error value should behave as a constant. It should be immutable and fungible. Let’s recap how the built in error interface works in Go.

type error interface {
        Error() string
}

Any type with an Error() string method fulfils the error interface. This includes user defined types, it includes types derived from primitives like string, and it includes constant strings. With that background, consider this error implementation:

type Error string

func (e Error) Error() string {
        return string(e)
}

We can use this error type as a constant expression:

const err = Error("EOF")

Unlike errors.errorString, which is a struct, a compact struct literal initialiser is not a constant expression and cannot be used.

const err2 = errors.errorString{"EOF"} // doesn't compile

As constants of this Error type are not variables, they are immutable.

const err = Error("EOF")
err = Error("not EOF")   // doesn't compile

Additionally, two constant strings are always equal if their contents are equal:

const str1 = "EOF"
const str2 = "EOF"
fmt.Println(str1 == str2) // true

which means two constants of a type derived from string with the same contents are also equal.

type Error string

const err1 = Error("EOF")
const err2 = Error("EOF")
fmt.Println(err1 == err2) // true```

Said another way, equal constant Error values are the same, in the way that the literal constant 1 is the same as every other literal constant 1.

Now we have all the pieces we need to make sentinel errors, like io.EOF, and rsa.ErrVerfication, immutable, fungible, constant expressions.

% git diff
diff --git a/src/io/io.go b/src/io/io.go
index 2010770e6a..355653b4b8 100644
--- a/src/io/io.go
+++ b/src/io/io.go
@@ -35,7 +35,12 @@ var ErrShortBuffer = errors.New("short buffer")
 // If the EOF occurs unexpectedly in a structured data stream,
 // the appropriate error is either ErrUnexpectedEOF or some other error
 // giving more detail.
-var EOF = errors.New("EOF")
+const EOF = ioError("EOF")
+
+type ioError string
+
+func (e ioError) Error() string { return string(e) }

This change is probably a bit of a stretch for the Go 1 contract, but there is no reason you cannot adopt a constant error pattern for your sentinel errors in the packages that you write.

In summary

Go’s constants are powerful. If you only think of them as immutable numbers, you’re missing out. Go’s constants let us compose programs that are more correct and harder to misuse.

Today I’ve outlined three ways to use constants that are more than your typical immutable number.

Now it’s over to you, I’m excited to see where you can take these ideas.

Eliminate error handling by eliminating errors

Go 2 aims to improve the overhead of error handling, but do you know what is better than an improved syntax for handling errors? Not needing to handle errors at all. Now, I’m not saying “delete your error handling code”, instead I’m suggesting changing your code so you don’t have as many errors to handle.

This article draws inspiration from a chapter in John Ousterhout’s, A philosophy of Software Design, “Define Errors Out of Existence”. I’m going to try to apply his advice to Go.


Here’s a function to count the number of lines in a file,

func CountLines(r io.Reader) (int, error) {
var (
br = bufio.NewReader(r)
lines int
err error
)

for {
_, err = br.ReadString('\n')
lines++
if err != nil {
break
}
}

if err != io.EOF {
return 0, err
}
return lines, nil
}

We construct a bufio.Reader, then sit in a loop calling the ReadString method, incrementing a counter until we reach the end of the file, then we return the number of lines read. That’s the code we wanted to write, instead CountLines is made more complicated by its error handling. For example, there is this strange construction:

                _, err = br.ReadString('\n')
lines++
if err != nil {
break
}

We increment the count of lines before checking the error—​that looks odd. The reason we have to write it this way is ReadString will return an error if it encounters an end-of-file—io.EOF—before hitting a newline character. This can happen if there is no trailing newline.

To address this problem, we rearrange the logic to increment the line count, then see if we need to exit the loop.1

But we’re not done checking errors yet. ReadString will return io.EOF when it hits the end of the file. This is expected, ReadString needs some way of saying stop, there is nothing more to read. So before we return the error to the caller of CountLine, we need to check if the error was not io.EOF, and in that case propagate it up, otherwise we return nil to say that everything worked fine. This is why the final line of the function is not simply

return lines, err

I think this is a good example of Russ Cox’s observation that error handling can obscure the operation of the function. Let’s look at an improved version.

func CountLines(r io.Reader) (int, error) {
sc := bufio.NewScanner(r)
lines := 0

for sc.Scan() {
lines++
}

return lines, sc.Err()
}

This improved version switches from using bufio.Reader to bufio.Scanner. Under the hood bufio.Scanner uses bufio.Reader adding a layer of abstraction which helps remove the error handling which obscured the operation of our previous version of CountLines 2

The method sc.Scan() returns true if the scanner has matched a line of text and has not encountered an error. So, the body of our for loop will be called only when there is a line of text in the scanner’s buffer. This means our revised CountLines correctly handles the case where there is no trailing newline, It also correctly handles the case where the file is empty.

Secondly, as sc.Scan returns false once an error is encountered, our for loop will exit when the end-of-file is reached or an error is encountered. The bufio.Scanner type memoises the first error it encounters and we recover that error once we’ve exited the loop using the sc.Err() method.

Lastly, buffo.Scanner takes care of handling io.EOF and will convert it to a nil if the end of file was reached without encountering another error.


My second example is inspired by Rob Pikes’ Errors are values blog post.

When dealing with opening, writing and closing files, the error handling is present but not overwhelming as, the operations can be encapsulated in helpers like ioutil.ReadFile and ioutil.WriteFile. However, when dealing with low level network protocols it often becomes necessary to build the response directly using I/O primitives, thus the error handling can become repetitive. Consider this fragment of a HTTP server which is constructing a HTTP/1.1 response.

type Header struct {
Key, Value string
}

type Status struct {
Code int
Reason string
}

func WriteResponse(w io.Writer, st Status, headers []Header, body io.Reader) error {
_, err := fmt.Fprintf(w, "HTTP/1.1 %d %s\r\n", st.Code, st.Reason)
if err != nil {
return err
}

for _, h := range headers {
_, err := fmt.Fprintf(w, "%s: %s\r\n", h.Key, h.Value)
if err != nil {
return err
}
}

if _, err := fmt.Fprint(w, "\r\n"); err != nil {
return err
}

_, err = io.Copy(w, body)
return err
}

First we construct the status line using fmt.Fprintf, and check the error. Then for each header we write the header key and value, checking the error each time. Lastly we terminate the header section with an additional \r\n, check the error, and copy the response body to the client. Finally, although we don’t need to check the error from io.Copy, we do need to translate it from the two return value form that io.Copy returns into the single return value that WriteResponse expects.

Not only is this a lot of repetitive work, each operation—fundamentally writing bytes to an io.Writer—has a different form of error handling. But we can make it easier on ourselves by introducing a small wrapper type.

type errWriter struct {
io.Writer
err error
}

func (e *errWriter) Write(buf []byte) (int, error) {
if e.err != nil {
return 0, e.err
}

var n int
n, e.err = e.Writer.Write(buf)
return n, nil
}

errWriter fulfils the io.Writer contract so it can be used to wrap an existing io.WritererrWriter passes writes through to its underlying writer until an error is detected. From that point on, it discards any writes and returns the previous error.

func WriteResponse(w io.Writer, st Status, headers []Header, body io.Reader) error {
ew := &errWriter{Writer: w}
fmt.Fprintf(ew, "HTTP/1.1 %d %s\r\n", st.Code, st.Reason)

for _, h := range headers {
fmt.Fprintf(ew, "%s: %s\r\n", h.Key, h.Value)
}

fmt.Fprint(ew, "\r\n")
io.Copy(ew, body)

return ew.err
}

Applying errWriter to WriteResponse dramatically improves the clarity of the code. Each of the operations no longer needs to bracket itself with an error check. Reporting the error is moved to the end of the function by inspecting the ew.err field, avoiding the annoying translation from io.Copy’s return values.


When you find yourself faced with overbearing error handling, try to extract some of the operations into a helper type.

Stack traces and the errors package

A few months ago I gave a presentation on my philosophy for error handling. In the talk I introduced a small errors package designed to support the ideas presented in the talk.

This post is an update to my previous blog post which reflects the changes in the errors package as I’ve put it into service in my own projects.

Wrapping and stack traces

In my April presentation I gave examples of using the Wrap function to produce an annotated error that could be unwrapped for inspection, yet mirrored the recommendations from Kernighan and Donovan’s book.

package main

import "fmt"
import "github.com/pkg/errors"

func main() {
        err := errors.New("error")
        err = errors.Wrap(err, "open failed")
        err = errors.Wrap(err, "read config failed")

        fmt.Println(err) // read config failed: open failed: error
}

Wraping an error added context to the underlying error and recorded the file and line that the error occurred. This file and line information could be retrieved via a helper function, Fprint, to give a trace of the execution path leading away from the error. More on that later.

However, when I came to integrate the errors package into my own projects, I found that using Wrap at each call site in the return path often felt redundant. For example:

func readconfig(file string) {
        if err := openfile(file); err != nil {
                return errors.Wrap(err, "read config failed")
        }
        // ...
}

If openfile failed it would likely annotate the error it returned with open failed, and that error would also include the file and line of the openfile function. Similarly, readconfig‘s wrapped error would be annotated with read config failed as well as the file and line of the call to errors.Wrap inside the readconfig function.

I realised that, at least in my own code, it is likely that the name of the function contains sufficient information to frequently make the additional context passed to Wrap redundant. But as Wrap requires a message, even if I had nothing useful to add, I’d still have to pass something:

if err != nil {
        return errors.Wrap(err, "") // ewww
}

I briefly considered making Wrap variadic–to make the second parameter optional–before realising that rather than forcing the user to manually annotate each stack frame in the return path, I can just record the entire stack trace at the point that an error is created by the errors package.

I believe that for 90% of the use cases, this natural stack trace–that is the trace collected at the point New or Errorf are called–is correct with respect to the information required to investigate the error’s cause. In the other cases, Wrap and Wrapf can be used to add context when needed.

This lead to a large internal refactor of the package to collect and expose this natural stack trace.

Fprint and Print have been removed

As mentioned earlier, the mechanism for printing not just the err.Error() text of an error, but also its stack trace, has also changed with feedback from early users.

The first attempts were a pair of functions; Print(err error), which printed the detailed error to os.Stderr, and Fprint(w io.Writer, err error) which did the same but allowed the caller to control the destination. Neither were very popular.

Print was removed in version 0.4.0 because it was just a wrapper around Fprint(os.Stderr, err) and was hard to test, harder to write an example test for, and didn’t feel like its three lines paid their way. However, with Print gone, users were unhappy that Fprint required you to pass an io.Writer, usually a bytes.Buffer, just to retrieve a string form of the error’s trace.

So, Print and Fprint were the wrong API. They were too opinionated, without it being a useful opinion. Fprint has been slowly gutted over the period of 0.5, 0.6 and now has been replaced with a much more powerful facility inspired by Chris Hines’ go-stack/stack package.

The errors package now leverages the powerful fmt.Formatter interface to allow it to customise its output when any error generated, or wrapped by this package, is passed to fmt.Printf. This extended format is activated by the %+v verb. For example,

func main() {
        err := parseArgs(os.Args[1:])
        fmt.Printf("%v\n", err)
}

Prints, as expected,

not enough arguments, expected at least 3, got 0

However if we change the formatting verb to %+v,

func main() {
        err := parseArgs(os.Args[1:])
        fmt.Printf("%+v\n", err)
}

the same error value now results in

not enough arguments, expected at least 3, got 0
main.parseArgs
        /home/dfc/src/github.com/pkg/errors/_examples/wrap/main.go:12
main.main
        /home/dfc/src/github.com/pkg/errors/_examples/wrap/main.go:18
runtime.main
        /home/dfc/go/src/runtime/proc.go:183
runtime.goexit
        /home/dfc/go/src/runtime/asm_amd64.s:2059

For those that need more control the Cause and StackTrace behaviours return values who have their own fmt.Formatter implementations. The latter is alias for a slice of Frame values which represent each frame in a call stack. Again, Frame implements several fmt.Formatter verbs that allow its output to be customised as required.

Putting it all together

With the changes to the errors package, some guidelines on how to use the package are in order.

  • In your own code, use errors.New or errors.Errorf at the point an error occurs.
    func parseArgs(args []string) error {
            if len(args) < 3 {
                    return errors.Errorf("not enough arguments, expected at least 3, got %d", len(args))
            }
            // ...
    }
  • If you receive an error from another function, it is often sufficient to simply return it.
    if err != nil {
           return err
    }
  • If you interact with a package from another repository, consider using errors.Wrap or errors.Wrapf to establish a stack trace at that point. This advice also applies when interacting with the standard library.
    f, err := os.Open(path)
    if err != nil {
            return errors.Wrapf(err, "failed to open %q", path)
    }
  • Always return errors to their caller rather than logging them throughout your program.
  • At the top level of your program, or worker goroutine, use %+v to print the error with sufficient detail.
    func main() {
            err := app.Run()
            if err != nil {
                    fmt.Printf("FATAL: %+v\n", err)
                    os.Exit(1)
            }
    }
  • If you want to exclude some classes of error from printing, use errors.Cause to unwrap errors before inspecting them.

Conclusion

The errors package, from the point of view of the four package level functions, New, Errorf, Wrap, and Wrapf, is done. Their API signatures are well tested, and now this package has been integrated into over 100 other packages, are unlikely to change at this point.

The extended stack trace format, %+v, is still very new and I encourage you to try it and leave feedback via an issue.

Don’t just check errors, handle them gracefully

This post is an extract from my presentation at the recent GoCon spring conference in Tokyo, Japan.


Don't just check errors, handle them gracefully

Errors are just values

I’ve spent a lot of time thinking about the best way to handle errors in Go programs. I really wanted there to be a single way to do error handling, something that we could teach all Go programmers by rote, just as we might teach mathematics, or the alphabet.

However, I have concluded that there is no single way to handle errors. Instead, I believe Go’s error handling can be classified into the three core strategies.

Sentinel errors

The first category of error handling is what I call sentinel errors.

if err == ErrSomething { … }

The name descends from the practice in computer programming of using a specific value to signify that no further processing is possible. So to with Go, we use specific values to signify an error.

Examples include values like io.EOF or low level errors like the constants in the syscall package, like syscall.ENOENT.

There are even sentinel errors that signify that an error did not occur, like go/build.NoGoError, and path/filepath.SkipDir from path/filepath.Walk.

Using sentinel values is the least flexible error handling strategy, as the caller must compare the result to predeclared value using the equality operator. This presents a problem when you want to provide more context, as returning a different error would will break the equality check.

Even something as well meaning as using fmt.Errorf to add some context to the error will defeat the caller’s equality test. Instead the caller will be forced to look at the output of the error‘s Error method to see if it matches a specific string.

Never inspect the output of error.Error

As an aside, I believe you should never inspect the output of the error.Error method. The Error method on the error interface exists for humans, not code.

The contents of that string belong in a log file, or displayed on screen. You shouldn’t try to change the behaviour of your program by inspecting it.

I know that sometimes this isn’t possible, and as someone pointed out on twitter, this advice doesn’t apply to writing tests. Never the less, comparing the string form of an error is, in my opinion, a code smell, and you should try to avoid it.

Sentinel errors become part of your public API

If your public function or method returns an error of a particular value then that value must be public, and of course documented. This adds to the surface area of your API.

If your API defines an interface which returns a specific error, all implementations of that interface will be restricted to returning only that error, even if they could provide a more descriptive error.

We see this with io.Reader. Functions like io.Copy require a reader implementation to return exactly io.EOF to signal to the caller no more data, but that isn’t an error.

Sentinel errors create a dependency between two packages

By far the worst problem with sentinel error values is they create a source code dependency between two packages. As an example, to check if an error is equal to io.EOF, your code must import the io package.

This specific example does not sound so bad, because it is quite common, but imagine the coupling that exists when many packages in your project export error values, which other packages in your project must import to check for specific error conditions.

Having worked in a large project that toyed with this pattern, I can tell you that the spectre of bad design–in the form of an import loop–was never far from our minds.

Conclusion: avoid sentinel errors

So, my advice is to avoid using sentinel error values in the code you write. There are a few cases where they are used in the standard library, but this is not a pattern that you should emulate.

If someone asks you to export an error value from your package, you should politely decline and instead suggest an alternative method, such as the ones I will discuss next.

Error types

Error types are the second form of Go error handling I want to discuss.

if err, ok := err.(SomeType); ok { … }

An error type is a type that you create that implements the error interface. In this example, the MyError type tracks the file and line, as well as a message explaining what happened.

type MyError struct {
        Msg string
        File string
        Line int
}

func (e *MyError) Error() string { 
        return fmt.Sprintf("%s:%d: %s”, e.File, e.Line, e.Msg)
}

return &MyError{"Something happened", “server.go", 42}

Because MyError error is a type, callers can use type assertion to extract the extra context from the error.

err := something()
switch err := err.(type) {
case nil:
        // call succeeded, nothing to do
case *MyError:
        fmt.Println(“error occurred on line:”, err.Line)
default:
// unknown error
}

A big improvement of error types over error values is their ability to wrap an underlying error to provide more context.

An excellent example of this is the os.PathError type which annotates the underlying error with the operation it was trying to perform, and the file it was trying to use.

// PathError records an error and the operation
// and file path that caused it.
type PathError struct {
        Op   string
        Path string
        Err  error // the cause
}

func (e *PathError) Error() string

Problems with error types

So the caller can use a type assertion or type switch, error types must be made public.

If your code implements an interface whose contract requires a specific error type, all implementors of that interface need to depend on the package that defines the error type.

This intimate knowledge of a package’s types creates a strong coupling with the caller, making for a brittle API.

Conclusion: avoid error types

While error types are better than sentinel error values, because they can capture more context about what went wrong, error types share many of the problems of error values.

So again my advice is to avoid error types, or at least, avoid making them part of your public API.

Opaque errors

Now we come to the third category of error handling. In my opinion this is the most flexible error handling strategy as it requires the least coupling between your code and caller.

I call this style opaque error handling, because while you know an error occurred, you don’t have the ability to see inside the error. As the caller, all you know about the result of the operation is that it worked, or it didn’t.

This is all there is to opaque error handling–just return the error without assuming anything about its contents. If you adopt this position, then error handling can become significantly more useful as a debugging aid.

import “github.com/quux/bar”

func fn() error {
        x, err := bar.Foo()
        if err != nil {
                return err
        }
        // use x
}

For example, Foo‘s contract makes no guarantees about what it will return in the context of an error. The author of Foo is now free to annotate errors that pass through it with additional context without breaking its contract with the caller.

Assert errors for behaviour, not type

In a small number of cases, this binary approach to error handling is not sufficient.

For example, interactions with the world outside your process, like network activity, require that the caller investigate the nature of the error to decide if it is reasonable to retry the operation.

In this case rather than asserting the error is a specific type or value, we can assert that the error implements a particular behaviour. Consider this example:

type temporary interface {
        Temporary() bool
}
 
// IsTemporary returns true if err is temporary.
func IsTemporary(err error) bool {
        te, ok := err.(temporary)
        return ok && te.Temporary()
}

We can pass any error to IsTemporary to determine if the error could be retried.

If the error does not implement the temporary interface; that is, it does not have a Temporary method, then then error is not temporary.

If the error does implement Temporary, then perhaps the caller can retry the operation if Temporary returns true.

The key here is this logic can be implemented without importing the package that defines the error or indeed knowing anything about err‘s underlying type–we’re simply interested in its behaviour.

Don’t just check errors, handle them gracefully

This brings me to a second Go proverb that I want to talk about; don’t just check errors, handle them gracefully. Can you suggest some problems with the following piece of code?

func AuthenticateRequest(r *Request) error {
        err := authenticate(r.User)
        if err != nil {
                return err
        }
        return nil
}

An obvious suggestion is that the five lines of the function could be replaced with

return authenticate(r.User)

But this is the simple stuff that everyone should be catching in code review. More fundamentally the problem with this code is I cannot tell where the original error came from.

If authenticate returns an error, then AuthenticateRequest will return the error to its caller, who will probably do the same, and so on. At the top of the program the main body of the program will print the error to the screen or a log file, and all that will be printed is: No such file or directory.
No such file or directory
There is no information of file and line where the error was generated. There is no stack trace of the call stack leading up to the error. The author of this code will be forced to a long session of bisecting their code to discover which code path trigged the file not found error.

Donovan and Kernighan’s The Go Programming Language recommends that you add context to the error path using fmt.Errorf

func AuthenticateRequest(r *Request) error {
        err := authenticate(r.User)
        if err != nil {
                return fmt.Errorf("authenticate failed: %v", err)
        }
        return nil
}

But as we saw earlier, this pattern is incompatible with the use of sentinel error values or type assertions, because converting the error value to a string, merging it with another string, then converting it back to an error with fmt.Errorf breaks equality and destroys any context in the original error.

Annotating errors

I’d like to suggest a method to add context to errors, and to do that I’m going to introduce a simple package. The code is online at github.com/pkg/errors. The errors package has two main functions:

// Wrap annotates cause with a message.
func Wrap(cause error, message string) error

The first function is Wrap, which takes an error, and a message and produces a new error.

// Cause unwraps an annotated error.
func Cause(err error) error

The second function is Cause, which takes an error that has possibly been wrapped, and unwraps it to recover the original error.

Using these two functions, we can now annotate any error, and recover the underlying error if we need to inspect it. Consider this example of a function that reads the content of a file into memory.

func ReadFile(path string) ([]byte, error) {
        f, err := os.Open(path)
        if err != nil {
                return nil, errors.Wrap(err, "open failed")
        } 
        defer f.Close()
 
        buf, err := ioutil.ReadAll(f)
        if err != nil {
                return nil, errors.Wrap(err, "read failed")
        }
        return buf, nil
}

We’ll use this function to write a function to read a config file, then call that from main.

func ReadConfig() ([]byte, error) {
        home := os.Getenv("HOME")
        config, err := ReadFile(filepath.Join(home, ".settings.xml"))
        return config, errors.Wrap(err, "could not read config")
}
 
func main() {
        _, err := ReadConfig()
        if err != nil {
                fmt.Println(err)
                os.Exit(1)
        }
}

If the ReadConfig code path fails, because we used errors.Wrap, we get a nicely annotated error in the K&D style.

could not read config: open failed: open /Users/dfc/.settings.xml: no such file or directory

Because errors.Wrap produces a stack of errors, we can inspect that stack for additional debugging information. This is the same example again, but this time we replace fmt.Println with errors.Print

func main() {
        _, err := ReadConfig()
        if err != nil {
                errors.Print(err)
                os.Exit(1)
        }
}

We’ll get something like this:

readfile.go:27: could not read config
readfile.go:14: open failed
open /Users/dfc/.settings.xml: no such file or directory

The first line comes from ReadConfig, the second comes from the os.Open part of ReadFile, and the remainder comes from the os package itself, which does not carry location information.

Now we’ve introduced the concept of wrapping errors to produce a stack, we need to talk about the reverse, unwrapping them. This is the domain of the errors.Cause function.

// IsTemporary returns true if err is temporary.
func IsTemporary(err error) bool {
        te, ok := errors.Cause(err).(temporary)
        return ok && te.Temporary()
}

In operation, whenever you need to check an error matches a specific value or type, you should first recover the original error using the errors.Cause function.

Only handle errors once

Lastly, I want to mention that you should only handle errors once. Handling an error means inspecting the error value, and making a decision.

func Write(w io.Writer, buf []byte) {
        w.Write(buf)
}

If you make less than one decision, you’re ignoring the error. As we see here, the error from w.Write is being discarded.

But making more than one decision in response to a single error is also problematic.

func Write(w io.Writer, buf []byte) error {
        _, err := w.Write(buf)
        if err != nil {
                // annotated error goes to log file
                log.Println("unable to write:", err)
 
                // unannotated error returned to caller
                return err
        }
        return nil
}

In this example if an error occurs during Write, a line will be written to a log file, noting the file and line that the error occurred, and the error is also returned to the caller, who possibly will log it, and return it, all the way back up to the top of the program.

So you get a stack of duplicate lines in your log file, but at the top of the program you get the original error without any context. Java anyone?

func Write(w io.Write, buf []byte) error {
        _, err := w.Write(buf)
        return errors.Wrap(err, "write failed")
}

Using the errors package gives you the ability to add context to error values, in a way that is inspectable by both a human and a machine.

Conclusion

In conclusion, errors are part of your package’s public API, treat them with as much care as you would any other part of your public API.

For maximum flexibility I recommend that you try to treat all errors as opaque. In the situations where you cannot do that, assert errors for behaviour, not type or value.

Minimise the number of sentinel error values in your program and convert errors to opaque errors by wrapping them with errors.Wrap as soon as they occur.

Finally, use errors.Cause to recover the underlying error if you need to inspect it.

Constant errors

This is a thought experiment about sentinel error values in Go.

Sentinel errors are bad, they introduce strong source and run time coupling, but are sometimes necessary. io.EOF is one of these sentinel values. Ideally a sentinel value should behave as a constant, that is it should be immutable and fungible.

The first problem is io.EOF is a public variable–any code that imports the io package could change the value of io.EOF. It turns out that most of the time this isn’t a big deal, but it could be a very confusing problem to debug.

fmt.Println(io.EOF == io.EOF) // true
x := io.EOF
fmt.Println(io.EOF == x)      // true
	
io.EOF = fmt.Errorf("whoops")
fmt.Println(io.EOF == io.EOF) // true
fmt.Println(x == io.EOF)      // false

The second problem is io.EOF behaves like a singleton, not a constant. Even if we follow the exact procedure used by the io package to create our own EOF value, they are not comparable.

err := errors.New("EOF")   // io/io.go line 38
fmt.Println(io.EOF == err) // false

Combine these properties and you have a set of weird behaviours stemming from the fact that sentinel error values in Go, those traditionally created with errors.New or fmt.Errorf, are not constants.

Constant errors

Before I introduce my solution, let’s recap how the error interface works in Go. Any type with an Error() string method fulfils the error interface. This includes primitive types like string, including constant strings.

With that background, consider this error implementation.

type Error string

func (e Error) Error() string { return string(e) }

It looks similar to the errors.errorString implementation that powers errors.New. However unlike errors.errorString this type is a constant expression.

const err = Error("EOF") 
const err2 = errorString{"EOF"} // const initializer errorString literal is not a constant

As constants of the Error type are not variables, they are immutable.

const err = Error("EOF") 
err = Error("not EOF") // error, cannot assign to err

Additionally, two constant strings are always equal if their contents are equal, which means two Error values with the same contents are equal.

const err = Error("EOF") 
fmt.Println(err == Error("EOF")) // true

Said another way, equal Error values are the same, in the way that the constant 1 is the same as every other constant 1.

const eof = Error("eof")

type Reader struct{}

func (r *Reader) Read([]byte) (int, error) {
        return 0, eof
}

func main() {
        var r Reader
        _, err := r.Read([]byte{})
        fmt.Println(err == eof) // true
}

Could we change the definition of io.EOF to be a constant? It turns out that this compiles just fine and passes all the tests, but it’s probably a stretch for the Go 1 contract.

However this does not prevent you from using this idiom in your own code. Although, you really shouldn’t be using sentinel errors anyway.

Error handling vs. exceptions redux

Revisiting my post about error handling and exceptions, written well before Go hit 1.0, I’m pleased that it stands the test of time.

Java has comprehensively demonstrated that checked exceptions (actually having both checked and unchecked exceptions) has been a disaster for the evolution of the language.

Checked exceptions have placed a suffocating yoke of backward compatibility on the architects trying to modernise Java’s decades old design.

I can see no future language designers making the same decision, no matter how well meaning, as the Java designers in 1995.

C++ exceptions, remain as difficult to use safely as they did three decades ago. When any part of your call stack can explode without warning, it is no wonder so many C++ shops mandate that exceptions not be used.

Where does this leave Go, with its sometimes long winded, but always predictable error values?

Two things

The first is an observation made by Rob Pike at Gophercon 2014

Error values in Go aren’t special, they are just values like any other, and so you have the entire language at your disposal.

I think this is something so fundamental that it escapes the notice of most Go programmers.

The second, which I ran across a close to a year after my first post, was this presentation by Andrei Alexandrescu, where he noted (around the 11 minute mark):

… exceptional code is hopelessly serial. There is only one exception in flight, how quaint is that ? There can be only one exception at any moment in flight. … [they] require immediate an exclusive attention. [The exception] comes to the fore, you must handle it right now.

To me this is the argument that seals the case in the favour of errors over exceptions.

Consider this simple example that consumes the contents of an io.Reader.

func ReadAll(r io.Reader) ([]byte, error) {
        var buf = make([]byte, 1024)
        var result []byte
        for {
                n, err := r.Read(buf)
                result = append(result, buf[:n]...)
                if err == io.EOF {
                        return result, nil
                }
                if err != nil {
                        return nil, err
                }
        }
}

In Go, handling any returned data, as well as an error, is second nature. I cannot begin to think of how you could handle this as simply in an exception based workflow.

Conclusion

Everything that I wrote then, nearly three years ago, I believe to be true today. So in conclusion, stealing a line from Churchill,

Returning error values is the worst form of error handling, except all the others that have been tried.