Category Archives: Small ideas

Containers versus Operating Systems

What does a distro provide?

The most popular docker base container image is either busybox, or scratch. This is driven by a movement that is equal parts puritanical and pragmatic. The puritan asks “Why do I need to run init(1) just to run my process?” The pragmatist asks “Why do I need a 700 meg base image to deploy my application?” And both, seeking immutable deployment units ask “Is it a good idea that I can ssh into my container?” But let’s step back for a second and look at the history of how we got to the point where questions like this are even a thing.

In the very beginnings, there were no operating systems. Programs ran one at a time with the whole machine at their disposal. While efficient, this created a problem for the keepers of these large and expensive machines. To maximise their investment, the time between one program finishing and another starting must be kept to an absolute minimum; hence monitor programs and batch processing was born.

Monitors started as barely more than watchdog timers. They knew how to load the next program off tape, then set an alarm if the program ran too long. As time went on, monitors became job control–quasi single user operating systems where the operators could schedule batch jobs with slightly more finesse than the previous model of concatenating them in the card reader.1

In response to the limitations of batch processing, and with the help of increased computing resources, interactive computing was born. Interactive computing allowing multiple users to interact with the computer directly, time slicing, or time sharing, the resources between users to present the illusion of each program having a whole computer to itself.

“The UNIX kernel is an I/O multiplexer more than a complete operating system. This is as it should be.” — Ken Thompson, BSTJ, 1978.

interactive computing in raw terms was less efficient than batch, however it recognised that the potential to deliver programs faster outweighed a less than optimal utilisation of the processor; a fact borne out by the realisation that programming time was not benefiting from the same economies of scale that Moore’s law was delivering for hardware. Job control evolved to became what we know as the kernel, a supervisor program which sits above the raw hardware, portioning it out and mediating access to hardware devices.

With interactive users came the shell, a place to start programs, and return once they completed. The shell presented an environment, a virtual work space to organise your work, communicate with others, and of course customise. Customise with programs you wrote, programs you got from others, and programs that you collaborated with your coworkers on.

Interactive computing, multi user systems and then networking gave birth to the first wave of client/server computing–the server was your world, your terminal was just a pane of glass to interact with it. Thus begat userspace, a crowded bazaar of programs, written in many languages, traded, sold, swapped and sometimes stolen. A great inter breeding between the UNIX vendors produced a whole far larger than the sum of its parts.

Each server was an island, lovingly tended by operators, living for years, slowly patched and upgraded, becoming ever more unique through the tide of software updates and personnel changes.

Skip forward to Linux and the GNU generation, a kernel by itself does not serve the market, it needs a user space of tools to attract and nurture users accustomed to the full interactive environment.

But that software was hard, and messy, and spread across a million ftp, tucows, sourceforge, and cvs servers. Their installation procedures are each unique, their dependencies are unknown or unmanaged–in short, a job for an expert. Thus distributions became experts at packaging open source software to work together as a coherent interactive userspace story.

Container sprawl

We used to just have lots of servers, drawing power, running old software, old operating systems, hidden under people’s desks, and sometimes left running behind dry wall. Along came virtualisation to sweep away all the old, slow, flaky, out of warranty hardware. Yet the software remained, and multiplied.

Vmsprawl, it was called. Now free from a purchase order and a network switch port, virtual machines could spawn faster than rabbits. But their lifespan would be much longer.

Back when physical hardware existed, you could put labels on things, assign them to operators, have someone to blame, or at least ask if the operating system was up to date, but virtual machines became ephemeral, multitudinous, and increasingly, redundant,

Now that a virtual machines’ virtual bulk has given way to containers, what does that mean for the security and patching landscape? Surely it’s as bad, if not worse. Containers can multiply even faster than VMs and at such little cost compared to their bloated cousins that the problem could be magnified many times over. Or will it?

The problem is maintaining the software you didn’t write. Before containers that was everything between you and the hardware; obviously a kernel, that is inescapable, but the much larger surface area (in recent years ballooning to a DVD’s girth) was the userland. The gigabytes of software that existed to haul the machine onto the network, initialise its device drivers, scrub its /tmp partition, and so on.

But what if there was no userland? What if the network was handled for you, truly virtualised at layer 3, not layer 1. Your volumes were always mounted and your local storage was fleeting, so nothing to scrub. What would be the purpose of all those decades of lovingly crafted userland cruft?

If interactive software goes unused, was it ever installed at all?

Immutable images

Netflix tells us that immutable images are the path to enlightenment. Built it once, deploy it often. If there is a problem, an update, a software change, a patch, or a kernel fix, then build another image and roll it out. Never change something in place. This mirrors the trend towards immutability writ large by the functional programming tidal wave.

While Netflix use virtual machines, and so need software to configure their (simulated) hardware and software to plumb their (simulated) network interfaces to get to the point of being able to launch the application, containers leave all these concerns to the host. A container is spawned with any block devices or network interfaces required already mounted or plumbed a priori.

So, if you remove the requirement, and increasingly, the ability, to change the contents of the running image, and you remove the requirement to prepare the environment before starting the application, because the container is created with all its facilities already prepared, why do you need a userland inside a container?

Debugging? Possibly.

Today there are many of my generation who would feel helpless without being about to ssh to a host, run their favourite (and disparate) set of inspection tools. But a container is just a process inside a larger host operating system, so do you diagnosis there instead. Unlike virtual machines, these are not black boxes, the host operating system has far more capability to inspect and diagnose a guest than the guest itself–so leave your diagnosis tools on the host. And your ssh daemon, for good measure.

Updates, updates. Updates!

Why do we have operating system distros? In a word, outsourcing.

Sure, every admin could subscribe to the mailing lists of all the software packages installed on the servers they maintain (you do know all the software installed on the machines you are responsible for, right?) and then download, test, certify, upgrade the software promptly after being notified. Sound’s simple. Any admin worth hiring should be able to do this.

Sure, assuming you can find an admin who wants to do this grunt work, and that they can keep up with the workload, and that they can service more than a few machines before they’re hopelessly chasing their tails.

No, of course not, we outsource this to operating system vendor. In return for using outdated versions of software, distros will centralise the triage, testing and preparation of upgrades and patches.

This is the reason that a distro and its package management tool of choice are synonymous. Without a tool to automate the dissemination, installation and upgrade of packaged software, distro vendors would have no value. And without someone to marshal unique snowflake open source software into a unified form, package management software would have no value.

No wonder that the revenue model for all open source distro vendors centers around tooling that automates the distribution of update packages.

The last laugh

Ironically, the last laugh in this tale may be the closed source operating system vendors. It was Linux and open source that destroyed the proprietary UNIX market after the first dot com crash.

Linux rode Moore’s law to become the server operating system for the internet, and made kings of the operating system distributors. But it’s Linux that is driving containers, at least in their current form, and Linux containers, or more specifically a program that communicates directly with the kernel syscall api inside a specially prepared process namespace, is defining the new normal for applications.

Containers are eating the very Linux distribution market which enabled their creation.

OSX and Windows may be relegated to second class citizens–the clients in the client/server or client/container equation–but at least nobody is asking difficult questions about the role of their userspace.

Whither distros

What is the future of operating system distributions? Their services, while mature, scalable, well integrated, and expertly staffed, will unfortunately be priced out of the market. History tells us this.

In the first dot com bust, companies retreated from expensive proprietary software, not because it wasn’t good, not because it wasn’t extensible or changeable, but because it was too expensive. With no money coming in, thousands of dollars of opex walking out the door in licence fees was unsustainable.

The companies that survived the crash, or were born in its wreckage, chose open source software. Software that was arguably less mature, less refined, at the time, but with a price tag that was much more approachable. They kept their investors money in the bank, rode the wave of hardware improvements, and by pulling together in a million loosely organised software projects created a free (as in free puppy) platform to build their services on top–trading opex for some risk that they may have no-one to blame if their free software balloon sprang a leak.

Now, it is the Linux distributors who are chasing the per seat or per cpu licence fees. Offering scaled out professional services in the form of a stream of software updates and patches, well tested and well integrated.

But, with the exception of the kernel–which is actually provided by the host operating system, not the container–all those patches and updates are for software that is not used by the container, and in the case of our opening examples, busybox and scratch. not present. The temptation to go it alone, cut out the distro vendors, backed by the savings in licence fees is overwhelming.

What can distros do?

What would you do if you woke up one day to find that you owned the best butchers shop in a town that had decided to become vegetarian en mass?

Never edit a method, always rewrite it

At a recent RubyConf, Chad Fowler presented his ideas for writing software systems that mirror the process of continual replacement observed in biological systems.

The first principal of this approach is, unsurprisingly, to keep the components of the software system small–just as complex organisms like human beings are constituted from billions of tiny cells which are constantly undergoing a process of renewal.

Following from that Fowler proposed this idea:

What would happen if you had a rule on your team that said you never edit a method after it was written, you only rewrote it again from scratch?

Fowler quickly walked back this suggestion as possibly not a good idea, nevertheless the idea has stuck in my head all day. What would happen if we developed software this way? What benefits could it bring?

  • Would it have benefits for software reuse? Opening up a method to add another branch condition or switch clause would become more expensive, and having rewritten the same function over and over again, the author might be tempted to make it more generalisable over a class of problems.
  • Would it have an impact on function complexity? If you knew that changing a long, complex, function required writing it again from scratch, would it encourage you to make is smaller? Perhaps you would pull non critical setup or checking logic into other functions to limit the amount you had to rewrite.
  • Would it have an impact on the tests you write? Some functions are truly complex, they contain a core algorithm that can’t be reduced any further. If you had to rewrite them, how would you know you got it right? Are there tests? Do they cover the edge cases? Are there benchmarks so you could ensure your version ran comparably to the previous?
  • Would it have an impact on the name you chose? Is the name of the current function sufficient to describe how to re-implement it? Would a comment help? Does the current comment give you sufficient guidance?

I agree with Fowler that the idea of immutable source code is likely unworkable. But even if you never actually followed this rule in practice, what would be the impact on the quality, reliability, and usability of your programs if you always wrote your functions with the mindset of it being immutable?


Note: Fowler talks about methods, because in Ruby, everything is a method. I prefer to talk about functions, because in Go, methods are a syntactic sugar over functions. For the purpose of this article, please treat functions and methods as interchangable.

Please, vote Yes for marriage equality in Australia

I wanted to write a few words about the postal survey on marriage law currently underway in Australia.

As an Australian, our country and our government do so many things that make me ashamed as a citizen; the poverty of our indigenous population, the inhumane treatment of refugees on Manus Island, and the maniacal desire to burn every last ounce of coal in the country, come hell and high water, to name just a few.

It is, quite frankly, overwhelming how institutionally cruel our government, which is after all a representation of the majority of Australians, can be, and nothing has sharpened this meanness to a point than the way the Liberal government have approached this survey.

With everything that is wrong in the world right now; climate change, the threat of nuclear war, and an unqualified narcissist running the White House, voting yes to the survey’s simple question is, quite literally, the smallest thing you could do to bring joy to two people.

So please, when you get your postal survey, vote yes.

Thank you.

Why I joined Heptio

Everyone gets the same set of tools

Something that had long puzzled me was the question “Why do some people [in the organisation] have root, and others do not?” It seemed to me that the reason the sysadmins had the root passwords, and everyone else had to raise tickets, was a tooling problem. Giving everyone root would permit anyone in the organisation to fix their own problems, deploy their own software, or, less charitably, cowboy things or be downright naughty. And while everyone had root, it usually turned out that only the operations team had the on call pager.

After the wholesale failure of organisations to understand Devops, I’m a big fan of the “You build it, you run it” movement. So when George Barnett and I built the Atlassian OnDemand Cloud we made a deliberate decision that everyone would get the same tools, and (modulo permissions and audit logs) be empowered to use the platform to the full extent. There wouldn’t be one set of tools for regular users, and a super set of “power tools” reserved for operators.

To me you build it, you run it, means if you have a problem, we’ll help you learn to use the tools better, not fix your problem for you.

Virtualise the operating system, not the hardware

I remember playing with VMware in 1999 or early 2000. I thought it was an amazing trick, especially as the drivers for my sound card worked way better in virtualized Windows than the real thing.

Fast forward a few years and I was using VMware to maintain a fleet of foreign language Windows installations for testing. Skip forward a few more and the industry had figured out that virtualisation was a solution to the sprawl of single use Windows servers that cluttered up wiring cupboards and data centres.

Virtualisation is a neat trick taken well beyond the point of a joke, but it did shine a light on the dark corners of systems administration. Back when turning up a server involved purchase orders, waiting for hardware to be shipped, contract negotiations, and trips to the data centre, what was a few hours spent installing the operating system? But when virtual hardware could be conjured out of thin air in seconds, it cast a long shadow over the need to automate operating system installation and management.

This was the age of Puppet and Chef, who re-plowed the ground sowed a decade earlier by CFEngine. Now sysadmins could configure and manage servers at the speed they could be virtually provisioned. I remember, thinking back to when I started to use Puppet, and imagining about what it would have been like to have those tools in previous jobs, where automation involved SVN repositories full of perl scripts, and crontab entries lovingly copy pasta’d between machines. And so everything was good for a time in the age of configuration as code.

But, simulating the entirety of an x86 host on another, just so people can share a computer, is a ridiculous waste. This shouldn’t be a surprise, FreeBSD Jails and Solaris Zones (rest in peace) had been coughing loudly about this for decades. Bryan Cantrill said it best when he exclaimed that we should “virtualise the operating system, not the hardware“, or as we’ve come to know them: containers.

The death of the operating system

I remember where I saw Docker for the first time. The product wasn’t even a year old and they were carpet bombing any meetup that would have them to promote it. Canonical were sprinting at a hotel near SFO and I convinced several of my teammates to squeeze into a taxi for the first meetup in San Mateo. What I saw that night shook me to my core. It wasn’t just the speed–oh the speed, after spending two years waiting for EC2 and slow apt mirrors–it was the clarity of that Californian mindset. What would happen if I checked my entire application deployment into git?

It was clear to me that night that virtual machines were virtualising stuff that people didn’t care about; virtual video cards, virtual floppy drives, virtual ram that swapped to virtual disks. What people wanted was a virtual kernel–their own pid 1. Orchestration tools like Chef, Puppet, and Juju were trying to orchestrate an entire operating system when what developers really wanted was a way to take a single program, the one that they had written, and deploy it to a server. Filesystems, crontabs, init/upstart/systemd, apt-get and dpkg-reconfigure, weren’t just someone else’s problem, they were irrelevant.

Anyone who’s endured to my rants about product knows my unwavering belief in the Innovator’s Dilemma. Through the window of Christensen’s logic, it was clear that the server orchestration market had been upended in that moment. Squeezed between Docker images at the low end and Netflix’s “everything is an AMI” model at the top end, was a large middle ground filled with orchestration tools that expected to be given a running operating system to configure. The Chefs and Puppets and whatnots would be desperately trying to convince the biggest orchestration users–the Netflixs of the world, with their CI/CD pipelines that pooped AMIs–to adopt agent based tooling, while all the while each developer faced with the question “How should I deploy my application?” would default to docker push.

Orchestration as table stakes

If you’re building your own orchestration layer, then you are betting on the wrong horse–I say this as someone who’s built a bespoke container based PaaS.

Within the next year or two you’ll be able to buy access to a Kubernetes API server at every price point; on your laptop, shared as a VPS, in your own VPC, or even as an appliance. Building on top of the Kubernetes primitives is where the value lies. Building on top of the shared tooling the Kubernetes API provides the level playing field that every development team who is responsible for supporting their own software in production is entitled to.

Why did I join Heptio? Because I believe that the administration of operating systems has reached its endgame. Kubernetes is going to revolutionise the way software is developed, and deployed, and I’m honoured to be given the opportunity to join the company that is going to make that happen.

Simplicity Debt

Fifteen years ago Python’s GIL wasn’t a big issue. Concurrency was something dismissed as probably unnecessary. What people really was needed was a faster interpreter, after all, who had more than one CPU? But, slowly, as the requirement for concurrency increased, the problems with the GIL increased.

By the time this decade rolled around, Node.js and Go had arrived on the scene, highlighting the need for concurrency as a first class concept. Various async contortions papered over the single threaded cracks of Python programs, but it was too late. Other languages had shown that concurrency must be a built-in facility, and Python had missed the boat.

When Go launched in 2009, it didn’t have a story for templated types. First we said they were important, but we didn’t know how to implement them. Then we argued that you probably didn’t need them, instead Go programmers should focus on interfaces, not types. Meanwhile Rust, Nim, Pony, Crystal, and Swift showed that basic templated types are a useful, and increasingly, expected feature of any language—just like concurrency.

There is no question that templated types and immutability are on their way to becoming mandatory in any modern programming language. But there is equally no question that adding these features to Go would make it more complex.

Just as efforts to improve Go’s dependency management situation have made it easier to build programs that consume larger dependency graphs, producing larger and more complex pieces of software, efforts to add templated types and immutability to the language would unlock the ability to write more complex, less readable software. Indeed, the addition of these features would have a knock on effect that would profoundly alter the way error handling, collections, and concurrency are implemented.

I have no doubt that adding templated types to Go will make it a more complicated language, just as I have no doubt that not adding them would be a mistake–lest Go find itself, like Python, on the wrong side of history. But, no matter how important and useful templated types and immutability would be, integrating them into a hypothetical Go 2 would decrease its readability and increase compilation times—two things which Go was designed to address. They would, in effect, impose a simplicity debt.

If you want generics, immutability, ownership semantics, option types, etc, those are already available in other languages. There is a reason Go programmers choose to program in Go, and I believe that reason stems from our core tenets of simplicity and readability. The question is, how can we pay down the cost in complexity of adding templated types or immutability to Go?

Go 2 isn’t here yet, but its arrival is a lot more certain than previously believed. As it stands now, generics or immutability can’t just be added to Go and still call it simple. As important as the discussions on how to add these features to Go 2 would be, equal weight must be given to the discussion of how to first offset their inherent complexity.

We have to build up a bankroll to spend on the complexity generics and immutability would add, otherwise Go 2 will start its life in simplicity debt.

Next: Simplicity Debt Redux

Go, without package scoped variables

This is a thought experiment, what would Go look like if we could no longer declare variables at the package level? What would be the impact of removing package scoped variable declarations, and what could we learn about the design of Go programs?

I’m only talking about expunging var, the other five top level declarations would still be permitted as they are effectively constant at compile time. You can, of course, continue to declare variables at the function or block scope.

Why are package scoped variables bad?

But first, why are package scoped variables bad? Putting aside the problem of globally visible mutable state in a heavily concurrent language, package scoped variables are fundamentally singletons, used to smuggle state between unrelated concerns, encourage tight coupling and makes the code that relies on them hard to test.

As Peter Bourgon wrote recently:

tl;dr: magic is bad; global state is magic → [therefore, you want] no package level vars; no func init.

Removing package scoped variables, in practice

To put this idea to the test I surveyed the most popular Go code base in existence; the standard library, to see how package scoped variables were used, and assessed the effect applying this experiment would have.

Errors

One of the most frequent uses of public package level var declarations are errors; io.EOF,
sql.ErrNoRowscrypto/x509.ErrUnsupportedAlgorithm, and so on. Removing the use of package scoped variables would remove the ability to use public variables for sentinel error values. But what could be used to replace them?

I’ve written previously that you should prefer behaviour over type or identity when inspecting errors. Where that isn’t possible, declaring error constants removes the potential for modification which retaining their identity semantics.

The remaining error variables are private declarations which give a symbolic name to an error message. These error values are unexported so they cannot be used for comparison by callers outside the package. Declaring them at the package level, rather than at the point they occur inside a function negates the opportunity to add additional context to the error. Instead I recommend using something like pkg/errors to capture a stack trace at the point the error occurs.

Registration

A registration pattern is followed by several packages in the standard library such as net/http, database/sql, flag, and to a lesser extent log. It commonly involves a package scoped private map or struct which is mutated by a public function—a textbook singleton.

Not being able to create a package scoped placeholder for this state would remove the side effects in the image, database/sql, and crypto packages to register image decoders, database drivers and cryptographic schemes. However, this is precisely the magic that Peter is referring to–importing a package for the side effect of changing some global state of your program is truly spooky action at a distance.

Registration also promotes duplicated business logic. The net/http/pprof package registers itself, via a side effect with net/http.DefaultServeMux, which is both a potential security issue—other code cannot use the default mux without exposing the pprof endpoints—and makes it difficult to convince the net/http/pprof package to register its handlers with another mux.

If package scoped variables were no longer used, packages like net/http/pprof could provide a function that registers routes on a supplied http.ServeMux, rather than relying on side effects to altering global state.

Removing the ability to apply the registry pattern would also solve the issues encountered when multiple copies of the same package are imported in the final binary and try to register themselves during startup.

Interface satisfaction assertions

The interface satisfaction idiom

var _ SomeInterface = new(SomeType)

occurred at least 19 times in the standard library. In my opinion these assertions are tests. They don’t need to be compiled, only to be eliminated, every time you build your package. Instead they should be moved to the corresponding _test.go file. But if we’re prohibiting package scoped variables, this prohibition also applies to tests, so how can we keep this test?

One option is to move the declaration from package scope to function scope, which will still fail to compile if SomeType stop implementing SomeInterface

func TestSomeTypeImplementsSomeInterface(t *testing.T) {
       // won't compile if SomeType does not implement SomeInterface
       var _ SomeInterface = new(SomeType)
}

But, as this is actually a test, it’s not hard to rewrite this idiom as a standard Go test.

func TestSomeTypeImplementsSomeInterface(t *testing.T) {
       var i interface{} = new(SomeType)
       if _, ok := i.(SomeInterface); !ok {
               t.Fatalf("expected %t to implement SomeInterface", i)
       }
}

As a side note, because the spec says that assignment to the blank identifier must fully evaluate the right hand side of the expression, there are probably a few suspicious package level initialisation constructs hidden in those var declarations.

It’s not all beer and skittles

The previous sections showed that avoiding package scoped variables might be possible, but there are some areas of the standard library which have proved more difficult to apply this idea.

Real singletons

While I think that the singleton pattern is generally overplayed, especially in its registration form, there are always some real singleton values in every program. A good example of this is  os.Stdout and friends.

package os 

var (
        Stdin  = NewFile(uintptr(syscall.Stdin), "/dev/stdin")
        Stdout = NewFile(uintptr(syscall.Stdout), "/dev/stdout")
        Stderr = NewFile(uintptr(syscall.Stderr), "/dev/stderr")
)

There are a few problems with this declaration. Firstly Stdin, Stdout, and Stderr are of type *os.File, not their respective io.Reader or io.Writer interfaces. This makes replacing them with alternatives problematic. However the notion of replacing them is exactly the kind of magic that this experiment seeks to avoid.

As the previous constant error example showed, we can retain the singleton nature of the standard IO file descriptors, such that packages like log and fmt can address them directly, but avoid declaring them as mutable public variables with something like this:

package main

import (
        "fmt"
        "syscall"
)

type readfd int

func (r readfd) Read(buf []byte) (int, error) {
        return syscall.Read(int(r), buf)
}

type writefd int

func (w writefd) Write(buf []byte) (int, error) {
        return syscall.Write(int(w), buf)
}

const (
        Stdin  = readfd(0)
        Stdout = writefd(1)
        Stderr = writefd(2)
)

func main() {
        fmt.Fprintf(Stdout, "Hello world")
}

Caches

The second most common use of unexported package scoped variables are caches. These come in two forms; real caches made out of maps (see the registration pattern above) and sync.Pool, and quasi constant variables that ameliorate the cost of a compilation.

As an example the crypto/ecsda package has a zr type whose Read method zeros any buffer passed to it. The package keeps a single instance of zr around because it is embedded in other structs as an io.Reader, potentially escaping to the heap each time it is instantiated.

package ecdsa 

type zr struct {
        io.Reader
}

// Read replaces the contents of dst with zeros.
func (z *zr) Read(dst []byte) (n int, err error) {
        for i := range dst {
                dst[i] = 0
        }
        return len(dst), nil
}

var zeroReader = &zr{}

However zr doesn’t embed an io.Reader, it is an io.Reader, so the unused zr.Reader field could be eliminated, giving zr a width of zero. In my testing this modified type can be created directly where it is used without performance regression.

        csprng := cipher.StreamReader{
                R: zr{},
                S: cipher.NewCTR(block, []byte(aesIV)),
        }

Perhaps some of the caching decision could be revisited as the inlining and escape analysis options available to the compiler have improved significantly since the standard library was first written.

Tables

The last major use of  common use of private package scoped variables is for tables, as seen in the unicode, crypto/*, and math packages. These tables either encode constant data in the form of arrays of integer types, or less commonly simple structs and maps.

Replacing package scoped variables with constants would require a language change along the lines of #20443. So, fundamentally, providing there is no way to modify those tables at run time, they are probably a reasonable exception to this proposal.

A bridge too far

Even though this post was just a thought experiment, it’s clear that forbidding all package scoped variables is too draconian to be workable as a language precept. Addressing the bespoke uses of private var usage may prove impractical from a performance standpoint, would be akin to pinning a “kick me” sign to ones back and inviting all the Go haters to take a free swing.

However, I believe there are a few concrete recommendations that can be drawn from this exercise, without going to the extreme of changing the language spec.

  • Firstly, public var declarations should be eschewed. This is not a controversial conclusion and not one that is unique to Go. The singleton pattern is discouraged, and an unadorned public variable that can be changed at any time by any party that knows its name should be a design, and concurrency, red flag.
  • Secondly, where public package var declarations are used, the type of those variables should be carefully constructed to expose as little surface area as possible. It should not be the default to take a type expected to be used on a per instance basis, and assign it to a package scoped variable.

Private variable declarations are more nuanced, but certain patterns can be observed:

  • Private variables with public setters, which I labelled registries, have the same effect on the overall program design as their public counterparts. Rather than registering dependencies globally, they should instead be passed in during declaration using a constructor function, compact literal, config structure, or option function.
  • Caches of []byte vars can often be expressed as consts at no performance cost.  Don’t forget the compiler is pretty good at avoiding string([]byte) conversions where they don’t escape the function call.
  • Private variables that hold tables, like the unicode package, are an unavoidable consequence of the lack of a constant array type. As long as they are unexported, and do not expose any way to mutate them, they can be considered effectively constant for the purpose of this discussion.

The bottom line; think long and hard about adding package scoped variables that are mutated during the operation of your program. It may be a sign that you’ve introduced magic global state.

Why Slack is inappropriate for open source communications

Full disclosure: my employer makes a Slack alternative. All my concerns about the use of Slack type chat services apply equally to its competitors, including my employer’s.


I’ve tweeted a few times about my frustration with the movement of open source projects from open, asynchronous, communication tools like forums, mailing lists, and issue trackers, to closed, synchronous communication services like Slack. This post is a long form version of my gripe.

The text of this post is also available in Russian (thank you Softdroid).

What is Slack good for?

Before I stick the boot in, let’s talk about the good things about synchronous chat applications like Slack, HipChat, and so on.

In a work context, chat applications take the place of @staff email blasts about fire system testing, broken lifts, and spontaneous availability of baked goods. This is a good thing as this kind of company spam is often impossible to unsubscribe from.

In the context of an open source project, Slack, HipChat, Gitter, etc, provide a forum for advocacy, gossip, informal discussion, and support. My complaints start when Slack and friends are promoted as the recommended way to communicate with the project.

Why is Slack bad for open source communication?

My complaint about the growing use of chat services like Slack, HipChat, and so on, for communication by open source projects is that these services are not open. As I see it there are two issues:

  1. Slack, et al, are paid services with closed memberships. Sure, there are lots of little apps running on Heroku dyno’s that automate the “send me an invite” process, but fundamentally these are closed systems.

    This means that the content inside those systems is closed. I cannot link to a discussion in a Slack channel in a tweet. I cannot refer to it in an issue report, and I cannot cite it in a presentation. Knowledge is silo’d to those who have the time and ability to participate in chat services in real time.

  2. Slack, et al, are based on synchronous communication, which discriminate against those who do not or can not take part of the conversation in real time. For example, real time chat discriminates against those who aren’t in the same time zone–you can’t participate fully in an open source project if all the discussion happens while you’re asleep.

    Even if you are in the same time zone, real time chat assumes a privilege that you have the spare time–or an employer who doesn’t mind you being constantly distracted–to be virtually present in a chat room. Online chat clients are resource hogs, and presume the availability of a fast computer and ample, always on, internet connection, again raising the bar for participation.

In my view these issues are inseparable. Calls to use IRC instead, miss the point that IRC is similarly real-time, just as efforts to create a post facto log of a Slack channel miss the fact that this is a record of a conversation which others cannot contribute equally. There is no solution for equitable open source communication that does not address both simultaneously.

Prefer asynchronous communication for open source projects

Instead of closed, synchronous, systems I recommend open source projects stick to asynchronous communication tools that leave a publicly linkable, searchable, url. The tools that fit this requirement best are; mailing list, issue trackers, and forums.

How to write a successful conference proposal

As an organiser of a large programming conference and a speaker who’s pitched talk ideas to many conferences, I’ve been on both sides of the selection process. Last month I published a piece on writing a proposal for GopherCon. I wanted to revisit that post in the form of more general advice to give some insight into the why, not just the how, of writing a good conference proposal.

Presentations and proposals are different things

Your talk and the proposal to give that talk are different because they target different audiences. The former is what you are going to present on stage, the latter is a pitch to the reviewers to let you give that presentation.

Writing a good conference proposal is a different skill than writing the presentation itself. This article is aimed at writing a good proposal with a focus on the reviewer of your proposal as the audience.

Focus on the audience

Speaking of audiences, good public speakers start planning a presentation by identifying the audience they want to address. Presenting at a conference is like teaching a class, you have to present the material at the level of the people in the room.

It’s not just a question of beginner, advanced, or expert, you also have to consider the kinds of people at the conference. If it’s a vendor conference, there are probably going to be lots of managers, (pre) sales people, and business decision makers in the audience. While they might also be competent engineers, they’re at that conference wearing their business leader hat. They want to hear a different story; reliability, ease of maintenance, or evidence of widespread adoption, than an audience of software engineers who are more likely interested in things such as performance, orthogonality, and extensiblity.

So, if a proposal is for the conference reviewers, not the audience, should you pitch the presentation at the reviewers? Well no, but you should focus on what the reviewers want.
Your goal in writing a proposal is to convince the reviewers that, as well as thinking about your idea, and how to present it, you’ve considered the people who will come to see your talk.

Who are the reviewers and what do they want?

For smaller conferences it’ll be the organiser, or organisers, of the conference you’re applying too. For larger conferences it will likely be a group of reviewers who the organisers have invited to review proposals, this is the model that GopherCon follows. For really large conferences, such as OSCON, they will have a group of reviewers per track who funnel their recommendations up to a programme chair or set of program coordinators.

Regardless of their size, conference reviewers are charged with recommending to the organisers a set of talks they think are interesting and appropriate for the audience of the conference.

Most review panels are confidential, so you shouldn’t know anything about the individual reviewers, although you can probably guess that they will be experienced in the subject of your conference.

Most proposal are reviewed anonymously, at least in the initial rounds. This means the reviewers must judge your proposal, and your ability to present it, using only the fields provided on the submission form.

it’s important to remember that at least in part, all conferences are commercial enterprises. Venue owners have bills to pay just like the rest of us, and at a minimum speakers need to be compensated for their travel and lodging, otherwise the programme will be filled with people who are paid by their employer to speak.

To put it bluntly, reviewers are looking for talks that people will pay to see. This might sound capitalistic, but it turns out that this is what the audience want as well. At GopherCon we cover the travel and accommodation expenses of all our speakers. We think this is important because we want to hear what the speaker thinks, not their marketing department.

All of these are factors that reviewers will be considering when reading your proposal.

What to put in a proposal

Almost every conference call for proposals will ask for the following; title, abstract, and description. They may ask for other things like a biography, questions about AV requirements for your talk, and so on, but with respect to successful acceptance, these three items are key.

Title

A title is mandatory on almost every talk submission system I’ve seen. It’s your one line elevator pitch to entice the audience to come to your talk.

Keeping the title a little vague, or quixotic is popular, but I tend to stay away from 11 things that will make your proposal sound like a buzzfeed article. I’m not saying never do that, but if you do, you’d better pack a heck of a proposal behind your braggadocios

Abstract

Conference organisers usually ask you to provide a talk abstract as they often don’t feel it is appropriate to summarise your proposal for you. This abstract will be printed in the program or placed on the website so potential visitors to the event know what they’ll be seeing.

There are usually restrictions on the size of the abstract. One sentence that describes the topic that you’ll be talking about, and one sentence that describes what the audience will take away from listening to your talk or participating in your workshop, is all you need.

Together with the title, these are the two pieces of information the eventual conference audience will use to decide if they want to come to your session or not.

Talk description

This is where you sell your talk idea, and this is the place as a reviewer, I have seen so many good proposals with interesting ideas fail to make the cut because they simply didn’t include enough detail.

This is where my advice differs from other’s you’ll read on the web. Many pieces of advice encourage you to write less in your description, sometimes out of recognition that the organisers are busy and you don’t wish to burden them. I wanted to take some time to explain why I push every speaker to write more detail.

You are looking to do three things when writing a description of your talk:

  1. Make it clear to the reviewers that you know what you are talking about.
  2. That you have a plan to communicate what you know to the audience and you’ve thought about how to do this within the time limit of the speaking slot.
  3. Answer all the selection criteria for the conference.

The first point is self explanatory, but you still need to make sure that you communicate this clearly to your reviewers. For example, if you’re talking about how to manage a large open source project, then make sure you mention that in the proposal, “as the maintainer of a large open source project”. If you plan to talk about a subject in the third person, then you should cite your sources, “for my PhD thesis I studied the day to day interactions of the top 10 projects on GitHub”. You don’t have to be an expert, but if your goal is to communicate something new to the audience, you should demonstrate that you know more about the topic than they do.

The second point relates to how likely you are to effectively communicate your ideas. The reviewers want to feel comfortable that you have a plan. It is all too common to see a proposal for an hour long session with only a sentence or two for the description. The less you write in a conference proposal, the more the reviewers are left to take it on faith that you’ll do a good job.

The opposite is also true. Occasionally I see a proposal for a talk that includes every possible aspect of a subject. Reviewers are generally wary that the speaker cannot cover all their material in the time available–few conferences can afford C++Con’s multi-part multi-hour format.

A presenter that doesn’t manage their time, rambles without conclusion, or covers a lot of material in common knowledge is going to waste the audience’s time. That’s not just unfair to the audience, but unfair on the speakers that follow who must deal with a disgruntled audience.

One thing that I recommend to anyone considering submitting a proposal is to include an outline of your talk in the proposal. This can be literally the headings of your slides, or your ideas in bullet points. As a reviewer this makes it crystal clear that you’ve not only thought about your idea, but how to present it.

The last point, address all the selection criteria, I cannot emphasise enough. Review committees strive to be fair and often rate all proposal by a common standard. It crucial to address the selection criteria clearly as these are the ground rules by which every proposal are judged.

This point is probably the trickiest as not all conferences publish their selection criteria. Sometimes conferences ask for talks along a particular theme and these can be substituted for criteria in a pinch. If there are no criteria available–don’t guess, ask the organisers. If they don’t have any to share, which can happen with smaller conferences, then think about the audience and the wider ecosystem of the conference’s focus and ask yourself “if I were thinking about coming to this conference, what would I like to hear about?”

If you take away one thing from this section it is this–proposals with less detail loose out to proposals that provide more–as they do not provide the reviewer with sufficient evidence to be confident in their recommendations.

Don’t sell snow to Eskimos

Before closing I want to highlight a very common mistake I see in both conference proposals, and conference presentations, which is a speaker selling their audience on a thing the audience already likes.

To give an example, you wouldn’t go to the JVM Language Summit and give a presentation about how great the JVM is and they should use it. Instead, you’d go to the JVM language summit and show the audience the JVM is great by telling them about your project which was only possible because you chose to base it on the JVM.

Don’t take my word for it

Finally, if you’ve read this far, I encourage you to read what others have written on the topic, especially where their advice differs.

Karolina Szczur recently wrote a great article on writing conference proposals and includes many references to similar articles for further reading.

Conclusion

Reviewers are looking to put together the best conference they can. They want to see your talk on stage, but you have to give them the evidence they need to feel confident in recommending you. Show the reviewers you’ve thought about the audience, and you’ll make their decision a lot simpler.

Thinking about $GOPATH

This is a short blog post about my thoughts on using Go in anger through several workplaces, as a developer and an advocate.

What is $GOPATH?

Back when Go was first announced we used Makefiles to compile Go code. These Makefiles referenced some shared logic stored in the Go distribution. This is where $GOROOT comes from.

Back then, if you wrote Go code, you’d probably also used these Makefiles, and while you could check out your source code anywhere, most people would put their own Go code in what today we’d call $GOROOT/src as you must’ve compiled Go from source, so this directory was always going to be present.

Towards the 1.0 release goinstall, then go get, solidified the use of domain names in import paths to provide a globally unique namespace. These tools introduced a new location into which Go code would be fetched. This location was separate from $GOROOT to make clear the distinction between code provided by the Go project, and code written by the developer. By the time Go 1.1 was released in 2013, $GOROOT was removed as a fallback option.

Why does $GOPATH exist?

$GOPATH exists for two main reasons:

  1. In Go, the import declaration references a package via its fully qualified import path. $GOPATH exist so that from any directory inside $GOPATH/src the go tool can compute the absolute import path of the package in question.1
  2. A location to store dependencies fetched by go get.

Having a per user $GOPATH environment variable also means developers could use the go tool from any directory on their system to build, test and install code, but I suspect only a minority utilise this feature.

What’s wrong with $GOPATH?

In my experience, many newcomers to Go are frustrated with the single workspace $GOPATH model. They are confused that $GOPATH doesn’t let them check out the source of a project in a directory of their choice like they are used to with other languages. Additionally, $GOPATH does not let the developer have more than one copy of a project (or its dependencies)  checked out at the same time without having to update $GOPATH constantly.

I think it is important to recognise that these issues are legitimate points of confusion for many newcomers (including those on the Go team) and act as a drag on Go adoption. As we’re on the cusp of a blessed dependency management tool for Go, I think it’s equally important to continue to question the base assumptions that this new tool will build on, namely requiring a $GOPATH.

In my opinion, any Go build tool needs to provide (in addition to actually building and testing code) a way for Go code checked out in an arbitrary location on disk to recover its intended fully qualified import path; the path other code will import it as.

The $GOPATH model answers this question by subtracting the prefix of $GOPATH/src from the path to the directory of the current package; the remainder is the package’s fully qualified import path. This is why if you check out a package outside a $GOPATH workspace, the go tool cannot figure out the packages’ fully qualified import path and everything falls apart.

What are some alternatives to $GOPATH?

I attempted to address both issues with gb, which gives developers the ability to check out a project anywhere you want, but has no solution for libraries, and gb projects were not go gettable. However gb showed that writing a new build tool that did not wrap the go tool meant it was not forced to reorganise the world to fit into the $GOPATH model allowing gb users to include the source of all their dependencies in their project without the pitfalls of the Go 1.6’s vendor/ directory.

Recently, on a suggestion from Bill Kennedy, I built an experimental build tool that recorded the expected import prefix in a manifest file. That prefix, rather than one computed by $GOPATH directory arithmetic, is used to determine the fully qualified import path.

I’m working on a similar tool (unfinished) based on a suggestion from Brad Fitzpatrick that uses the .git directory as a sentinel to determine the root of the project and hopefully infer the full import path from the git remote configuration.

While these experiments are unfinished, both demonstrate that you can avoid the $GOPATH restrictions and retain compatibility with the go get ecosystem. Potentially in the case of Kodos, even avoid a manifest file.

Conclusion

Kang and Kodos use a lot of forked code from gb, which I hope to rectify over the new years’ break. If you are interesting in contributing or better yet, building your own Go tool to explore this problem space, Kang, Kodos, and gb are permissively licensed.


Notes:

  1. This is notably different from the way imports work in scripting languages like Python and Ruby, which use directly scanning and inserting onto a global search path source code directories.

Do not fear first class functions

This is the text of my dotGo 2016 presentation. A recording and slide deck are also available.


firstclass-functions-763

Hello, welcome to dotGo.

Two years ago I stood on a stage, not unlike this one, and told you my opinion for how configuration options should be handled in Go. The cornerstone of my presentation was Rob Pike’s blog post, Self-referential functions and the design of options.

Since then it has been wonderful to watch this idea mature from Rob’s original blog post, to the gRPC project, who in my opinion have continued to evolve this design pattern into its best form to date.

But, when talking to Gophers at a conference in London a few months ago, several of them expressed a concern that while they understood the notion of a function that returns a function, the technique that powers functional options, they worried that other Go programmers—I suspect they meant less experienced Go programmers—wouldn’t be able to understand this style of programming.

And this made me a bit sad because I consider Go’s support of first class functions to be a gift, and something that we should all be able to take advantage of. So I’m here today to show you, that you do not need to fear first class functions.

Functional options recap

To begin, I’ll very quickly recap the functional options pattern

type Config struct{ ... }

func WithReticulatedSplines(c *Config) { ... }

type Terrain struct {
        config Config
}

func NewTerrain(options ...func(*Config)) *Terrain {
        var t Terrain
        for _, option := range options {
                option(&t.config)
        }
        return &t

}

func main() {
        t := NewTerrain(WithReticulatedSplines)
        // [ simulation intensifies ]
}

We start with some options, expressed as functions which take a pointer to a structure to configure. We pass those functions to a constructor, and inside the body of that constructor each option function is invoked in order, passing in a reference to the Config value. Finally, we call NewTerrain with the options we want, and away we go.

Okay, everyone should be familiar with this pattern. Where I believe the confusion comes from, is when you need an option function which take a parameter. For example, we have WithCities, which lets us add a number of cities to our terrain model.

 // WithCities adds n cities to the Terrain model
func WithCities(n int) func(*Config) { ... }

func main() {        
        t := NewTerrain(WithCities(9))      
        // ...
}

Because WithCities takes an argument, we cannot simply pass WithCities to NewTerrain, its signature does not match. Instead we evaluate WithCities, passing in the number of cities to create, and use the result as the value to pass to NewTerrain.

Functions as first class values

What’s going on here? Let’s break it down. Fundamentally, evaluating a function returns a value. We have functions that take two numbers and return a number.

package math

func Min(a, b float64) float64

We have functions that take a slice, and return a pointer to a structure.

package bytes

func NewReader(b []byte) *Reader

and now we have a function which returns a function.

func WithCities(n int) func(*Config)

The type of the value that is returned from WithCities is a function which takes a pointer to a Config. This ability to treat functions as regular values leads to their name: first class functions.

interface.Apply

Another way to think about what is going on here is to try to rewrite the functional option pattern using an interface.

type Option interface {
        Apply(*Config)
}

Rather than a function type we declare an interface, we’ll call it Option, and give it a single method, Apply which takes a pointer to a Config.

func NewTerrain(options ...Option) *Terrain {
        var config Config
        for _, option := range options {
                option.Apply(&config)
        }
        // ...
}

Whenever we call NewTerrain we pass in one or more values that implement the Option interface. Inside NewTerrain, just as before, we loop over the slice of options and call the Apply method on each.

This doesn’t look too different to the previous example. Rather than ranging over a slice of functions and calling them, we range over a slice of interface values and call a method on each. Let’s take a look at the other side, declaring the WithReticulatedSplines option.

type splines struct{}

func (s *splines) Apply(c *Config) { ... }

func WithReticulatedSplines() Option {
        return new(splines)
}

Because we’re passing around interface implementations, we need to declare a type to hold the Apply method. We also need to declare a constructor function to return our splines option implementation–you can already see that this is going to be more code.

To write WithCities using our Option interface we need to do a bit more work.

type cities struct {
        cities int
}

func (c *cities) Apply(c *Config) { ... }

func WithCities(n int) Option {
        return &cities{
                cities: n,
        }
}

In the previous, functional, version the value of n, the number of cities to create, was captured lexically for us in the declaration of the anonymous function. Because we’re using an interface we need to declare a type to hold the count of cities and we need a constructor to assign the field during construction.

func main() {
        t := NewTerrain(WithReticulatedSplines(), WithCities(9))
        // ...
}

Putting it all together, we call NewTerrain with the results of evaluating WithReticulatedSplines and WithCities.

At GopherCon last year Tomás Senart spoke about the duality of a first class function and an interface with one method. You can see this duality play out in our example; an interface with one method and a function are equivalent.

But, you can also see that using functions as first class values involves much less code.

Encapsulating behaviour

Let’s leave interfaces for a moment and talk about some other properties of first class functions.

When we invoke a function or a method, we do so passing around data. The job of that function is often to interpret that data and take some action. Function values allow you to pass behaviour to be executed, rather that data to be interpreted. In effect, passing a function value allows you to declare code that will execute later, perhaps in a different context.

To illustrate this, here is a simple calculator.

type Calculator struct {
        acc float64
}

const (
        OP_ADD = 1 << iota
        OP_SUB
        OP_MUL
)

It has a set of operations it understands.

func (c *Calculator) Do(op int, v float64) float64 {
        switch op {
        case OP_ADD:
                c.acc += v
        case OP_SUB:
                c.acc -= v
        case OP_MUL:
                c.acc *= v
        default:
                panic("unhandled operation")
        }
        return c.acc
}

It has one method, Do, which takes an operation and an operand, v. For convenience, Do also returns the value of the accumulator after the operation is applied.

func main() {
        var c Calculator
        fmt.Println(c.Do(OP_ADD, 100))     // 100
        fmt.Println(c.Do(OP_SUB, 50))      // 50
        fmt.Println(c.Do(OP_MUL, 2))       // 100
}

Our calculator only knows how to add, subtract, and multiply. If we wanted to implement division, we’d have to allocate an operation constant, then open up the Do method and add the code to implement division. Sounds reasonable, it’s only a few lines, but what if we wanted to add square root and exponentiation?

Each time we did this, Do grows longer and become harder to follow, because each time we add an operation we have to encode into Do knowledge of how to interpret that operation.

Let’s rewrite our calculator a little.

type Calculator struct {
        acc float64
}

type opfunc func(float64, float64) float64

func (c *Calculator) Do(op opfunc, v float64) float64 {
        c.acc = op(c.acc, v)
        return c.acc
}

As before we have a Calculator, which manages its own accumulator. The Calculator has a Do method, which this time takes an function as the operation, and a value as the operand. Whenever Do is called, it calls the operation we pass in, using its own accumulator and the operand we provide.

So, how do we use this new Calculator? You guessed it, by writing our operations as functions.

func Add(a, b float64) float64 { return a + b }

This is the code for Add. What about the other operations? It turns out they aren’t too hard either.

func Sub(a, b float64) float64 { return a - b }
func Mul(a, b float64) float64 { return a * b }

func main() {
        var c Calculator
        fmt.Println(c.Do(Add, 5))       // 5
        fmt.Println(c.Do(Sub, 3))       // 2
        fmt.Println(c.Do(Mul, 8))       // 16
}

As before we construct a Calculator and call it passing operations and an operand.

Extending the calculator

Now we can describe operations as functions, we can try to extend our calculator to handle square root.

func Sqrt(n, _ float64) float64 {
        return math.Sqrt(n)
}

But, it turns out there is a problem. math.Sqrt takes one argument, not two. However our Calculator’s Do method’s signature requires an operation function that takes two arguments.

func main() {
        var c Calculator
        c.Do(Add, 16)
        c.Do(Sqrt, 0) // operand ignored
}

Maybe we just cheat and ignore the operand. That’s a bit gross, I think we can do better.

Let’s redefine Add from a function that is called with two values and returns a third, to a function which returns a function that takes a value and returns a value.

func Add(n float64) func(float64) float64 {
        return func(acc float64) float64 {
                return acc + n
        }
}

func (c *Calculator) Do(op func(float64) float64) float64 {
        c.acc = op(c.acc)
        return c.acc
}

Do now invokes the operation function passing in its own accumulator and recording the result back in the accumulator.

func main() {
        var c Calculator
        c.Do(Add(10))   // 10
        c.Do(Add(20))   // 30
}

Now in main we call Do not with the Add function itself, but with the result of evaluating Add(10). The type of the result of evaluating Add(10) is a function which takes a value, and returns a value, matching the signature that Do requires.

func Sub(n float64) func(float64) float64 {
        return func(acc float64) float64 {
                return acc - n
        }
}

func Mul(n float64) func(float64) float64 {
        return func(acc float64) float64 {
                return acc * n
        }
}

Subtraction and multiplication are similarly easy to implement. But what about square root?

func Sqrt() func(float64) float64 {
        return func(n float64) float64 {
                return math.Sqrt(n)
        }
}

func main() {
        var c Calculator
        c.Do(Add(2))
        c.Do(Sqrt())   // 1.41421356237
}

This implementation of square root avoids the awkward syntax of the previous calculator’s operation function, as our revised calculator now operates on functions which take and return only one value.

Hopefully you’ve noticed that the signature of our Sqrt function is the same as math.Sqrt, so we can make this code smaller by reusing any function from the math package that takes a single argument.

func main() {
        var c Calculator
        c.Do(Add(2))      // 2
        c.Do(math.Sqrt)   // 1.41421356237
        c.Do(math.Cos)    // 0.99969539804
}

We started with a model of hard coded, interpreted logic. We moved to a more functional model, where we pass in the behaviour we want. Then, by taking it a step further, we generalised our calculator to work for operations regardless of their number of arguments.

Let’s talk about actors

photofunia-1475700854-1253
Let’s change tracks a little and talk about why most of us are here at a Go conference; concurrency, specifically actors. To give due credit, the examples here are inspired by Bryan Boreham’s talk from GolangUK, you should check it out.

Suppose we’re building a chat server, we plan to be the next Hipchat or Slack, but we’ll start small for the moment.

type Mux struct {
        mu    sync.Mutex
        conns map[net.Addr]net.Conn
}

func (m *Mux) Add(conn net.Conn) {
        m.mu.Lock()
        defer m.mu.Unlock()
        m.conns[conn.RemoteAddr()] = conn
}

We have a way to register new connections.

func (m *Mux) Remove(addr net.Addr) {
        m.mu.Lock()
        defer m.mu.Unlock()
        delete(m.conns, addr)
}

Remove old connections.

func (m *Mux) SendMsg(msg string) error {
        m.mu.Lock()
        defer m.mu.Unlock()
        for _, conn := range m.conns {
                err := io.WriteString(conn, msg)
                if err != nil {
                        return err
                }
        }
        return nil
}

And a way to send a message to all the registered connections. Because this is a server, all of these methods will be called concurrently, so we need to use a mutex to protect the conns map and prevent data races. Is this what you’d call idiomatic Go code?

Don’t communicate by sharing memory, share memory by communicating.

Our first proverb–don’t mediate access to shared memory with locks and mutexes, instead share that memory by communicating. So let’s apply this advice to our chat server.

Rather than using a mutex to serialise access to the Mux‘s conns map, we can give that job to a goroutine, and communicate with that goroutine via channels.

type Mux struct {
        add     chan net.Conn
        remove  chan net.Addr
        sendMsg chan string
}

func (m *Mux) Add(conn net.Conn) {
        m.add <- conn
}

Add sends the connection to add to the add channel.

func (m *Mux) Remove(addr net.Addr) {
        m.remove <- addr
}

Remove sends the address of the connection to the remove channel.

func (m *Mux) SendMsg(msg string) error {
        m.sendMsg <- msg
        return nil
}

And send message sends the message to be transmitted to each connection to the sendMsg channel.

func (m *Mux) loop() {
        conns := make(map[net.Addr]net.Conn)
        for {
                select {
                case conn := <-m.add:
                        m.conns[conn.RemoteAddr()] = conn
                case addr := <-m.remove:
                        delete(m.conns, addr)
                case msg := <-m.sendMsg:
                        for _, conn := range m.conns {
                                io.WriteString(conn, msg)
                        }
                }
        }
}

Rather than using a mutex to serialise access to the conns map, loop will wait until it receives an operation in the form of a value sent over one of the add, remove, or sendMsg channels and apply the relevant case. We don’t need a mutex anymore because the shared state, our conns map, is local to the loop function.

But, there’s still a lot of hard coded logic here. loop only knows how to do three things; add, remove and broadcast a message. As with the previous example, adding new features to our Mux type will involve:

  • creating a channel.
  • adding a helper to send the data over the channel.
  • extending the select logic inside loop to process that data.

Just like our Calculator example we can rewrite our Mux to use first class functions to pass around behaviour we want to executed, not data to interpret. Now, each method sends an operation to be executed in the context of the loop function, using our single ops channel.

type Mux struct {
        ops chan func(map[net.Addr]net.Conn)
}

func (m *Mux) Add(conn net.Conn) {
        m.ops <- func(m map[net.Addr]net.Conn) {
                m[conn.RemoteAddr()] = conn
        }
}

In this case the signature of the operation is a function which takes a map of net.Addr’s to net.Conn’s. In a real program you’d probably have a much more complicated type to represent a client connection, but it’s sufficient for the purpose of this example.

func (m *Mux) Remove(addr net.Addr) {
        m.ops <- func(m map[net.Addr]net.Conn) {
                delete(m, addr)
        }
}

Remove is similar, we send a function that deletes its connection’s address from the supplied map.

func (m *Mux) SendMsg(msg string) error {
        m.ops <- func(m map[net.Addr]net.Conn) {
                for _, conn := range m {
                        io.WriteString(conn, msg)
                }
        }
        return nil
}

SendMsg is a function which iterates over all connections in the supplied map and calls io.WriteString to send each a copy of the message.

func (m *Mux) loop() {

        conns := make(map[net.Addr]net.Conn)
        for op := range m.ops {
                op(conns)
        }
}

You can see that we’ve moved the logic from the body of loop into anonymous functions created by our helpers. So the job of loop is now to create a conns map, wait for an operation to be provided on the ops channel, then invoke it, passing in its map of connections.

But there are a few problems still to fix. The most pressing is the lack of error handling in SendMsg; an error writing to a connection will not be communicated back to the caller. So let’s fix that now.

func (m *Mux) SendMsg(msg string) error {
        result := make(chan error, 1)
        m.ops <- func(m map[net.Addr]net.Conn) {
                for _, conn := range m.conns {
                        err := io.WriteString(conn, msg)
                        if err != nil {
                                result <- err
                                return
                        }
                }
                result <- nil
        }
        return <-result
}

To handle the error being generated inside the anonymous function we pass to loop we need to create a channel to communicate the result of the operation. This also creates a point of synchronisation, the last line of SendMsg blocks until the function we passed into loop has been executed.

func (m *Mux) loop() {
        conns := make(map[net.Addr]net.Conn)
        for op := range m.ops {
                op(conns)
        }
}

Note that we didn’t have the change the body of loop at all to incorporate this error handling. And now we know how to do this, we can easily add a new function to Mux to send a private message to a single client.

func (m *Mux) PrivateMsg(addr net.Addr, msg string) error {
        result := make(chan net.Conn, 1)
        m.ops <- func(m map[net.Addr]net.Conn) {
                result <- m[addr]
        }
        conn := <-result
        if conn == nil {
                return errors.Errorf("client %v not registered", addr)
        }
        return io.WriteString(conn, msg)
}

To do this we pass a “lookup function” to loop via the ops channel, which will look in the map provided to it—this is loop‘s conns map—and return the value for the address we want on the result channel.

In the rest of the function we check to see if the result was nil—the zero value from the map lookup implies that the client is not registered. Otherwise we now have a reference to the client and we can call io.WriteString to send them a message.

And just to reiterate, we did this all without changing the body of loop, or affecting any of the other operations.

Conclusion

In summary

  • First class functions bring you tremendous expressive power. They let you pass around behaviour, not just dead data that must be interpreted.
  • First class functions aren’t new or novel. Many older languages have offered them, even C. In fact it was only somewhere along the lines of removing pointers did programmers in the OO stream of languages lose access to first class functions. If you’re a Javascript programmer, you’ve probably spent the last 15 minutes wondering what the big deal is.
  • First class functions, like the other features Go offers, should be used with restraint. Just as it is possible to make an overcomplicated program with the overuse of channels, it’s possible to make an impenetrable program with an overuse of first class functions. But that does not mean you shouldn’t use them at all; just use them in moderation.
  • First class functions are something that I believe every Go programmer should have in their toolbox. First class functions aren’t unique to Go, and Go programmers shouldn’t be afraid of them.
  • If you can learn to use interfaces, you can learn to use first class functions. They aren’t hard, just a little unfamiliar, and unfamiliarity is something that I believe can be overcome with time and practice.

So next time you define an API that has just one method, ask yourself, shouldn’t it really just be a function?