Go, without package scoped variables

This is a thought experiment, what would Go look like if we could no longer declare variables at the package level? What would be the impact of removing package scoped variable declarations, and what could we learn about the design of Go programs?

I’m only talking about expunging var, the other five top level declarations would still be permitted as they are effectively constant at compile time. You can, of course, continue to declare variables at the function or block scope.

Why are package scoped variables bad?

But first, why are package scoped variables bad? Putting aside the problem of globally visible mutable state in a heavily concurrent language, package scoped variables are fundamentally singletons, used to smuggle state between unrelated concerns, encourage tight coupling and makes the code that relies on them hard to test.

As Peter Bourgon wrote recently:

tl;dr: magic is bad; global state is magic → [therefore, you want] no package level vars; no func init.

Removing package scoped variables, in practice

To put this idea to the test I surveyed the most popular Go code base in existence; the standard library, to see how package scoped variables were used, and assessed the effect applying this experiment would have.

Errors

One of the most frequent uses of public package level var declarations are errors; io.EOF,
sql.ErrNoRowscrypto/x509.ErrUnsupportedAlgorithm, and so on. Removing the use of package scoped variables would remove the ability to use public variables for sentinel error values. But what could be used to replace them?

I’ve written previously that you should prefer behaviour over type or identity when inspecting errors. Where that isn’t possible, declaring error constants removes the potential for modification which retaining their identity semantics.

The remaining error variables are private declarations which give a symbolic name to an error message. These error values are unexported so they cannot be used for comparison by callers outside the package. Declaring them at the package level, rather than at the point they occur inside a function negates the opportunity to add additional context to the error. Instead I recommend using something like pkg/errors to capture a stack trace at the point the error occurs.

Registration

A registration pattern is followed by several packages in the standard library such as net/http, database/sql, flag, and to a lesser extent log. It commonly involves a package scoped private map or struct which is mutated by a public function—a textbook singleton.

Not being able to create a package scoped placeholder for this state would remove the side effects in the image, database/sql, and crypto packages to register image decoders, database drivers and cryptographic schemes. However, this is precisely the magic that Peter is referring to–importing a package for the side effect of changing some global state of your program is truly spooky action at a distance.

Registration also promotes duplicated business logic. The net/http/pprof package registers itself, via a side effect with net/http.DefaultServeMux, which is both a potential security issue—other code cannot use the default mux without exposing the pprof endpoints—and makes it difficult to convince the net/http/pprof package to register its handlers with another mux.

If package scoped variables were no longer used, packages like net/http/pprof could provide a function that registers routes on a supplied http.ServeMux, rather than relying on side effects to altering global state.

Removing the ability to apply the registry pattern would also solve the issues encountered when multiple copies of the same package are imported in the final binary and try to register themselves during startup.

Interface satisfaction assertions

The interface satisfaction idiom

var _ SomeInterface = new(SomeType)

occurred at least 19 times in the standard library. In my opinion these assertions are tests. They don’t need to be compiled, only to be eliminated, every time you build your package. Instead they should be moved to the corresponding _test.go file. But if we’re prohibiting package scoped variables, this prohibition also applies to tests, so how can we keep this test?

One option is to move the declaration from package scope to function scope, which will still fail to compile if SomeType stop implementing SomeInterface

func TestSomeTypeImplementsSomeInterface(t *testing.T) {
       // won't compile if SomeType does not implement SomeInterface
       var _ SomeInterface = new(SomeType)
}

But, as this is actually a test, it’s not hard to rewrite this idiom as a standard Go test.

func TestSomeTypeImplementsSomeInterface(t *testing.T) {
       var i interface{} = new(SomeType)
       if _, ok := i.(SomeInterface); !ok {
               t.Fatalf("expected %t to implement SomeInterface", i)
       }
}

As a side note, because the spec says that assignment to the blank identifier must fully evaluate the right hand side of the expression, there are probably a few suspicious package level initialisation constructs hidden in those var declarations.

It’s not all beer and skittles

The previous sections showed that avoiding package scoped variables might be possible, but there are some areas of the standard library which have proved more difficult to apply this idea.

Real singletons

While I think that the singleton pattern is generally overplayed, especially in its registration form, there are always some real singleton values in every program. A good example of this is  os.Stdout and friends.

package os 

var (
        Stdin  = NewFile(uintptr(syscall.Stdin), "/dev/stdin")
        Stdout = NewFile(uintptr(syscall.Stdout), "/dev/stdout")
        Stderr = NewFile(uintptr(syscall.Stderr), "/dev/stderr")
)

There are a few problems with this declaration. Firstly Stdin, Stdout, and Stderr are of type *os.File, not their respective io.Reader or io.Writer interfaces. This makes replacing them with alternatives problematic. However the notion of replacing them is exactly the kind of magic that this experiment seeks to avoid.

As the previous constant error example showed, we can retain the singleton nature of the standard IO file descriptors, such that packages like log and fmt can address them directly, but avoid declaring them as mutable public variables with something like this:

package main

import (
        "fmt"
        "syscall"
)

type readfd int

func (r readfd) Read(buf []byte) (int, error) {
        return syscall.Read(int(r), buf)
}

type writefd int

func (w writefd) Write(buf []byte) (int, error) {
        return syscall.Write(int(w), buf)
}

const (
        Stdin  = readfd(0)
        Stdout = writefd(1)
        Stderr = writefd(2)
)

func main() {
        fmt.Fprintf(Stdout, "Hello world")
}

Caches

The second most common use of unexported package scoped variables are caches. These come in two forms; real caches made out of maps (see the registration pattern above) and sync.Pool, and quasi constant variables that ameliorate the cost of a compilation.

As an example the crypto/ecsda package has a zr type whose Read method zeros any buffer passed to it. The package keeps a single instance of zr around because it is embedded in other structs as an io.Reader, potentially escaping to the heap each time it is instantiated.

package ecdsa 

type zr struct {
        io.Reader
}

// Read replaces the contents of dst with zeros.
func (z *zr) Read(dst []byte) (n int, err error) {
        for i := range dst {
                dst[i] = 0
        }
        return len(dst), nil
}

var zeroReader = &zr{}

However zr doesn’t embed an io.Reader, it is an io.Reader, so the unused zr.Reader field could be eliminated, giving zr a width of zero. In my testing this modified type can be created directly where it is used without performance regression.

        csprng := cipher.StreamReader{
                R: zr{},
                S: cipher.NewCTR(block, []byte(aesIV)),
        }

Perhaps some of the caching decision could be revisited as the inlining and escape analysis options available to the compiler have improved significantly since the standard library was first written.

Tables

The last major use of  common use of private package scoped variables is for tables, as seen in the unicode, crypto/*, and math packages. These tables either encode constant data in the form of arrays of integer types, or less commonly simple structs and maps.

Replacing package scoped variables with constants would require a language change along the lines of #20443. So, fundamentally, providing there is no way to modify those tables at run time, they are probably a reasonable exception to this proposal.

A bridge too far

Even though this post was just a thought experiment, it’s clear that forbidding all package scoped variables is too draconian to be workable as a language precept. Addressing the bespoke uses of private var usage may prove impractical from a performance standpoint, would be akin to pinning a “kick me” sign to ones back and inviting all the Go haters to take a free swing.

However, I believe there are a few concrete recommendations that can be drawn from this exercise, without going to the extreme of changing the language spec.

  • Firstly, public var declarations should be eschewed. This is not a controversial conclusion and not one that is unique to Go. The singleton pattern is discouraged, and an unadorned public variable that can be changed at any time by any party that knows its name should be a design, and concurrency, red flag.
  • Secondly, where public package var declarations are used, the type of those variables should be carefully constructed to expose as little surface area as possible. It should not be the default to take a type expected to be used on a per instance basis, and assign it to a package scoped variable.

Private variable declarations are more nuanced, but certain patterns can be observed:

  • Private variables with public setters, which I labelled registries, have the same effect on the overall program design as their public counterparts. Rather than registering dependencies globally, they should instead be passed in during declaration using a constructor function, compact literal, config structure, or option function.
  • Caches of []byte vars can often be expressed as consts at no performance cost.  Don’t forget the compiler is pretty good at avoiding string([]byte) conversions where they don’t escape the function call.
  • Private variables that hold tables, like the unicode package, are an unavoidable consequence of the lack of a constant array type. As long as they are unexported, and do not expose any way to mutate them, they can be considered effectively constant for the purpose of this discussion.

The bottom line; think long and hard about adding package scoped variables that are mutated during the operation of your program. It may be a sign that you’ve introduced magic global state.

If a map isn’t a reference variable, what is it?

In my previous post I showed that Go maps are not reference variables, and are not passed by reference. This leaves the question, if maps are not references variables, what are they?

For the impatient, the answer is:

A map value is a pointer to a runtime.hmap structure.

If you’re not satisfied with this explanation, read on.

What is the type of a map value?

When you write the statement

m := make(map[int]int)

The compiler replaces it with a call to runtime.makemap, which has the signature

// makemap implements a Go map creation make(map[k]v, hint)
// If the compiler has determined that the map or the first bucket
// can be created on the stack, h and/or bucket may be non-nil.
// If h != nil, the map can be created directly in h.
// If bucket != nil, bucket can be used as the first bucket.
func makemap(t *maptype, hint int64, h *hmap, bucket unsafe.Pointer) *hmap

As you see, the type of the value returned from runtime.makemap is a pointer to a runtime.hmap structure. We cannot see this from normal Go code, but we can confirm that a map value is the same size as a uintptr–one machine word.

package main

import (
	"fmt"
	"unsafe"
)

func main() {
	var m map[int]int
	var p uintptr
	fmt.Println(unsafe.Sizeof(m), unsafe.Sizeof(p)) // 8 8 (linux/amd64)
}

If maps are pointers, shouldn’t they be *map[key]value?

It’s a good question that if maps are pointer values, why does the expression make(map[int]int) return a value with the type map[int]int. Shouldn’t it return a *map[int]int? Ian Taylor answered this recently in a golang-nuts thread1.

In the very early days what we call maps now were written as pointers, so you wrote *map[int]int. We moved away from that when we realized that no one ever wrote `map` without writing `*map`.

Arguably renaming the type from *map[int]int to map[int]int, while confusing because the type does not look like a pointer, was less confusing than a pointer shaped value which cannot be dereferenced.

Conclusion

Maps, like channels, but unlike slices, are just pointers to runtime types. As you saw above, a map is just a pointer to a runtime.hmap structure.

Maps have the same pointer semantics as any other pointer value in a Go program. There is no magic save the rewriting of map syntax by the compiler into calls to functions in runtime/hmap.go.


Notes

  1. If you look far enough back in the history of Go repository, you can find examples of maps created with the new operator.

There is no pass-by-reference in Go

My post on pointers provoked a lot of debate about maps and pass by reference semantics. This post is a response to those debates.

To be clear, Go does not have reference variables, so Go does not have pass-by-reference function call semantics.

What is a reference variable?

In languages like C++ you can declare an alias, or an alternate name to an existing variable. This is called a reference variable.

#include <stdio.h>

int main() {
        int a = 10;
        int &b = a;
        int &c = b;

        printf("%p %p %p\n", &a, &b, &c); // 0x7ffe114f0b14 0x7ffe114f0b14 0x7ffe114f0b14
        return 0;
}

You can see that a, b, and c all refer to the same memory location. A write to a will alter the contents of b and c. This is useful when you want to declare reference variables in different scopes–namely function calls.

Go does not have reference variables

Unlike C++, each variable defined in a Go program occupies a unique memory location.

package main

import "fmt"

func main() {
        var a, b, c int
        fmt.Println(&a, &b, &c) // 0x1040a124 0x1040a128 0x1040a12c
}

It is not possible to create a Go program where two variables share the same storage location in memory. It is possible to create two variables whose contents point to the same storage location, but that is not the same thing as two variables who share the same storage location.

package main

import "fmt"

func main() {
        var a int
        var b, c = &a, &a
        fmt.Println(b, c)   // 0x1040a124 0x1040a124
        fmt.Println(&b, &c) // 0x1040c108 0x1040c110
}

In this example, b and c hold the same value–the address of a–however, b and c themselves are stored in unique locations. Updating the contents of b would have no effect on c.

But maps and channels are references, right?

Wrong. Maps and channels are not references. If they were this program would print false.

package main

import "fmt"

func fn(m map[int]int) {
        m = make(map[int]int)
}

func main() {
        var m map[int]int
        fn(m)
        fmt.Println(m == nil)
}

If the map m was a C++ style reference variable, the m declared in main and the m declared in fn would occupy the same storage location in memory. But, because the assignment to m inside fn has no effect on the value of m in main, we can see that maps are not reference variables.

Conclusion

Go does not have pass-by-reference semantics because Go does not have reference variables.

Next: If a map isn’t a reference variable, what is it?

Understand Go pointers in less than 800 words or your money back

This post is for programmers coming to Go who are unfamiliar with the idea of pointers or a pointer type in Go.

What is a pointer?

Simply put, a pointer is a value which points to the address of another. This is the textbook explanation, but if you’re coming from a language that doesn’t let you talk about address of a variable, it could very well be written in Cuneiform.

Let’s break this down.

What is memory?

Computer memory, RAM, can be thought of as a sequence of boxes, placed one after another in a line. Each box, or cell, is labeled with a unique number, which increments sequentially; this is the address of the cell, its memory location.

Each cell holds a single value. If you know the memory address of a cell, you can go to that cell and read its contents. You can place a value in that cell; replacing anything that was in there previously.

That’s all there is to know about memory. Everything the CPU does is expressed as fetching and depositing values into memory cells.

What is a variable?

To write a program that retrieves the value stored in memory location 200, multiples it by 3 and deposits the result into memory location 201, we could write something like this in pseudocode:

  • retrieve the value stored in address 200 and place it in the CPU.
  • multiple the value stored in the CPU by 3.
  • deposit the value stored in the CPU into memory location 201.


This is exactly how early programs were written; programmers would keep a list of memory locations, who used it, when, and what the value stored there represented.

Obviously this was tedious and error prone, and meant every possible value stored in memory had to be assigned an address during the construction of the program. Worse, this arrangement made it difficult to allocate storage to variables dynamically as the program ran– just imagine if you had to write large programs using only global variables.

To address this, the notion of a variable was created. A variable is just a convenient, alphanumeric pseudonym for a memory location; a label, or nickname.

Now, rather than talking about memory locations, we can talk about variables, which are convenient names we give to memory locations. The previous program can now be expressed as:

  • Retrieve the value stored in variable a and place it in the CPU.
  • multiple it by 3
  • deposit the value into the variable b.

This is the same program, with one crucial improvement–because we no longer need to talk about memory locations directly, we no longer need to keep track of them–that drudgery is left to the compiler.

Now we can write a program like

var a = 6
var b = a * 3

And the compiler will make sure that the variables a and b are assigned unique memory locations to hold their value for as long as needed.

What is a pointer?

Now that we know that memory is just a series of numbered cells, and variables are just nicknames for a memory location assigned by the compiler, what is a pointer?

A pointer is a value that points to the memory address of another variable.

The pointer points to memory address of a variable, just as a variable represents the memory address of value.

Let’s have a look at this program fragment

func main() {
	a := 200
	b := &a
	*b++
	fmt.Println(a)
}

On the first line of main we declare a new variable a and assign it the value 200.

Next we declare a variable b and assign it the address a. Remember that we don’t know the exact memory location where a is stored, but we can still store a‘s address in b.

The third line is probably the most confusing, because of the strongly typed nature of Go. b contains the address of variable a, but we want to increment the value stored in a. To do this we must dereference b, follow the pointer from b to a.

Then we add one the value, and store it back in the memory location stored in b.

The final line prints the value of a, showing that it has increased to 201.

Conclusion

If you are coming from a language with no notion of pointers, or where every variable is implicitly a pointer don’t panic, forming a mental model of how variables and pointers relate takes time and practice. Just remember this rule:

A pointer is a value that points to the memory address of another variable.

Next: There is no pass-by-reference in Go

Why Slack is inappropriate for open source communications

Full disclosure: my employer makes a Slack alternative. All my concerns about the use of Slack type chat services apply equally to its competitors, including my employer’s.


I’ve tweeted a few times about my frustration with the movement of open source projects from open, asynchronous, communication tools like forums, mailing lists, and issue trackers, to closed, synchronous communication services like Slack. This post is a long form version of my gripe.

The text of this post is also available in Russian (thank you Softdroid).

What is Slack good for?

Before I stick the boot in, let’s talk about the good things about synchronous chat applications like Slack, HipChat, and so on.

In a work context, chat applications take the place of @staff email blasts about fire system testing, broken lifts, and spontaneous availability of baked goods. This is a good thing as this kind of company spam is often impossible to unsubscribe from.

In the context of an open source project, Slack, HipChat, Gitter, etc, provide a forum for advocacy, gossip, informal discussion, and support. My complaints start when Slack and friends are promoted as the recommended way to communicate with the project.

Why is Slack bad for open source communication?

My complaint about the growing use of chat services like Slack, HipChat, and so on, for communication by open source projects is that these services are not open. As I see it there are two issues:

  1. Slack, et al, are paid services with closed memberships. Sure, there are lots of little apps running on Heroku dyno’s that automate the “send me an invite” process, but fundamentally these are closed systems.

    This means that the content inside those systems is closed. I cannot link to a discussion in a Slack channel in a tweet. I cannot refer to it in an issue report, and I cannot cite it in a presentation. Knowledge is silo’d to those who have the time and ability to participate in chat services in real time.

  2. Slack, et al, are based on synchronous communication, which discriminate against those who do not or can not take part of the conversation in real time. For example, real time chat discriminates against those who aren’t in the same time zone–you can’t participate fully in an open source project if all the discussion happens while you’re asleep.

    Even if you are in the same time zone, real time chat assumes a privilege that you have the spare time–or an employer who doesn’t mind you being constantly distracted–to be virtually present in a chat room. Online chat clients are resource hogs, and presume the availability of a fast computer and ample, always on, internet connection, again raising the bar for participation.

In my view these issues are inseparable. Calls to use IRC instead, miss the point that IRC is similarly real-time, just as efforts to create a post facto log of a Slack channel miss the fact that this is a record of a conversation which others cannot contribute equally. There is no solution for equitable open source communication that does not address both simultaneously.

Prefer asynchronous communication for open source projects

Instead of closed, synchronous, systems I recommend open source projects stick to asynchronous communication tools that leave a publicly linkable, searchable, url. The tools that fit this requirement best are; mailing list, issue trackers, and forums.

RC2017/4: Introducing Arudino6502

This weekend I polished up my Arduino Day project and published it to GitHub for Retrochallenge 2017/04.

Introducing Arduino6502

https://github.com/davecheney/arduino6502

The repository contains an Arduino sketch that can be loaded on Arduino Mega boards (Arduino Uno’s can be accommodated by lowering the RAMSIZE value).

The sketch includes ROM images for AppleSoft Lite and Krusader symbolic assembler.

To run AppleSoft BASIC, enter

\
E000R

And you will be dropped into the BASIC prompt

>

Remember that the Apple 1 was an upper case only machine, so you should enter all letters in upper case.

Krusader is available at $F000

\
F000R

What’s next?

The project currently targets the 8 bit Atmel chips due to the tight coupling between the Atmega 2560’s hardware serial port and the simulated 6821 PIA. The mapping is good enough that, as long as you insert a delay of 200ms or greater between newlines, you can paste HEX files into the Woz monitor and run the resulting program.

However, while the Apple 1 came with 4k or 8k of RAM, modern recreations like the Replica 1 assume at least 32k of RAM, making most of the software written for these nouveau Apple 1’s out of reach of this project.

To overcome this my next effort will be to port the project to run on Cortex-M platforms like the Teensy 3.x or Arduino Due to get access to more SRAM.

Retrochallenge 2017/04: 6502 on Arduino

This is my late entry for the 2017/04 retrochallenge. Notwithstanding my 2015 failure to launch, I plan to work on Arduino based emulators for various 6502 computers.

A few years ago I built an Arduino shield to host a real 6502 using the Arduino as RAM, PIA, and  glue logic. To some extent the software that this project ran was an afterthought, as it turned out the Apple 1 Woz monitor was perfect a proof of concept.

For the recent Arduino Day I brushed off my old code and reworked it to use an emulated CPU core so the sketch can run on an unadorned Arduino Uno or Mega.

For retrochallenge 2017/04 I plan to continue this work.

Roadmap

  • Clean up my sketch and publish the code to GitHub. I plan to target the Arduino Uno and Arduino Mega natively.
  • Hopefully add support for other monitors like Microsoft 8k BASIC.
  • Stretch goal: Rockwell made a 6502 variant called the R65F11 which was a 6502 with FORTH built into on chip ROM. Details are sketchy, but I found this pdf online and maybe if the ROM can be uncovered I can emulate that system.

Why Go?

A few weeks ago I was asked by a friend, “why should I care about Go”? They knew that I was passionate about Go, but wanted to know why I thought other people should care. This article contains three salient reasons why I think Go is an important programming language.

Safety

As individuals, you and I may be perfectly capable of writing a program in C that neither leaks memory or reuses it unsafely. However, with more than 40 years of experience, it is clear that collectively, programmers working in C are unable to reliably do so en masse.

Despite static code analysis, valgrind, tsan, and -Werror being available for a decades, there is scant evidence those tools have achieved widespread acknowledgement, let alone widespread adoption. In aggregate, programmers have shown they simply cannot safely manage their own memory. It’s time to move away from C.

Go does not rely on the programmer to manage memory directly, instead all memory allocation is managed by the language runtime, initialized before use, and bounds checked when necessary. It’s certainly not the first mainstream language that offered these safety guarantees, Java (1995) is probably a contender for that crown. The point being, the world has no appetite for unsafe programming languages, thus Go is memory safe by default.

Developer productivity

The point at which developer time became more expensive than hardware time was crossed back in the late 1970s. Developer productivity is a sprawling topic but it boils down to this; how much time do you spend doing useful work vs waiting for the compiler or hopelessly lost in a foreign codebase.

The joke goes that Go was developed while waiting for a C++ program to compile. Fast compilation is a key feature of Go and a key recruiting tool to attract new developers. While compilation speed remains a constant battleground, it is fair to say that compilations which take minutes in other languages, take seconds in Go.

More fundamental to the question of developer productivity, Go programmers realise that code is written to be read and so place the act of reading code above the act of writing it. Go goes so far as to enforce, via tooling and custom, that all code by formatted in a specific style. This removes the friction of learning a project specific language sub-dialect and helps spot mistakes because they just look incorrect.

Due to a focus on analysis and mechanical assistance, a growing set of tools that exist to spot common coding errors have been adopted by Go developers in a way that never struck a chord with C programmers—Go developers want tools to help them keep their code clean.

Concurrency

For more than a decade, chip designers have been warning that the free lunch is over. Hardware parallelism, from the lowliest mobile phone to the most power hungry server, in the form of more, slower, cpu cores, is only available if your language can utilise them. Therefore, concurrency needs to be built into the software we write to run on today’s hardware.

Go takes a step beyond languages that expose the operating system’s multi-process or multi-threading parallelism models by offering a lightweight concurrency model based on coroutines, or goroutines as they are known in Go. Goroutines allows the programmer to eschew convoluted callback styles while the language runtime makes sure that there will be just enough threads to keep your cores active.

The rule of three

These were my three reasons for recommending Go to my friend; safety, productivity, and concurrency. Individually, there are languages that cover one, possibly two of these domains, but it is the combination of all three that makes Go an excellent choice for mainstream programmers today.

How to write a successful conference proposal

As an organiser of a large programming conference and a speaker who’s pitched talk ideas to many conferences, I’ve been on both sides of the selection process. Last month I published a piece on writing a proposal for GopherCon. I wanted to revisit that post in the form of more general advice to give some insight into the why, not just the how, of writing a good conference proposal.

Presentations and proposals are different things

Your talk and the proposal to give that talk are different because they target different audiences. The former is what you are going to present on stage, the latter is a pitch to the reviewers to let you give that presentation.

Writing a good conference proposal is a different skill than writing the presentation itself. This article is aimed at writing a good proposal with a focus on the reviewer of your proposal as the audience.

Focus on the audience

Speaking of audiences, good public speakers start planning a presentation by identifying the audience they want to address. Presenting at a conference is like teaching a class, you have to present the material at the level of the people in the room.

It’s not just a question of beginner, advanced, or expert, you also have to consider the kinds of people at the conference. If it’s a vendor conference, there are probably going to be lots of managers, (pre) sales people, and business decision makers in the audience. While they might also be competent engineers, they’re at that conference wearing their business leader hat. They want to hear a different story; reliability, ease of maintenance, or evidence of widespread adoption, than an audience of software engineers who are more likely interested in things such as performance, orthogonality, and extensiblity.

So, if a proposal is for the conference reviewers, not the audience, should you pitch the presentation at the reviewers? Well no, but you should focus on what the reviewers want.
Your goal in writing a proposal is to convince the reviewers that, as well as thinking about your idea, and how to present it, you’ve considered the people who will come to see your talk.

Who are the reviewers and what do they want?

For smaller conferences it’ll be the organiser, or organisers, of the conference you’re applying too. For larger conferences it will likely be a group of reviewers who the organisers have invited to review proposals, this is the model that GopherCon follows. For really large conferences, such as OSCON, they will have a group of reviewers per track who funnel their recommendations up to a programme chair or set of program coordinators.

Regardless of their size, conference reviewers are charged with recommending to the organisers a set of talks they think are interesting and appropriate for the audience of the conference.

Most review panels are confidential, so you shouldn’t know anything about the individual reviewers, although you can probably guess that they will be experienced in the subject of your conference.

Most proposal are reviewed anonymously, at least in the initial rounds. This means the reviewers must judge your proposal, and your ability to present it, using only the fields provided on the submission form.

it’s important to remember that at least in part, all conferences are commercial enterprises. Venue owners have bills to pay just like the rest of us, and at a minimum speakers need to be compensated for their travel and lodging, otherwise the programme will be filled with people who are paid by their employer to speak.

To put it bluntly, reviewers are looking for talks that people will pay to see. This might sound capitalistic, but it turns out that this is what the audience want as well. At GopherCon we cover the travel and accommodation expenses of all our speakers. We think this is important because we want to hear what the speaker thinks, not their marketing department.

All of these are factors that reviewers will be considering when reading your proposal.

What to put in a proposal

Almost every conference call for proposals will ask for the following; title, abstract, and description. They may ask for other things like a biography, questions about AV requirements for your talk, and so on, but with respect to successful acceptance, these three items are key.

Title

A title is mandatory on almost every talk submission system I’ve seen. It’s your one line elevator pitch to entice the audience to come to your talk.

Keeping the title a little vague, or quixotic is popular, but I tend to stay away from 11 things that will make your proposal sound like a buzzfeed article. I’m not saying never do that, but if you do, you’d better pack a heck of a proposal behind your braggadocios

Abstract

Conference organisers usually ask you to provide a talk abstract as they often don’t feel it is appropriate to summarise your proposal for you. This abstract will be printed in the program or placed on the website so potential visitors to the event know what they’ll be seeing.

There are usually restrictions on the size of the abstract. One sentence that describes the topic that you’ll be talking about, and one sentence that describes what the audience will take away from listening to your talk or participating in your workshop, is all you need.

Together with the title, these are the two pieces of information the eventual conference audience will use to decide if they want to come to your session or not.

Talk description

This is where you sell your talk idea, and this is the place as a reviewer, I have seen so many good proposals with interesting ideas fail to make the cut because they simply didn’t include enough detail.

This is where my advice differs from other’s you’ll read on the web. Many pieces of advice encourage you to write less in your description, sometimes out of recognition that the organisers are busy and you don’t wish to burden them. I wanted to take some time to explain why I push every speaker to write more detail.

You are looking to do three things when writing a description of your talk:

  1. Make it clear to the reviewers that you know what you are talking about.
  2. That you have a plan to communicate what you know to the audience and you’ve thought about how to do this within the time limit of the speaking slot.
  3. Answer all the selection criteria for the conference.

The first point is self explanatory, but you still need to make sure that you communicate this clearly to your reviewers. For example, if you’re talking about how to manage a large open source project, then make sure you mention that in the proposal, “as the maintainer of a large open source project”. If you plan to talk about a subject in the third person, then you should cite your sources, “for my PhD thesis I studied the day to day interactions of the top 10 projects on GitHub”. You don’t have to be an expert, but if your goal is to communicate something new to the audience, you should demonstrate that you know more about the topic than they do.

The second point relates to how likely you are to effectively communicate your ideas. The reviewers want to feel comfortable that you have a plan. It is all too common to see a proposal for an hour long session with only a sentence or two for the description. The less you write in a conference proposal, the more the reviewers are left to take it on faith that you’ll do a good job.

The opposite is also true. Occasionally I see a proposal for a talk that includes every possible aspect of a subject. Reviewers are generally wary that the speaker cannot cover all their material in the time available–few conferences can afford C++Con’s multi-part multi-hour format.

A presenter that doesn’t manage their time, rambles without conclusion, or covers a lot of material in common knowledge is going to waste the audience’s time. That’s not just unfair to the audience, but unfair on the speakers that follow who must deal with a disgruntled audience.

One thing that I recommend to anyone considering submitting a proposal is to include an outline of your talk in the proposal. This can be literally the headings of your slides, or your ideas in bullet points. As a reviewer this makes it crystal clear that you’ve not only thought about your idea, but how to present it.

The last point, address all the selection criteria, I cannot emphasise enough. Review committees strive to be fair and often rate all proposal by a common standard. It crucial to address the selection criteria clearly as these are the ground rules by which every proposal are judged.

This point is probably the trickiest as not all conferences publish their selection criteria. Sometimes conferences ask for talks along a particular theme and these can be substituted for criteria in a pinch. If there are no criteria available–don’t guess, ask the organisers. If they don’t have any to share, which can happen with smaller conferences, then think about the audience and the wider ecosystem of the conference’s focus and ask yourself “if I were thinking about coming to this conference, what would I like to hear about?”

If you take away one thing from this section it is this–proposals with less detail loose out to proposals that provide more–as they do not provide the reviewer with sufficient evidence to be confident in their recommendations.

Don’t sell snow to Eskimos

Before closing I want to highlight a very common mistake I see in both conference proposals, and conference presentations, which is a speaker selling their audience on a thing the audience already likes.

To give an example, you wouldn’t go to the JVM Language Summit and give a presentation about how great the JVM is and they should use it. Instead, you’d go to the JVM language summit and show the audience the JVM is great by telling them about your project which was only possible because you chose to base it on the JVM.

Don’t take my word for it

Finally, if you’ve read this far, I encourage you to read what others have written on the topic, especially where their advice differs.

Karolina Szczur recently wrote a great article on writing conference proposals and includes many references to similar articles for further reading.

Conclusion

Reviewers are looking to put together the best conference they can. They want to see your talk on stage, but you have to give them the evidence they need to feel confident in recommending you. Show the reviewers you’ve thought about the audience, and you’ll make their decision a lot simpler.

I’m speaking at GopherChina and GopherCon Singapore

In April and May I’ll be speaking at GopherChina and GopherCon Singapore, respectively. This post is a teaser for the talks that were selected by the organisers. If you’re in the area, I hope you’ll come and hear me speak.

GopherChina

GopherChina is the third event in this conference series and this year will return to Shanghai. I was lucky to attend the event in 2016 and am looking forward to 2017.

The hidden #pragmas of Go

Go isn’t like C. It doesn’t have a preprocessor, it doesn’t have macros, and it certainly doesn’t have #define, but Go does have pragmas.

What are pragmas? The name come from the #pragma declaration that tells C compilers to alter their interpretation of a piece of code. Now, Go doesn’t have a #pragma directive, but it does have ways of altering the operation of the Go compiler via directive syntax hidden in comments.

This talk will explore the history of these directives, how and why they are used, and how you can, but probably shouldn’t, use them in your own code.

GopherCon Singapore

GopherCon Singapore is the latest in the GopherCon franchise, and as flight times go, relatively close to home. I’m delighted to have the opportunity to present at their inaugural conference in May.

Concurrency made Easy

In my experience, many people who come to Go do so because they have a problem where being able to run more than one task at a time in their program would be beneficial. Ruby and Python programmers come to Go because the concurrency story is much better, the same is true of Node programmers; the event loop is still inherently single threaded.

But, most programmers who stick with Go for a while tend to look back on their early efforts and say things like “wow, I really went overboard with channels” or “I went crazy with goroutines when I started writing Go. It was impossible to understand what the program did”. For people who learn Go formally from an instructor or a book, the concurrency section is always the last section they cover.

So there is a dichotomy here. Go’s headline feature is simple, lightweight concurrency. As a product the language sells itself on that feature alone. On the other hand, there is a narrative that concurrency isn’t actually that easy to use, otherwise people wouldn’t make it the last things in their books or classes, or perhaps more accurately, concurrency is not the solution to every problem.

With this as a background, I’d like to explore some strategies for using concurrency in Go without the pitfalls of convoluted code, the importance of memory ownership, and the best way to structure a Go program using goroutines.