Tag Archives: open source

Talk, then code

The open source projects that I contribute to follow a philosophy which I describe as talk, then code. I think this is generally a good way to develop software and I want to spend a little time talking about the benefits of this methodology.

Avoiding hurt feelings

The most important reason for discussing the change you want to make is it avoids hurt feelings. Often I see a contributor work hard in isolation on a pull request only to find their work is rejected. This can be for a bunch of reasons; the PR is too large, the PR doesn’t follow the local style, the PR fixes an issue which wasn’t important to the project or was recently fixed indirectly, and many more.

The underlying cause of all these issues is a lack of communication. The goal of the talk, then code philosophy is not to impede or frustrate, but to ensure that a feature lands correctly the first time, without incurring significant maintenance debt, and neither the author of the change, or the reviewer, has to carry the emotional burden of dealing with hurt feelings when a change appears out of the blue with an implicit “well, I’ve done the work, all you have to do is merge it, right?”

What does discussion look like?

Every new feature or bug fix should be discussed with the maintainer(s) of the project before work commences. It’s fine to experiment privately, but do not send a change without discussing it first.

The definition of talk for simple changes can be as little as a design sketch in a GitHub issue. If your PR fixes a bug, you should link to the bug it fixes. If there isn’t one, you should raise a bug and wait for the maintainers to acknowledge it before sending a PR. This might seem a little backward–who wouldn’t want a bug fixed–but consider the bug could be a misunderstanding in how the software works or it could be a symptom of a larger problem that needs further investigation.

For more complicated changes, especially feature requests, I recommend that a design document be circulated and agreed upon before sending code. This doesn’t have to be a full blown document, a sketch in an issue may be sufficient, but the key is to reach agreement using words, before locking it in stone with code.

In all cases you shouldn’t proceed to send code until there is a positive agreement from the maintainer that the approach is one they are happy with. A pull request is for life, not just for Christmas.

Code review, not design by committee

A code review is not the place for arguments about design. This is for two reasons. First, most code review tools are not suitable for long comment threads, GitHub’s PR interface is very bad at this, Gerrit is better, but few have a team of admins to maintain a Gerrit instance. More importantly, disagreements at the code review stage suggests there wasn’t agreement on how the change should be implemented.

Talk about what you want to code, then code what you talked about. Please don’t do it the other way around.

Go project contributors by the numbers

In December 2014 the Go project moved from Google Code to GitHub. Along with the move to GitHub, the Go project moved from Mercurial to Git, which necessitated a move away from Rietveld to Gerrit for code review.

A healthy open source project lives and dies by its contributors. People come and people go as time, circumstance, their jobs, and their interests change. I wanted to investigate if these moves were a net positive for the project.

Go project contributors

As of writing 744 people have contributed to the Go project. This number is slightly lower than OpenHub‘s count, which also includes the golang.org/x sub-repositories that this analysis ignores. The number is slightly higher than GitHub‘s count for reasons which are unclear.

Go project contributors (click to enlarge)

Go project contributors (click to enlarge)

This graph shows contributors over time. As a new contributor lands their first commit, a line representing the current count of contributors is placed on the graph. The graph also records the number of contributors whose email addresses end in golang.org or google.com as a proxy for the Go team and Google employees respectively.

Did the move to Git, Gerrit, and GitHub increase the number of contributors, and thereby contributions? Almost certainly. The graph shows a clear up-tick in the number of new contributors to the project from January 2015 onwards. However with so many factors in play it is not possible to identify a single cause.

It should also be noted that even prior to the move, the project has always attracted a healthy stream of new contributors.

Runtime contributors

Runtime contributors (click to enlarge)

Runtime contributors (click to enlarge)

From June 2014 to December 2014, the Go runtime was rewritten (with the help of some automated tools) from C to Go. This graph records the number of contributors to the parts of the standard library considered the runtime. This has changed over time as paths have been renamed, but is currently what we think of as the tree rooted at $GOROOT/src/runtime.

Did the rewrite of the runtime from a dialect of C that was compiled with the project’s own C compiler to Go increase the number of individual contributors willing to work on the runtime? Unfortunately not, although the number of Googlers contributing to the the runtime did increase slightly.  An increase in individual runtime contributions did not occur till the following January once the move to GitHub was complete.

Compiler contributors

Compiler contributors (click to enlarge)

Compiler contributors (click to enlarge)

After the move to GitHub, the Go compiler was translated from C to Go for the Go 1.5 release. This process was completed by May 2015. Did this have an impact on the number of new contributors specifically targeting the compiler? Possibly, there was a short lived spurt of new contributions around July 2015. The reason for this could be attributed to the strong message from the Go team that the 1.5 release was focusing on the runtime, specifically the garbage collector, and that the current compiler was to be replaced with the SSA back end being developed for the 1.6 time frame (since delayed til 1.7).

This latter point is supported by the clear spike in contributors after February 2016, when the 1.7 tree opened for development with the SSA back end attracting a number of talented new contributors.

An open source project

At the moment there is no question that the largest contributors by total commits to Go are the Go team themselves. This stands to reason as they are sponsored by Google itself to develop the language. However, out of the current top 16 contributors to the project, the number 7th, 9th, 11th, 14th, and 16th contributors are neither members of the Go team or employed by Google.

The charge is commonly levelled at Google that Go is not an open source project. This analysis shows that claim to be false. The number of contributors from the Go team, or Google, continues to be a dwindling fraction of the total number of contributors to the project.

Suggestions for contributing to an Open Source project

Occasionally I am asked for advice on how to get started contributing to an Open Source project. I thought it may be useful to write down my suggestions.

These points were written in the context of the Go programming language, but I think this advice is applicable to the majority of modern Open Source projects.

  1. Pick an issue you know how to solve. The best way to get started with a project is to fix a bug. You’ll need to be self sufficient, so do some research and investigate the history behind a bug. Don’t pick an issue you have no familiarity with and then ask “Who can tell me how to solve this bug?”
  2. Ask for more detail. Many bugs lack enough detail to be addressed, so promoting the reporter for more information is in itself a useful service. You may discover that the bug is a duplicate of another, in which case it can be closed. If you can distil the bug report into a reproduction or a test case that is a valuable contribution in itself.
  3. Discuss your change first. When you have chosen a bug, discuss your change before starting to code. You can experiment privately, but do not send a change without discussing it first. Your can probably skip this with very trivial changes, like typos or adding a small test case to an existing package, but for anything larger the rule is: discuss, then code.
  4. Always include a test. One of the first things a reviewer will do is patch in your test and verify that it fails before even looking at your fix. You should therefore write the failing test case first, then write the fix. It may be that you need to refactor the code to be able to write a failing test, which is fine, but brings me back to point 3; discuss your change first. If the project does not have a strong testing regime then you should describe how you went about verifying the fix so someone reviewing your change can do the same.
  5. Change as little as possible. All things being equal, smaller changes are easier to review and are merged faster than large ones. You should aim to change as little as possible to keep the size of the change as small as possible. Avoid the temptation to include a bunch of unrelated changes.
  6. Follow the existing style. Even with tools like gofmt, large projects will commonly exhibit minor stylistic differences. My rule of thumb is: always follow the predominant style of the file in question; if they use long identifiers, use long identifiers, if they use short ones, do so too, and so on. Above all, resist the temptation to include a large stylistic change along with your bug fix.
  7. Be polite, but persistent. If you haven’t received feedback on your proposal after a few days, politely ask for a response. It may be that your proposal was overlooked, or that the project is currently in a feature freeze. Assuming you have followed the advice above, you should expect to get actionable advice on how to improve your change so it can be reviewed.