Test Package Naming in Go

I’m still – still – playing with Google’s Go language, making the occasional one-off utility and also working on a Crazy Web Server Project. I hesitate to call it a Framework, since all Frameworks are ultimately doomed. It’s nothing too extreme, but it’s (hopefully) going to solve some of my web-server problems for small sites; and it’s (absolutely) helping me learn Go “for reals.”

I recently discovered something very simple, more or less by accident, that has big implications. It is this: tests for package foo should declare package foo_test.

To me this was counterintuitive, since one of the big hairy things you have to adjust to when learning go is the flat package directory. A package is made up of a bunch of .go files, and they all live in one flat directory. If you want to organize the files in a hierarchical set of directories, you must also have a corresponding set of packages (though they need not be hierarchical).

Incidentally, as I get better at Go I find myself hating such conventions less, because they are useful in forcing the programmer’s hand: be idiomatic damn you! And that certainly has its upsides.

Thus, logically, every .go file in a given directory declares the same package. If it doesn’t, you get a compile-time error!

Except, that is, for tests.

The Test Exception

For a directory with a package foo, you may have two packages: foo and foo_test. You may not have any others. So, why would you want to use foo_test?

There is a very good reason, which I’ll get to in a moment. If you already have a lot of unit-testing experience and know a little Go you may already have guessed.

Let’s first consider the way I had been doing it until recently. Apologies in advance for the lack of syntax highlighting.

foo.go

package foo

func Bar() string { return "BAZ"}

foo_test.go

package foo

import "testing"

func Test_Bar(t *testing.T) {

    exp := "BAZ"
    got := Bar()
    if got != exp {
        t.Errorf("Expected '%s', got '%s'", exp, got)
    }
}

OK, great. This makes perfect sense so far. But now consider this, which in fact seems quite the obvious thing to do:

foo.go

package foo

func Bar() string { return ook(1) }

func ook(i int) string {
    if i > 0 {
        return "BAZ"
    } else {
        return "BAT"
    }
}

foo_test.go

package foo

import "testing"

func Test_Bar(t *testing.T) {

    exp := "BAZ"
    got := Bar()
    if got != exp {
        t.Errorf("Expected '%s', got '%s'", exp, got)
    }
}

func Test_ook(t *testing.T) {

    exp0 := "BAT"
    got0 := ook(0)
    if got0 != exp0 {
        t.Errorf("For 0, expected '%s', got '%s'", exp0, got0)
    }

    exp1 := "BAZ"
    got1 := ook(1)
    if got1 != exp1 {
        t.Errorf("For 1, expected '%s', got '%s'", exp1, got1)
    }

}

The example above shows a very common pattern, about whose virtue we could certainly argue: you can have much more thorough test coverage if you have robust unit tests for private functions. (Remember, in Go only Capitalized functions are exported.)

However, there’s a cost in clarity: you should be testing all things that could happen in the use of the package, i.e. you should be testing all variations of the public API. If there are conditions your public API can not trigger, and which thus can not be tested via the public API, then remove them from the code.

Go is strongly biased towards writing only the code that is actually used. Speculative code, while possible, is definitely not idiomatic.

As somebody who writes code for a lot of “what-if” scenarios in Perl, I can certainly apprecitate that discipline.

This is why you should use the foo_test package: Go will then throw a compile time error if you try to test a private function, and in your quest for 100% code coverage you will see the unreachable parts of your code.

So let’s get back to our example, changing the test first:

foo_test.go

package foo_test

import (
    "./"
    "testing"
)

func Test_Bar(t *testing.T) {

    exp := "BAZ"
    got := foo.Bar()
    if got != exp {
        t.Errorf("Expected '%s', got '%s'", exp, got)
    }
}

If Bar is the only thing that ever calls ook then you’ll have to change ook or you’ll never get 100% coverage. In this contrived example, the compiler will most likely abstract away ook entirely, but bear with me.

foo.go

package foo

func Bar() string { return ook() }

func ook() string {
    return "BAZ"
}

And there we have it, in silly little examples. Always use a _test package name for your Go tests. Testing will be harder. Deal with it: your tests will also be better.

Never Pay Marhaba In Advance.

I recently flew to lovely, hazy Singapore, with a stopover in Dubai. It was a trip of many firsts: my first time in Asia, let alone Singapore; my first time flying Emirates; my first flight on an Airbus A380; and my first time in the famous Dubai Airport.

Since I had arranged for a four-hour stopover, I thought it would be nice to kill the time in a lounge. Since my lovely wife and I were traveling coach – not half bad on the A380 – the business lounges were out of the question. But fortunately there was a pay lounge avaialable: Marhaba.

After first getting a bite to eat elsewhere, we went to the Marhaba desk and asked about buying in. The attendants were quite friendly, and told us the lounge was rather full and thus we might not have a seat. I looked around, and while it was indeed full, there were some bistro-style chairs at a table. Good enough. We paid about 100 EUR for the two of us, and enjoyed the lounge.

I was planning on recommending it: it’s not as nice as a real business lounge, but it’s not too expensive and it beats just killing time in the airport. Plus, they mix a strong G & T.

However, the trip home went rather differently, and hence instead of a glowing recommendation I give you this advice:

Never

Pay

Marhaba

In

Advance

Why not, you ask? Wouldn’t it be cheaper that way, you ask?

Well yes, it would be cheaper, except for the part about Marhaba keeping your money if your flight is delayed.

Our flight from Singapore to Dubai was significantly delayed after boarding. We sat on the plane for over two hours waiting for the engines to be replaced, or whatever technical defect they needed to fix (the Captain left us guessing). Then we were further delayed in the air; and just to make sure everyone was in the best possible mood, Emirates canceled the dinner service, at least in Coach.

When we arrived in Dubai, with clearly insufficient time to make our connection, we hustled through security and asked an attendant what we should do. He told us we should hurry. And so we jogged to the gate, and found the plane waiting.

About ten minutes later we were boarded, along with a few slower joggers, and on our way back to Europe.

Marhaba Will Not Refund Your Money.

I had booked the Marhaba lounge, pre-paid at a small discount, back in Singapore this time, as I figured it would be better that way. I had entered my flight number on their web site, so their system was fully aware of the delay.

Once back at a computer, I sent them an e-mail politely reminding them that my flight had been so delayed it would have been physically impossible to go to the lounge before making my connection, and asked them to refund my payment if they had not done so yet. (Honestly I half expected them to have issued a refund already, since they knew about the flight, but then I am a computer programmer.)

Marhaba got back to me promptly and told me that unfortunately, since I failed to cancel the booking eight hours in advance – i.e., since I failed to call them from the air – they would be keeping my money. Have a nice day.

I thought, well, that can’t really be your policy if you’re providing airport services, can it? This must be a pretty common occurrence: flight delayed, services can’t be used, at least issue a credit for the next time, right?

Bear in mind that lounge access is only one of the services Marhaba provides. You can easily shell out hundreds of Euros (thousands of Dirham) on Marhaba services, and then what happens when your flight is delayed, or for that matter canceled?

So I asked for clarification. And I got it:

We regret to inform you that no refund will be processed for a no show booking since any amendments or cancellation should be done at least 10hrs prior to the flight. Please be advised that all no show bookings has no refund.

Never Pay Marhaba In Advance.

Maybe that’s just how they roll in the Emirates. Maybe it’s usually somebody else’s money. Maybe everyone’s rich enough they don’t care about such things.

But if you care about your money, or about the principle of the matter, then it’s imperative that you never pay Marhaba in advance, for anything.

If I fly through Dubai again I might use their walk-in lounge service, since a strong drink is a strong drink, but my credit card shall never again darken the Marahaba website’s door. Neither should yours.

Test Post from Classeur

Mostly nothing to see here folks, move along, move along…

This is just a test post from Classeur, which is a very slick in-browser Markdown editor with support for some popular blogging platforms.

I usually compose my blog posts here in Markdown first, then copy the generated HTML, then make a few corrections to get around Blogger’s buggy HTML-to-HTML converter, then paste it into the blogger UI.

Cumbersome, to say the least. Maybe this is a better way.

Also, maybe it’s an easy way to collaborate on content creation. Well, for open-source stuff it would be pretty easy to come up with a workflow, and the price is way right: $5/mo I believe.

But what if you’re, say, the editor of an online newspaper, and you want your writers to compose in Classeur?

The web site suggests you can publish to DropBox, which might be a solution, but that seems to not be implemented yet. Or you can share documents as read-write among Premium users, which would be a solution for getting it as far as the editor. The editor would then need to export it somewhere.

Yes, I realize Real Newspapers would have online workflows for all this, but first: I’m not a Real Newspaper, and second: I bet their workflows really really suck. Oh, and yes, I realize most online publications don’t employ editors, but humor me please.

So let’s assume this workflow is OK and will actually work:

1. Reporter, who has a Classeur Premium account, writes a story.
2. Reporter sends its link to the Editor, who also has a Premium account.
3. Editor edits, if necessary sends it back, and so on until it’s ready.
4. Editor exports the Markdown source and sticks it in a database, or source control, or whatever.

I like this so far. Not perfect, but the editor is really quite nice, and in my experience it’s not an easy thing to get non-techies to use plain text editors, much less parse Markdown in their minds.

The problem, though, is in the pictures. I can add a nice picture of a squash ballsquash ballsquash ball here:

The problem is that it’s on Imgur, an image hosting site very popular with the young and bored. And not a place I want my internet-breaking Paparazzi shots of the Donald.

But hey, if I’m a literary editor maybe I don’t need pictures. And if I’m just running some random click -baiting Bored Wombat site, maybe I don’t care.

Again: nothing to see here, test post mostly, &c., YMMV, E&OE, and the like. But I think Classeur is pretty interesting, and I will keep it in mind for future use.

Go for Minor Utilities

I continue to play around with Google’s Go language, and I continue to have very mixed feelings about it.

It’s clearly not built for You and Me, i.e. for the programmers of the world; rather it’s built for Google and shared with us. This is fine as far as it goes, but as with so many other Google products the joy of the user is mostly “out of scope.”

Also, presumably because the whole Go ecosystem is so young, there are all kinds of quirky things being done in the many, many, many third-party packages scattered around the Githubbernets. I may write more later about the Goverse’s tendency towards Database Worst Practices and its shunning of the Null. (Ironic for a language in which every thirteenth line is a nil check.)

So far, I’m pretty sure I would not use Go for any big, complex, long-term project. But it’s good enough at some things that I’d definitely consider it for any straightforward, easy-to-run subcomponents of a big project. I wouldn’t choose Go for code shared with a bunch of other developers, but if I were dropped into such a project I’d probably do just fine. I might use Go if I planned on bringing in outside help, because as others have pointed out, one of the Google use-cases it seems optimized for is bringing inexperienced programmers into a project.

But there is one thing for which I absolutely love Go, and for which I could picture myself using it for a long time to come: Go is really great for writing simple utility programs.

Tonight I found myself needing some random-ish strings for a goofy little side-project I’m doing. After not finding quite what I wanted at the otherwise wonderful Random.org, I took an hour to throw together a little utility. It will probably only ever be utilitous to me, and I may or may not bother cleaning it up and testing it, but now at least it exists:

https://github.com/biztos/randstr

And that means I can, should I need to on some other computer, simply do this:

$ go get github.com/biztos/randstr
$ which randstr
/Users/Shared/gocode/bin/randstr
$ randstr --help

…assuming my $GOPATH is in order, and subject to dependencies I have not yet “vendored in,” and so on.

There are, I think, four things that make it especially attractive to write such programs in Go:

1. It compiles, quickly no less, into a single binary.
2. It’s reasonably cross-platform.
3. The standard library, while sometimes bizarre, is very rich.
4. Docopt!

The Joy of DocOpt

Command-line utilities, and Unix-like programs in general, usually take options and other arguments, and they usually have some rudimentary help text explaining these things.

DocOpt parses a structured, but very human-readable, help text in order to determine what the available options are. It’s not perfect but it more than covers the reasonable use-cases of most utilities I’ll ever need to write.

For any nerds who have not yet heard of DocOpt, here is the canonical example:

Naval Fate.

Usage:
  naval_fate ship new <name>...
  naval_fate ship <name> move <x> <y> [--speed=<kn>]
  naval_fate ship shoot <x> <y>
  naval_fate mine (set|remove) <x> <y> [--moored|--drifting]
  naval_fate -h | --help
  naval_fate --version

Options:
  -h --help     Show this screen.
  --version     Show version.
  --speed=<kn>  Speed in knots [default: 10].
  --moored      Moored (anchored) mine.
  --drifting    Drifting mine.

There are many implementations of DocOpt, but the one I have been using with Go is the excellent docopt-go by Keith Batten et al.

I do still love the completeness and flexibility of Perl’s Getopt::Long, and I’ve not found anything similar for Go; but now that I’ve gotten used to DocOpt I would think twice before committing to anything more complicated.

With DocOpt in your toolkit, Go is an awesome language for writing quick little utilities. Whatever else it may be, and whatever it may lack, this is reason enough to know some Go.

Go First Impressions

Over the last couple of weeks I’ve been playing with Go, an open-source programming language from Google.

My motivation was pretty simple: I felt like trying out a new server language, something modern and perhaps even trendy. Partly because it’s always a good idea to learn new things, and partly because I was growing frustrated with the limitations of Perl and Javascript/Node. Those are the two languages I use for most of my work, professional and otherwise. While I’d happily sing the praises of either one for many, many domains, every language has its baggage and I was lately feeling the weight of theirs.

Also, a highly respected member of the Perl community recently published a soul-searching article, The Mid-Career Crisis of the Perl Programmer. One thing it reminded me of is that I never considered myself a “Perl Programmer” any more than I consider myself an “Oil Painter.” I earn my keep as a software engineer. I have certain preferences and biases as would any serious professional, but I’d much sooner go to war over text editors than programming languages.

When I look into the crystal ball at my coding future, I see some Perl and some SQL (since 1994...) and that sure looks like Swift out there past the liquor store, but beyond that it’s a little fuzzy as we get closer to the beach. Maybe one of the fuzzies is a gopher, maybe not.

Without further ado, my first impressions of Go follow below.

Pro Go

Productivity Now!

I’m very impressed with how quickly I was able to be at least minimally productive in Go.

When trying out a new language, like most people I want to see if I can use it for something in the real world. In this case I tried a simple web application, as I’ve been writing a bunch of those lately.

A web app should be easy to prototype in this day and age, but it should also be obvious how you can go deep and scale up to a complex and high-performance system. That said, it doesn’t exercise features of the language so much as the available libraries.

After a week of part-time hacking and quite a bit of reading around the web, I feel like I know how to build a robust web app using Go and some of the popular libraries; but also, and here is where Go stands out, I’m fairly confident I could do the same with just the standard library. We’ll soon see why that’s important, but it’s worth noting that I couldn’t do the equivalent in Perl or Javascript without a lot of brute force, and I’ve been writing in those languages since before some Silicon Valley billionaires got to high school.

In fact I can’t remember the last time I played with a new language and so quickly found myself able to write real software, software I understand, and not some Thy-CRUD-Be-Thus templated Sinatrail tiger-trap.

This is a big deal: you might need to hire people, and assuming you are not an idiot you will probably hire people who don’t yet have expertise in the particular software stack you’re using, because that’s not a proxy for intelligence or productivity. (Hiring that way is, as we say in the coding trenches, an anti-pattern, like hiring only Ivy Leaguers or only those who can solve math puzzles in an interview.) These smart people you’ve hired need to get “up to speed” quickly, and it looks to me like Go is a language that would give one a leg up in that inevitable struggle.

The main criticism of Go I’ve encountered so far on the Web is that it may be a practical and a useful language, but it is also ugly and lacks this or that feature. This sounds about right, but for me ugliness exists on a sliding scale: show me the language that gets things done and I’ll show you the language that’s ugly and lacking some feature. Productivity is its own kind of beauty, at least for us Engineers.

Static Types.

As a Perl guy I didn’t expect to find myself saying this, but here it is: I’m starting to prefer static typing. It’s harder to do some things that way, but when I think of all the boilerplate crap I have to put into my dynamically typed code to make it reasonably safe, and all the tests I have to write to make sure I didn’t screw it up, static types start to look pretty good. In my experiments with Swift the static types have been more helpful than not.

And it’s not like you can’t deal with arbitrary types when you need to. Go makes it easy to write things like func DoSomething(m interface{}) {...} – though of course it would be a bad habit to do that very often.

What I’ve grown to like about statically typed languages is what we might call the enforcement of intent: if I write a function to turn Gophers into Marmots then the compiler or interpreter, by default, will make sure that I only ever give it gophers and that I know marmots, specifically, will be returned. This is probably as valuable for its encouragement of design clarity as for its bug-reduction potential.

Readability.

I wouldn’t say Go is a particularly beautiful language, in an aesthetic sense. But it is fairly expressive and once you get used to a few quirks it’s quite easy to read. This is obviously a key part of productivity: every time you write public static void or my ($self,$params) = @_ a poet dies.

You do have bad things, things that are ugly and non-obvious and will be the object of ridicule when Go is a bit older: map[string]reflect.Type is, for example, a perfectly reasonable return value in Go. But most of it is fairly obvious and easily grokable if you’ve done a bit of programming already. (And if you have not, maybe you should see the turtle before you talk to the gopher.)

Compiling and Statically Linking.

This is another one of those things that might sound odd coming from a Perl guy, but when I think about deploying Go programs on servers – possibly on very many servers – I really like the idea that my program, including all its dependencies, will be compiled into a single binary file.

That’s probably not the only file I’d have to deploy, but it removes a lot of hassle from my deployment story, whether that’s just me deploying to a Linode or a team of IT operations experts deploying to a server farm on SeaLand.

Sure, this introduces the problem that you need to compile on the same platform you intend to deploy on; but isn’t that what virtual machines are for? And if you’re not already testing on your destination platform then you’re not really testing, so to my mind this is the smallest of inconveniences. Even for a one-off little project you ought to ssh into your host for a final compile-and-test anyway.

Big Fat Standard Library.

One of the great strengths of Perl is CPAN, a vast repository of free software that will save you immeasurable effort in writing your programs. Go has nothing quite like that (see below); but it does have a very deep standard library. This means you can:

1. Build a lot without external requirements.
2. Have a common frame of reference in talking about Go programming.
3. Fall back to a reasonable midpoint if you find an external package buggy.

The frame of reference might be the most important part of this. If I’m building a web server I can use the net/http package and chances are very good I will be able to get help, should I need it, on sites like Stack Overflow. Because just about everyone who “knows Go” will know how net/http works.

Building software with fewer external requirements can be a great advantage in itself, simply because every dependency introduces a risk of bugs. If I find something genuinely wrong with net/http I am very confident it will get fixed, and that the fix will then be part of the standard software. If I find something wrong with github.com/biztos/gogogo/http then maybe the author will fix it, or maybe not; or maybe I’ll fork it, or maybe somebody else already forked it, or maybe I’ll just throw some sand into the wind and see where it lands, which from a stability point of view isn’t much less useful.

It’s also nice to know you can fall back if you need to, and not have to fall back all the way to sockets. How far this is realistically possible depends on the domain in which you’re operating, but one could at least bring up a decent web API without much external code beyond the database drivers. This might be irrelevant if you’re building the Twitter-Enabled Platform for Instagram Drivesharing (TEPID.IO, YC16). But if you’re stuck in that old-school world of solving actual problems with software it should help you sleep at night.

Scalable and Fast (they say).

I have not yet done anything at all with the features of Go that should, and purportedly do, make it scalable and fast. Unless you count compiling my code as a minimalist demonstration of speed:

$ go build hello.go
$ time ./hello
Hello world.

real    0m0.005s
use     0m0.001s
sys     0m0.003s
$ time echo Hello world.
Hello world.

real    0m0.000s
user    0m0.000s
sys     0m0.000s 

Hmm... But seriously, there’s a lot of chatter online about Go programs being pretty fast, and until I’m in a position to judge (not likely soon) I will take the Internet’s word for it. In any case it ought to be fast-ish just by virtue of its being compiled, right? Right?

The thing that looks most promising for higher-performance programming in Go is its concurrency model. I have not used this at all yet, but I like the look of it, and the fact that its keyword is go makes me think they gave it some serious thought. In fact it looks so promising that I find myself trying to think up a programming task for which I would need it. Some kind of batch processing I suppose...

Cloud Friendly, and Trendy!

What is cloud friendliness anyway? A buzzword, a buzzphrase; maybe it’s meaningless. But when you are thinking about writing things that might need “cloud scale” it’s worth asking how friendly the “cloud” is to your language. Amazon likes Go. Rackspace likes Go. Heroku tolerates Go, in a friendly sort of way. Google App Engine supports Go, in a tentative sort of way.

The concurrency model should make it fairly straightforward to write programs that scale with the size of a compute host, and that ought to help in cloud-style applications. But in the end the trendiness may be the key here: trendiness gets you support (Heroku is Trendy!) and trendiness gets you free software mashups with other trendy things. (The Cloud is Trendy!)

The other thing trendiness gets you is enthusiasm. This might sound shallow, but at the end of the day you will not be the only one writing your software. Having people excited about working with the latest, trendiest thing will improve their work. And that will make you money.

Standards You Can Live With.

With Perl, there’s always more than one way to do anything, but there are some established best practices for common scenarios. With Javascript, there’s usually more than one way to do anything, and there are dozens of competing practices all claiming to be the best. With Go, as far as I can tell, there is a consensus about the “right” way to do most things.

In coding patterns, the Go folks call this idiomatic programming, by which they mean that which follows the language team’s preferences. This is probably a good thing, but it’s hard to see how to enforce it other than to have an appointed Arbiter of the Idiom. That can work in open-source projects, where the Go Community will happily play that role. Internal projects are another story. As far as I know there is no equivalent of PerlCritic, but that hardly puts Go at a disadvantage to any language other than Perl.

Other standards are more actionable. For instance, unit testing is to be done with the built-in testing software. Code formatting is to be done with the gofmt command – a pale, sickly shadow of Perltidy but one that for better or worse brooks no argument.

Project layout also follows a standard, but it’s a very problematic one as I’ll discuss below.

In the end there is a “Go Way” and you can save a lot of time by just submitting to it instead of arguing about its merits. When this works, it’s probably worth having it not be perfect. I’m right about indentation and they are wrong, but if I have to prove I’m right to one more f---ing nerd then I might as well save the effort and live with their wrong indentation. When it doesn’t work, however, it leads to nonstandard standards, which is a slippery slope indeed.

Plan Nine!

Once upon a time, we computer types thought the world needed new and better operating systems. It turned out the world didn’t exactly need them: the Apple computer I’m writing this on, and the server rendering it for you, and the iPhone on which I’m listening to music right now, are all running their fancy software on top of UNIX, which is venerable and beautiful and stable and definitely not modern. Windows limps along apace.

Before the rise of Linux and Steve Jobs’ return to Apple, people really did think there would be something new running the world’s computers by 2015. One of the most exiting new things was Plan 9, named after a classic low-budget sci-fi movie.

It was brilliant and advanced and promising and it looked a lot like the future. But Linux rose, Steve Jobs used NextStep to save Apple, Microsoft stayed in business, and new operating systems like Plan 9 didn’t come to much.

But it was a great idea. And some of the same people who made Plan 9 are behind Go.

No Go

Living with Half-Baked Standards.

I’m sure there are others, but so far I’ve repeatedly been stymied by two poorly thought-out Go standards: project layout and packaging. The standard is in both cases so oddly incomplete that I wonder about the bubble in which it must have been created. Nice, comfortable bubble, no mess here, ah the purity of the bubble...

Project Layout

Go projects are supposed to live in workspaces, and a workspace has a few standard directories: src, pkg, and bin, with pkg holding compiled packages so you don’t have to recompile every last thing every time. You keep your workspaces in your $GOPATH environment variable, which supports multiple workspaces. Multiple workspaces can be useful in development if you want all your third-party packages in one place.

Here for example is a workspace in which I’ve built and installed the foo package, which has a foo/bar package within it and uses one external package, github.com/kr/text:

bin/foo
pkg/darwin_amd64/
                foo/boo.a
                github.com/kr/text.a
src/
    foo/
        main.go
        boo/booing.go
    github.com/kr/text/
                      .git/[shit-ton of git stuff]
                      License
                      Readme
                      doc.go
                      indent.go
                      ...et al. 

You will have noticed that the License and Readme files are thrown in with the source code, which is a bit messy but not unreasonable. And the shit-ton of git stuff is there because I fetched the text package with the standard go get command, and that clones the git repo.

So far so good, but what happens when I have a Real Life Project, in which I have a bunch of things that are essential but are not source code? Templates, configuration files, resources, test data. Where should it live?

Obviously I should have something like this, more or less, for a simple web app:

src/foo/main.go
src/bar/some/other/package.go
templates/index.tmpl
static/foo.css
config/foo-dev.conf 

As far as I can tell, that’s not what Go wants. Go wants me to do this, which is not wise:

src/foo/main.go
src/bar/some/other/package.go
src/foo/templates/index.tmpl
src/foo/static/foo.css
src/foo/config/foo-dev.conf 

I have yet to encounter any defense of this craziness, nor any official way to sanely lay out real-world projects in which the source code is but one part among many. I’m an optimist so I’ll keep looking, but so far it looks like it’s up to me to hack my way around Go’s deficient workspace concept.

If I’m lucky somebody smarter than me will come up with a great solution and it will become the de-facto standard. This often happens when somebody writes a popular Web framework and ships it with a sample application and/or an application generator, the default layout of which is carried forward by thousands of people using the framework.

But if we’re going to look for de-facto standards to emerge from the community, and in the meantime roll our own, then what is the point of having an Official Standard?

Packaging

Go’s default package manager is simply the go get command, which fetches a package from a public repository, does any necessary unpacking and installing, and drops the results under the first directory in your $GOPATH. It’s very clever about cloning Git repositories, and thus gives you a lot of flexibility when dealing with GitHub-hosted open-source packages, which seem to be the majority.

This is great up to a point. That point would be when you want to ship software that’s supposed to work, the not-working of which would be your problem, and which has external dependencies. Then suddenly you realize – oh shit! – you forgot to specify the exact version of that cool parser package from GitHub. And it’s at 0.31 so even assuming malice and human error do not exist in the world, your code still might break when cool/parser hits 0.33 and the API changes.

I keep hoping I will run across some justification for this, since it’s such an obviously bad design yet Go was written by obviously smart people. I have read that “stable code on HEAD” is the answer, but whoever says that doesn’t understand the question. It’s not about hygiene, it’s about predictability. The official justification isn’t much help either.

My best guess is that Go was written in a context, in-house software development at Google, in which package versioning was not an issue, and as a result the authors simply didn’t think of it.

Standard Anarchy.

One of the wonderful things about the open-source world, especially in this age of cheap internet infrastructure, is the way problems are often solved for you before you are even aware of them.

In the case of Go, we have GoPkg.cc, which very elegantly solves the package versioning problem for packages hosted on GitHub (the majority, I believe).

We also have GoDoc for package discovery and documentation. Between these two things Go almost has a CPAN.

I get the sense that Go revels in its anarchy at the moment, while also trying to insist that there are standard ways of doing most things. Presumably something like GoPkg will become a standard, de-facto or otherwise, in time.

The problem I see with this mix of anarchy and imperative is that not everyone is going to agree on the original standards. This problem grows with the user base, particularly as older, more experienced (and more opinionated) programmers get involved. The standard documentation, for example, is ugly and hard to use. Let’s say I make a better documentation package: should I refrain? Should the best solution win, or the factory standard?

Much has been made of the time saved by having gofmt format source code in the One True Standard Way, thus obviating the need to argue about indentation. But if there’s no standard manager for versioned external packages, and no standard Posix-compliant option parser, then why shouldn’t I make a competing code formatter? Let’s say I do, and it catches on. This could be a very good thing, but reduces the point of gofmt to simply saying “automated code formatters are good.” We’ve been saying that in the Perl world for a long long time.

Objects, Not Objects, Deja Vu!

I almost put this in the Pro Go section, because after half a million years writing object-oriented code I welcome the challenge of not thinking in “OO” terms, and I very much look forward to learning more about good objectless design. And anyway Go is kinda-sorta object-y to begin with.

In the real world, however, somebody will devise a clever hack that messily glues object inheritance onto Go’s structs and interfaces, which is where its non-inheriting yes-and-no object-orientation already lives.

In the real world, this object hack will be derided as nonsense by Go purists, will be more or less disowned by the Go community, will be exposed as borderline fraudulent in six dozen blog posts and half that many conference presentations, and will nonetheless be used, largely out of sight, in thousands of Go projects.

I think it was a big mistake of the Go authors to not build in proper object orientation. I don’t see why they couldn’t have. I get that they don’t want people to write Go that way, but sadly I believe a lot of people eventually will; and there won’t be any official way to do it, so somebody’s unofficial solution will end up as the base of some large and indispensible package that sooner or later everybody, or at least you, has to use.

Google: The Glass is Half.

To paraphrase jricher42’s Reddit comment: Google is an advertising company that realizes it must produce excellent software in order to continue dominating the advertising market.

It would be a little ridiculous to expect such a company to over-commit to any particular technology. If you’re a Go lover, and particularly if you’re one who doesn’t love Java and Python, then you probably wish Google would “Go” all-in and make Go their primary language. Then the outside world would love Go even more, because after all it’s good enough to run Google; and Google would get a bunch of free training for potential engineering hires.

But that’s not how this works, and for Go’s sake that’s mostly a good thing. First, given that Google’s support of Go is a little tepid even now, I wouldn’t assume it has any special prominence in Mountain View. As recently as last year I had a Google recruiter pitch me without once mentioning Go: he said I could code in the language of my choice as long as it was Java or Python. Second, Google for obvious reasons insists on the flexibility to discard its experiments, no matter how much “the community” might care about them (viz. Reader), and for that reason alone it’s better if Go isn’t a “Google Thing” per se. Third, Google is every bit as closed-source and poker-faced as Apple when it comes to any software that actually makes it money. If Google comes to depend on Go, then the internal Google fork of Go will start growing proprietary features if it hasn’t already. Good for Google, bad for Go.

This adds up to a risk, and one that we must hope Go outgrows. Google is capricious, and exerts an influence on the language. It might overcommit, it might meddle, it might abandon all support of Go; and in each of these cases the language might suffer, to an unknown degree.

Youth Itself.

Go is a very young language. It’s been publicly available since November 2009, which makes it five years old as of this writing.

Five years is time enough to mature some but not really time enough to prove any staying power. These have been faddish years in computer languages, and while Go’s popularity is growing it’s hard to guess how far it’ll, er, go.

Consider Perl: so very not trendy, yet so well established that an enormous amount of very serious code is written in it every day, making a lot of people a lot of money (and the programmers some too). Most of it isn’t powering startups, and I suppose none of it is powering Google, but Perl remains a reliable industry workhorse. I would much sooner trust Perl’s DBI than anything gluing databases onto Go.

Will Go become that reliable? Will an investment in Go expertise pay dividends in ten years? In twenty? Will there be a canonical Oracle driver, Perforce and Atlassian integration, that sort of thing?

I like Go largely because it solves problems I actually have, and the “canonical Go program” is a web application, of which I expect to write many more in years to come. But Go’s youth is a risk and will be for at least a few years. It would be helpful to hear success stories about large-scale adoptions of Go at major companies other than Google. Are there any such stories?

Limited Sense of Humor.

Much like Google itself, I get the feeling that desipite its cute (?) gopher mascot the Goosphere takes itself a bit too seriously. The Perl folks are much wittier.

Case in point:

$ go ogle
go: unknown subcommand “ogle”
Run ’go help’ for usage. 

Further Reading

If you’re interested in using Go, I recommend you start by playing with it, or following the official tutorial, or reading a free online book. If you’re the learn-by-doing type that will probably be enough to get you excited or repulsed, depending on whether Go is a natural match for you.

I also strongly recommend reading a talk the Go authors published in 2012, Go at Google: Language Design in the Service of Software Engineering. That explains the why of Go as well as much of the how,
and I don’t think the FAQ really makes sense until you’ve read it.

Finally, I found it enlightening to read a few “Go vs X” writeups, such as Scala vs. Go on Quora.

Now go build something!