Ask HN: What are you using Go for?
I'm interested in knowing what HNers are doing with Go (other than learning about it, though it would be interesting to hear about learning projects too). Thanks.
10 comments
A collaborative music jukebox that lets multiple people listen to the stream over a network. Because cloud-based jukeboxes seem to be all the rage, but none of them let you play music with other people at the same time. We gave it a fun name too: https://github.com/turbowookie/turbo-wookie. Go is used for the HTTP server, and is basically a layer over interacting with MPD.
Also working on a flat-file blogging system with an online editor, because flat-file is the middle ground between a static-site generator and a traditional database-based blogging engine.
Also working on a flat-file blogging system with an online editor, because flat-file is the middle ground between a static-site generator and a traditional database-based blogging engine.
After dabbling with it for a couple of weeks, I wrote the backend (search & indexing implementation using the Go standard library's suffix array implementation) of our websites' autocomplete feature. It's been running for 7 months without a crash and without leaking memory, serving several million requests/day.
I'd like to use it more, but we have so many layers of legacy Perl code, that it's difficult to replace single components because of their dependencies (rewriting anything worthwhile would mean rewriting 4-5 libraries also).
I'd like to use it more, but we have so many layers of legacy Perl code, that it's difficult to replace single components because of their dependencies (rewriting anything worthwhile would mean rewriting 4-5 libraries also).
It's been running for 7 months without a crash and without leaking memory, serving several million requests/day.
Sounds cool. What kind of server are you running this on?
Sounds cool. What kind of server are you running this on?
> What kind of server are you running this on?
It runs on a single node of a 4-in-1 supermicro server with dual Opteron 6136 CPUs. Because it uses so little CPU time (about 15% of one core on average), we are now using the server for other things as well ...
It runs on a single node of a 4-in-1 supermicro server with dual Opteron 6136 CPUs. Because it uses so little CPU time (about 15% of one core on average), we are now using the server for other things as well ...
Cool ...
As the public entry point towards many internal APIs. Its lighter and faster than any scripting language. Gorilla toolbox made it very easy to write.
Interesting use case.
It was to simplify a rather complicated system that used bots. Each bot would report home by requesting different URLs. That seemed overly complicated to me, because one URI change would break every single bot. I then created the public interface and simplified it to one URI. Then the data structure being sent to the APIs got an additional parameter. It told the public interface which API had to get that data. This was a very simple change, because the data structure was defined by the APIs and common among all. It ended up being very fast and allowed the bots to spend less time waiting for a response. Was planning to add a messaging queue but then I left the project (I was a consultant).
>Then the data structure being sent to the APIs got an additional parameter. It told the public interface which API had to get that data.
Nice technique.
Nice technique.
I call it the "Boarding Pass Pattern." I modeled it after the typical airplane boarding pass where everyone has the same data structure yet is going to different places/doing different things. :)
Good name :) Another possible name could be the Differentiator Pattern (heh, just a wild try), since it differentiates between calls which otherwise have the same data. Or even the Call Router pattern, for obvious reasons ...
I'm using it to scrape a popular gaming API endpoint to get info on hundreds of matches per day. I chose Go because it's fast, easily deployable and I enjoy it. :)
Dotabuff? Awesome work if its you.
Just checked the github page for it. Looks interesting and useful. Could be used to push system monitoring data to a browser (among tons of other uses, of course). Trying it out ...
Here's an example of using it to build a quick Linux system monitoring dashboard: https://github.com/joewalnes/web-vmstats
Wrote something similar but without much GUI, here:
http://jugad2.blogspot.in/2014/01/websocketd-and-python-for-...
websocketd is a good tool.
http://jugad2.blogspot.in/2014/01/websocketd-and-python-for-...
websocketd is a good tool.
Thanks, I'm checking that one out.
I work with streaming data and Go has been great reducing the number of backend workers (formerly in Ruby) to handle about 1 TB/day in real time. It's very stable and great for building an internal data aggregator.
A friend has also been using Go to create a board game as his side project.
A friend has also been using Go to create a board game as his side project.
Bytengine : https://github.com/johnwilson/bytengine
Work in progress but man is Go fun!
Had a quick look at Bytengine. Like the idea of using Linux-like file paths to refer to documents.
>Files are JSON documents to which you can attach any binary/text data (docx, txt, pdf, jpeg, etc...).
I'm guessing you store the filename(s) of the attached data (docx, txt, pdf, ...) in the JSON fragment?
>Files are JSON documents to which you can attach any binary/text data (docx, txt, pdf, jpeg, etc...).
I'm guessing you store the filename(s) of the attached data (docx, txt, pdf, ...) in the JSON fragment?
Docker is writing in Go (http://www.docker.io/)
Yes, had seen that some days ago, but thanks.
I had blogged about Docker using LXC, as part of this post about Domino, a Python PaaS for data science:
http://jugad2.blogspot.in/2013/12/domino-paas-for-data-scien...
and had seen then that Docker uses Go, though I didn't say so in the post.
Cool.
I had blogged about Docker using LXC, as part of this post about Domino, a Python PaaS for data science:
http://jugad2.blogspot.in/2013/12/domino-paas-for-data-scien...
and had seen then that Docker uses Go, though I didn't say so in the post.
Cool.
I've been using it primarily for a little bit of systems programming and writing internal utilities at work.
Utilities seems like a good use of Go, apart from the others I'm aware of. I mean command-line ones.
> Utilities seems like a good use of Go
Can you elaborate? I'm unfamiliar with Go, but most of my Python (and Perl, sigh) coding could be classified as "utilities for work". I would have thought that any language with an explicit compile step would be at a disadvantage in this space.
Can you elaborate? I'm unfamiliar with Go, but most of my Python (and Perl, sigh) coding could be classified as "utilities for work". I would have thought that any language with an explicit compile step would be at a disadvantage in this space.
>Can you elaborate?
Sure.
>I would have thought that any language with an explicit compile step would be at a disadvantage in this space.
Not exactly, and not always. In fact it can be the opposite. It depends on a few factors, and there are tradeoffs.
Consider that so many of the command-line utilities that come bundled with Unixen (Unix, Linux, BSD) are written in C - just look in /bin, /usr/bin, etc (some of the default dirs in $PATH). (Do a "file * | less" in those dirs.)
(Though there definitely are utilities that are written purely in *sh, or Perl, or Python, and more so these days than some years ago. Still, some, particularly shell scripts, tend to make calls to compiled utilities such as sed, awk, grep, ls, du, df, ps, and many other commands.)
For utilities used often or heavily, or where performance is important, a compiled binary can often be a good choice: lower startup time (don't have to start the interpreter to run the script, often faster run time (because running compiled machine code vs. interpreted or JIT'ed).
Go has performance closer to C than Python does, and also has some aspects of HLL's (High Level Languages) like Python. The compile only has to be done once per machine or platform, remember; also, Go compilation is quite fast, which was a design goal of the language. And if you compile once on one machine, you can just copy the binary (in an automated way, too) to any number of other machines running the same OS and version. With a self-sufficient binary, you avoid both the issues of missing or incorrect dynamically loaded libraries of languages like C (if the C program uses them), and the issues of missing or wrong versions of required libraries, with languages like Python.
Those are some reasons I could think of, off the top of my head. There may be others, both pro and con.
Sure.
>I would have thought that any language with an explicit compile step would be at a disadvantage in this space.
Not exactly, and not always. In fact it can be the opposite. It depends on a few factors, and there are tradeoffs.
Consider that so many of the command-line utilities that come bundled with Unixen (Unix, Linux, BSD) are written in C - just look in /bin, /usr/bin, etc (some of the default dirs in $PATH). (Do a "file * | less" in those dirs.)
(Though there definitely are utilities that are written purely in *sh, or Perl, or Python, and more so these days than some years ago. Still, some, particularly shell scripts, tend to make calls to compiled utilities such as sed, awk, grep, ls, du, df, ps, and many other commands.)
For utilities used often or heavily, or where performance is important, a compiled binary can often be a good choice: lower startup time (don't have to start the interpreter to run the script, often faster run time (because running compiled machine code vs. interpreted or JIT'ed).
Go has performance closer to C than Python does, and also has some aspects of HLL's (High Level Languages) like Python. The compile only has to be done once per machine or platform, remember; also, Go compilation is quite fast, which was a design goal of the language. And if you compile once on one machine, you can just copy the binary (in an automated way, too) to any number of other machines running the same OS and version. With a self-sufficient binary, you avoid both the issues of missing or incorrect dynamically loaded libraries of languages like C (if the C program uses them), and the issues of missing or wrong versions of required libraries, with languages like Python.
Those are some reasons I could think of, off the top of my head. There may be others, both pro and con.
More reasons: some of the Go language creators were heavily involved in the creation of Unix and C, and while I don't know it for a fact, it's pretty likely that they intended and designed Go to be useful for command-line tools as well, e.g. servers, utilities, etc. Also, Go has the flags package for handling command-line arguments (though other languages like Python do have various libs for that too).
Also, though not directly related to your question, Torbit (later acquired by Walmart Labs), used Go, due to its combination of productivity and performance. Josh Fraser, Torbit co-founder, told me that.
http://www.torbit.com/
http://www.onlineaspect.com/about/ (Josh's site)
http://www.torbit.com/
http://www.onlineaspect.com/about/ (Josh's site)
> a compiled binary can often be a good choice
Uh, well, sure. From the context, I thought little ad hoc utilities were meant. The kind that are changed frequently. We keep many right in our svn repository, so you make a check-out and make use of a little script right in the same directory as your project source code.
Uh, well, sure. From the context, I thought little ad hoc utilities were meant. The kind that are changed frequently. We keep many right in our svn repository, so you make a check-out and make use of a little script right in the same directory as your project source code.
>I thought little ad hoc utilities were meant.
The thing is, given Go's compilation speed, little utilities will compile so fast that you'll barely notice it (on any decently powerful machine), and on the other side there's the overhead of starting up the Python or Perl interpreter.
There's also the "go run" command, so you don't even have to give two separate commands (compile and run).
http://golang.org/cmd/go/
The command "go run filename.go" compiles the specified Go source file and then runs the resulting binary (assuming no compile-time errors, of course).
The thing is, given Go's compilation speed, little utilities will compile so fast that you'll barely notice it (on any decently powerful machine), and on the other side there's the overhead of starting up the Python or Perl interpreter.
There's also the "go run" command, so you don't even have to give two separate commands (compile and run).
http://golang.org/cmd/go/
The command "go run filename.go" compiles the specified Go source file and then runs the resulting binary (assuming no compile-time errors, of course).