Interfaces in Go make it hard to navigate code(swageroo.com)
swageroo.com
Interfaces in Go make it hard to navigate code
http://www.swageroo.com/wordpress/googles-go-and-implicit-interface-declaration/?h
61 comments
> I assure readers that this kind of problem is only faced by beginners.
No, this problem is faced by anyone that needs to read code code written by other developers.
If you are faced by a code base written by a team of 30+ developers, good luck trying to find out which types implement which interface.
At least LiteIDE (http://code.google.com/p/liteide) is a possible help that eases the problem.
No, this problem is faced by anyone that needs to read code code written by other developers.
If you are faced by a code base written by a team of 30+ developers, good luck trying to find out which types implement which interface.
At least LiteIDE (http://code.google.com/p/liteide) is a possible help that eases the problem.
Like the author, I had the same problem when I was going through the GO tutorial as well (I have to say it's not the greatest tutorial for someone coming from a totally different programming background, I've never done C or Java for example). Interface in GO can be a little bit tricky to understand at first, but then they become kind of like a nice way of thinking about coding.
I can't help to think like what @tomp said. It's really up to the developers to annotate their code and document their Interfaces, specially when using a systems programming language like GO, comments are essential as the code is not as self-explanatory like Python or Ruby. GO provides an excellent tool to generate documentation from internal or external packages: http://golang.org/cmd/godoc/.
There's this great write-up on GO's Interface by Jordan Orelli: http://jordanorelli.tumblr.com/post/32665860244/how-to-use-i...
I can't help to think like what @tomp said. It's really up to the developers to annotate their code and document their Interfaces, specially when using a systems programming language like GO, comments are essential as the code is not as self-explanatory like Python or Ruby. GO provides an excellent tool to generate documentation from internal or external packages: http://golang.org/cmd/godoc/.
There's this great write-up on GO's Interface by Jordan Orelli: http://jordanorelli.tumblr.com/post/32665860244/how-to-use-i...
> No, this problem is faced by anyone that needs to read code code written by other developers.
No, it's not a problem for code readers. If you see a file being passed as an argument to a function which expects a Reader, you know that it implements Reader. Otherwise it wouldn't have compiled.
No, it's not a problem for code readers. If you see a file being passed as an argument to a function which expects a Reader, you know that it implements Reader. Otherwise it wouldn't have compiled.
How to answer the question "Given type X what are ALL interfaces that it implements from the overall code"?
First off, why do you need to know? The whole point is that interfaces are in no way tied to the things that implement them.
Second, you can search for the function definition(s) that define the interface. Search for Write(p []byte) if you're looking for Writers.
If you're in control of the interface, you can just modify it slightly and see what breaks.
Second, you can search for the function definition(s) that define the interface. Search for Write(p []byte) if you're looking for Writers.
If you're in control of the interface, you can just modify it slightly and see what breaks.
You still haven't said why you would need to find all implementations of an interface. "Because I'm new to the company" is not a reason.
The thing about interfaces is that the function that takes the interface is the thing that defines how it is used. The implementation of the type that implements the interface is irrelevant.
log.Logger writes logging output to an io.Writer. It doesn't care what the implementation of any particular writer is... the writer could be sending data across the network, it could be writing it to the console, could be writing to a file... doesn't matter. Logger just calls Write() and merrily goes about its business. Trying to look up all the implementations of the io.Writer interface is not useful when looking at log.Logger. They're mostly unrelated.
Now, if you're trying to figure out how to do some specific thing, like log to a Windows named pipe, then you'd want to start looking around your named pipe library to see if it implements Write().... which it almost certainly does.
The thing about interfaces is that the function that takes the interface is the thing that defines how it is used. The implementation of the type that implements the interface is irrelevant.
log.Logger writes logging output to an io.Writer. It doesn't care what the implementation of any particular writer is... the writer could be sending data across the network, it could be writing it to the console, could be writing to a file... doesn't matter. Logger just calls Write() and merrily goes about its business. Trying to look up all the implementations of the io.Writer interface is not useful when looking at log.Logger. They're mostly unrelated.
Now, if you're trying to figure out how to do some specific thing, like log to a Windows named pipe, then you'd want to start looking around your named pipe library to see if it implements Write().... which it almost certainly does.
Because in the enterprise world we have these boring projects with 60+ developers scattered around the world, high rotation of developers and new guys need to be able to jump into the code of several megabytes of text?
I think this could mostly be solved by automatic documentation generation tools and/or IDEs. Just like Java IDEs and JavaDoc index the world and show relations (i.e. classes that implement an interface), so could (and probably will) Go tools do so.
"I assure readers that this kind of problem is only faced by beginners"
I disagree. I've even brought this up before in passing on HN:
https://news.ycombinator.com/item?id=5639846
I've written many thousands of lines of Go code and have been using the language for more than a year and I still think the gist of the author's argument is a valid pain point when it comes to Go documentation. I'm now very familiar with the standard Go library, so the problem doesn't impact me as much as it used to, but I do find auto-generated Go documentation for new libraries I haven't yet used to be much harder to get a quick mental overview of than code autogenerated from other languages I've programmed in (including languages which are OO, functional or procedural).
Go is still my favorite language to program in, but I think there is a valid issue here (one that can be fixed with improved tooling)
I disagree. I've even brought this up before in passing on HN:
https://news.ycombinator.com/item?id=5639846
I've written many thousands of lines of Go code and have been using the language for more than a year and I still think the gist of the author's argument is a valid pain point when it comes to Go documentation. I'm now very familiar with the standard Go library, so the problem doesn't impact me as much as it used to, but I do find auto-generated Go documentation for new libraries I haven't yet used to be much harder to get a quick mental overview of than code autogenerated from other languages I've programmed in (including languages which are OO, functional or procedural).
Go is still my favorite language to program in, but I think there is a valid issue here (one that can be fixed with improved tooling)
I am someone who has shipped code in many "less popular" languages (Erlang, Eiffel, Lisp...) and am currently exploring / building apps with Go. As someone with some actually time with Golang, could you elaborate on the pain point a bit?
I had a similar issue on my first toy program with Golang, but found it trivial to check the implemented methods, or assign it and let the compiler blow up if it doesn't fulfill the needs.
I had a similar issue on my first toy program with Golang, but found it trivial to check the implemented methods, or assign it and let the compiler blow up if it doesn't fulfill the needs.
I find the author's particular problem utterly baffling from either direction. When presented with io.Reader's specification:
This seems like a basic failure to understand what a Go interface is, nevermind any understanding of the standard library.
type Reader interface {
Read(p []byte) (n int, err error)
}
This seems like a basic failure to understand what a Go interface is, nevermind any understanding of the standard library.
I'm not a Go programmer, but this seems like implicit interfaces could bite you in the ass pretty easily.
For example if I have a Book class that implements a Read method (to mark the book as having been read, for example) then have I just created something which implements io.Reader?
For example if I have a Book class that implements a Read method (to mark the book as having been read, for example) then have I just created something which implements io.Reader?
The signature must match as well, not only the name. And even if it matched by coincidence, which is rare enough, in order to mix them up, you would have to be a programmer without any clue what he's programming, randomly combining objects and calling methods without knowing what they're supposed to do.
You just described how off-shoring projects work.
Only if you had a method with that exact signature: taking a []byte argument and returning an int and an error. You'd have to stretch to have that happen by chance.
Ok, I did think that would be part of it but the only reference I could find in the documentation was at http://golang.org/doc/faq#overloading which suggested that names are the only thing considered. (The other document I looked at before was http://golang.org/doc/effective_go.html which didn't help)
Fair enough, I did say I was only speculating and I've learned something.
Fair enough, I did say I was only speculating and I've learned something.
Aside from the accurate response that the signature needs to match, you must understand that Go's type system is deliberately a meld of a very strict type system, and tools that allow you to beat it into a form of duck typing.
It's trying to merge the best parts of two very different worlds. A modest loss of safety is therefore tolerated.
It's trying to merge the best parts of two very different worlds. A modest loss of safety is therefore tolerated.
Honestly, it's only a loss of safety if you randomly throw types you don't understand into functions you don't understand. And actually... most of the time this will do exactly what you want it to do, especially if you're talking about io.Writer and io.Reader.
Only if your Book.Read method has the same signature as Reader.Read (i.e., not just the same name).
This was one of those problems I had in the first month of using Go... along with import dependency cycles (as a consequence of code organisation), and not understanding the idiomatic approach to some things and writing completely unnecessary tracts of code.
Now that I have unlearned some of the approaches from other languages I actually feel this is a blessing. To oversimplify it: Anything that shares the same signature can pretty much be called as if it were that thing.
This gives the illusion of some fairly dynamic code (being able to assign some function or interface at runtime within compiled code) and allows for extremely low-coupling and for radical re-factoring to be extraordinarily simple to achieve.
Now that the penny has dropped I no longer find this as mysterious and worrying as the first few times I encountered it (and io.Reader was the most frequent instance of encountering it).
All that said... if the tooling (GoSublime via SublimeText2 in my case) were to show what matched these interfaces then that penny would have dropped a lot sooner.
Now that I have unlearned some of the approaches from other languages I actually feel this is a blessing. To oversimplify it: Anything that shares the same signature can pretty much be called as if it were that thing.
This gives the illusion of some fairly dynamic code (being able to assign some function or interface at runtime within compiled code) and allows for extremely low-coupling and for radical re-factoring to be extraordinarily simple to achieve.
Now that the penny has dropped I no longer find this as mysterious and worrying as the first few times I encountered it (and io.Reader was the most frequent instance of encountering it).
All that said... if the tooling (GoSublime via SublimeText2 in my case) were to show what matched these interfaces then that penny would have dropped a lot sooner.
Can this information be automatically deduced by tools? If so, it's a great opportunity for tools to help out with understanding, while maintaing all the benefits of duck typing when the writing code.
> Can this information be automatically deduced by tools?
Yes, it can and it should be fairly easy. The Go standard library already has a "go/ast" package. Would probably be similar to this documentation tool: http://code.google.com/p/rspace/source/browse/doc/doc.go?rep...
Yes, it can and it should be fairly easy. The Go standard library already has a "go/ast" package. Would probably be similar to this documentation tool: http://code.google.com/p/rspace/source/browse/doc/doc.go?rep...
Given that it's automatically deduced by the compiler, I would hope so :)
It's not that simple. I mean, given an os.File and a function that requires an io.Reader, it's pretty straightforward to discover that square peg goes in square hole. But just given either half of the relationship it's not currently so easy:
"This function needs an io.Reader, how would I get my hands on one of those?" Go doesn't currently have an easy way to tell you.
"This os.File has a bunch of stuff implemented. What can I do with it?" Go doesn't currently have an easy way to tell you.
The hard problem that remains to be solved is discovery. Someone somewhere needs to take a good long look at your project, the libraries available on your computer, and the standard library and connect the dots. It should be possible, most IDEs have this for other languages, when Go grows up you would expect these to be flagship features: "Find all known implementations of a given interface" and "Find all interfaces this type satisfies."
For example, you might eventually expect the locally generated versions of these two pieces of documentation to refer to each other, even though they currently do not:
http://golang.org/pkg/io/#Reader http://golang.org/pkg/os/#File
godoc already aggregates all your local packages' documentation into one place, adding cross-references to known implementations seems like an obvious incremental addition.
"This function needs an io.Reader, how would I get my hands on one of those?" Go doesn't currently have an easy way to tell you.
"This os.File has a bunch of stuff implemented. What can I do with it?" Go doesn't currently have an easy way to tell you.
The hard problem that remains to be solved is discovery. Someone somewhere needs to take a good long look at your project, the libraries available on your computer, and the standard library and connect the dots. It should be possible, most IDEs have this for other languages, when Go grows up you would expect these to be flagship features: "Find all known implementations of a given interface" and "Find all interfaces this type satisfies."
For example, you might eventually expect the locally generated versions of these two pieces of documentation to refer to each other, even though they currently do not:
http://golang.org/pkg/io/#Reader http://golang.org/pkg/os/#File
godoc already aggregates all your local packages' documentation into one place, adding cross-references to known implementations seems like an obvious incremental addition.
How many interfaces are there? You can iterate over all of them and see if the module implements them.
If you are doing this for many modules at a time you can do a lot better by caching signatures.
If you are doing this for many modules at a time you can do a lot better by caching signatures.
First, the grievances expressed by the author can be solved by better documentation tools: http://godoc.org/image/gif#Decode clicking "io.Reader" will take you to its definition, pretty useful when navigating messes like xgb.
The author has avoided reading the "Learning Go" documents on the Go website which explains these differences from other languages http://golang.org/doc/ .
I'd like to argue against writing explicit implementation annotations in code unless really necessary. If the author had read the documents I mentioned earlier, he would have read this: http://golang.org/doc/effective_go.html#interface-names and realised that any value that implements io.Reader will have a function called Reader, no need for messy lists of code annotations describing lists of interface implementations. The authors complaint is somewhat like complaining that a car is not sold with the text "this goes into a garage" because otherwise they couldn't know- you'd expect the Car to implement the Volume interface the Garage takes so you could check.
To me, the way interfaces function is more intuitive. The author's problem is that they are solving the problem in reverse, it is strange to me that they would start the problem of reading a GIF from the end of 'decoding it' and not the end of 'reading it from something', the point of io.Reader is that it represents anything that can be read, whatever method he uses to read a GIF, be it out of a buffer, a connection, or a file will return a value that implements io.Reader if it is good code.
The author has avoided reading the "Learning Go" documents on the Go website which explains these differences from other languages http://golang.org/doc/ .
I'd like to argue against writing explicit implementation annotations in code unless really necessary. If the author had read the documents I mentioned earlier, he would have read this: http://golang.org/doc/effective_go.html#interface-names and realised that any value that implements io.Reader will have a function called Reader, no need for messy lists of code annotations describing lists of interface implementations. The authors complaint is somewhat like complaining that a car is not sold with the text "this goes into a garage" because otherwise they couldn't know- you'd expect the Car to implement the Volume interface the Garage takes so you could check.
To me, the way interfaces function is more intuitive. The author's problem is that they are solving the problem in reverse, it is strange to me that they would start the problem of reading a GIF from the end of 'decoding it' and not the end of 'reading it from something', the point of io.Reader is that it represents anything that can be read, whatever method he uses to read a GIF, be it out of a buffer, a connection, or a file will return a value that implements io.Reader if it is good code.
Whenever I read something like this I think that every programmer should be required to spend a significant amount of time working in a large well-written dynamically typed code base. It would certainly help them see that languages and tools are not the culprit when it comes to navigability.
The problem is the "well written" part.
One reason I don't like dynamic languages for large scale applications, is that I have already seen how it works in the world of multi-site enterprise software with outsourcing partners around the world.
The type of companies where unit testing are seen as buzzwords from Silicon Valey startups and are only written if the customer explicitly requires them as part of the contract.
So I am yet to see any "large well-written dynamically typed code" out in the wild, at least in my area of work.
One reason I don't like dynamic languages for large scale applications, is that I have already seen how it works in the world of multi-site enterprise software with outsourcing partners around the world.
The type of companies where unit testing are seen as buzzwords from Silicon Valey startups and are only written if the customer explicitly requires them as part of the contract.
So I am yet to see any "large well-written dynamically typed code" out in the wild, at least in my area of work.
Outsourcing is the problem here. I've seen some really terrible code written by outsourcing companies, even in static typed languages that are as simple to write as C#. It's not the language, it's the writers.
There can definitely be a problem here if you've built very complex interfaces - but that isn't the "Go Way." The documentation suggests make one- or two- method interfaces, with interface names that suggest the methods you must implement. The interface name is the required function + "er". Hence if you have an interface "Peeker" you can expect that the method types must implement is "Peek".
I agree with the OP. The root of the problem, as I see it, is that you can't state in the documentation which interfaces a data type implements, because the interfaces may be declared outside of the "field of view". I think Haskell's solution to this problem is particularly elegant.
You can state in documentation what interfaces a type implements. It is not uncommon to do that. For instance, the os package could have written something like:
// File represents an open file.
// It implements the io.Reader, io.Writer and io.Closer interfaces.
type File struct {
...
}Relying on people to do things consistently is not a good strategy. I'm sitting in front of a machine that I'd like to do that for me. Having to document this information manually is a disadvantage of Go when compared to other languages.
If I make a new interface that is an mix of io.Reader, io.Writer and io.Closer and I call this interface YouDidNotSeeMeComing -- then the file struct would also implement it... without even knowing it!
I read the article this way:
I'm brand new to Go. I read neither the os.File documentation, nor the io.Reader documentation, and I'm baffled why I couldn't figure out how to make a File into a Reader.
I'm brand new to Go. I read neither the os.File documentation, nor the io.Reader documentation, and I'm baffled why I couldn't figure out how to make a File into a Reader.
There's a real complaint here, which is that there's not (yet) any tooling to find all known implementations of an interface, or all known interfaces satisfied by a type. Someday I expect there will be, and maybe this blog post helps spur some of that to be created, so it's certainly not without value.
That's true. However, there's no such tooling for any other language, either. There's nothing in Java that says "If you have a file and you need to pass it into something that needs a Reader, use FileReader". You have to do some digging to find the right class. (fudge my class names, I'm not a java guy)
"How do I convert a Foo into a Bar" is a problem in all languages.
"How do I convert a Foo into a Bar" is a problem in all languages.
Sure there is. For example the FileReader class documentation[1] has a listing of "All implemented interfaces," and the Readable interface documentation[2] has a listing of "All Known Implementing Classes." Pretty much every IDE I know of extends that to include classes from the current package and referenced libraries.
So if you need a Readable in Java ("Reader"[3] is an abstract base class, which incidentally lists all of its known subclasses, including FileReader if you follow through InputStreamReader), you have a ready listing available of all the common implementations, which solves this issue straight away, in a way that Go currently lacks.
There's nothing stopping Go from having this tooling too (with the caveat that interface cross-references would have to be "All known implemented interfaces" instead of "All implemented interfaces"), it's just not there yet.
[1]: http://docs.oracle.com/javase/7/docs/api/java/io/FileReader....
[2]: http://docs.oracle.com/javase/7/docs/api/java/lang/Readable....
[3]: http://docs.oracle.com/javase/7/docs/api/java/io/Reader.html
So if you need a Readable in Java ("Reader"[3] is an abstract base class, which incidentally lists all of its known subclasses, including FileReader if you follow through InputStreamReader), you have a ready listing available of all the common implementations, which solves this issue straight away, in a way that Go currently lacks.
There's nothing stopping Go from having this tooling too (with the caveat that interface cross-references would have to be "All known implemented interfaces" instead of "All implemented interfaces"), it's just not there yet.
[1]: http://docs.oracle.com/javase/7/docs/api/java/io/FileReader....
[2]: http://docs.oracle.com/javase/7/docs/api/java/lang/Readable....
[3]: http://docs.oracle.com/javase/7/docs/api/java/io/Reader.html
Tooling should get us part of the way - it surely cannot be that difficult to interactively show which interfaces a type satisfies (and conversely, what you'd need to add to a type to have it satisfy an interface).
In general it would be neat if the "go" command supported some more advanced static analysis stuff beyond go fix, fmt, and vet.
A lot of this stuff could be done by reflect -- if only there was a repl!
In general it would be neat if the "go" command supported some more advanced static analysis stuff beyond go fix, fmt, and vet.
A lot of this stuff could be done by reflect -- if only there was a repl!
Kind of feels like a round hole, square peg kind of complaint (Go doesn't really even pretend to be like Java, and Java has an exceptional level of abstraction for these sorts of things), but it probably would be nice to have a canonical way of annotating known interfaces implemented.
Often times the documentation on a function will state
// Read implements io.Reader
// Read implements io.Reader
> So all this time wasted trying to figure out what to pass into the Decode function could have been easily saved if os.File simply wrote “implements io.Reader” in the code itself.
imho, an "implements" keyword would be a bad choice to describe what interface a type implements partly because it introduces dependencies between types which might not exist. also, you don't have to design this type-hierarchy from the very beginning. you may notice a particular pattern in a bunch of libraries after they are written, and noticing this you can describe an interface which captures it. and in some cases it might not even be possible to annotate this type information, as the source-code might even be available...
imho, an "implements" keyword would be a bad choice to describe what interface a type implements partly because it introduces dependencies between types which might not exist. also, you don't have to design this type-hierarchy from the very beginning. you may notice a particular pattern in a bunch of libraries after they are written, and noticing this you can describe an interface which captures it. and in some cases it might not even be possible to annotate this type information, as the source-code might even be available...
It's only hard to navigate because the linguistic relativity of the languages you're used to programming in. The world as seen through Go is not conceptually comparable to Java-like languages, thus it is difficult to navigate with a java-like worldview.
> The world as seen through Go is not conceptually comparable to Java-like languages, thus it is difficult to navigate with a java-like worldview.
What a load of hand-wavey nonsense.
What a load of hand-wavey nonsense.
Exactly, his issue could (and eventually probably will) be resolved by added "interface implementation awareness" to tooling such as gocode (the IDE helper) or GoDoc.
LiteIDE (http://code.google.com/p/liteide) does it.
I had the same problem as a beginner (http://stackoverflow.com/q/14577162/317915) and I think it's a valid criticism. If the documentation (go doc) could help that would be great.
This guy should really try Python and figure out that the lack of "implements Something" is not really a big deal...
If the author had gone through the documentation or even asked around the mailing list, the first thing he would learnt was to expect common packages to implement "common-sensical" interfaces.For e.g: when I first used os.File, my first thought was, "Oh, it must be implementing the io.Reader in there somewhere, since you know, its a File package". Next thing was to look up the os.File and io.Reader package and confirming it. In time maybe this "common sense" can be further improved by having a hand-holding IDE.
That's what people used to say about "script languages". Now they are called dynamic languages and nobody seems to mind...
I wish the go documentation would provide a list of all the interfaces which are implemented by functions or vice versa, it would make finding usable functions much easier.
The lack of clearly defined interfaces in the documentation was one of the big turn offs when I tried go a while back.
The lack of clearly defined interfaces in the documentation was one of the big turn offs when I tried go a while back.
That's impossible because godoc cannot possibly know about all interfaces that is in all the Go code out there.
It can know about all the interfaces in your current $GOROOT and $GOPATH, which is the set of code you'd be interested in anyway.
Installing packages is now quadratic time. Go doesn't scale.
The doc could only display implemented interfaces that it knows about, so when you generate the docs they show which interfaces a type implements from within that particular library (and it's dependencies, since go takes it all as source code). You wouldn't, however, know which interfaces from github.com/foo are implemented by types of bitbucket.org/bar unless their docs were somehow generated together.
Cached version:
http://webcache.googleusercontent.com/search?q=cache:http://...
Website is working again
I assure readers that this kind of problem is only faced by beginners. If the author approached the problem from the other direction, i.e. by asking "how do I extract a file's contents?", he would have looked up the documentation for os.File, and found it had a Read() method.
It's not often that you come across an arbitrary interface you have to implement, but aren't sure what to provide. If the documentation doesn't say, then presumably there's an type in the same package that you're meant to use. If not, domain-specific knowledge will help.
In this case it's necessary to have a basic idea of the way the standard library works - which as a beginner, the author doesn't have.
We rarely want a list of all possible types that can implement an interface. There are many different types that implement io.Reader in the standard library, it's not necessary for any particular package to be aware of all others ("slick"). And anyway, your editor might be able to help.
http://golang.org/pkg/os/#File.Read