Ask HN: Ways for reading open source code bases?
9 comments
Once upon a time, a client of the day job dropped a substantial Perl codebase on our desk and said "Tell us what this does." My boss gave me the job, and expected me to actually READ all the code, but given that I had no desire to read through 100 kloc of Perl code commented only in Japanese, I went for visualization instead:
1) Inspect several files for commonalities. Thankfully, the author was obsessive compulsive about coding standards.
2) Write a parser for the Perl they used. Use it to glean what pages of the site were connected to each other and what the flow control was like. (a -> b, b -> c, b -> d, etc)
3) Plot that on a graph (all hard work already done for me: http://rgl.rubyforge.org/rgl/index.html )
4) Visually inspect the graph to learn non-obvious things about the codebase like "Oh, there is an English language version of the site embedded in here. Isn't that TOTALLY UNDOCUMENTED." Do a bit more code to chop the graph into subgraphs by related functionality (signup flow, admin functions, etc etc).
5) Spit out all the code into HTML pages with appropriate autogenerated navigation, inline flow control graphs, and syntax highlighting. Do a bit of quality control, add in some comments about notable things I had learned, burn on CD and hand to customer.
6) Charge customer $X0,000 for the CD. The customer was overjoyed they got it done so cheaply. (Did I mention 100 kloc of Perl?)
1) Inspect several files for commonalities. Thankfully, the author was obsessive compulsive about coding standards.
2) Write a parser for the Perl they used. Use it to glean what pages of the site were connected to each other and what the flow control was like. (a -> b, b -> c, b -> d, etc)
3) Plot that on a graph (all hard work already done for me: http://rgl.rubyforge.org/rgl/index.html )
4) Visually inspect the graph to learn non-obvious things about the codebase like "Oh, there is an English language version of the site embedded in here. Isn't that TOTALLY UNDOCUMENTED." Do a bit more code to chop the graph into subgraphs by related functionality (signup flow, admin functions, etc etc).
5) Spit out all the code into HTML pages with appropriate autogenerated navigation, inline flow control graphs, and syntax highlighting. Do a bit of quality control, add in some comments about notable things I had learned, burn on CD and hand to customer.
6) Charge customer $X0,000 for the CD. The customer was overjoyed they got it done so cheaply. (Did I mention 100 kloc of Perl?)
(Did I mention 100 kloc of Perl?)
Hey, you got off easy. Just imagine how much more work it would have been if everything had been done in a single line of Perl. :-)
Hey, you got off easy. Just imagine how much more work it would have been if everything had been done in a single line of Perl. :-)
Or it might have been Klingon.
(Lest I be considered entirely off topic: http://www.perl.com/lpt/a/560 http://www.perlmonks.org/?node_id=88704 and search "klingon". Better the salariman, than Conway in a playful mood.)
(Lest I be considered entirely off topic: http://www.perl.com/lpt/a/560 http://www.perlmonks.org/?node_id=88704 and search "klingon". Better the salariman, than Conway in a playful mood.)
70 hour workweeks, 3 hour daily commutes, salary below the median for a starting graduate from my alma mater, and debugging code written in India based on instructions translated by Babelfish: all of these I will endure without complaint. But I will not blackbox analyze Perl one-liners. That is just abusive.
Yes. When I write anything in Perl I frequently try to compact what would be several lines in just a few obscure expressions.
But as the Freebsd security head, what do you do to understand a large code base you probably aren't familiar with and its security implications?
But as the Freebsd security head, what do you do to understand a large code base you probably aren't familiar with and its security implications?
But as the Freebsd security head, what do you do to understand a large code base you probably aren't familiar with and its security implications?
I ask someone to explain it to me.
Being FreeBSD Security Officer, I have an uncommon luxury: I have about 200 highly qualified developers available who will stop what they're doing and help me out if I say "so, umm, I sort of need to understand how this code works...".
I ask someone to explain it to me.
Being FreeBSD Security Officer, I have an uncommon luxury: I have about 200 highly qualified developers available who will stop what they're doing and help me out if I say "so, umm, I sort of need to understand how this code works...".
Google Code Search:
http://www.google.com/codesearch
I use it for working my way around Google's codebase, which is a few orders of magnitude bigger than that.
Also, there's no substitute for getting your hands dirty and diving into the code. You don't really understand something until you've changed it a few times. Grab a couple of low-priority bugs and write some patches for them; you'll learn far more than if you just sit down and study things.
http://www.google.com/codesearch
I use it for working my way around Google's codebase, which is a few orders of magnitude bigger than that.
Also, there's no substitute for getting your hands dirty and diving into the code. You don't really understand something until you've changed it a few times. Grab a couple of low-priority bugs and write some patches for them; you'll learn far more than if you just sit down and study things.
ruby:
http://eigenclass.org/hiki/rcodetools
http://railroad.rubyforge.org/
http://www.pathf.com/blogs/2008/12/read-the-source-luke-a-re...
http://stackoverflow.com/questions/37105/how-do-you-actually...
---------------------------
(you'll see lots of questions on stackoverflow: navigate/inspect/read source repo's)
http://stackoverflow.com/questions/1623906/programmatically-...
http://stackoverflow.com/questions/935516/how-does-macos-dev...
python:
http://code.activestate.com/recipes/213898/
http://stackoverflow.com/questions/1568544/given-a-python-cl...
emacs, vim, ctags, etags,
http://stackoverflow.com/questions/1220456/navigating-effect...
http://eigenclass.org/hiki/rcodetools
http://railroad.rubyforge.org/
http://www.pathf.com/blogs/2008/12/read-the-source-luke-a-re...
http://stackoverflow.com/questions/37105/how-do-you-actually...
---------------------------
(you'll see lots of questions on stackoverflow: navigate/inspect/read source repo's)
http://stackoverflow.com/questions/1623906/programmatically-...
http://stackoverflow.com/questions/935516/how-does-macos-dev...
python:
http://code.activestate.com/recipes/213898/
http://stackoverflow.com/questions/1568544/given-a-python-cl...
emacs, vim, ctags, etags,
http://stackoverflow.com/questions/1220456/navigating-effect...
Itd be good to get a debugger, connect it to the process and run through the code. You will find a pattern emerging when playing with the application, say, for different clicks of a webapp. After a few days, you will know how the flow works.
That said, Ive been more successful looking at the forums of an open source project, figure out what problems folks have and trying to solve them. You will be amazed to see how much you can learn about the code base and undocumented features solving those problems.
That said, Ive been more successful looking at the forums of an open source project, figure out what problems folks have and trying to solve them. You will be amazed to see how much you can learn about the code base and undocumented features solving those problems.
Is it a web app? Open up Firebug, look for interesting URLs, and grep for segments of the URL. Find the function that implements it, read bottom-up.
I guess you could also take a look at a tool called "LXR Cross Referencer" at http://sourceforge.net/projects/lxr/. Page http://fxr.watson.org includes LXR-generated cross references for multiple operating systems.
Maybe try this book? http://www.amazon.com/Code-Reading-Open-Source-Perspective/d... I'm not sure how good it is.
Can any suggest a good way of understanding a largish code base (thousands of lines)? Is there any tools to help you visualize/understand a code base? In particular, i'm looking for tools for javascript/ruby/python code bases. Thank you very much!