Ask HN: Computer science for the self taught programmer
Can you list the essential readings for a self taught programmer - I'm guessing this will include things like Big O, and recursive functions, etc.
40 comments
Computer Science is not just algorithms. Generally speaking, Comp Sci consists of:
Algorithms/Data Structures
Computational Complexity
Discrete Math
Programming Languages (both application and the concepts behind them)
System Architecture
Networks
Compilers
Operating Systems
Databases
Parallel Programming
Artificial Intelligence
Graphics
As for what to learn, here is THE BEST link: http://matt.might.net/articles/what-cs-majors-should-know/. It covers all the topics I mentioned, and then some, and provides links to books/resources about everything. If you're serious about self studying Comp sci, that page has all the links and info you need to start.
Algorithms/Data Structures
Computational Complexity
Discrete Math
Programming Languages (both application and the concepts behind them)
System Architecture
Networks
Compilers
Operating Systems
Databases
Parallel Programming
Artificial Intelligence
Graphics
As for what to learn, here is THE BEST link: http://matt.might.net/articles/what-cs-majors-should-know/. It covers all the topics I mentioned, and then some, and provides links to books/resources about everything. If you're serious about self studying Comp sci, that page has all the links and info you need to start.
"Programmer Competency Matrix" is also a great resource: http://www.indiangeek.net/wp-content/uploads/Programmer%20co...
I was going to come in here announcing some random algorithm book, but you know, you're right. When I'm writing code for speed, I have to start thinking about how the data is laid out in memory (even in languages that abstract much of this away from you, like Java); and I start thinking about how long different operations take in their assembly form. These are both pretty important things to know, especially as you get close to the metal in languages like C. Discrete Math is useful, too; as is at least a basic understanding of networking and compilers; and, nowadays, knowing how to work with ~at least~ SQL databases is greatly beneficial. Most of that whole list is useful :)
To be thorough, you might include Sound Synthesis if you're going to include Graphics. Or you might swap one for the other depending on your interests. While Graphics will have you learn applied linear algebra, Sound Synthesis will have you learn digital signal processing, which includes Sampling Theory, Filter Theory, Fourier Analysis and Synthesis Methods.
Last time I checked with a university, they dropped half of the hardcore stuff, and teach more Java/Python. YMMV.
Which university? I can tell you that's not true of my university (Northwestern) and most of the peer institutions.
To a certain extent I agree with Joel Spolsky's old article on Java schools[1]. The one gripe I would have to make is that I believe that there are many useful tasks which are hard more than for the sake of being hard. My compiler's course was quite difficult but at the same time very informative. One thing we didn't do is write a parser. When I asked my professor (now advisor) why not, he responded that writing a parser is more of a character-building exercise. I tend to agree. While very tricky, I find that in the context of university classes so much time is spent writing a parser that more interesting topics in compilers are completely omitted. There's always benefit in deeper understanding, but one must try to identify the richest areas for learning.
[1] http://www.joelonsoftware.com/articles/ThePerilsofJavaSchool...
To a certain extent I agree with Joel Spolsky's old article on Java schools[1]. The one gripe I would have to make is that I believe that there are many useful tasks which are hard more than for the sake of being hard. My compiler's course was quite difficult but at the same time very informative. One thing we didn't do is write a parser. When I asked my professor (now advisor) why not, he responded that writing a parser is more of a character-building exercise. I tend to agree. While very tricky, I find that in the context of university classes so much time is spent writing a parser that more interesting topics in compilers are completely omitted. There's always benefit in deeper understanding, but one must try to identify the richest areas for learning.
[1] http://www.joelonsoftware.com/articles/ThePerilsofJavaSchool...
A sample size of 1 is not representative. If the university is more interested in teaching "programming," then that's fine, but if you want a solid grasp of computer science, then most of those subjects are still necessary.
I do. Thank you for the link.
It doesn't really matter what universities do or don't teach. If a subject is of interest you should learn it. I thought everything you and the linked article listed made sense.
It doesn't really matter what universities do or don't teach. If a subject is of interest you should learn it. I thought everything you and the linked article listed made sense.
Yeah... so one thing I love about being self taught was that I've learned because I wanted to do a project, and I learned the most simple tools to accomplish those projects. I'm sure I missed some stuff beacuse I was never forced to do it in a class. I think what would be most helpful is - what are the topics that made you most well rounded, that you would be a crappier coder without, that you took during a CS degree. Also, what are some important theoretical things that paradigm shifted your brain cells for the benefit of every following line of code.
[deleted]
I'll start out with the classic:
"Introduction to Algorithms" a.k.a. "CLRS" http://www.amazon.com/Introduction-Algorithms-Thomas-H-Corme...
I think this is all (and probably more than) you really "need" if you want to know CS for programming, and an enjoyable read too.
"Introduction to Algorithms" a.k.a. "CLRS" http://www.amazon.com/Introduction-Algorithms-Thomas-H-Corme...
I think this is all (and probably more than) you really "need" if you want to know CS for programming, and an enjoyable read too.
I would strongly urge the OP to not use this book. I used it as my undergrad algorithms book and it is, quite frankly, far too obtuse and hard to understand. At first, I thought it was just me and that other people found the book readable. I eventually realized that the book itself is just bad for autodidacts.
Last month (about 5 years after I took the course), I went through to review some stuff that I'd looked at more recently. Specifically, I had worked out Dijkstra's algorithm on my own and finally understood how the heck it actually worked. I then opened CLRS to check their proof. What I saw blew my mind in terms of sheer obfuscation. Despite having a very clear understanding in my head of the core concept behind the algorithm, I read their proof and was just amazed at how little it actually explained about the simple idea that ties the whole thing together. That by itself isn't a huge deal, because some proofs are just hard to follow. What really upset me was that there was practically no surrounding discussion of the intuition behind the algorithm. Behind all the rigor of mathematics lies very simple and powerful ideas. This book did nothing to emphasize those simple ideas. It's a real travesty to be honest :(
If you're looking for something more accessible, I've heard that The Algorithm Design Manual[1] by Steve Skiena[2] is better. Whatever you choose, though, please stay away from CLRS.
[1] http://www.amazon.com/exec/obidos/ASIN/0387948600/ref=ase_th...
[2] http://www.cs.sunysb.edu/~skiena/
edit: I just found this review of CLRS on Amazon (http://www.amazon.com/review/R2FI8CA368KWGE/ref=cm_cr_pr_per...). It's the top comment for this book and sort of gets at the issue:
With that said, this book falls short in one MAJOR area, explanations. Too often explanations are left out and left as exercises and there are no solutions to the exercises! Or details are replaced by ambiguous statements such as of "cleary, this works", or "it is easy to see that this ...". I get the concept of learning by doing, really I do, but there should be some kind of solutions so the student can CHECK his/her understanding of the material and sometimes the exercises are not about advanced aspects of a concept, sometimes it is the core material. Even if the solution manual only contained a simple answer without the work. Not only would it help tremendously but the purpose of doing the exercises would be preserved; that is the student getting his/her "hands dirty" and working out a problem.
For the love everything good and pure in this universe, I really wish writers of mathematical books would stop using statements like "clearly this works" or "it is easy to see", "it is obvious" etc. While that may be true for you and your brilliant circle of colleagues, everything is not always clear and obvious to your readers. Save all of that ambiguity for your research paper.
A great book should deliver in two areas; it should challenge and it should inform. The challenge is there, no doubt. However in some ways it fails to inform the reader. The authors should really think about releasing a students solution manual to help students learn better. I take away two stars for the reasons stated about
Last month (about 5 years after I took the course), I went through to review some stuff that I'd looked at more recently. Specifically, I had worked out Dijkstra's algorithm on my own and finally understood how the heck it actually worked. I then opened CLRS to check their proof. What I saw blew my mind in terms of sheer obfuscation. Despite having a very clear understanding in my head of the core concept behind the algorithm, I read their proof and was just amazed at how little it actually explained about the simple idea that ties the whole thing together. That by itself isn't a huge deal, because some proofs are just hard to follow. What really upset me was that there was practically no surrounding discussion of the intuition behind the algorithm. Behind all the rigor of mathematics lies very simple and powerful ideas. This book did nothing to emphasize those simple ideas. It's a real travesty to be honest :(
If you're looking for something more accessible, I've heard that The Algorithm Design Manual[1] by Steve Skiena[2] is better. Whatever you choose, though, please stay away from CLRS.
[1] http://www.amazon.com/exec/obidos/ASIN/0387948600/ref=ase_th...
[2] http://www.cs.sunysb.edu/~skiena/
edit: I just found this review of CLRS on Amazon (http://www.amazon.com/review/R2FI8CA368KWGE/ref=cm_cr_pr_per...). It's the top comment for this book and sort of gets at the issue:
With that said, this book falls short in one MAJOR area, explanations. Too often explanations are left out and left as exercises and there are no solutions to the exercises! Or details are replaced by ambiguous statements such as of "cleary, this works", or "it is easy to see that this ...". I get the concept of learning by doing, really I do, but there should be some kind of solutions so the student can CHECK his/her understanding of the material and sometimes the exercises are not about advanced aspects of a concept, sometimes it is the core material. Even if the solution manual only contained a simple answer without the work. Not only would it help tremendously but the purpose of doing the exercises would be preserved; that is the student getting his/her "hands dirty" and working out a problem.
For the love everything good and pure in this universe, I really wish writers of mathematical books would stop using statements like "clearly this works" or "it is easy to see", "it is obvious" etc. While that may be true for you and your brilliant circle of colleagues, everything is not always clear and obvious to your readers. Save all of that ambiguity for your research paper.
A great book should deliver in two areas; it should challenge and it should inform. The challenge is there, no doubt. However in some ways it fails to inform the reader. The authors should really think about releasing a students solution manual to help students learn better. I take away two stars for the reasons stated about
As far as The Algorithm Design Manual and CLRS is concerned I would not say that they are comparable.
CLRS is about algorithm analysis and formally proving that they behave in certain ways - like proving that quicksort has a guaranteed worst case of O(n^2) and an average case of O(n log n). Being able to work with proofs such as those is fairly fundamental to being a Computer Scientist in the strict, academic, sense. That being said, it should not be used as book to teach data structures or algorithms without formal proofs having been taught first - without the maths background the book is going to be very hard going.
The Algorithm Design Manual on the other hand, is more like a literature review for a collection of algorithms/families of algorithms (75, claims the copy infront of me). For each of them it gives an overview of the class of problems you'd apply the algorithm to, a run through of the algorithm (some with pseudo code) and discussion about the algorithm and times when the author has used it, and then pointers to more in depth sources. CRLS and Knuth are cited considerably, which should give you an idea about the relative stature of the two books being compared here.
Ultimately, they're both good books but for different audiences. The Algorithm Design Manual is by far the easier read, not least of which because it's not trying to be an academic text and is replete with anecdotes from the author. It's an excellent overview of the techniques available to the practicing developer - the ones who work in fields where you might need to whip up a quick Bin Packing routine anyway. CLRS is a lot harder to get through, but it will teach you how to prove that your algorithms will do what they should.
When it comes down to it, I'd say that CLRS is for the Computer Scientist while TADM is for the practitioner and I'm glad I own both.
CLRS is about algorithm analysis and formally proving that they behave in certain ways - like proving that quicksort has a guaranteed worst case of O(n^2) and an average case of O(n log n). Being able to work with proofs such as those is fairly fundamental to being a Computer Scientist in the strict, academic, sense. That being said, it should not be used as book to teach data structures or algorithms without formal proofs having been taught first - without the maths background the book is going to be very hard going.
The Algorithm Design Manual on the other hand, is more like a literature review for a collection of algorithms/families of algorithms (75, claims the copy infront of me). For each of them it gives an overview of the class of problems you'd apply the algorithm to, a run through of the algorithm (some with pseudo code) and discussion about the algorithm and times when the author has used it, and then pointers to more in depth sources. CRLS and Knuth are cited considerably, which should give you an idea about the relative stature of the two books being compared here.
Ultimately, they're both good books but for different audiences. The Algorithm Design Manual is by far the easier read, not least of which because it's not trying to be an academic text and is replete with anecdotes from the author. It's an excellent overview of the techniques available to the practicing developer - the ones who work in fields where you might need to whip up a quick Bin Packing routine anyway. CLRS is a lot harder to get through, but it will teach you how to prove that your algorithms will do what they should.
When it comes down to it, I'd say that CLRS is for the Computer Scientist while TADM is for the practitioner and I'm glad I own both.
Unfortunately, at least in my experience, most textbooks are rather unfriendly for self-teaching (including the ones very frequently recommended), especially in the more theoretical fields like algorithms, theory of computation etc. (no solutions to exercises, not enough worked out examples, general strong reliance on classroom usage), it takes a long time to find good ones and even then a very long time to get to a decent speed in a new field, surely much longer then when taking a course. Thus I would recommend relying more on some online courses at least initially:
http://ocw.mit.edu/courses/electrical-engineering-and-comput... - Very good course covering the most crucial theoretical things every programmer should now
http://aduni.org/courses/ - Good discrete mathematics, algorithms, theory of computation courses (I recommend "taking" them in this order) without assuming any prerequisites
http://ocw.mit.edu/courses/electrical-engineering-and-comput... - More sophisticated "CS 101" kind of course
I think almost everyone with some programming experience should be able to work through them in the sequence listed in one year perhaps, at a satisfying level of detail. At the same time, many people programming professionally lack mastery of this basic material.
http://ocw.mit.edu/courses/electrical-engineering-and-comput... - Very good course covering the most crucial theoretical things every programmer should now
http://aduni.org/courses/ - Good discrete mathematics, algorithms, theory of computation courses (I recommend "taking" them in this order) without assuming any prerequisites
http://ocw.mit.edu/courses/electrical-engineering-and-comput... - More sophisticated "CS 101" kind of course
I think almost everyone with some programming experience should be able to work through them in the sequence listed in one year perhaps, at a satisfying level of detail. At the same time, many people programming professionally lack mastery of this basic material.
Suggested books HN post from about a month ago:
http://news.ycombinator.com/item?id=3595599
I've found The Little Schemer[1] to be very approachable and effective for grokking common CS concepts.
[1]: http://www.amazon.com/The-Little-Schemer-4th-Edition/dp/0262...
[1]: http://www.amazon.com/The-Little-Schemer-4th-Edition/dp/0262...
I'm reading/working through this book now and can't recommend it enough, especially for those with limited time (or limited ability to focus and study due to environmental factors). I've worked through most of the first two chapters of SICP, but honestly felt a little overwhelmed at times. TLS has been enlightening AND fun. The format is genuinely unique and well-suited for reading/thinking in the small slivers of free time I have between work, kids, freelance, Khan academy, etc.
Most exercises don't require an interpreter, which is nice if you need a break from screentime. But I do enjoy working through some of them in Racket or even Pixie Scheme. TLS isn't free or available in a digital format (AFAIK), but the book is inexpensive and very portable.
Most exercises don't require an interpreter, which is nice if you need a break from screentime. But I do enjoy working through some of them in Racket or even Pixie Scheme. TLS isn't free or available in a digital format (AFAIK), but the book is inexpensive and very portable.
I agree with this suggestion.
Also, I want to point out that "essential readings" is not the right attitude. In order to really make progress you're going to need to get your hands dirty and work though some examples. One very good exercise is to write an interpreter for a language you invent.
Also, I want to point out that "essential readings" is not the right attitude. In order to really make progress you're going to need to get your hands dirty and work though some examples. One very good exercise is to write an interpreter for a language you invent.
TLS series is wonderful - HTDP is great too, it assumes the student doesn't have programming experience which means they start out with simple stuff (and use Scheme for the implementation language!).
Ummm. I read this after SICP. While I found it to be an easy way to pick up a lot of "gotchas," I definitely wouldn't give this to someone as an intro to CS or programming. It's a good book though.
Learn you lots of Algebra; then go learn you some functional programming (in particular LISP, HTDP: How To Design Programs and SICP are great for this).
Then I would pick up Randall Hyde's books on "How to Write Great Code" series - he starts out with machine fundamentals (stuff you would typically learn in an Assembly or computer engineering course minus the Assembly).
Then I would learn some type of Assembly (again I recommend Randall Hyde).
Then, IMHO follow @BlackJack's comment of subject matter.
NOTE: I'm a self educated programmer.
Then I would pick up Randall Hyde's books on "How to Write Great Code" series - he starts out with machine fundamentals (stuff you would typically learn in an Assembly or computer engineering course minus the Assembly).
Then I would learn some type of Assembly (again I recommend Randall Hyde).
Then, IMHO follow @BlackJack's comment of subject matter.
NOTE: I'm a self educated programmer.
If you're not intimidated by a little C, I highly recommend "Computer Systems: A Programmer's Perspective"[1] by Bryant and O'Hallaron from Carnegie Mellon. It was personally recommended to me by a PhD student from CM when I inquired about CS resources that would be directly pertinent to programmers, and it completely exceeded my expectations.
[1] http://csapp.cs.cmu.edu/
[1] http://csapp.cs.cmu.edu/
I took the class that this book was written for when I was at CMU, and it's an excellent book. If you're a programmer with some understanding of C (read: pointers don't fill you with terror), this book will give you a solid overview of systems programming (how programs are laid out in memory, basic assembly, optimization, cache behavior, sockets, multithreading).
I'm not clear if you are looking for books, links, or topics. If you are looking for topics or books, I'd suggest starting by taking a look at the syllabi from degree programs that you respect. You may want to pay particular attention to the courses that have the word "Theory" and "Computation" in them.
You are going to find topics like:
I written about some of these topics in an article last year that might give you a very small introduction to finite state machines: http://blog.markwshead.com/869/state-machines-computer-scien...
You are going to find topics like:
* Regular Languages/Expressions
* Automata
* Nonregular Languages
* Context Free Languages
* Turing Machines
* Decidable vs. Undecidable Problems
* Halting Problem
I written about some of these topics in an article last year that might give you a very small introduction to finite state machines: http://blog.markwshead.com/869/state-machines-computer-scien...
What is it that you wish to learn? If you're a self-taught programmer who wants to learn (to use an extreme example) bricklaying, then you won't have much use for the standard undergrad CS reading list. Similarly, if you want to be a better programmer, the list will be different to that of the standard undergrad CS reading list. Can you tell us what your goal here is? Do you actually want to do CS, or do you want to fill in some of the theoretical gaps that might be useful to you as a coder, or do you actually not care so much about the theory and just want to be a better programmer?
> do you want to fill in some of the theoretical gaps that might be useful to you as a coder, or do you actually not care so much about the theory and just want to be a better programmer
Definitely yes, and yes. I suppose I want to see a few examples of writing better code, see how and why things break when they scale, understand useful principles that I just "worked around" before (ie recursive functions), and then also some primers on other subjects - so for example I studied poli sci, but they made me take anthropology, geography, economics and some other related but not completely aligned fields. What are some examples to complement web dev?
Definitely yes, and yes. I suppose I want to see a few examples of writing better code, see how and why things break when they scale, understand useful principles that I just "worked around" before (ie recursive functions), and then also some primers on other subjects - so for example I studied poli sci, but they made me take anthropology, geography, economics and some other related but not completely aligned fields. What are some examples to complement web dev?
Same boat here. I've really only skimmed 'Think Complexity: http://greenteapress.com/complexity/index.html but it looks like a nice, digestible introduction to a broad range of topics. If you want to get more (way more) in-depth, have a look at 'Mathematics for Computer Science': http://courses.csail.mit.edu/6.042/spring12/mcsfull.pdf
I'd be interested in your recommendations for primarily becoming a better programmer.
Programming, Learn by Doing:
For programming specifically you can learn by doing more than reading. Just like Maths you have to have solved a problem or two. So hope the following inspires you and in the case of codeschool's rails for zombies or code academy - it means you can learn by doing.
http://www.codeschool.com/ Kicks ass. Unless you are scared of zombies.
http://www.codecademy.com// YC Startup if I remember correctly. Beautiful.
Computer Science, Turing Turing Turing:
For some serious inspiration the first ever essay we wrote at Manchester in Year 1 of CS was The Life and Times of Alan Turing. His huge impact still hadn't hit me then. The story of Alan Turing is told in so many different ways that to learn about him is to learn about the type of mind that led us to where we are now. His impact at Bletchley Park breaking the German Enigma code, then at Manchester his work not only seeded AI as we know it but showed how Mathematics could help us understand the code behind nature. These deep roots of CS are what keep you going through the more incomprehensible CS books and difficult challenges.
http://plato.stanford.edu/entries/turing/ http://en.wikiquote.org/wiki/Alan_Turing http://www.bbc.co.uk/history/people/alan_turing
I just found this:
Breaking the Code: Biography of Alan Turing (Derek Jacobi, BBC, 1996) http://www.youtube.com/watch?v=S23yie-779k
Looks like I am staying up working a little longer! If only BBC Programs could be streamed on demand I would pay for one daily. There were numerous programs on Turing including an episode from the series The Code - Numbers which was the type of programme that should always be available to the world and makes you wonder why the BBC doesn't unleash it's potential... Another story...
http://www.bbc.co.uk/programmes/b012xppj http://www.bbc.co.uk/history/people/alan_turing
I really value all the helpful comments here - thank you for your practical advice guys ;-)
For programming specifically you can learn by doing more than reading. Just like Maths you have to have solved a problem or two. So hope the following inspires you and in the case of codeschool's rails for zombies or code academy - it means you can learn by doing.
http://www.codeschool.com/ Kicks ass. Unless you are scared of zombies.
http://www.codecademy.com// YC Startup if I remember correctly. Beautiful.
Computer Science, Turing Turing Turing:
For some serious inspiration the first ever essay we wrote at Manchester in Year 1 of CS was The Life and Times of Alan Turing. His huge impact still hadn't hit me then. The story of Alan Turing is told in so many different ways that to learn about him is to learn about the type of mind that led us to where we are now. His impact at Bletchley Park breaking the German Enigma code, then at Manchester his work not only seeded AI as we know it but showed how Mathematics could help us understand the code behind nature. These deep roots of CS are what keep you going through the more incomprehensible CS books and difficult challenges.
http://plato.stanford.edu/entries/turing/ http://en.wikiquote.org/wiki/Alan_Turing http://www.bbc.co.uk/history/people/alan_turing
I just found this:
Breaking the Code: Biography of Alan Turing (Derek Jacobi, BBC, 1996) http://www.youtube.com/watch?v=S23yie-779k
Looks like I am staying up working a little longer! If only BBC Programs could be streamed on demand I would pay for one daily. There were numerous programs on Turing including an episode from the series The Code - Numbers which was the type of programme that should always be available to the world and makes you wonder why the BBC doesn't unleash it's potential... Another story...
http://www.bbc.co.uk/programmes/b012xppj http://www.bbc.co.uk/history/people/alan_turing
I really value all the helpful comments here - thank you for your practical advice guys ;-)
Not to discourage further discussion but there are a lot of good resources here:
http://www.hnsearch.com/search#request/all&q=title%3A(se...
http://www.hnsearch.com/search#request/all&q=title%3A(se...
A list for a programmer and a list for a computer scientist would look quite different.
This is not a helpful comment. Additionally I do not agree - while there is a difference between academically focused individuals and industry focused individuals they absolutely should not be mutually exclusive. That's a social generalization in our industry everyone should be working to dismantle.
I'm an industry programmer and pursued a lot of Comp Sci self-study in order to become a "better programmer", an "effective programmer", and most importantly "better able to reason about my programs".
Being versed in both is important so the reading list, IMHO, should be mixed!
I'm an industry programmer and pursued a lot of Comp Sci self-study in order to become a "better programmer", an "effective programmer", and most importantly "better able to reason about my programs".
Being versed in both is important so the reading list, IMHO, should be mixed!
Very well put. I would say the opportunity when self taught is that you can focus on the areas or challenges that excite you the most. "Getting close to the metal with C" is a good example of someone interested in taking things apart and seeing the gears turning the wheel. I studied CS at university specifically to understand how a machine could make chuckie egg jump or dragon ninja kick. You start with Binary, Machine Learning play with some C or Java and suddenly the magic was more than magic - mathematics. Immerse yourself and enjoy it is the most important thing. Back to the point - agreed - programming and computer science can never be mutually exclusive. Everyone's comments on here have me officially addicted to HN. About time.
I asked a similar question a while back:
http://news.ycombinator.com/item?id=3226708
I'd look at the online courses that are starting to be offered. Coursera, MITx, etc. There's a lot of foundational stuff in those courses.
what do you guys think of http://bottomupcs.com ? I just started reading it and feel I've learned a lot already but it feels very unfinished. I wish they would keep the same pace but go more in depth in each topic.
hackershelf.com is a good place to source for hacker books, most of them are free.
[deleted]