Ask HN: project ideas for a noob
26 comments
My first real programming project was a DOS TSR keylogger. When you start your programming journey in the computer underground: 1) you have no idea the magnitude of the problems being thrown around by your peers, everything is trivialized and you're expected to know a great deal of info about the inner workings of your programming languages and their execution model, application software, OS/kernel internals, compilers, network, hardware, file formats and .. bugs and 2) there is an absolute joy to everything you do; never underestimate the power of mischief as a great motivator.
I don't expect you to dawn a blackhat at this stage, you're probably too mature, and the scene is pretty much lame nowadays anyway. What you can do however is join a community that enjoys and fosters a healthy hacking attitude. Something very geeky and very focused, like the demo scene (if they're still as innovative as they used to be.) You need a group of friends, all of whom are hacking for fun and giving each other feedback. IRC is an excellent place to find such people. Something focused on a given subject and a given technology. Start with your favorite libraries and join their IRC channels. The Allegro game library scene was cool, write 2D games for fun. Once you master the basic usage of the library, you will see what more experienced people have done with it. There is a different, unique taste to seeing a master craftsman make something great out of the ordinary ("wow, he did that in 4k" or "wow, fake 3D".) This will motivate you to no end :-)
Take out the manual of your "battery included" language of choice (Python, PLT or Chicken Scheme) and step through the module list. Write small programs that use each module/library and pretty soon you will have tons of ideas. Just take a GUI library, a network library, a regex parser, a mime/XML/html parser, and an audio library; taken into any combination, you will have something that solves an interesting problem. Something as "big" as a web server can be written with just the system calls built into every unix :-)
Finding your own problems, to keep you busy, is also something you will eventually develop as you continue hacking.
P.S. DON'T start with a janitorial position cleaning up other people's code or doing manuals, as the "Hacker HOWTO" advocates. Fuck that, NIH and all, go out there and create your own bugs to fix. Have fun, eh? :-)
I don't expect you to dawn a blackhat at this stage, you're probably too mature, and the scene is pretty much lame nowadays anyway. What you can do however is join a community that enjoys and fosters a healthy hacking attitude. Something very geeky and very focused, like the demo scene (if they're still as innovative as they used to be.) You need a group of friends, all of whom are hacking for fun and giving each other feedback. IRC is an excellent place to find such people. Something focused on a given subject and a given technology. Start with your favorite libraries and join their IRC channels. The Allegro game library scene was cool, write 2D games for fun. Once you master the basic usage of the library, you will see what more experienced people have done with it. There is a different, unique taste to seeing a master craftsman make something great out of the ordinary ("wow, he did that in 4k" or "wow, fake 3D".) This will motivate you to no end :-)
Take out the manual of your "battery included" language of choice (Python, PLT or Chicken Scheme) and step through the module list. Write small programs that use each module/library and pretty soon you will have tons of ideas. Just take a GUI library, a network library, a regex parser, a mime/XML/html parser, and an audio library; taken into any combination, you will have something that solves an interesting problem. Something as "big" as a web server can be written with just the system calls built into every unix :-)
Finding your own problems, to keep you busy, is also something you will eventually develop as you continue hacking.
P.S. DON'T start with a janitorial position cleaning up other people's code or doing manuals, as the "Hacker HOWTO" advocates. Fuck that, NIH and all, go out there and create your own bugs to fix. Have fun, eh? :-)
There are more things I want to touch on, but I'm afraid my posts are long and opinionated as they are. Just a few quick pointers and hints, things that must be passed on:
* Learn how to find good information. Hone your bullshit detector, learn who knows what they're talking about and who is full of it (many will try to sell you things, if not an actual product or service, then their personality: good hackers are selfless and rarely sign their work.)
* Get heroes. Find good hackers to imitate. I grew up wanting to be "like Terje Mathisen" ..
* Avoid fluff mediums. Best information is found in ASCII mediums :-P just my finding. Avoid the blog scene, most of it is written to pad an ego or a resume. Usenet is good, sparingly. Citeseer is good when you know better. The underground zines are good, with a grain of salt.
* Discover the past; everything you need to know about computing was invented/discovered before 1980. Mine the past, seek the originators, not the imitators. Nothing good happened since 1999; the greatest minds left the Science and went to industry, behind NDAs, patents and sold to "investors".
* Don't do the web, yet. Start working in the console, then perhaps do a tangible GUI. Don't do Unicode
* ACM, IEEE and Elsevier are bastards. If you need a paper, google the authors' websites and you can usually find it there.
* It's OK to be in hackmode. If you need to get away from society, you are allowed to forgo all human contact and basic hygiene until you figure shit out.
* Mathematics is easy. The smaller and denser the text the better. It's the long, repetitive hammer-into-your-head crap that probably bored you in school. Real mathematics is neat and compact. Get math books with answers to the exercises at the back. Don't cheat yourself.
* Save your first programs, even print them out. You will be willing to give up everything just to get them back, should you lose them (and remain a programmer ;-)
* Learn assembly language. Learn C. Learn how to write your own tools. Programming tools should be the first systems you investigate. Levine's book "Linkers and Loaders" is free online. You can find "Advanced DOS" on Alexie Frounze's site. K&R is online. Unix is good. Learn it well, at the kernel level. Familiarize yourself executable file formats. Get into compiler hacking early, it will enlighten you, but will also force you to learn so many different languages you will have no time for anything else.
* Share what you learned, even if you have to cringe at your own verbosity :-)
* Learn how to find good information. Hone your bullshit detector, learn who knows what they're talking about and who is full of it (many will try to sell you things, if not an actual product or service, then their personality: good hackers are selfless and rarely sign their work.)
* Get heroes. Find good hackers to imitate. I grew up wanting to be "like Terje Mathisen" ..
* Avoid fluff mediums. Best information is found in ASCII mediums :-P just my finding. Avoid the blog scene, most of it is written to pad an ego or a resume. Usenet is good, sparingly. Citeseer is good when you know better. The underground zines are good, with a grain of salt.
* Discover the past; everything you need to know about computing was invented/discovered before 1980. Mine the past, seek the originators, not the imitators. Nothing good happened since 1999; the greatest minds left the Science and went to industry, behind NDAs, patents and sold to "investors".
* Don't do the web, yet. Start working in the console, then perhaps do a tangible GUI. Don't do Unicode
* ACM, IEEE and Elsevier are bastards. If you need a paper, google the authors' websites and you can usually find it there.
* It's OK to be in hackmode. If you need to get away from society, you are allowed to forgo all human contact and basic hygiene until you figure shit out.
* Mathematics is easy. The smaller and denser the text the better. It's the long, repetitive hammer-into-your-head crap that probably bored you in school. Real mathematics is neat and compact. Get math books with answers to the exercises at the back. Don't cheat yourself.
* Save your first programs, even print them out. You will be willing to give up everything just to get them back, should you lose them (and remain a programmer ;-)
* Learn assembly language. Learn C. Learn how to write your own tools. Programming tools should be the first systems you investigate. Levine's book "Linkers and Loaders" is free online. You can find "Advanced DOS" on Alexie Frounze's site. K&R is online. Unix is good. Learn it well, at the kernel level. Familiarize yourself executable file formats. Get into compiler hacking early, it will enlighten you, but will also force you to learn so many different languages you will have no time for anything else.
* Share what you learned, even if you have to cringe at your own verbosity :-)
[Edit: Update 2
* Help yourself to other people's servers. Get a free shell account on a unix server and try to use it to the fullest. Chat with people in whatever MUD system they have in place. Try to circumvent security, build software on your account to go around network restrictions .. but don't be abusive. Try to tunnel a banned network protocol over a permitted one. Have your programs in your various account talk to each other. Leave backdoors, but not big, nasty gaping ones.
* Automate your social life. Write scripts to check your email, or parse RSSes of sites you frequent, or see any replies to your threads.
* Get accounts on shells that run on non x86 archs. Learn different assembly languages and have fun. Hack your router and game consoles, install a different OS than the vendor's ;-]
Update!
* Clone the basic unix commands in python and scheme. Coreutils from the GNU project should keep you busy for a while.
* Keep your code off of other people's boxes, including Google (with the appengine recommendations and all.) Sandboxed environments will not teach you how NOT to shoot yourself in the foot. This also goes for hacking in an emulator or a VPS. Learn on a real machine that you can touch and physically fuck up. A desktop is highly recommended, it will force you to open it up and change hardware. The first time you google a motherboard manual to flip a jumper or change a dead CMOS battery will be the day you realize the benefit of having total mastery over your machines.
* Help yourself to other people's servers. Get a free shell account on a unix server and try to use it to the fullest. Chat with people in whatever MUD system they have in place. Try to circumvent security, build software on your account to go around network restrictions .. but don't be abusive. Try to tunnel a banned network protocol over a permitted one. Have your programs in your various account talk to each other. Leave backdoors, but not big, nasty gaping ones.
* Automate your social life. Write scripts to check your email, or parse RSSes of sites you frequent, or see any replies to your threads.
* Get accounts on shells that run on non x86 archs. Learn different assembly languages and have fun. Hack your router and game consoles, install a different OS than the vendor's ;-]
Update!
* Clone the basic unix commands in python and scheme. Coreutils from the GNU project should keep you busy for a while.
* Keep your code off of other people's boxes, including Google (with the appengine recommendations and all.) Sandboxed environments will not teach you how NOT to shoot yourself in the foot. This also goes for hacking in an emulator or a VPS. Learn on a real machine that you can touch and physically fuck up. A desktop is highly recommended, it will force you to open it up and change hardware. The first time you google a motherboard manual to flip a jumper or change a dead CMOS battery will be the day you realize the benefit of having total mastery over your machines.
Some of your suggestions are a bit over the top for a noob...
Nearly every interesting programming problem is a bit over the top to you, at your current knowledge level. The point is to apply yourself, learn things as you go, research, and conquer the problem.
There are no cookie-cutter "noob" problems; there are problems whose solutions you can visualize in your head and see them happen, and problems that make you sit back and go "hmmmm". As you gather many technologies, more and more previously impossible problems begin to fall into the first category. This is what allows you, as an experienced programmer, to rule many application domains as boring and trivial. Even if you have never written a ticket ordering application for a concert hall, if you have done the most basic database application, you can pretty much visualize the solution :-)
There are no cookie-cutter "noob" problems; there are problems whose solutions you can visualize in your head and see them happen, and problems that make you sit back and go "hmmmm". As you gather many technologies, more and more previously impossible problems begin to fall into the first category. This is what allows you, as an experienced programmer, to rule many application domains as boring and trivial. Even if you have never written a ticket ordering application for a concert hall, if you have done the most basic database application, you can pretty much visualize the solution :-)
Even though he is a "noob" the term coined by pg and appropriate here is to be relentlessly resourceful. That is if he wants to become a good hacker from a nubile state he will need to become relentlessly resourceful in the effort to get things done. I agree with pg that the good/great hackers are extremely resourceful.
There are more things I want to touch on, but I'm afraid my posts are long and opinionated as they are
Yes, and they're very good. More of your posts, more posts like yours!
Yes, and they're very good. More of your posts, more posts like yours!
Save your first programs, even print them out.
Yes! I wish I had done that. I vividly remember writing my first programs when I was a kid; alas, the code is gone forever now. I really should've printed it out.
Yes! I wish I had done that. I vividly remember writing my first programs when I was a kid; alas, the code is gone forever now. I really should've printed it out.
Completely agree, mischief motivates you like none other.
I started out by making batte.net bots for Starcraft. They were super hard but there was plenty of open source code to leech and copy off of in the beginning. Later on, I managed to learn how it all worked and pieced together which allowed me to be able to write them from scratch. This was no easy task and I started proramming like this. It entailed understanding network programming, reverse engineering, how to code in general, how to send/receive packets, and all the fundementals associated.
Make a bot for anything or any game, scripting bots to do automated things that are annoying to do manually generally has a great sense of sanctification. Making an IRC bot is easy, start to get into more challenging things like non-plaintext protocols.
If that's too complex join a community that has already done this so you have something to go off of.
The most important thing is to do something you'll have fun with and still learn. Create something that either fixes a need for yourself or for you to gain an advantage in or something you'll get to see people use.
I started out by making batte.net bots for Starcraft. They were super hard but there was plenty of open source code to leech and copy off of in the beginning. Later on, I managed to learn how it all worked and pieced together which allowed me to be able to write them from scratch. This was no easy task and I started proramming like this. It entailed understanding network programming, reverse engineering, how to code in general, how to send/receive packets, and all the fundementals associated.
Make a bot for anything or any game, scripting bots to do automated things that are annoying to do manually generally has a great sense of sanctification. Making an IRC bot is easy, start to get into more challenging things like non-plaintext protocols.
If that's too complex join a community that has already done this so you have something to go off of.
The most important thing is to do something you'll have fun with and still learn. Create something that either fixes a need for yourself or for you to gain an advantage in or something you'll get to see people use.
Red Code is also an interesting hackish programming environment one could clone in python or scheme.
I have to 2nd this post. I learned a great deal about systems programming by learning about binary hacking and what not. See http://www.overthewire.org/wargames/
Just take a GUI library, a network library, a regex parser, a mime/XML/html parser, and an audio library; taken into any combination, you will have something that solves an interesting problem. Something as "big" as a web server can be written with just the system calls built into every unix :-)
Great advice; unlike an artist who is burdened with a blank canvas, a programmer does not share this burden. You don't need to start with this blank canvas, instead heed mahmud's words and get your finger-tips dirty with some libraries and modules as a place to start.
Great advice; unlike an artist who is burdened with a blank canvas, a programmer does not share this burden. You don't need to start with this blank canvas, instead heed mahmud's words and get your finger-tips dirty with some libraries and modules as a place to start.
Strongly recommending project euler.
http://projecteuler.net/
Start from problem 1 and work your way up - they get gradually harder to solve as complete more. They are simple yet require flexing your algorithm-development muscles, which also help you to think about coding in an "efficient enough" manner: some problems can be implemented in ways that take 10 hours to run, or .001 seconds.
http://projecteuler.net/
Start from problem 1 and work your way up - they get gradually harder to solve as complete more. They are simple yet require flexing your algorithm-development muscles, which also help you to think about coding in an "efficient enough" manner: some problems can be implemented in ways that take 10 hours to run, or .001 seconds.
These might be good exercises, say, for getting the basics of a new programming language. But not really for interesting projects. They don't involve making configuration files, command-line or graphical interfaces, networking, etc. They're hardly something you'll want to use later.
A lot of those problems require an advanced understanding of mathematics to properly solve; i.e., there is a "trick" that you would only know if you had familiarized yourself with it earlier in the course of your mathematical journey. But +1 for Project Euler!
I've found it a great way to learn those tricks. Once I've solved a problem, working through the way somebody else did it, perhaps in an order of magnitude less time or in an idiomatically different language, has often given me a deeper understanding of the problem. I've learned a couple algorithmic tricks that way, too.
Also, there are some paired problems where the first is small enough to be feasible by a naive solution, but the second would take far too long, and you have to figure out the trick. There's a pair that requires finding the maximum path down a pyramid of numbers, for example, and the second pyramid is several times taller.
Also, there are some paired problems where the first is small enough to be feasible by a naive solution, but the second would take far too long, and you have to figure out the trick. There's a pair that requires finding the maximum path down a pyramid of numbers, for example, and the second pyramid is several times taller.
Many, however, just need the realization that "there has to be a better way to implement this". If you're just brute forcing the answer, and have no clue, turning to Google for efficient Fibonacci routines or whatnot is fine, because you've just taught yourself some new mathematical concept.
A few standard 'toy' learning projects come to mind:
Try building a client/server chat program, using TCP/IP, a mass-file Renamer, or mp3 tagger, including a GUI for your OS of choice.
In python, try building something using either google app engine, or implementing a full blog using django (with packages for all the useful features).
Build something that downloads stock data from yahoo or google, parses it, and charts it.
Most of those are more towards the 'trivial' end of the spectrum, but they'll make you familiar with the standard libraries, and the last 10% of 'polish' will sharpen your skills.
Try building a client/server chat program, using TCP/IP, a mass-file Renamer, or mp3 tagger, including a GUI for your OS of choice.
In python, try building something using either google app engine, or implementing a full blog using django (with packages for all the useful features).
Build something that downloads stock data from yahoo or google, parses it, and charts it.
Most of those are more towards the 'trivial' end of the spectrum, but they'll make you familiar with the standard libraries, and the last 10% of 'polish' will sharpen your skills.
I'd recommend spending some time with other peoples code. Open Source provides mountains of great (and bad) examples. The only way you'll really learn to grok the difference is to spend time enhancing other peoples code. You get exposure to a lot more code that way, since you can't possibly write hundreds of thousands of lines of code by yourself in a short period of time...but you can learn to navigate projects of that magnitude in a few weeks, and learn a lot of what those kinds of projects have to teach you.
Pick a project that interests you and has a large/friendly community, and make a plugin or a patch for it. You'll learn a lot about working with other developers, and a lot about the ancillary tools that are needed to build big and useful software but are rarely taught in school.
Pick a project that interests you and has a large/friendly community, and make a plugin or a patch for it. You'll learn a lot about working with other developers, and a lot about the ancillary tools that are needed to build big and useful software but are rarely taught in school.
What are your interests?
When I was learning to program as a kid (not counting Basic), I was really into text adventure games like this (http://www.rickadams.org/adventure/). I wrote an interpreter for text game worlds. It wasn't that great (I was 12 and it was in C++), but I learned quite a bit, and it gave me confidence for other stuff.
If you like math puzzles, Project Euler (http://projecteuler.net/) is great. Python Challenge (http://www.pythonchallenge.com/) is a series of fun puzzles that will also lead you through its standard library.
But, really, go with your interests. If you like logic puzzles, or making music, or 3D game stuff, or MUDs, etc., just pick something and run with it. Also, feel free to start small, you can hit the ground running once you have the basics down.
When I was learning to program as a kid (not counting Basic), I was really into text adventure games like this (http://www.rickadams.org/adventure/). I wrote an interpreter for text game worlds. It wasn't that great (I was 12 and it was in C++), but I learned quite a bit, and it gave me confidence for other stuff.
If you like math puzzles, Project Euler (http://projecteuler.net/) is great. Python Challenge (http://www.pythonchallenge.com/) is a series of fun puzzles that will also lead you through its standard library.
But, really, go with your interests. If you like logic puzzles, or making music, or 3D game stuff, or MUDs, etc., just pick something and run with it. Also, feel free to start small, you can hit the ground running once you have the basics down.
My first 'major' project was writing a Lisp interpreter from scratch. Looking back on it, my implementation was horrible, the design atrocious, and the extensibility null. However, I remember how AWESOME it was at the time to be able to run my code on my own interpreter. What mattered most about that project wasn't the program itself: it was what I learned in the process. I guess my point is, no matter how shitty your code is, you learned something from writing it. So don't wait for a 'firm foundation' to start hacking away, just dive in and feel free to screw up. Just don't screw up in the same way the next time you do it.
I've got to agree with this. Writing Lexers/Parsers/Interpreters is some of the most fun I've had programming. When your program can run other programs, that's a huge thrill.
Read EoPL, it's awesome (and free) and LiSP is awesome but expensive. In fact almost every Lisp related textbook seems to have several lisp interpreters in it.
edit: As noted below EoPL is apparently not free, and it seems that I've inadvertently pirated it from somewhere.
http://www.cs.indiana.edu/eopl/
http://tinyurl.com/lispinsmallpieces
I'm working on a toy (for now) lisp in javascript at the moment and it's profoundly unimpressive but has been hugely fun and educational for me.
Read EoPL, it's awesome (and free) and LiSP is awesome but expensive. In fact almost every Lisp related textbook seems to have several lisp interpreters in it.
edit: As noted below EoPL is apparently not free, and it seems that I've inadvertently pirated it from somewhere.
http://www.cs.indiana.edu/eopl/
http://tinyurl.com/lispinsmallpieces
I'm working on a toy (for now) lisp in javascript at the moment and it's profoundly unimpressive but has been hugely fun and educational for me.
"Read EoPL, it's awesome (and free)"
EoPL is not free afaik. I agree that it is an awesome book.
EoPL is not free afaik. I agree that it is an awesome book.
You should make a blog. It was my first project, and to this day i write my own blog software.
I just launched the latest incarnation as a Google App Engine blog using Django/Python: http://andrewljohnson.appspot.com
I just launched the latest incarnation as a Google App Engine blog using Django/Python: http://andrewljohnson.appspot.com
Find something that has a limited scope. ie: something that you /know/ you can do from start to end. A lot of things seem "too simple" but often, there are interesting ways to solve the problem which involve more complex thinking.
Sadly i don't have time to write a fuller post on this, to my regret.
But i definitely agree with Paul Graham on this subject: http://www.paulgraham.com/ideas.html
If one pays attention carefully there are many opportunities for projects that present themselves weekly. Some of your ideas will be bad, some of them good. But just keep thinking of things that may be cool to work on.
I think the important part of finding ideas has nothing to do with programming. You should find a subject you're interested in and something that you think would be cool to accomplish to help others.
But i definitely agree with Paul Graham on this subject: http://www.paulgraham.com/ideas.html
If one pays attention carefully there are many opportunities for projects that present themselves weekly. Some of your ideas will be bad, some of them good. But just keep thinking of things that may be cool to work on.
I think the important part of finding ideas has nothing to do with programming. You should find a subject you're interested in and something that you think would be cool to accomplish to help others.
Try livecoding with Snd or CLM or Impromptu (or extending them). You'll get a sense for code structures and tweaking your dev environment, and produce really interesting things.
Are you familar with MUD/Mushes?
Try writing one. You'll learn about concurrency, networking, parsing (some mushes have a built-in scripting language) and you will end up writing a lot of code.
Try writing one. You'll learn about concurrency, networking, parsing (some mushes have a built-in scripting language) and you will end up writing a lot of code.
Thank you all for the great comments
IMHO, it is hard to find a domain that is challenging, useful, and small. But a couple of examples listed above are pretty good:
1) Write your own web server, blog software, etc: By leveraging existing libraries (or not!) you can get a lot of effect for very few lines
2) Script your daily chores: It does not matter what language you write the “scripts” in. Choose your current favorite and get a feel for how you can interact with the OS
If you like math or math problems or complicated problems that require some thought you should check out http://projecteuler.net/ the first 30 problems don't really require a large amount of math knowledge and can be very interesting.
Edit: I saw you were going for projects maybe something using pythons cgi library on a web server to do interesting tasks would be nice.
Edit: I saw you were going for projects maybe something using pythons cgi library on a web server to do interesting tasks would be nice.
Actually, my first programming attempts started with extremely simple projects: BMI calculator, recipe generator, mathematics puzzles game, a simple assembly language implementation.
If you want to start with web, start coding a blog or a url shortner. If you want to have something more complex, start with existing web APIs and make a mashup!
If you want to start with web, start coding a blog or a url shortner. If you want to have something more complex, start with existing web APIs and make a mashup!
after making a few API`s using java, after coding in c and cpp for about two years, i moved on to explore the capabilities of various other languages. i took me about 6 months; but it was well worth the time.
I suggest you try out networking projects; maybe a chat client coupled with a file transfer mechanism; or maybe a P2P file sharing project.
i don`t agree with the comment which says "mischief motivated you like none other". it is true that many are interested towards activities like circumventing security systems, but as long as you do not understand the fundamental concepts behind the working of a particular exploit or shellcode; you`ll be wasting your time and will remain a script-kiddie learning little or nothing.
Be more observant in your daily routine. Analyze everything your friends and family say and don't overlook any problem they may have, because it could become your idea and something people want. Just ask questions and listen and something is bound to become salient.
If you have no ideas, then maybe just try a small unoriginal project to improve your skills for now. For example, try building something for the Google App Engine in python.
Stop procrastinating. Just make something.
Wasn't this done over and over on Proggit?
the first two comments mentioned by mahmud are very interesting; real good advice for any programming beginner.
Write your own URL shortener ;)
[deleted]
[deleted]
[deleted]
PS, I prefer writing in python or scheme.