Ask HN: Managing my own server.
Recently got my own VPS from prgmr.com which offers a do-it-yourself sort of support package (great for a beginner who wants to learn more). I've gotten pretty far by using man and apropos. What's the most efficient way for me to learn to manage my own server?
57 comments
Make backups (for whatever definition of "backup" you need in your context) religiously, and store them somewhere that isn't your primary provider. Test your ability to construct a new running system (preferably on another VPS provider) from your backup. Do this regularly, not just once.
Now you have a safety net, and you can go nuts with whatever hare-brained schemes you happen to find in a how-to. :) Both Slicehost and Linode have excellent libraries of articles for setting up typical services, although neither one give ongoing maintenance (backups, patching, monitoring, log reviews, etc) nearly enough attention.
If you really want a learning experience, find an old machine, and spend some time with "Linux From Scratch": http://www.linuxfromscratch.org/ . You'll learn an enormous amount about how the whole system fits together by going through that exercise. (I did that with a few machines back in the day before there was something like LFS to point people to; having a roadmap like that would have sped the learning process up significantly.)
Now you have a safety net, and you can go nuts with whatever hare-brained schemes you happen to find in a how-to. :) Both Slicehost and Linode have excellent libraries of articles for setting up typical services, although neither one give ongoing maintenance (backups, patching, monitoring, log reviews, etc) nearly enough attention.
If you really want a learning experience, find an old machine, and spend some time with "Linux From Scratch": http://www.linuxfromscratch.org/ . You'll learn an enormous amount about how the whole system fits together by going through that exercise. (I did that with a few machines back in the day before there was something like LFS to point people to; having a roadmap like that would have sped the learning process up significantly.)
Make should you have a notebook (Notepad will do) ready before you login the first time. Record down every single step as you setup your box.
Review back the steps, optimized them if necessary. You will find them extremely handy the next time you setup a new box.
Review back the steps, optimized them if necessary. You will find them extremely handy the next time you setup a new box.
Oh Yeah. I installed LFS once. Then I installed Gentoo. Then Arch. Learn't a lot of things along the way. Definitely recommended.
When I got my first VPS, I read through the articles at Slicehost: http://articles.slicehost.com/
The walkthroughs cover a variety of distros including Ubuntu, Debian, and CentOS and can be quite useful regardless of where your VPS is hosted. The articles emphasize good security practices as well which can be helpful for a beginner to understand.
The other thing to do is dig into the docs for anything specific you plan to work on. I tend to look for tools with good documentation before getting started. I've always been impressed with Apache HTTP Server's documentation (http://httpd.apache.org/) and have learned a lot by referencing it when I've had issues.
Whenever you have questions, ask friendly geeks for help. I always do best with real-life assistance, but IRC is a good way to go as well.
The walkthroughs cover a variety of distros including Ubuntu, Debian, and CentOS and can be quite useful regardless of where your VPS is hosted. The articles emphasize good security practices as well which can be helpful for a beginner to understand.
The other thing to do is dig into the docs for anything specific you plan to work on. I tend to look for tools with good documentation before getting started. I've always been impressed with Apache HTTP Server's documentation (http://httpd.apache.org/) and have learned a lot by referencing it when I've had issues.
Whenever you have questions, ask friendly geeks for help. I always do best with real-life assistance, but IRC is a good way to go as well.
Read on slicehost, ask questions on server fault -> repeat
nginx documentation is also really good actually: http://wiki.nginx.org/NginxModules
I got my first VPS with them too, and the articles there have been really useful and straight-forward. Once you've completed all the steps you're pretty much able to build up a functional webserver. Their chatroom is really useful too, if you have any question there's always somebody to help you out. Generally speaking, this is my favourite approch, Do lots of stuff, and learn to do only what you need to do.
Here's a short checklist sorta thing in no particular order.
1. Go get Nemeth's Linux Sysadmin Handbook. Read it and learn the tricks of the trade.
2. I cannot stress this enough, get a desktop with the exact Linux distro you put up. You never know what you'll learn just working in it day by day.
3. Pick up Sobell's book on the Linux shells. Running a VPS means learning the in's and out's of the shell.
4. Learn about the Apache server if you are using it (or another web server), everything you can get your hands on. Get to know how and at what access level is it running. What its config files are, what tweak goes in it. If you're an avid reader, go get the Apache Cookbook.
5. Make, make, make backups along checkpoints.
6. Don't fear the IRC. Sometimes even just listening in on some channels like where Slicehost is discussed, you'll pick up a lot.
7. Use 'info' where available. And read up on bugtraq if you have the time.
8. Tweak/tune one thing at a time. Starting with the things you know best. Take a backup at each step.
1. Go get Nemeth's Linux Sysadmin Handbook. Read it and learn the tricks of the trade.
2. I cannot stress this enough, get a desktop with the exact Linux distro you put up. You never know what you'll learn just working in it day by day.
3. Pick up Sobell's book on the Linux shells. Running a VPS means learning the in's and out's of the shell.
4. Learn about the Apache server if you are using it (or another web server), everything you can get your hands on. Get to know how and at what access level is it running. What its config files are, what tweak goes in it. If you're an avid reader, go get the Apache Cookbook.
5. Make, make, make backups along checkpoints.
6. Don't fear the IRC. Sometimes even just listening in on some channels like where Slicehost is discussed, you'll pick up a lot.
7. Use 'info' where available. And read up on bugtraq if you have the time.
8. Tweak/tune one thing at a time. Starting with the things you know best. Take a backup at each step.
Sounds to me like you're already doing it.
The other thing you'll want to do is understand why things work, not just be a "configuration file programmer".
The other thing you'll want to do is understand why things work, not just be a "configuration file programmer".
The Linode Library is great: http://library.linode.com/
Break stuff often. Be aware that prgmr's out of band management console is very spartan: if you break your system, using prgmr's console is analogous to loading a Live CD in a computer and attempting to recover the installation manually, so be aware of that.
This sounds borderline zealous, but I recommend installing Gentoo or Arch Linux as your first VPS system if you're just starting out on Linux; either that, or start off with NetBSD if you're willing to go down that route as well. The default install options for each of those distros/OS's are very minimal, and so you're forced to think about exactly what you need to install in order to get a working system.
I second bediger: be sure to investigate _why_ things work, and not _how_ to get it to work. The latter is very important for getting things done, but the former is what trains you in diagnosing obscure system issues.
This sounds borderline zealous, but I recommend installing Gentoo or Arch Linux as your first VPS system if you're just starting out on Linux; either that, or start off with NetBSD if you're willing to go down that route as well. The default install options for each of those distros/OS's are very minimal, and so you're forced to think about exactly what you need to install in order to get a working system.
I second bediger: be sure to investigate _why_ things work, and not _how_ to get it to work. The latter is very important for getting things done, but the former is what trains you in diagnosing obscure system issues.
It helps to use one of the more flexible and less user-friendly Linux distros on an old desktop or in a VM. I kind of like ArchLinux.
The problem with Linux, though, is that there are several main families, and even distros in the same family don't really adhere to any specific standard. Some might use syslog, others syslog-ng. Some sendmail, others postfix. It's kind of a mess.
Familiarizing yourself with a few flavors (and maybe FreeBSD or OpenBSD for good measure) on a machine you force yourself to use on occasion, you will find your comfort level for basic UNIX power-user and sysadmin stuff will grow.
The problem with Linux, though, is that there are several main families, and even distros in the same family don't really adhere to any specific standard. Some might use syslog, others syslog-ng. Some sendmail, others postfix. It's kind of a mess.
Familiarizing yourself with a few flavors (and maybe FreeBSD or OpenBSD for good measure) on a machine you force yourself to use on occasion, you will find your comfort level for basic UNIX power-user and sysadmin stuff will grow.
First, learn how to reduce your attack surface. This article is a pretty good introduction:
http://www.freesoftwaremagazine.com/articles/hardening_linux
http://www.freesoftwaremagazine.com/articles/hardening_linux
Also, TCP/IP stack hardening http://www.cromwell-intl.com/security/security-stack-hardeni...
Get cheap hardware and set up a Unix distro on it. Poke around and see how it works. You'll be better off with a distro that tries to use a relatively straightforward text-file-based configuration rather than a GUI/menu system. And when I say cheap hardware - the dusty, yellowing box in your friend/relative's closet with 200 mhz, 32 mb RAM, no video card, and a broken hard drive is very much an option. Unix was designed for far less, and flaky hardware will teach you about how to do real backups. :)
I really like OpenBSD (and the minimalism and excellent documentation are real advantages for learning systems administration), but since you're using prgmr.com, you would probably be better off using a distro they provide* - Debian, perhaps. While there are probably faster ways to learn sysadmin skills, I learned what I know by just dealing with all the issues that have come up over a decade. (And sooner or later, knowing ed will pay off.)
* Though I've found experience with BSD to transfer better to Unix-in-general than Linux. This is probably an even mix of cultural factors and history - BSD is pretty old-school.
I really like OpenBSD (and the minimalism and excellent documentation are real advantages for learning systems administration), but since you're using prgmr.com, you would probably be better off using a distro they provide* - Debian, perhaps. While there are probably faster ways to learn sysadmin skills, I learned what I know by just dealing with all the issues that have come up over a decade. (And sooner or later, knowing ed will pay off.)
* Though I've found experience with BSD to transfer better to Unix-in-general than Linux. This is probably an even mix of cultural factors and history - BSD is pretty old-school.
I first learned to manage a server using a Slicehost VPS. They have a good collection of tutorials at http://articles.slicehost.com/
A lot of it depends on what you are trying to 'serve' :)
EDIT: Make sure you spend some time reading about security issues (iptables and such). It's easy to miss, but certainly as important as any other sys-admin task
A lot of it depends on what you are trying to 'serve' :)
EDIT: Make sure you spend some time reading about security issues (iptables and such). It's easy to miss, but certainly as important as any other sys-admin task
Make sure you're also running Linux as your desktop machine. You'll be able to run almost exactly the same environment as your server, and it's a good way to be able to play around with all the different things available to you in that environment.
I'd suggest playing around in a VM instead; you don't want to hose your laptop.
I'd disagree, but that's because it's way too easy to walk away from the VM when you need to do something and don't want to futz around.
Jump feet first and use Linux as your main OS. If you are doing development work, it can easily surpass Windows, and is comparable (or superior) to OSX as well. Of course, this doesn't apply if you are doing graphic design stuff, however, Windows can live inside a VM for that sort of stuff. If you are doing web dev, you'll need a VM for Vista/Win7 and for XP anyways.
Jump feet first and use Linux as your main OS. If you are doing development work, it can easily surpass Windows, and is comparable (or superior) to OSX as well. Of course, this doesn't apply if you are doing graphic design stuff, however, Windows can live inside a VM for that sort of stuff. If you are doing web dev, you'll need a VM for Vista/Win7 and for XP anyways.
Even if Linux is your main OS, I would still use a VM, because you can reproduce a server environment more closely, isolates that work from your "personal stuff", allows powerful features such as snapshotting, and enables you to simulate multiple servers and/or clients (virtualization) on a single box, when for example playing with a distributed date store.
Or just Wubi to make an Ubuntu install you don't have to hose anything.
Which VM software would you suggest? Let's say I'm on a Mac.
http://virtualbox.org
I wouldn't have recommend it about a year ago but nowadays it's caught up with others quite nicely feature wise, is very fast, supports snapshots and USB, and is completely free.
I wouldn't have recommend it about a year ago but nowadays it's caught up with others quite nicely feature wise, is very fast, supports snapshots and USB, and is completely free.
I second this! I wrote a post ( http://goosey.posterous.com/the-good-news-is-you-dont-have-t... ) recently about this development strategy. It has many advantages beyond the "you don't have to worry about hosing your laptop" one, for sure!
I've been using (Oracle) VirtualBox and am happy with it.
I'm on Snow Leopard and have 4GB RAM.
/Second VBox. I'm using it on Ubuntu to play around with Arch, Nexenta, OpenSolaris, and WinXP. It just works.
I'm pretty happy with VMware Fusion.
http://www.vmware.com/products/fusion/
http://www.vmware.com/products/fusion/
I like VMware fusion provided you use the isolated mode. Trying to "fuse" windows 7 into OS X is the only thing thats crashed my macbook so far.
Just an FYI, VirtualBox images are not easily transferred to VMware, and vice versa.
If you're only ever on a Mac, I vote VMware Fusion, but if you're going to change vm host environments, I'd suggest VirtualBox.
If you're only ever on a Mac, I vote VMware Fusion, but if you're going to change vm host environments, I'd suggest VirtualBox.
[deleted]
[deleted]
I replied with the same advice before noticing you already suggested it - definitely the best way to get comfortable fast!
Above and beyond, install the same distro you are using on the server as your primary operating system at home. There is no better way to learn something than to surround yourself by it and work at it repeatedly. Linux on the desktop will get you comfortable enough on your server in a very quick way.
Oh, and above and beyond, don't even think about trying to mess with things that will require a reboot quite yet. Wait until you are much more capable. I've been using Linux for 15 years and OpenBSD for 8, and I still get cold palms when working remotely on my server if it requires a reboot!
Oh, and above and beyond, don't even think about trying to mess with things that will require a reboot quite yet. Wait until you are much more capable. I've been using Linux for 15 years and OpenBSD for 8, and I still get cold palms when working remotely on my server if it requires a reboot!
If you're a programmer I recommend you (try to) create your own console like Webmin. I learned everything I've ever needed to know about various mail servers, web servers, databases, various configuration files and system level stuff. You don't have to use it or go very far with the actual project in terms of polishing, but it makes for an excellent educational project.
Get in there and do something with your VPS. You can always reset/reinstall. The process of following through and doing it - until it works as you originally envisioned is the best way to learn and improve. During the process understanding how things work and why is more important than finding that perfect configuration (see bediger's comment).
Now as soon as you have users or a business, etc... things change quickly. Your goal will change from learning to maintaining or stability. That's why multiple environments are so common. Most people in IT are not learning they are optimizing, maintaining and protecting. So you have to understand when viewing most comments and information on the internet that you have a different goal (learning) than most of the IT industry.
Almost all the most useful ideas that I learned about managing a server, I learned due to an experience. Opposed to just reading about it on serverfault or superuser :)
Now as soon as you have users or a business, etc... things change quickly. Your goal will change from learning to maintaining or stability. That's why multiple environments are so common. Most people in IT are not learning they are optimizing, maintaining and protecting. So you have to understand when viewing most comments and information on the internet that you have a different goal (learning) than most of the IT industry.
Almost all the most useful ideas that I learned about managing a server, I learned due to an experience. Opposed to just reading about it on serverfault or superuser :)
The Ubuntu Server Guide is an awesome resource if you happen to be running Ubuntu on your server. It's got exactly the right recipes ready for cut 'n' paste.
https://help.ubuntu.com/10.04/serverguide/C/index.html
https://help.ubuntu.com/10.04/serverguide/C/index.html
I am also very interested in this. I need a JRuby hosting solution and there are not very many out there. Plus I would like to learn how to run my own server.
prgmr.com is out of space though. Does anyone know of anything else like this out there?
prgmr.com is out of space though. Does anyone know of anything else like this out there?
Linode is my favorite having used both slicehost and linode. They recently doubled the ram offered compared to the same price point of slicehost. Though the admin interface of slicehost is better.
Blue Box Group, EC2, Linode, Rackspace Cloud, Slicehost, ... there are a lot of options out there depending on the features you require and the level of support.
Getting a VPS is your best bet. Oddly, even though RAM is cheap these days, they are still stingy with it.
For learning, I'd start with the 256 or lowest RAM setup and plan to upgrade later if you get traffic.
Some of the most popular VPS hosts are linode and slicehost but there are many.
For learning, I'd start with the 256 or lowest RAM setup and plan to upgrade later if you get traffic.
Some of the most popular VPS hosts are linode and slicehost but there are many.
RAM being cheap is a relative thing.
It's cheap and easy to upgrade your laptop from 2GB to 4GB.
It can be expensive to upgrade your laptop from 4GB to 8GB, no matter how cheap the RAM itself is, because you might have to buy a new laptop to accomodate.
It's a much more expensive (and logistically difficult) task to roll out that kind of upgrade across a large hosting environment with hardware purchased over multiple deployments.
It's cheap and easy to upgrade your laptop from 2GB to 4GB.
It can be expensive to upgrade your laptop from 4GB to 8GB, no matter how cheap the RAM itself is, because you might have to buy a new laptop to accomodate.
It's a much more expensive (and logistically difficult) task to roll out that kind of upgrade across a large hosting environment with hardware purchased over multiple deployments.
No doubt it's expensive, but it has been several years. A natural hardware refresh should have helped. Check out these guys getting it right:
http://blog.linode.com/2010/06/16/linode-turns-7-big-ram-inc...
http://blog.linode.com/2010/06/16/linode-turns-7-big-ram-inc...
yeah... generally when you buy a box for VPS hosting, you fill it to capacity with ram... of late, the price/density of ram hasn't changed much, so usually you'll throw out the box and replace it when it's time for an upgrade.
None the less, new servers with lots of ram aren't that expensive. Registered ecc ddr3 is about $130 for each 4GiB stick, give or take, and that comprises about half the cost of a server with 8 cores and 32giB ram. (on the G34 platform, that's a single socket server)
None the less, new servers with lots of ram aren't that expensive. Registered ecc ddr3 is about $130 for each 4GiB stick, give or take, and that comprises about half the cost of a server with 8 cores and 32giB ram. (on the G34 platform, that's a single socket server)
For development, 256 works pretty well. If you host more than one site or have a lot of dynamically generated content, you can quickly surpass the 256MB limit and need 512MB.
This depends a lot on the stack you're using and what software your site depends on.
This depends a lot on the stack you're using and what software your site depends on.
prgmr.com is often out of space. check every day or so, they're bound to offer more VPSs pretty soon.
prgmr.com should have a new server up within the next couple days.
To start out free, google app engine is a good way to go. It is a little quirky and different from most solutions, but it's great to start paying nothing.
I don't have any specific advice for learning but if you're having some trouble researching a specific topic or you have a difficult question drop by #slicehost on irc.freenode.net and ask your question. Someone should be along within an hour, usually faster. (IRC can take some time to get answers, I drop by a few hours a day to read what's happened while I was busy with real life)
Get some books on Unix admin. Nemeth et al's "Unix System Administration Handbook", and its Linux version are usually a good start. The most recent edition is coming out in 17 days: http://amzn.com/0131480057
You will also need a bunch of O'Reilly books.
You will also need a bunch of O'Reilly books.
I run my servers on Linode. I've set up test servers on my desktop with virtualbox. I have a wiki in which I keep track over everything I do and how install/remove software. Read lot's of books and articles on the net. Get a ServerFault account to ask questions.
Depends on what you want to do with it. It might be useful to install webmin or virtualmin. Both can help you do the things you already know more conveniently, and both can expose you to some configuration options you didn't know existed.
I've used virtualmin in the past, and while it might provide a certain level of convenience, resolving server related problems becomes, well, a problem. It pays to learn to do things the hard way when the inevitable server issues arise.
One of the things I like about virtualmin and usermin is that they do very little in terms of non-standard config file manipulation (relative to, say, plesk). Agreed that resolving problems from usermin can sometimes be problematic, it is convenient for a lot of things too.
Being able to use the proper tool (control panel or straight command line) is best. I'd found that using a tool to do X would let me then go look at the config files before and after X to see what/how the tool was doing - that was a great learning exercise for me. Might not be for others though.
Being able to use the proper tool (control panel or straight command line) is best. I'd found that using a tool to do X would let me then go look at the config files before and after X to see what/how the tool was doing - that was a great learning exercise for me. Might not be for others though.
I second this... http://news.ycombinator.com/item?id=1482244
also, google is your friend for knowing what but often not why.
also, google is your friend for knowing what but often not why.
[deleted]