How To Use Amazon EC2 as Your Desktop(blog.restbackup.com)
blog.restbackup.com
How To Use Amazon EC2 as Your Desktop
http://blog.restbackup.com/how-to-use-amazon-ec2-as-your-desktop
90 comments
I do the majority of development locally and I find it works out just fine. I have ssh open to my machine and have a bunch of git repos offloaded to a server somewhere so I can work from practically anywhere, too. You don't have to have a network of four-five computers to work this way. :)
This is the setup I aspire to one day have. I've spent the better part of the past 6 months getting very comfortable with the command line and editing within the shell via emacs - now if only I could convince work to make the server I run all my analysis on be accessible from my home...
it's great to see people finally starting to get it 20 years after initial Plan 9 development :)
I have been doing this for a year with a Linode small instance and it works like a charm - although I only use it through a Putty terminal.
I had initially set it up to code small ruby scripts remotely that help me with my day job (mostly to scrape websites with the excellent Scrapi gem). Since IT would not let anyone install anything on their computers, but still left port 443 wide open, I gave it a try.
The added benefit is that working on a linux box through Putty forces you to learn a real terminal text editor - in my case Emacs. I guess I can thank IT now ;)
I had initially set it up to code small ruby scripts remotely that help me with my day job (mostly to scrape websites with the excellent Scrapi gem). Since IT would not let anyone install anything on their computers, but still left port 443 wide open, I gave it a try.
The added benefit is that working on a linux box through Putty forces you to learn a real terminal text editor - in my case Emacs. I guess I can thank IT now ;)
I've been using my linode box now for about a year and a half as my primary development machine. The reason here is twofold: I can access the machine from anywhere, and I have much better bandwidth on the machine than I'd get from where I currently live. That last bit is key - I live in Hawaii and the internet speeds here leave much to be desired (which is expected for an island in the middle of the pacific). If I need the latest build of ruby, it's not a 20 minute download, it's a 20 second one.
I do all of my work over ssh now. Key based auth allows me to hop in and out as needed, and more importantly it allows me to use libnotify + growl to alert me when things happen on my server. Code finished compiling? Growl pop up to let me know. Message via irssi? Pop up. Makes things feel very seamless. I use git to sync things as needed, vim for development, screen as my multiplexer, and irssi+bitlbee for messaging. It's lovely.
I do all of my work over ssh now. Key based auth allows me to hop in and out as needed, and more importantly it allows me to use libnotify + growl to alert me when things happen on my server. Code finished compiling? Growl pop up to let me know. Message via irssi? Pop up. Makes things feel very seamless. I use git to sync things as needed, vim for development, screen as my multiplexer, and irssi+bitlbee for messaging. It's lovely.
Note, if you like Windows, you can create an instance that runs Windows and log in with RDP. This is particularly nice because it's dead easy to install and configure an RDP client on Linux, Windows or MacOS... Whereas getting an X-Windows installation working perfectly on a non-Unix machine is a lot of work.
I'm pretty amazed that it took Microsoft about two years to realize how much fun it is to use Windows/RDP in the cloud and they still haven't quite added this to Azure. A lot of what I like about AWS is that you can do an awful lot of things the way you're used to doing them.
That said, I mainly use Linux instances, and normally work with an ssh text console from Linux, Windows and MacOS clients. I use jedit on the client machine to edit files through sftp. Some of my instances are production instances that run 24-7, although I commission development machines and machines to run big batch jobs from time to time.
I'm pretty amazed that it took Microsoft about two years to realize how much fun it is to use Windows/RDP in the cloud and they still haven't quite added this to Azure. A lot of what I like about AWS is that you can do an awful lot of things the way you're used to doing them.
That said, I mainly use Linux instances, and normally work with an ssh text console from Linux, Windows and MacOS clients. I use jedit on the client machine to edit files through sftp. Some of my instances are production instances that run 24-7, although I commission development machines and machines to run big batch jobs from time to time.
This is how I've been working on my side project. Not with EC2 or Linode, but with a Chunkhost (they're running a beta of very small, 20gb disk / 512mb ram) VPS. Using my Cr48 to develop it has been a dream (minus some minor issues with Cr48's tendency to lock up ssh terms if you lose connection, requiring a reboot to get them out of your alt-tab queue).
It's pretty great - not only can I work on it from anywhere with a working ssh client, but I get super fast speeds.
It's pretty great - not only can I work on it from anywhere with a working ssh client, but I get super fast speeds.
I've had a Linode for 3 years and love it, too. I spend about $240/year for a node with 512 MB of memory. But EC2 is more economical, providing 3x the memory for nearly the same price, plus much more CPU power and great disk IO performance.
I'm not sure EC2 is more economical in terms of CPU power, as I think Linode gives you access to up to 4 Xeon cores depending on load (which is generally quite low) so you probably get more raw performance from a Linode instance despite having less memory.
IMO, Linode is the best for this use. I pay around $22/month for 512MB/16GB/32-bit Linux instance and the instance is always on.
how is that any different from a micro-tier EC2 instance? seems to be roughly $12/month for about the same specs.
Well, don't forget that the lowest tier Linode comes with 200GB bandwidth/month whereas EC2 is pay as you go.
Plus it's easier to scale up EC2, with all their other cloud features available nearline....
4 Xeon cores on Linode vs. some leftover burst cycles on EC2 micro.
Just try to compile Ruby from source on an EC2 Micro instance vs. a $20/month Linode. In my experience (repeated builds at various times of day) the EC2 instance is an order of magnitude slower. And the Linode includes some bandwith, as someone else already mentioned.
I actually switched to using an EC2 Micro instance because my 256MB VPS didn't have enough RAM to compile some of the packages I needed. The compile took a while, but they were for libraries I rebuild maybe once a month.
As far as development goes I haven't had a problem yet with speed. I tend to do small incremental compiles, and when I need to a full compile I go get a cup of coffee.
As far as development goes I haven't had a problem yet with speed. I tend to do small incremental compiles, and when I need to a full compile I go get a cup of coffee.
Just to clarify, the $20/month Linode I'm referring to has 512MB ram.
I'm just using compiling as an illustration of CPU performance. You'll have the same problem when running unit tests, for example. You can only spend so much time on forced coffee breaks before it gets old. Or you can have a very tiny test suite.
I'm just using compiling as an illustration of CPU performance. You'll have the same problem when running unit tests, for example. You can only spend so much time on forced coffee breaks before it gets old. Or you can have a very tiny test suite.
Micro instances are super constrained on CPU in my experience. Unusably so.
Phoronix recently tried to benchmark a micro instance. It didn't do very well:
http://www.phoronix.com/scan.php?page=article&item=amazo...
Turns out that on average a micro's performance is somewhere between a netbook and an ARM tablet for most tasks, but the performance varies wildly.
http://www.phoronix.com/scan.php?page=article&item=amazo...
Turns out that on average a micro's performance is somewhere between a netbook and an ARM tablet for most tasks, but the performance varies wildly.
Why would I need a desktop in the cloud?
Everything can be done much faster via an ssh command line. I don't get it.
Very nice tutorial though, good intro to EC2 for newbies.
Everything can be done much faster via an ssh command line. I don't get it.
Very nice tutorial though, good intro to EC2 for newbies.
Not everything can be done through a command line, that's all. I use a VNC desktop when travelling for browser access, image editing and to use my preferred mail client.
Persistent desktop accessible from anywhere via VPN.
He said he shuts the machine down on nights and weekends. I am not familiar with EC2, but doesn't that mean that the desktop isn't persistent from day to day? Yes, he can connect and disconnect at will throughout the day while it stays the same, but shutting down the machine at night shuts down all the apps he had open I assume. Am I wrong?
No differently than if you shut your computer down and boot it up again, if he's configured it accordingly with an elastic block store for the filesystem, etc.
Not if you shut down with "hibernate".
With no risk of hardware failure.
Well, minimal. http://www.google.com/search?q=amazon+ec2+down
I have that too. It costs me nothing (my electricity is comped currently), it's a faster machine and I have physical control of it if necessary.
What do you do when it breaks?
Who pays for the bandwdith? How much do you have?
What if your house burns down?
I agree, but not everyone is a command line ninja.
This works out to about $315 per year for a development machine.
Am I the only one who thinks that's ridiculous? I mean, after two years, you'll already be at the cost of a decent laptop. Three years will get you the cost of a MacBook. Unless you're upgrading your laptop every year, there is no way that this is cheaper than developing on localhost.
Am I the only one who thinks that's ridiculous? I mean, after two years, you'll already be at the cost of a decent laptop. Three years will get you the cost of a MacBook. Unless you're upgrading your laptop every year, there is no way that this is cheaper than developing on localhost.
its not like its massively more expensive or massively cheapter, so they aren't competing on price.
EC micro instances are 56$/year (reserved) and in a combination with version control tool like SVN I don't see why one would pay > $300/year
The price of resources drops over the years.
You could save for 3 years and buy a macbook.... or you could work in the cloud and use whatever cheap-ass machine you want to access it, with much less worry about backup.
THis isn't really a home-PC replacement - it's a developer hting.
You could save for 3 years and buy a macbook.... or you could work in the cloud and use whatever cheap-ass machine you want to access it, with much less worry about backup.
THis isn't really a home-PC replacement - it's a developer hting.
It's accessible from more places than your laptop is, unless you take your laptop with you everywhere. You could demo something to a friend for instance.
It's accessible from wherever you have a computer with the necessary software and a fast enough connection. Sure, if you don't have your laptop then this could be useful, but that's like saying that a land-line based system for managing your address book is good because it's accessible from more places than your phone is, unless you take your phone with you everywhere.
Um, exactly? Memorizing important numbers is a good idea precisely because then you can use a friend's phone in case yours isn't available.
There's this amazing technology called "paper". You should read up on it.
don't forget about energy consumption costs
Maybe I'm missing something, but he still needs a physical computer to connect to his development machine in the cloud right?
I personally think this is expensive for the benefits it brings but clearly for the author it is worth it.
I personally think this is expensive for the benefits it brings but clearly for the author it is worth it.
Are you paying for power for a fridge? Then laptop energy consumption is negligible.
on a similar note, I've been doing web development on my CR-48 with an ec2-hosted development environment. I documented my setup here: http://www.dustingetz.com/dev-etc/app-engine-web-development...
Re: your post. No need to kill ssh when it hangs. Type <ENTER> <~> <.> to kill the session. There's other commands available - type "~?" to see them.
Thanks for this - I've been running into similar problems. I'll try it out when I get back to my cr48.. didn't bring it into the office today.
One reason that someone might want to do this (besides learning about ec2) is if they are itinerant. For example, suppose you work for a company where you travel a lot and there are restrictions on vpn usage--you could start an instance of this (assuming that you can still comply with the IP/PII policies of your workplace) and access it from anywhere.
Another thought--if you are playing with web services such as twilio for example, you need an IP address for the response to your REST requests--this could be a problem for people doing development at home without a static ip (sure, you could use localtunnel, but this does have some appeal). This way, you can test this on your development machine without going live...
Anywho, just a few thoughts.
Another thought--if you are playing with web services such as twilio for example, you need an IP address for the response to your REST requests--this could be a problem for people doing development at home without a static ip (sure, you could use localtunnel, but this does have some appeal). This way, you can test this on your development machine without going live...
Anywho, just a few thoughts.
This is exactly why I set up my first (and now my second0 EC2 instance for development. I've been playing with services that need an external static IP address. My ISP is pretty hostile towards that kind of thing, and it's a PITA if you're travelling and need to do some work.
This does have the obvious problem that you need a PC / Tablet / Laptop to connect to Amazon EC2 in the first place so why not just use that.
In this specific example it's a problem because a fair bit of local software is required. If you set it up so that all you have to do is VNC or RDP, possibly over ssh, all you need is a USB key and access to a computer to get access to your computer.
Sorry but that seems a lot of effort to go through. Why would you bother?
Learning to use EC2 for development is a great way to learn to use it for production.
I disagree. Running EC2 isn't rocket science. With web console, they have made it more simple.
What is little hard is, auto load balancing, shutting down instances based on req/minute, monitoring and stuff. But you don't play with that, if you are using remote EC2 machine as desktop.
What is little hard is, auto load balancing, shutting down instances based on req/minute, monitoring and stuff. But you don't play with that, if you are using remote EC2 machine as desktop.
Development and production can run on the same network then, which depending upon what you're doing can make development way faster.
You might want that if you want to have your setup with you no matter where on Earth you happen to be.
I was thinking of using a dedicated server for this for a while, with visualization maybe: a VM for server, a VM for my desktop... Using EC2 for this instead gives you an option to relatively easily choose from several locations in different parts of the world. Also it forces you to learn using EC2.
I was thinking of using a dedicated server for this for a while, with visualization maybe: a VM for server, a VM for my desktop... Using EC2 for this instead gives you an option to relatively easily choose from several locations in different parts of the world. Also it forces you to learn using EC2.
I just learned how to do this! If anyone's interested I could write out the steps for doing this on Ubuntu.
I've been toying with the idea of offering managed dev environment using this. Some pain points it would solve: -no new installation/configuration when you buy a new laptop -let an outsourced sysadmin take care of annoying things like installing java, adding environment variables, installing mobile app sdk's etc. -environment is accessible anywhere you have Internet access
The ultimate benefit is: spend less time installing things and more time developing.
I probably don't know enough about UNIX to do this myself, but I think it's a worthy idea.
I've been toying with the idea of offering managed dev environment using this. Some pain points it would solve: -no new installation/configuration when you buy a new laptop -let an outsourced sysadmin take care of annoying things like installing java, adding environment variables, installing mobile app sdk's etc. -environment is accessible anywhere you have Internet access
The ultimate benefit is: spend less time installing things and more time developing.
I probably don't know enough about UNIX to do this myself, but I think it's a worthy idea.
It'd be difficult to trust the admin in that case. Same reason you see instructions for how to shrink the official ubuntu ami down to something smaller for a micro instance instead of a blog that say "here, mount this one I already shrunk"
IMO the problem you will run into with that is that most developers seem to heavily customize their dev environments. I'd love to be wrong about that though, because there are times when I'm setting up a new one when I'd love to be able to pay someone to do it for me.
Hmm, private key posted fully in the image...
You can type it in and try it out... :P
Probably not the one he actually uses.
One advantage of this setup not mentioned in the article: you needn't be afraid of ${airport_security} searching your laptop at the border.
What's the point of the cloud? Sure, EC2 scales well, but does desktop needs scaling? (Unless you want to buzzword that your desktop is web scale, haha)
it's accessible from anywhere at anytime (if you don't shut it down) from any device capable of executing a ssh client (iPhone, iPad, for example) also you can have a development environment exactly equal to your production environment and you are safe from hardware failures.
So is usual non-cloud hosting. Well, not turning off my home desktop PC is certainly not "cloud", but, given I have fast and stable network connection and power supply, it works equally (?) well.
That's why my question emphasized the "cloud" word.
That's why my question emphasized the "cloud" word.
It'd be neat to start dynamically adding more cores or RAM or storage as you start to hit capacity on an instance.
I did something like this with Dropbox:
http://valyagolev.net/article/develop_windows_linux/
bash on windows without Cygwin's lameness. problem solved
(I moved to ubuntu & emacs eventually, although their usability is crazy)
http://valyagolev.net/article/develop_windows_linux/
bash on windows without Cygwin's lameness. problem solved
(I moved to ubuntu & emacs eventually, although their usability is crazy)
Since you will still have your local computer running to actually use it, you have to add its electricity costs to the dev machine cloud hosting aswell. You actually also need to buy the local machine to do work anyway, so it doesnt save any money, but adds 300 USD to your current dev enviroment.
That said, i can get a plain old Vserver similar to an EC2 mini instance (i think its even more powerful) for 80 bucks a year and that would be running 24/7. If you dont need the benefits of EC2 which are scaling and such, there are much cheaper solutions.
Tried following the instructions - ended up with some "command not found" stuff when trying to connect with the NX BEta client for OSX (the older client requires rosetta, which most modern OSX machines don't have)
The instructions could use an update to the latest version of ubuntu too.... but this will make for a fun weekend project - excellent find (more for the EBS shortcuts and bootstrapping method + NX tips than the concept itself)
The instructions could use an update to the latest version of ubuntu too.... but this will make for a fun weekend project - excellent find (more for the EBS shortcuts and bootstrapping method + NX tips than the concept itself)
OS X will prompt you to download and install Rosetta from Apple, hassle-free, if you attempt to launch a PPC application and you don't have it installed already.
It only works if you're launching an app bundle, though, not if you're trying to run a bare binary from the command line. I just downloaded some old PPC-only shareware app and launched it.
Apparantly there's a .pkg file somewhere on the OS install DVD too, but I couldn't find it on my MacBook's restore USB stick.
It only works if you're launching an app bundle, though, not if you're trying to run a bare binary from the command line. I just downloaded some old PPC-only shareware app and launched it.
Apparantly there's a .pkg file somewhere on the OS install DVD too, but I couldn't find it on my MacBook's restore USB stick.
Too late to edit - works fine with the previous rosetta client -just not with the beta.
Cheers.
I did this a few weeks ago and tried to use it form a fairly old MacBook. I intended to use RubyMine which had a few minor issues while scrolling. What kille dit for me however was that all short keys would go to my local machine. Since using RubyMine lives from extensive short key usage this setup wasn't an option for me.
The nice thing however is that you can also stop your instance and only pay as you go.
The nice thing however is that you can also stop your instance and only pay as you go.
I'd actually tried to do this on a Linode machine before, but could never get NX to connect. VNC worked but is a bit slow, of course. Actually, I've never seen NX actually work. I'll have to try the directions here and see how it goes.
2noobs questions:
- Do you think that it's possible to use AWS Free Usage Tier with micro instance do to that ?
- Is it possible to have a graphic interface?
- Do you think that it's possible to use AWS Free Usage Tier with micro instance do to that ?
- Is it possible to have a graphic interface?
- A micro instance should work, although you'll only have 600 MB of memory. Also, I hear that micro instances have poor cpu scheduling, which would make your apps to feel jerky and sometimes freeze for 1-2 seconds. If you try it please let me know how it works.
- FreeNX provides a graphical interface. It makes your Linux program windows appear on your Windows machine.
- FreeNX provides a graphical interface. It makes your Linux program windows appear on your Windows machine.
I don't know about the FreeNX server, but if you use a lightweight window manager like fluxbox 600MB of memory will be more than you need for most things.
oh, i assumed the jerking was lag! that explains a lot!
Yes, the Linux Micro instance should be enough to run a development environment. And yes, you can have a graphic interface, but I'm not sure it's your best option (both for bw. And memory usage if you're in the micro instance might be a little bit to high.
I am trying to figure out exactly what restbackup.com does. Does it backup customer info from a desktop app or the web app?
Anyone know?
Anyone know?
RestBackup is a web service. You can HTTP PUT files to https://us.restbackup.com/ and then HTTP GET them later. The uploaded files are deleted automatically after a specified period of time.
The RestBackup Management API lets you create separate backup accounts for your users. Your app/script/cronjob can back up each user's data from their machine/device to a separate and private account.
Using RestBackup, you can have automatic backups without adding any new servers or equipment. It works for mobile apps, desktop apps, and hosted applications. It also works well for normal file backups.
At the moment, the only way to use RestBackup is with curl or your own program. More client software will be available soon.
The RestBackup Management API lets you create separate backup accounts for your users. Your app/script/cronjob can back up each user's data from their machine/device to a separate and private account.
Using RestBackup, you can have automatic backups without adding any new servers or equipment. It works for mobile apps, desktop apps, and hosted applications. It also works well for normal file backups.
At the moment, the only way to use RestBackup is with curl or your own program. More client software will be available soon.
Ahh...I see.
Well, if you don't mind, can I give you some feedback (assuming that you are the founder).
On the landing page, talk about benefits - not technology. No matter how much I read and searched the site, I couldn't figure out what it does and who it does it for. It's not because I am stupid either, well...could be but I doubt it.
I would choose one niche, and communicate that in the copy on the main page.
Also, pricing is an important value communicator. Honestly, I think your pricing is too cheap. Perhaps you should figure out another way to charge more. $9.99 is WAYY too cheap, and it worries me that you might not be around - especially for my backup purposes.
I don't want to be worried that you are going to disappear.
Hope that helps!
Well, if you don't mind, can I give you some feedback (assuming that you are the founder).
On the landing page, talk about benefits - not technology. No matter how much I read and searched the site, I couldn't figure out what it does and who it does it for. It's not because I am stupid either, well...could be but I doubt it.
I would choose one niche, and communicate that in the copy on the main page.
Also, pricing is an important value communicator. Honestly, I think your pricing is too cheap. Perhaps you should figure out another way to charge more. $9.99 is WAYY too cheap, and it worries me that you might not be around - especially for my backup purposes.
I don't want to be worried that you are going to disappear.
Hope that helps!
this is pretty fascinating.
we're already seeing people switch over to the new mac air which runs IOS.
once it becomes more cost efficient to run your desktop in the cloud its clear this is a major trend which is about to start and permanently change computing. far for the better IMHO, i'm sick of handling upgrades myself.
we're already seeing people switch over to the new mac air which runs IOS.
once it becomes more cost efficient to run your desktop in the cloud its clear this is a major trend which is about to start and permanently change computing. far for the better IMHO, i'm sick of handling upgrades myself.
A few years on and I really don't think about my computer, but rather a network of computers. It is not uncommon for me to be logged into four different systems at a time, running different things on each. If I need more processing power, I log into one of the beefier systems and start a job. If I'm just editing email it's local, but if I'm editing web files, it's probably on a server hosting my dev instance so my changes are live. I just do each task wherever it is most appropriate.
The real change is that you stop having one entry point to your "computer". I can access the same resources from almost any laptop or desktop. I keep desktops with large dual monitors at work and at home and rarely worry about synchronizing them because the work is actually being done in the same place.
It really is liberating to build your own set of cloud resources and realize that your desktop is just another resource at your disposal in your network.