Scaling PHP Up, Out, and Around(blog.phpfog.com)
blog.phpfog.com
Scaling PHP Up, Out, and Around
http://blog.phpfog.com/2011/03/16/scaling-php-up-out-and-around/
6 comments
Web servers are limited in the number of concurrent connections they can maintain (tubes aren’t big enough).
This doesn't really make sense -- maintaining a TCP connection requires no "tubes"; the limit comes from finite state storage: a finite state table, a finite number of file descriptors, and a finite amount of RAM in which to store the application-level connection contexts. These numbers turn out to be Really Fucking Big, so this is probably not affecting your application unless you are Google's IMAP servers or something.
This doesn't really make sense -- maintaining a TCP connection requires no "tubes"; the limit comes from finite state storage: a finite state table, a finite number of file descriptors, and a finite amount of RAM in which to store the application-level connection contexts. These numbers turn out to be Really Fucking Big, so this is probably not affecting your application unless you are Google's IMAP servers or something.
By "maintain", he likely means that bandwidth to any particular server is constrained. Yes, you may be able to hold 10k connections open on your Wifi'd laptop, but if each connection only gets 1B/s is it really a useful webserver?
Any modern web server is going to be able to handle tons of connections, much more than a wifi laptop that can not yet be considered a modern web server. Beyond that, bandwidth is rarely the limiting factor. Framework/language weight, slow db queries, and javascript performance in the user's browser are much more prevalent concerns. In my experience it's always the db that falls over first, well before connection limits are reached.
Slow DB queries, yes. If your application does something stupid with SQL, your whole awesome C10K-solving epoll-based framework comes to a halt waiting for the database to read 1M rows off disk. Worst of all, your ORM likely hides the facts from you, producing left joins when you need inner ones, or worse not grouping the results so that a full join'ed result is returned when you only need the first and last tables.
More on topic, scaling web servers is not hard, just tedious. Scaling storage, especially if you need to use RDBMS's is the real trick.
More on topic, scaling web servers is not hard, just tedious. Scaling storage, especially if you need to use RDBMS's is the real trick.
If your application does something stupid with SQL, your whole awesome C10K-solving epoll-based framework comes to a halt waiting for the database to read 1M rows off disk.
No, a database connection is just another socket in your "awesome epoll-based framework".
Worst of all, your ORM likely hides the facts from you
Maybe if you're writing your app in BASIC? A good ORM doesn't hide the query logic from you, and there are plenty of popular languages that have good ORMs. The difference between using an ORM and not using an ORM is that you don't have a bunch of code to map SQL rows to application domain objects in your app. That's it.
(Note: Active Record and Class::DBI from 5 years ago aren't good ORMs. But people have written good ones in the intervening time, like DBIx::Class.)
No, a database connection is just another socket in your "awesome epoll-based framework".
Worst of all, your ORM likely hides the facts from you
Maybe if you're writing your app in BASIC? A good ORM doesn't hide the query logic from you, and there are plenty of popular languages that have good ORMs. The difference between using an ORM and not using an ORM is that you don't have a bunch of code to map SQL rows to application domain objects in your app. That's it.
(Note: Active Record and Class::DBI from 5 years ago aren't good ORMs. But people have written good ones in the intervening time, like DBIx::Class.)
To your first point, at least libmysqlclient blocks and any wrapper around it does too. To your second point, my most recent experience is with sqlalchemy, which is really pretty powerful, but also produces some hideous SQL when using the ORM layer. It does not hide the query logic from you, but conceals it just enough that I have to check every time that it is not lazy-loading 10,000 rows or not returning 10,000 rows when I only need 10.
libmysqlclient sucks. :)
Thing I don't get about things like PHPFog, is how do you manage dependencies (my app depends on my shared libraries), data storage (postgresql, for instance), Memcached, application configuring, etc?
Agreed, i asked on Twitter about these kind of stuff (more specifically about Redis, but that applies to anything), never got an answer.
Right now, i have an access to PhpFog, it's "nice", but, without the ability of getting Redis, or Sphinx prevents me to use it as my little laboratory for small apps and stuff.
I guess i'll stick with AWS for now...
Heroku doesn't put your applications on their own instances. I imagine they are also getting around the problem of crap low end EC2 instances by distributing all apps across clusters of the largest instances.
Not to bag on PHP Fog, they have created a lot of excitement in the PHP space. Personally, I prefer Heroku's model and for PHP I would rather just cut out the middle man and run my own VPS. It's nice not to have to deal with administration, but that's just one benefit of using Heroku, and not the most important for me.