Cloudflare’s CDNJS vs. Google Hosted Libraries(baldnerd.com)
baldnerd.com
Cloudflare’s CDNJS vs. Google Hosted Libraries
http://www.baldnerd.com/make-your-site-faster-cloudflares-cdnjs-vs-google-hosted-libraries-shocking-results/
13 comments
Could you post the exact commands you used? That high connection time for google looks like SSL setup.
Non concurrent:
ab -n 100 http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min...
ab -n 100 http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.mi...
Concurrent case:
ab -c 5 -n 100 http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.mi...
ab -c 5 -n 100 http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min...
ab -n 100 http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min...
ab -n 100 http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.mi...
Concurrent case:
ab -c 5 -n 100 http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.mi...
ab -c 5 -n 100 http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min...
Interesting results, but let's think about a few things before we all switch to Cloudflare CDNJS.
1) Time to first byte. In my experience the biggest lag is the connection, not actually downloading the file (for small js, images, etc...) I don't know how to test this.
2) Caching, by using google for common libraries like jQuery your chance of the end client already having a local cached copy are much greater.
3) Reliability. I know google has been recently killing off it's products, and even I tweeted how they could break the internet by shutting off their hosted library API, but it's probably not going to happen. Can you say the same about cloudflare?
I'm not saying I like one more than the other, but these are some things I would like to address before switching my and my clients sites over.
One thing I will give a +1 to Cloudflare is the ability to add js to the CDN via github. https://github.com/cdnjs/cdnjs
1) Time to first byte. In my experience the biggest lag is the connection, not actually downloading the file (for small js, images, etc...) I don't know how to test this.
2) Caching, by using google for common libraries like jQuery your chance of the end client already having a local cached copy are much greater.
3) Reliability. I know google has been recently killing off it's products, and even I tweeted how they could break the internet by shutting off their hosted library API, but it's probably not going to happen. Can you say the same about cloudflare?
I'm not saying I like one more than the other, but these are some things I would like to address before switching my and my clients sites over.
One thing I will give a +1 to Cloudflare is the ability to add js to the CDN via github. https://github.com/cdnjs/cdnjs
Using wget for a test is just terrible. Here's a better test (and I'll put it here, so everyone can criticize it):
1) start chrome in incognito mode.
2) press f12, and then click on network
3) load each js
Google:
DNS Lookup: 14ms
Connecting: 24ms
Sending: 0
Waiting: 46ms
Receiving: 42ms
TOTAL: 128ms (86ms latency)
Cloudflare:
DNS Lookup: 15ms
Connecting: 43ms
Sending: 0
Waiting: 46ms
Receiving: 49ms
TOTAL: 154ms (105ms latency)
It probably varies enormously depending on your location.
Here in Vancouver, I get 1.25MB/s for Google's CDN and 1.15MB/s for CloudFare. Unless we have a distributed tool to test that on a massive scale, it will only reflect the greatly-varying differences between people's connections, "distance to CDN", DNS metrics and whatnot which I guess greatly impact the speeds observed.
Nevertheless I really like that this actually "exercises" the different services and provides some form of measuring and poking that improves the overall transparency and accountability.
Here in Vancouver, I get 1.25MB/s for Google's CDN and 1.15MB/s for CloudFare. Unless we have a distributed tool to test that on a massive scale, it will only reflect the greatly-varying differences between people's connections, "distance to CDN", DNS metrics and whatnot which I guess greatly impact the speeds observed.
Nevertheless I really like that this actually "exercises" the different services and provides some form of measuring and poking that improves the overall transparency and accountability.
Another point to make is server response time. I'm not making a claim one is better overall than the other, but at my location (boston) cloudflare's servers respond a bit faster. Just grabbing a couple different files between the two shows up to 300ms difference, but usually <100ms.
[edit] oh yeah, and this is all moot if your user already has it in cache, which is currently more likely from google.
[edit] oh yeah, and this is all moot if your user already has it in cache, which is currently more likely from google.
Disclaimer: I'm not offering a perfect solution to the ideal test.
1.) Depends on how everyone is routing. The people replying directly to the OP's page are checking from different servers with god-knows-what kind of routing tables that the average web browser on the net does not have access to.
1b.) A better test would be to have people from different ISP's and different geographic locations testing this out. Routing is everything. In other words, let's see what results are seen by mom-and-pop from India, UK, Texas, Rhode Island, Montana, Nunavut, Brazil, China, and Nigeria.
2.) Already said, but worth pounding into the OP's head, Google is the most widely used for jQuery because everyone else is using it and harddrive cache is 1.21 jiggawatts faster than any networked CDN.
3.) Latency and overall speed need to be measured. What use is 88Mbps if you've got horrible latency? Sure, there are times when a courier on a bike is faster than FTP, but for small files that latency is a bigger deal.
1.) Depends on how everyone is routing. The people replying directly to the OP's page are checking from different servers with god-knows-what kind of routing tables that the average web browser on the net does not have access to.
1b.) A better test would be to have people from different ISP's and different geographic locations testing this out. Routing is everything. In other words, let's see what results are seen by mom-and-pop from India, UK, Texas, Rhode Island, Montana, Nunavut, Brazil, China, and Nigeria.
2.) Already said, but worth pounding into the OP's head, Google is the most widely used for jQuery because everyone else is using it and harddrive cache is 1.21 jiggawatts faster than any networked CDN.
3.) Latency and overall speed need to be measured. What use is 88Mbps if you've got horrible latency? Sure, there are times when a courier on a bike is faster than FTP, but for small files that latency is a bigger deal.
That is impressive, but you've basically just checked whether cloudfare has a POP in Ontario, or atleast within 2-3ms of your datacenter.
They both have POPs local to me:
They both have POPs local to me:
eggplant:~ tsl$ wget http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.min.js
--2013-03-22 19:29:21-- http://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.min.js
Resolving cdnjs.cloudflare.com... 141.101.123.8, 190.93.240.8, 190.93.241.8, ...
Connecting to cdnjs.cloudflare.com|141.101.123.8|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-javascript]
Saving to: ‘jquery.min.js’
[ <=> ] 92,629 --.-K/s in 0.04s
2013-03-22 19:29:21 (2.03 MB/s) - ‘jquery.min.js’ saved [92629]
eggplant:~ tsl$ wget http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js
--2013-03-22 19:42:05-- http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js
Resolving ajax.googleapis.com... 173.194.75.95, 2607:f8b0:400c:c01::5f
Connecting to ajax.googleapis.com|173.194.75.95|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/javascript]
Saving to: ‘jquery.min.js.1’
[ <=> ] 92,629 --.-K/s in 0.05s
2013-03-22 19:42:05 (1.62 MB/s) - ‘jquery.min.js.1’ saved [92629]
tsl@beast:/volumes/fast$ wget http://nas.***.com/jquery.min.js
--20:00:05-- http://nas.***.com/jquery.min.js
=> `jquery.min.js'
Resolving nas.***.com... 10.10.10.2, fd2b:2048:4633:10::2
Connecting to nas.***.com|10.10.10.2|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 92,629 (90K) [application/javascript]
100%[====================================>] 92,629 --.--K/s
20:00:05 (63.00 MB/s) - `jquery.min.js' saved [92629/92629]
Never seen that before - what is nas.*.com?
He censored the domain name with * :-)
> my single core ARM6
You need to double-check your methodology:
http://sworddance.com/blog/2013/03/22/always-check-twice/
The difference is almost certainly an initial poor DNS resolution the first time for which ever service had the slower time.
That said, a more complete js cdn is always nice.
http://sworddance.com/blog/2013/03/22/always-check-twice/
The difference is almost certainly an initial poor DNS resolution the first time for which ever service had the slower time.
That said, a more complete js cdn is always nice.
I ran the test, did it five times for Google, five times for Cloudflare and for each and every single one Google won out (for the current location I am at ... work), the script I used is here:
https://gist.github.com/bertjwregeer/5225202
This way it will also output how long it spent in each step, including connect, appconnect (if you try the SSL version), how long it spends in pretransfer and how long it takes to start the transfer at all. Followed by the size of the download and the header size.
This way you can get a much better idea as to why it is taking longer for one service over the other. The files themselves are relatively small so are not an accurate indicator of transfer speed.
https://gist.github.com/bertjwregeer/5225202
This way it will also output how long it spent in each step, including connect, appconnect (if you try the SSL version), how long it spends in pretransfer and how long it takes to start the transfer at all. Followed by the size of the download and the header size.
This way you can get a much better idea as to why it is taking longer for one service over the other. The files themselves are relatively small so are not an accurate indicator of transfer speed.
For small files like this the latency is much more important than throughput, not to mention more users will have Google's version cached already.
Indeed. Past 5 Mbps, bandwidth is basically irrelevant, and Google knows it.[1]
[1]: https://docs.google.com/a/chromium.org/viewer?a=v&pid=si...
[1]: https://docs.google.com/a/chromium.org/viewer?a=v&pid=si...
Don’t forget that one of the advantages of CDNs is that the user may already have the file cached in their browser. This is why I use Google’s CDN for jQuery and CDNJS for everything else – more web sites use Google’s CDN for jQuery, so the user is more likely to have that cached. Not to mention the HTTP limitation of 3 simultaneous requests from one domain…
the point isn't how fast your CDN is, the point is how likely it is that the user already has the asset in browser cache.
network effect of caching aside, I decided to run tests from 95 nodes around the world. Most of these have above average connectivity, however some are true "last-mile" nodes. The following data set consists of just over 2000 runs over one hour of testing.
#tests avgTotal avgDNS avgPingRT
CDNJS 1105 187ms 58ms 27ms
GoogleHostedAPIs 1120 219ms 30ms 38ms
(IE min is 496k/s, max is 895k/s, average is about 600k/s)
Using wget is a completely useless benchmark, from what I can tell. Using apache's little benchmark tool I get about 255ms vs 180ms average for 100 requests to each.
The interesting part is that for ajax.googleapis.com I get:
and for cloudflare I get:
IE for google, all the time is in actually getting a connection and getting the bits, whereas, for cloudflare, there is actually some time waiting for their servers.
Using 5 concurrent requests actually gives me a massive advantage for google (cloudflare takes roughly the same time, google goes 4 times faster)