Unpkg CDN down causing dependent website outages(github.com)
github.com
Unpkg CDN down causing dependent website outages
https://github.com/mjackson/unpkg/issues/384
8 comments
You're looking for a CDN with an integrated storage service, bunny.net has that feature[1]. They also have a volume tier for cheaply serving files[2] which seems to meet your need for a low PoP count (with some discounts to go along with it).
[1] https://support.bunny.net/hc/en-us/articles/360017093479-Und...
[2] https://bunny.net/network/
[1] https://support.bunny.net/hc/en-us/articles/360017093479-Und...
[2] https://bunny.net/network/
> What I wish existed (or maybe does) is something like a CDN with ~8 global locations that promised to keep your static site in a warm cache for a small fee
cloudflare cache reserve or bunny.net as someone else mentioned
cloudflare cache reserve or bunny.net as someone else mentioned
> What I wish existed (or maybe does) is something like a CDN with ~8 global locations that promised to keep your static site in a warm cache for a small fee
IIRC this is what Cloudflare Argo is supposed to do? https://www.cloudflare.com/en-gb/application-services/produc...
Rather than routing cache misses to origin, it'll route to other PoPs with the cached content?
IIRC this is what Cloudflare Argo is supposed to do? https://www.cloudflare.com/en-gb/application-services/produc...
Rather than routing cache misses to origin, it'll route to other PoPs with the cached content?
“Works on my machine”
Depend on external services, die by external services. If you can’t afford redundancy, then you must accept downtime.
I build all of my dependent libraries locally and bundle with my app. For one application, I maintain and build the entire OS from source. I depend on aws,azure cloud storage for my own hosted cdn. I can fail back to my own hosts if necessary.
Depend on external services, die by external services. If you can’t afford redundancy, then you must accept downtime.
I build all of my dependent libraries locally and bundle with my app. For one application, I maintain and build the entire OS from source. I depend on aws,azure cloud storage for my own hosted cdn. I can fail back to my own hosts if necessary.
This is one thing I never understood about web development.
I would have thought the risks from 3rd party production dependencies would make devs do all in their power to minimize them. But IIUC it's a somewhat common practice.
Is there something about the market dynamics that make this a reasonable tradeoff? E.g., the problems are rare and this somehow allows faster time to market?
(Asking as a systems programmer, very much not a web developer.)
I would have thought the risks from 3rd party production dependencies would make devs do all in their power to minimize them. But IIUC it's a somewhat common practice.
Is there something about the market dynamics that make this a reasonable tradeoff? E.g., the problems are rare and this somehow allows faster time to market?
(Asking as a systems programmer, very much not a web developer.)
> But IIUC it's a somewhat common practice.
I don’t have any data but I really don’t think it actually is that common. I definitely heard it used a lot more in the 2000s jQuery era but not really today. Browsers even have isolated storage these days so the benefits really are infinitesimal.
I don’t have any data but I really don’t think it actually is that common. I definitely heard it used a lot more in the 2000s jQuery era but not really today. Browsers even have isolated storage these days so the benefits really are infinitesimal.
I think it's fairly common. I was reviewing an implementation of a phone number input component last month. It looked great, changing the country code automatically showed the correct country's flag or vice versa select your flag and the country code updated, plus a bunch of useful phone number validation logic was provided, you just add the NPM package and use the resulting component and magic happened. But open up dev tools and you see that all the flags are being brought in from a CDN. The only fix is to fork the library and change asset paths in their code to self host. Many libraries are like this, icon collections, design system web components, whatever. If you're not vigilant it's easy to lazily include CDN hosted assets without explicit intent.
This is wild. I don't think this is very common.
Part of it is just because it's easier. There's a bootstrap theme I've used that relies on a Google font. Rather than package the font, which would require extra work on the webdev's part to serve locally, they just stuck a CDN URL in the css.
But the webfont is not crucial to website UX as a blocking 3rd-party <script src="..."> without async/defer in <head> is.
> I would have thought the risks from 3rd party production dependencies would make devs do all in their power to minimize them.
Tell that to any developer (not just web) that outsources everything these days e.g. authentication, storage, analytics, database, etc - you name it.
They think it's less risk (and this includes managers and directors in larger companies). Apparently in-house would take forever, cost more and be unable to maintain anything these days.
I don't get it either but it's the current train of thought / trend and is not tied to individual developers.
Tell that to any developer (not just web) that outsources everything these days e.g. authentication, storage, analytics, database, etc - you name it.
They think it's less risk (and this includes managers and directors in larger companies). Apparently in-house would take forever, cost more and be unable to maintain anything these days.
I don't get it either but it's the current train of thought / trend and is not tied to individual developers.
> the problems are rare and this somehow allows faster time to market?
Yep. Entire teams work on these products. You have a couple of options:
1. Use them
2. Make it internally with less staffing, resulting in an inferior product
Option 2 is viable if you really only need a small subset of features/deep customization, but if you run into the edges the problem space will explode on you. At that point, you either have to throw more resources at it or go 3rd party.
Yep. Entire teams work on these products. You have a couple of options:
1. Use them
2. Make it internally with less staffing, resulting in an inferior product
Option 2 is viable if you really only need a small subset of features/deep customization, but if you run into the edges the problem space will explode on you. At that point, you either have to throw more resources at it or go 3rd party.
I'd imagine for many developers, the probability of something like Google's Hosted Libraries (https://developers.google.com/speed/libraries) going down or being hacked is far, far lower than the probability of their own site hosting having a problem.
Personally if I'm going to rely on someone to host it, I check if my hosting provider does it for me already. I believe Microsoft pre-hosts some JS packages. Google hosts fonts and such as well.
I also miss being able to host random CSS and assets on Tumblr.
Otherwise, it should be on my own CDN. Especially for enterprise sites.
I also miss being able to host random CSS and assets on Tumblr.
Otherwise, it should be on my own CDN. Especially for enterprise sites.
It is because the data (web logs containing web pages, timestamps, ip addresses, cookies, etc) can be sold.
It's also because the big reputable third party CDNs are more reliable hosts with better uptime and SLAs than many of the sites that use them.
Nonsense. Your site is obviously running if it served the page that wants the fonts, so there's absolutely no way they're more reliable than the site that uses it.
Meanwhile, that third-party is effectively paying your bandwidth bill: You might want to trade some reliability in exchange for a lower bandwidth bill, but you should wonder why exactly that third-party is willing to pay your bandwidth bill at all.
Meanwhile, that third-party is effectively paying your bandwidth bill: You might want to trade some reliability in exchange for a lower bandwidth bill, but you should wonder why exactly that third-party is willing to pay your bandwidth bill at all.
I invite everyone affected to consider switching to a production focused CDN https://www.jsdelivr.com/
There is also a tool to simplify migration https://www.jsdelivr.com/unpkg
There is also a tool to simplify migration https://www.jsdelivr.com/unpkg
I invite everyone (even those not affected in this instance) to consider switching away from relying on random third parties for your hosting. Like wtf how is did this ever become the norm even when there was a slight performance advantage from a shared cache, which doesn't even exist anymore these days.
A very specific problem with third party hosts : if you use noscript and only allow main domain script as a default, you have to either whitelist those CDN or authorize them on every site using it. Not the best experience. Especially with some sites having dozens of third party scripts so instead of trying to find the right one(s) to authorize you just close the tab and forget about it.
I do both. If you are loading a JS lib then check for its existence on the client. If it's not found load the resource from your server. You could also achieve this on the server side.
Or you could simplify the whole thing and just host it yourself. It isn’t worth that level of complication.
In this situation server bandwidth was an issue. Using an available CDN with the fall back was what we decided to do. It's not that complicated, from the client check if the object you expect to exist in the library exists, if not load it from your server.
I’m curious, in what scenario was bandwidth for static resources an issue? It’s one of the most trivial things to serve cheaply.
When the server is behind a pipe of limited size. When most of your new visits happen all at once over a short period of time. When you can't rely on a mobile phones cache like a desktop because 90% of your visitors are using one. A CDN is a easy win to off load resources, even if they are small.
> A CDN is an easy win to off load resources, even if they are small.
Yes and setting up your own CDN config (Cloudfront, whatever) is minutes of work and brings all of the benefits you’ve outlined. And costs pennies.
Yes and setting up your own CDN config (Cloudfront, whatever) is minutes of work and brings all of the benefits you’ve outlined. And costs pennies.
1. You realise caches, even for third party assets are partitioned by host domain in most modern browsers now? So you're not getting a benefit that users loaded the script from someone else's site.
2. For saving bandwidth on the server are you using proper cache setup with etags and If-Modified-Since support? It shouldn't have to download a first party asset again if it's in the cache. It's worth pointing out that non-ancient versions of all major web servers do this by default.
3. Haven't you now added latency by first making users download the loader script, have that execute, and then start loading the dependencies?
2. For saving bandwidth on the server are you using proper cache setup with etags and If-Modified-Since support? It shouldn't have to download a first party asset again if it's in the cache. It's worth pointing out that non-ancient versions of all major web servers do this by default.
3. Haven't you now added latency by first making users download the loader script, have that execute, and then start loading the dependencies?
1) Yes, the other benefit at the time was a browser would only do X number of requests per domain, using the CDN allowed more requests. Also it was a bandwidth issue from the server.
2) Yes. I should also mention at the time mobile browser cache was not consistent/reliable.
3) The "loader script" is a basic if(!lib.func) { do the magic }. Its one extra thing I suppose. For the overall page load there are lower hanging fruit.
2) Yes. I should also mention at the time mobile browser cache was not consistent/reliable.
3) The "loader script" is a basic if(!lib.func) { do the magic }. Its one extra thing I suppose. For the overall page load there are lower hanging fruit.
> The "loader script" is a basic if(!lib.func) { do the magic }. Its one extra thing I suppose. For the overall page load there are lower hanging fruit.
That's still a 100-200ms if the user is not on the same continent as your server.
That's still a 100-200ms if the user is not on the same continent as your server.
What does that actually mean? How do you check if a JS library exists on the client?
Why? If you care about being even remotely reliable (hinted at with "production-focused"), why would you do this? Just use the same host as where you serve everything else from.
Browsers have partitioned caches per origin for over 10 years, so there's no performance gains from getting cache hits from other websites.
Browsers have partitioned caches per origin for over 10 years, so there's no performance gains from getting cache hits from other websites.
its a fascinating little tech-anthropology quirk that front-end/web developers use the term "CDN" to mean a different thing from how devops/sre/infra people use the term.
in the strictest sense, unpkg is not a cdn. unpkg is a javascript library hosting service that uses cloudflare as a cdn. cloudflare is not down, therefore this headline "unpkg cdn down..." reads as a massive "oh shit!" to infra/devops type cloudflare customers who then have to immediately worry that their site is down, only to realize no, they're fine, this is just devs using the term to mean a different thing (roughly: "third party javascript host").
in the strictest sense, unpkg is not a cdn. unpkg is a javascript library hosting service that uses cloudflare as a cdn. cloudflare is not down, therefore this headline "unpkg cdn down..." reads as a massive "oh shit!" to infra/devops type cloudflare customers who then have to immediately worry that their site is down, only to realize no, they're fine, this is just devs using the term to mean a different thing (roughly: "third party javascript host").
> its a fascinating little tech-anthropology quirk that front-end/web developers use the term "CDN" to mean a different thing from how devops/sre/infra people use the term.
Do they? Same to me.
> in the strictest sense, unpkg is not a cdn. unpkg is a javascript library hosting service that uses cloudflare as a cdn.
This is just arguing over semantics. It even says on the website that unpkg CDN is powered by Cloudflare.
By your analogy if a website hosted on AWS is down, is the whole of AWS down? If I provide managed kubernetes on GCP and my service is down, does that make GKE or GCP down? No.
A CDN is a CDN just like a website is a website, whether it's backed by Wordpress, NextJs or any other thing.
Do they? Same to me.
> in the strictest sense, unpkg is not a cdn. unpkg is a javascript library hosting service that uses cloudflare as a cdn.
This is just arguing over semantics. It even says on the website that unpkg CDN is powered by Cloudflare.
By your analogy if a website hosted on AWS is down, is the whole of AWS down? If I provide managed kubernetes on GCP and my service is down, does that make GKE or GCP down? No.
A CDN is a CDN just like a website is a website, whether it's backed by Wordpress, NextJs or any other thing.
So what was the issue?
Unpkg runs on Fly with Cloudflare as CDN.
Unpkg runs on Fly with Cloudflare as CDN.
[deleted]
I assume the developers who use a free third party service for serving their dependencies are the same devs who :pikachu_face: when they are told they can’t statically link GPL libraries into their proprietary apps.
Personally I’d say the overlap between web developers linking out to a JS CDN and native developers statically linking GPL libraries is probably pretty small.
And you almost certainly don't need one for scaling purposes. I've handled frontpage traffic multiple times without.
What I wish existed (or maybe does) is something like a CDN with ~8 global locations that promises to keep your static site in a warm cache for a small fee, without me having to set up a fly.io app manually. That would let my blog always load fast anywhere on the planet even for the first hit in a region.