StackOverflow Is Down(stackstatus.net)
stackstatus.net
StackOverflow Is Down
https://www.stackstatus.net/incidents/9cbac2e1-6bdc-41c9-b520-8875d3ce5050
14 comments
I hope they are financially well because it is one of the few online things I think it works.
it is only bad they are having some many outages these days. I am not complaining for me. but it means something is happening there
it is only bad they are having some many outages these days. I am not complaining for me. but it means something is happening there
> "Still independently owned. Imagine AWS or Microsoft buying it. Oof."
Owned by a VC firm, isn't it? https://www.forbes.com/sites/vijaygurbaxani/2021/06/08/the-1...
after https://vcnewsdaily.com/stack-overflow/venture-capital-fundi...
Owned by a VC firm, isn't it? https://www.forbes.com/sites/vijaygurbaxani/2021/06/08/the-1...
after https://vcnewsdaily.com/stack-overflow/venture-capital-fundi...
You're right. That makes me very sad.
> Doesn't require me to log in to read an answer
The cookie dialog being different on every site is infuriating if you’re not logged in.
The cookie dialog being different on every site is infuriating if you’re not logged in.
It could sound like a joke, but when I applied years ago to be a sysadmin for an Antartica base, they let me travel with whatever I want.
My first option was a full copy of stackoverflow. No joke.
Now, I'm reading this 5 hours letter and everything is ok, but what a wonderful site.
My first option was a full copy of stackoverflow. No joke.
Now, I'm reading this 5 hours letter and everything is ok, but what a wonderful site.
They're having stack overflow?
Cue jokes like "... and today was 32% less productive for developers worldwide"
For anyone looking on this from the future, the error page has Content-Type set to text/html but literally returns the text: "The service is unavailable." and a newline character.
* Trying 151.101.1.69:80...
* Connected to stackoverflow.com (151.101.1.69) port 80 (#0)
> GET / HTTP/1.1
> Host: stackoverflow.com
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 503 Service Unavailable
< content-type: text/html
< Content-Length: 27
< Accept-Ranges: bytes
< Date: Wed, 03 Aug 2022 14:58:39 GMT
< Via: 1.1 varnish
< Connection: keep-alive
< X-Served-By: cache-lga21957-LGA
< X-Cache: MISS
< X-Cache-Hits: 0
< X-Timer: S1659538719.045867,VS0,VE2
< Vary: Fastly-SSL
< X-DNS-Prefetch-Control: off
< Set-Cookie: prov=f6b7f757-544e-1afc-3cc2-e13af9fc8932; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
<
* Connection #0 to host stackoverflow.com left intact
The service is unavailable.⏎That's valid HTML.
A title-element is required for the HTML to be valid. It is the only required element AFAIK.
Victerius(2)
It had the same issue the other day, at least for a few minutes, so possibly an ongoing glitch?
They were down for several hours a week ago. It seems they've been having some sustained issues maybe? https://twitter.com/StackStatus/
So what is their stack, or where can I find more about it? A totally custom webserver? Someone commented to this effect the yesterday [1], but I couldn't find much in the way of a good writeup, except this answer from 2009 [2].
1 https://news.ycombinator.com/item?id=32321726 2 https://stackoverflow.com/questions/676326/how-does-stackove...
1 https://news.ycombinator.com/item?id=32321726 2 https://stackoverflow.com/questions/676326/how-does-stackove...
Scott Hanselman had an interview with the Stack Overflow Director of Engineering just a few weeks ago. They cover some high-level details there [0]. As I understand it's an .NET Core monolith with a very-large (~1.5 TB RAM) SQL Server backend.
[0] https://hanselminutes.com/847/engineering-stack-overflow-wit...
[0] https://hanselminutes.com/847/engineering-stack-overflow-wit...
One thing about using SQL databases live in production for everything is that the architecture is amazingly simple, almost always performant, but if something goes wrong, it goes catastrophically wrong.
Amazon for new critical applications prefers NoSQL solutions for this reason since performance is flatter and not dependent on vagaries of the query optimizer.
Amazon for new critical applications prefers NoSQL solutions for this reason since performance is flatter and not dependent on vagaries of the query optimizer.
From what I've gleaned through speaking to current and former Amazon employees you basically have to use DynamoDB for all new services.
IMHO the logical progression for applications with typical relational models is starting with an RDBMS in combination with an ORM, breaking out of the ORM when necessary for more complex/slower queries, ditching the ORM altogether when it starts to break, then evaluating sharding or moving domains into a NoSQL solution piecemeal. A well-architected monolith or series of microservices suits this paradigm which allows you to realize the productivity benefits of SQL and only worry about the deduplication, indexing, and consistency issues that come with NoSQL when it becomes actually necessary.
Then again if I were Amazon I'd probably plan for scale up front and be jaded by their experience with Oracle too.
IMHO the logical progression for applications with typical relational models is starting with an RDBMS in combination with an ORM, breaking out of the ORM when necessary for more complex/slower queries, ditching the ORM altogether when it starts to break, then evaluating sharding or moving domains into a NoSQL solution piecemeal. A well-architected monolith or series of microservices suits this paradigm which allows you to realize the productivity benefits of SQL and only worry about the deduplication, indexing, and consistency issues that come with NoSQL when it becomes actually necessary.
Then again if I were Amazon I'd probably plan for scale up front and be jaded by their experience with Oracle too.
Excellent, thank you. "Stack Overflow: How We Do App Caching - 2019 Edition" is particularly informative.
I also just discovered that “new” entry while posting this link and it was indeed quite a good read.
Not specifically stackoverflow, but here is an answer on stackexchange [1] listing many of the parts that make up their stack.
I'm assuming stackoverflow runs as a part of all this, and not separately.
1 https://meta.stackexchange.com/a/10370
I'm assuming stackoverflow runs as a part of all this, and not separately.
1 https://meta.stackexchange.com/a/10370
I remember hearing that they are a .NET stack with C# and SQL Server.
The only reason I know that is because I was working at a company and discussing the issues the company was running into scaling SQL Server and Stack Overflow was used as a counter example of SQL Server scalability. It lead into a long discussion of caching on high read workloads.
I'll never touch SQL Server again if I can help it.
The only reason I know that is because I was working at a company and discussing the issues the company was running into scaling SQL Server and Stack Overflow was used as a counter example of SQL Server scalability. It lead into a long discussion of caching on high read workloads.
I'll never touch SQL Server again if I can help it.
builtwith.com gives a lot of information, although the results page comes back with a "Misleading Technology Profile Warning".
And this is at least the 4th or 5th time within 10 days that either the website is down or performance is badly degraded.
So bad for a website that is basically the go-to resource where IT professionals get their answers.
So bad that the whole platform is closed-source, so we can't even spin multiple instances.
The main resource used by developers and sysadmins around the world is locked and centralized, and when that central place goes down all the answers go down with it.
We need crawlers and scrapers to download EVERYTHING out of the SE platforms, and they need to do so on a daily basis. All content on SE must be mirrored across the world, and if they don't want to do it then we'll do the scraping for them.
And, most of all, we need alternatives that are open and decentralized, and we need them right now.
So bad for a website that is basically the go-to resource where IT professionals get their answers.
So bad that the whole platform is closed-source, so we can't even spin multiple instances.
The main resource used by developers and sysadmins around the world is locked and centralized, and when that central place goes down all the answers go down with it.
We need crawlers and scrapers to download EVERYTHING out of the SE platforms, and they need to do so on a daily basis. All content on SE must be mirrored across the world, and if they don't want to do it then we'll do the scraping for them.
And, most of all, we need alternatives that are open and decentralized, and we need them right now.
Stackoverflow's data is cc-by-sa 4.0 licensed and can be found here: https://archive.org/details/stackexchange
So nothing is stopping anyone to spin-up multiple instances.
So nothing is stopping anyone to spin-up multiple instances.
StackExchange is by no means perfect, but the copyleft content license is a key reason I feel comfortable contributing to the platform. (Which I assume is why they do it.)
It's much harder to work with the data if you don't have the code. SO is not open source.
https://www.rickcarlino.com/2020/keeping-an-offline-copy-of-...
not so sure about that
not so sure about that
Here's a working link to Kiwix since the link on the page 404s: https://www.kiwix.org/en/download/
That's kind of cool, thanks. How long does it take to do a web scrape versus a DB dump?
IMO it would be more interesting to not use their code and build some kind of specialized search service. One that learns to rank results from user metrics and feedback?
so like personalised google search, with the terrible results that that inevitably produces for code related content?
i am confused though. how do you think SO currently works? Both questions and answers are already ranked via user metrics (individual upvotes and historical upvotes) and feedback (comments)?
i am confused though. how do you think SO currently works? Both questions and answers are already ranked via user metrics (individual upvotes and historical upvotes) and feedback (comments)?
> We need crawlers and scrapers to download EVERYTHING out of the SE platforms, and they need to do so on a daily basis. All content on SE must be mirrored across the world, and if they don't want to do it then we'll do the scraping for them.
I think that's already the case. Whenever I look up a technical issue, the first result is usually from SO; the next results are usually from websites that copied that first SO page
I think that's already the case. Whenever I look up a technical issue, the first result is usually from SO; the next results are usually from websites that copied that first SO page
You're lucky, for me recently its the other way around, spam copies first.
If you use brave browser it usually is able to point you to corresponding web archive pages when SO is down. Nice to have it directly integrated into the browser
I think stack exchange provides data dumps via archive.org
https://archive.org/download/stackexchange
https://archive.org/download/stackexchange
A read-only, self-hostable copy of StackOverflow (and its sister sites) is already doable with Kiwix + the content packages they publish.
Content packages: https://wiki.kiwix.org/wiki/Content_in_all_languages
Example of how a site looks like (in this case SuperUser), hosted on my server: https://kiwix.ounapuu.ee/superuser.com_en_all_2021-04/A/inde...
And hosting it really isn't all that difficult: https://ounapuu.ee/posts/2021/12/09/self-hosting-wikipedia/
Content packages: https://wiki.kiwix.org/wiki/Content_in_all_languages
Example of how a site looks like (in this case SuperUser), hosted on my server: https://kiwix.ounapuu.ee/superuser.com_en_all_2021-04/A/inde...
And hosting it really isn't all that difficult: https://ounapuu.ee/posts/2021/12/09/self-hosting-wikipedia/
> we'll do the scraping for them
Awesome! Let us know when we can start using your new open source service, it must be mirrored and need it right now.
Awesome! Let us know when we can start using your new open source service, it must be mirrored and need it right now.
FWIW, the Codidact project has been plugging away at building an OSS replacement for SE:
https://www.codidact.org/
https://www.codidact.org/
[deleted]
You mean I gotta start thinking on my own?
Not sure that will work.
Not sure that will work.
I am using the Stackoverflow Keyboard. How am I supposed to type code now?
I had a good laugh, thanks. Never knew that was a thing[1] but it totally made my day.
[1] https://stackoverflow.blog/2021/09/28/become-a-better-coder-...
[1] https://stackoverflow.blog/2021/09/28/become-a-better-coder-...
Time to take a long lunch. Hopefully it’ll be back soon, otherwise I’ll have to invent a reason to give to my boss on why I can’t get back to the office!
Finally we can't thanks the spam copies, they assure the vital service of caching the site when it goes down.
- FREE
- Permissive content license
- Doesn't require me to log in to read an answer
- Few / limited ads
- Data available at archive.org
- Still independently owned. Imagine AWS or Microsoft buying it. Oof.
I think about LinkedIn, Github, ... the other large platforms we use for work, and SE is the only one that hasn't gone downhill.
No, I don't want them to open source the platform. No, I don't think you can do better. Yes, a few mirrors for redundancy would be excellent, but not urgent. Discoverability here would be the challenge for mirrors. They've been down less than a day total for me in 10+ years of using them.