Linode down in Newark data center(status.linode.com)
status.linode.com
Linode down in Newark data center
http://status.linode.com/
17 comments
"We have detected a large incoming DDoS attack to our Newark facility. We already have our administrators working to mitigate the attack. We will have service restored as soon as possible."
Interesting their reply says that, while their status page says it is due to a power outage.
Edit: Nevermind, j15e clarified.
Edit: Nevermind, j15e clarified.
Where is that quote from?
Same here, got this reply from support :
"Linode 1 hour ago
Hello there,
We have detected a large incoming DDoS attack to our Newark facility. We already have our administrators working to mitigate the attack. We will have service restored as soon as possible. Please refer to the following page for status updates:
http://status.linode.com
If you have any other questions please let me know.
Thank you.
Nick"
"Linode 1 hour ago
Hello there,
We have detected a large incoming DDoS attack to our Newark facility. We already have our administrators working to mitigate the attack. We will have service restored as soon as possible. Please refer to the following page for status updates:
http://status.linode.com
If you have any other questions please let me know.
Thank you.
Nick"
A support ticket I had opened before there was anything on their status page. That was the response at 11:53 PM Newark time.
New response from support :
"Based on the information we had at the time we believed it to be a DDoS attack. We are now aware that it was a power problem affecting our Newark data center. The backup system did not work and we are still looking for additional information from the data center staff.
We will be updating our status page with any further information.
Thank you."
"Based on the information we had at the time we believed it to be a DDoS attack. We are now aware that it was a power problem affecting our Newark data center. The backup system did not work and we are still looking for additional information from the data center staff.
We will be updating our status page with any further information.
Thank you."
Seems like it was caused by a power outage.
1:09AM EDT (UTC -4): The Newark datacenter has suffered from a power outage. We are working on bringing Linodes back up as soon as possible.As others have mentioned, we've seen reports of network and power outage, and now I've received a notice of hardware failure.
Just when I set up HA load-balancing, the whole data center outage takes it all down for 4 hours and counting.
Just when I set up HA load-balancing, the whole data center outage takes it all down for 4 hours and counting.
Hello,
The host that your Linode resides on experienced hardware
failure. As a result, drives are being transplanted to
separate, known-working standby hardware which we keep
available in case of events such as this.
Again, we're very sorry for any inconvenience caused by
this issue. Please feel free to update this ticket in the
event that you have any additional questions or concerns.They previously had a message that Linodes would be returned to their previous state. Mine was rebooted. Just FYI. Check your DBs and all that usual stuff.
Mine was rebooted 2hrs ago but still cant access it.
That happened to mine, too, though not for as long.
I bet they had some host filesystem corruption and they're dealing with it manually.
I bet they had some host filesystem corruption and they're dealing with it manually.
Thanks for the encouraging words ;) Still not up, 5+hrs outage so far...
EDIT: it was up, but the network interface was messed up for some reason. Rebooted again and its ok now.
Nodes are now being reported as stopped, but can't be started yet.
"An issue affecting the physical hardware this Linode resides on has been detected.
We are working to resolve the issue as quickly as possible and will update you as soon as we have more information.
Your Linode will return to its previous state once the issue is resolved.
Thank you for your patience and understanding."
"An issue affecting the physical hardware this Linode resides on has been detected.
We are working to resolve the issue as quickly as possible and will update you as soon as we have more information.
Your Linode will return to its previous state once the issue is resolved.
Thank you for your patience and understanding."
We have tens of servers across 4 Linode data centers. We get hit with both data-center-wide and single-server downtime pretty much on a weekly basis.
In the span of 3 months we had something like 25% of our servers go under "emergency maintenance" downtime. Just part of the ebb and flow of "the cloud", I guess.
In the span of 3 months we had something like 25% of our servers go under "emergency maintenance" downtime. Just part of the ebb and flow of "the cloud", I guess.
Wow, that sounds pretty unlucky. Do certain data centers/servers have more downtime for you than others?
update: "The Newark datacenter has suffered from a power outage. "
so, no DDoS after all?
so, no DDoS after all?
[deleted]
Is it just me or does it seem like there have been tons of DDoS attacks the past couple months.
DDoS attacks have been on the rise, I work at a hosting provider and last year we only saw somewhere around a thousand attacks, now we're seeing that monthly...
It sure feels that way, but I don't know if they are just being better reported now. Services to mitigate DDOS have been around for ages, so I imagine the attacks have as well.
My server auto-rebooted and is back up, and has been up for 30 minutes as of writing (approx 1:00am EDT / UTC -4). Strange that mine is up and fine now, but they haven't updated the status page yet. Maybe mine is an isolated case?
I feel like this should be included in every press release post down-time.
"As issues unfold, information changes and speculations can lead to improper actions based on incorrect information."
So very well said.
"As issues unfold, information changes and speculations can lead to improper actions based on incorrect information."
So very well said.
Hanging out on #linode, it appears as if some people's servers are coming back up. This is the second day this week I've woken up to unpleasant computer news :-(
... and mine are back up.
... and mine are back up.
Complete blackout for my nodes, can't reach them at all. Even Linode dashboard stats for my nodes are empty starting 11:51.
Almost 30 minutes now, I hope it get fix soon!
Almost 30 minutes now, I hope it get fix soon!
I have 12 websites there. zomg. Looks like they are alive now, after 2hrs outage.
Did someone cut the cable for scrap metal?
I think this is the first time I've ever noticed my own website being down.
[deleted]
Some of mine back up, others not. Not how I wanted to end my Saturday night.
Evidently DC issues are fixed, since some of my servers are up again. Others still down due to (related?) hardware issue. Linode support proactively opened tickets about those that are still down, and upon friendly questioning, expressed a high level of confidence that they'll be back up once drives are migrated to stand-by hardware.