General Discussion
Related: Editorials & Other Articles, Issue Forums, Alliance Forums, Region ForumsWant some Facebook schadenfreude?
The gist, as I understand it, is they did this to their own selves. It's in essence a configuration change which broke the entire routing system. Which, apparently due to poor planing on their part, cannot be remotely managed and has to be done in person.
Employees said they had trouble making calls from work-issued cellphones and receiving emails from people outside the company. Facebooks internal communications platform, Workplace, was also taken out, leaving many unable to do their jobs. Some turned to other platforms to communicate, including LinkedIn and Zoom as well as Discord chat rooms.
Some Facebook employees who had returned to working in the office were also unable to enter buildings and conference rooms because their digital badges stopped working. Security engineers said they were hampered from assessing the outage because they could not get to server areas.
Facebooks global security operations center determined the outage was a HIGH risk to the People, MODERATE risk to Assets and a HIGH risk to the Reputation of Facebook, the company memo said.
A small team of employees was soon dispatched to Facebooks Santa Clara, Calif., data center to try a manual reset of the companys servers, according to an internal memo.
https://www.nytimes.com/2021/10/04/technology/facebook-down.html
And a moment of silence is in order for ramenporn... they had to delete their 6 year 30k karma Reddit account for talking about this publicly. 😢
Link to tweet
Archive of his last moments here: https://archive.ph/sMgCi
Rorey
(8,445 posts)Absolutely absurd.
Comfortably_Numb
(3,777 posts)Thats FB top, bottom, and middle line deal.
rurallib
(62,344 posts)The part about being locked out of the buildings is hilarious!
Rorey
(8,445 posts)And that they resorted to using Zoom and LinkedIn. Truly embarrassing for them.
herding cats
(19,549 posts)And how they're having to use Twitter to try and reach out to their disgruntled users. 😂
Gidney N Cloyd
(19,781 posts)uponit7771
(90,225 posts)... denied that there was anything wrong for nearly 2 weeks while the other huge IT component team appeared on site with chief engineers.
Ocelot II
(115,276 posts)JohnQFunk
(408 posts)FakeNoose
(32,349 posts)He had to fire all the coders who were the only ones that knew how to fix this. Am I right?
lagomorph777
(30,613 posts)I hope this helps wean a few people away from the Fascist teat.