|
North American Network Operators Group Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical Re: Palo Alto Power Hit
Apparently my normal sources inside the carriers got stomped on by senior management, and aren't allowed to talk to me anymore. So here is a summary of unofficial reports from various unofficial sources. The West Coast is experience severe weather, including storms and flash flooding. There are multiple weather advisories up and down most of the coast. Approximately 55,000 people are without power. At the moment, no reported deaths due to the storms. Internet related incidents There were power problems through the early morning hours. A major power surge hit the Palo Alto grid at 7:16am PST. I haven't seen any SCADA information on just how large the surge was. This appears to be the instigating factor for several carrier outages. Several significant Internet trunks pass through the Palo Alto area. Customers of several carriers reported problems including MCI/Worldcom/UUNET/MFS, GTE/BBN, and Sprint. One person reported loss of residential phone service, which may indicate Pacific Bell was also affected. A lot of dead routes, and partially routed CIDR blocks seem to indicate those backbones may have been partitioned for some time. UUNET and MFS customers reported loss of a Palo Alto node. I don't know if this referred to their equipment in the PAIX building, or at another Palo Alto location. MFS had a power problem at PAIX a few years ago, but elected to use utility power instead of PAIX power. PAIX reported their power equipment absorbed the surge, and continues to operate normally. BBN at Stanford appeared to have some communication problems, no word if any rats were implicated. MAE-West-San Jose, Mae-West-Ames, and PacBell AADS appear unaffected. Sprint didn't appear to have any circuits down, but had enough problems with the few people they peer with externally, that Internet routing wasn't working well. A common report was the power surge either blew the fuses, or the entire power supply on a MFS OC192 piece of equipment. This may be the source for some reports of a fiber cut in the area. Most problems cleared up around 9:15PST after the fuse/equipment was repaired. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation
|