|
North American Network Operators Group Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical Building Evacuation [Re: BBN outage]
At 09:06 PM 12-10-96 -0700, maillists wrote: > >In hindsight, solutions to tolerate a given failure are >very easy to come by but shouldn't be talked down. I did not >blame BBN--shit happens-- One thing constructive I would note that I haven't seen yet is that while one can imagine many awful scenarios where your lovely NOC is absolutely leveled (*) by a horrendous accident, it is much more likely that your NOC will be forced to evacuate for some "trivial" reason. Perhaps asbestos. Perhaps a gas main leak in the street. Perhaps the fire alarms just keep going off. Perhaps the power went off and the fire marshall won't let anyone stay in the building without elevator power. It might very well have been the case that had BBN had a backup generator they might still have been forced to evacuate the building, if the fire dept was paying attention. You have to remember that it's harder to keep people in a given building than it is to safeguard the equipment there. You always have to be ready to send your NOC operators packing with a cell phone and a laptop for up to 48 hours, in case the building has to be evacuated for one of a dozen reasons. You should have a backup call center with an ACD set up in some other city or a contract with someone for a backup service. --Kent (*) My nightmare scenario back when I ran the NOC at BBN at 10 Moulton Street was a gasoline tanker turning over in the street out front. The gasoline runs down the stairwell into the sunken parking lot under the building. Someone exits the building into the parking lot and lights a cigarette ... Next moment my NOC and 300 people are ashes and there's a 60 foot crater where the building used to be. - - - - - - - - - - - - - - - - -
|