North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Network Outage

  • From: blitz
  • Date: Thu Mar 21 20:41:30 2002

Well, I used to be a contractor for MCI/WCOM, so I know all the numbers to call and the names of people to ask for. They as a rule are very mysterious about anything outage related. The first rule to get better service, is always say OK to intrusive testing. I've had jacks go bad, and the bridge point isn't a wholesome indicator and the fault can be 2 inches away. Besides, I was hard down, all zeros, so it didn't matter.
After some arm twisting at Bell Canada, they confirmed it wasn't on their side of the ditch, so I started calling people I know here. The usual characters at the WCOM NOC were calling and leaving useless messages on my answering machine, but I called Sardinia, and got them to check outbound, towards Canada, and they said the T3 looked wholesome, but he couldn't access it at T1 level. I did however, find my Ticket appended to some others, so I knew it had to be at a bigger pipe than my T1. I know where they can check at T1 level, at 3DE, (the old MFS site, where I used to work) so I called, and a girl I worked with told me they had a DS3 cross-connect cable failure earlier, and it was repaired. I called over to the transport desk, and they confirmed the circuit was outbound twords Canada. I put two and two together and when I called the NOC back (they hadn't called my pager as instructed when the cut was cleared) they said the problem had been cleared and the circuit restored. Turns out, the NYC stuff was independent of the Canada problem, (which was my top priority) and the NYC stuff cleared on its own about 15 mins after I talked to Sardinia. Never got a positive cause on that one, but its gone, so its ok by me. Theyre stretched so thin on manpower now, and rumors are flying theres to be another round of layoffs next week, groan.

Anyway, all the circuits are back and wholesome, so Im happy...back up at 4:10pm

At 19:59 3/21/02 -0500, you wrote:
What did you end up getting from them.  I lost some ccts out of 60 Hudson
and they would not cough up to a problem, only hour later during a co-op
test did the problem "clear on its own".  Any details you can provide would
be great.

----- Original Message -----
From: "blitz" <[email protected]>
To: <[email protected]>
Sent: Thursday, March 21, 2002 3:00 PM
Subject: Network Outage

> Good day,
>   There has been a MCI/Worldcom outage near the Sardinia, NY switching
> I have been in contact with them since around 12:00 EST and they are
> presently trying to localize the problem.
> I've lost a couple international circuits to Canada, and some to NYC. I
> don't know if anyone else is experiencing similar problems.
> This is just a heads up for the list (and my first post..hello all)
> Marc Blitz