North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Voluntary outage report, who gets blamed

  • From: Sean Donelan
  • Date: Tue Nov 30 18:07:48 1999

Robert Cannon <[email protected]> writes:
>Focus Group III of NRIC is the Focus Group working on this issue. 
>Subcommittee 2 is looking at future considerations.  This is the
>subcommittee that came up with the recommendation to have a trial
>period of ISP outage reporting.

Of course, some large companies will continue their current
practice of keeping bad news quiet.  But my intent hasn't been
to point out how bad some companies are.

My concern is the wild, unregulated Internet gets blamed for a
lot of outages which really aren't "Internet" outages. In the
last two weeks I've only noticed three multi-hour multi-ISP
affecting incidents:  1) a misconfigured router in Dallas spewed
bogus routing information, 2) an ILEC's DACS lost its configuration
in New Jersey, and 3) a MAE had a loopback which caused "network
congestion" and routes to flap for a few hours.  There were other
problems, a couple of fiber cuts and power failures, but they didn't
seem to cause problems for multiple ISPs.

Interestingly enough problem 1 affected multi-homed networks more
than single homed networks, which happen also to house the most popular
web services.

With so much hype this month about web retailers and the christmas
shopping season, some very high profile sites have been hit by various
problems and covered by the news media.  However, just because a credit
card processor goes down, and many web sites were shutdown, does not
mean its an "Internet" problem.  Its a credit card processor problem.

As I poked around several of the problems over the last couple of weeks,
I found most web site outages related to "back-end services."  Problems
with their database software, disk and server farms, and the ever popular
"operator error."  Yes, some network problems happened.  But ISPs are getting
pretty good at the packet delivery stuff.  The ISP can usually deliver
more packets than the server can handle :-)

Perhaps surprising, but since some folks are concerned e-commerce is
where the money is, none of the problem e-commerce web sites with whom
I spoke or read about attributed any down time to criminal or "cracker"
activity. But it may be a self-selecting group. Maybe the reason some people
wouldn't answer my questions was because their outages did involve suspicious
activity.