North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

ISP Y2K mailing list summary

  • From: Sean Donelan
  • Date: Sun Jan 02 05:41:19 2000

This was the final summary post for the NANOG ISP Y2K mailing list.
Special thanks to Merit for allowing the ad-hoc group of Internet
Service Providers to share the name for the NANOG ISP Y2K event.
This post is a couple of hours early, because I want to go to sleep.


                NANOG ISP Y2K Summary Status
                   2-JAN-2000 12:30 UTC
                       Y2K Summary

   Although the "hotlines" got a lot of press, much of the information
   came via e-mail.  It was difficult to dial into the international
   conference bridges near midnight in various countries due to
   congestion on the international voice circuits.

   The Internet is a dynamic place.  All numbers may vary depending
   on where you are on the Internet.  But the trends should be the
   same.

   The Internet global routing table showed a slow decline in
   networks from 71,200 on December 25, 1999 to 70,911 on
   December 30, 1999.  On December 31, 1999 the networks steadily
   declined to 69,812 by 1 hour before Midnight EST.  During
   this hour the routing table quickly dropped to 67,795 routes,
   and held steady from Midnight until 5am EST.  After 5am EST
   January 1, 2000 the number of networks started increasing
   steadily to 70,455 on January 2, 2000.

   The number of unique ASNs in the global route table declined
   from 6,380 on December 30, 1999 to 6,330 at January 1, 2000
   Midnight EST.  The number of terminating ASNs showed a similar
   decline from 5,180 to 5,130.  Which leads to the conclusion
   transit networks did not disconnect during the rollover.

   Peak traffic levels across MAE-East were lower on Friday December 31,
   about 1.6Gb versus 1.9Gb for a normal workday.  There was a slight
   but noticable 0.2Gb dip in traffic at Midnight EST.  Followed by
   a slight increase after Midnight EST.  All well below peak traffic
   capacity.  Traffic flow followed the normal sine wave pattern, except
   for the dip at Midnight EST.

Summary of ISPs
  2 ISPs reported a routing problem, isolated to normal circuit problems.

  1 ISP reported voice-call (POTS) congestion in New Zealand shortly
  after midnight through several voice carriers.  Internet backbone
  connectivity was not affected.  This was the pattern through most
  time zones.

  Several ISPs reported trouble synchronizing NTP servers.  Congestion
  on the NIST network is believed the cause.  NTP servers elsewhere
  did not show any problems.

  1 ISP reported messages sent in elm showed year as 100 instead
  of 2000.  Patch was available prior to Y2K.

  Several reports of user-written scripts containing poor date handling
  practices.

  Report of one web site defacement after close of business.

  Report of some unauthorized domain name transfers at the beginning of
  the holiday weekend.  Not unusual, seems to happen every holiday
  weekend.

  Two ccTLDs were down for part of the rollover.  13 ccTLDs had partial
  problems.  No Y2K faults were found.  Circuit problems, administrators
  turning off servers, and ordinary DNS problems.

  Exchange points worldwide reported a decrease in traffic immediately
  before their midnight local time.  A sharp increase immediately after
  midnight local time, but less than peak capacity.

  Certain "event" web sites saw traffic increase 20 times over normal
  levels.  Some event sites report being overwhelmed.  But overall,
  traffic appears to have been lower than peak business day capacity.

  The use of many country-specific information sites, and the rolling
  nature of the Y2K event helped distribute the traffic worldwide.  In
  general public interest waned about 30 minutes after local Midnight
  in each timezone, leaving more bandwidth for each successive timezone.

  Midnight US Eastern time did dominate changes in traffic and routing.

  A few rumors sprung up throughout the night.  Most were quickly determined
  to be unfounded.  But overall, not as many as I thought would spread.
 
Issues known/predicted prior to the Y2K rollover

  Some sites plan to shutdown or disconnect from the Internet
    over the New Year's weekend

  Some certificate authority certificates expire on
    December 31, 1999

  Increase in voice and cellular calls immediately around local
    midnight may cause some congestion on circuits

Information from the media (Note: ISP names are from publicly
   announced information.  Names will not be included from
   non-public sources.)

   C I Host (www.cihost.com) issued a press release about a name
   server problem.  No client data files were damaged. The data
   corruption that occurred Dec. 29 is isolated to the [company
   local] nameservers only and the restoration remains unrelated
   to any Y2K issues.

   Keynote Internet Performance Update #2 reports slower access
   to New Zealand after rollover, 6.2 seconds.  No information
   what the measurement was before the rollover.  Appears to be
   localized to specific web sites (i.e. congestion)

   France's National Weather Service Internet site had a display
   problem with a date.  The page shows 01/01/19100 instead of
   01/01/2000.

   United States Naval Observatory web time site had a display
   problem with a date.  The page shows 19100 instead of 2000.

   Hacker target Japan Y2K Center.  No intrusion or damage
   reported.

   Several stories based on Keynote press releases report the
   Internet passed through the Y2K rollover unaffected, other than
   a few spots of congestion around particular web sites.

   The www.y2k.gov web servers were predicted able to handle 40 million
   hits.  Only 3 million showed on January 1.

   Many, MANY, ISPs, web hosting, and access providers issued press
   releases throughout January 1, 2000 announcing they had no Y2K
   problems (and I think they all sent me a copy).

Summary from the Y2KCC/JP

  The number of reported troubles       25

    Troubles with Stratum1 NTP server    4
    Trouble with ICQ                     1
    Troubles with ccTLD                 15
           small partial troubles       13
           others                        2
    Trouble with router                  1
    Trouble with Y2KCC/JP system         1
    Latency of a particular site         1
    Suspect of cracking                  1
    Trouble with NNTP server             1