North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Internic does it again

  • From: David Stoddard
  • Date: Mon Jul 28 12:57:30 1997

Eric Germann writes:
> Well our friends at Internic apparently forgot how to update their servers
> 
> *.root-servers.net from here is showing a serial of 1997072500 with no
> updates made Friday night showing  (11:30am EDT July 27).

	While is is easy to blame the InterNIC and NSI for all the evils
	in the world, the InterNIC of five years ago (or even one year ago)
	is not the InterNIC that exists today.  Last week I went to Network
	Solutions to meet with Dave Holtzman (SVP of Engineering) on an
	unrelated matter, and while I was there I got a chance to observe
	first hand the metamorphosis that has transpired over the past year.
	Here are just a few points that everyone should be aware of:

	*  NSI completely replaced senior level management about nine
	   months ago.  Most of those folks were responsible for the arrogant,
	   unresponsive attitude that NSI displayed for so many years.  NSI
	   replaced those folks with the most qualified people they could
	   find.  For example, Gabe Battista, NSI's CEO, was President of
	   Cables and Wireless before coming to NSI last year.  Dave Holtzman
	   was one of IBM's most-talented senior engineers, and also has a
	   substantial business background.  There are dozens of other
	   examples throughout the organization.

	*  Since the transistion in the management structure, the call
	   support area has gone from three people to 50 people, and the
	   engineering support desk has gone from two to over 30 (and they
	   are trying to hire more people right now).  Hours have been
	   extended from 5 PM EDT to 9 PM EDT, and plans are underway to
	   make InterNIC support a 24x7 operation very soon.

	*  The operations area is much like you would expect any large
	   industrial organization to have -- a controlled environment 
	   computer room with raised floors, fire and water detection alarms,
	   racks of equipment and systems, hot spare systems, several large
	   Cisco routers including a spare Cisco 7500, three T3 links to
	   separate backbone carriers, a UPS system with 200 lead batteries
	   capable of running the entire facility for more than 24 hours,
	   and a large deisel generator with enough fuel capacity to run
	   for two weeks.  The entire facility is secured with access cards,
	   limiting people to those areas that they work in.

	   [BTW, the entire InterNIC moved from its old location to its new
	   location one month ago without any outage -- this is a true test
	   of the InterNICs resources and expertise to be able to move an
	   operation that large without impacting the rest of us.]

	*  Effort is underway to define and document formal procedures for
	   every activity that goes on in the InterNIC.  Systems are being
	   implemented everywhere to make the InterNIC as fault tolerant as
	   possible.  Based on what I saw, problems like the partial DNS
	   update two weeks ago will be a thing of the past very shortly.
	   I had the opportunity to talk with Richard Walsh, the Director
	   of Operations, about the DNS problem and I am convinced it will
	   never happen again -- there are too many automated systems and
	   check procedures in place to allow a repeat of the previous
	   partial update problem.

	*  There is a complete focus on customer service in the InterNIC
	   now.  With a 1600+ percent increase in customer supprt personnel
	   in just the past few months (which continues to grow as they are
	   able to find and train people), the folks at NSI now talk about
	   quality of service, response time, escalation procedures, and other
	   customer support concepts that were anathma to the organization
	   just one year ago.

	   [Case in point -- before Dave Holtzman started posting on NANOG
	   a few weeks ago, I can't remember anytime when an SVP at the
	   InterNIC cared enough to post anything to the NANOG list.]

	Before I started US Net four years ago, I spent 16 years managing
	software engineering projects for KPMG Peat Marwick and American
	Management Systems (before that I taught CS at UofMD) -- I know
	what it takes to build, operate, and manage an industrial-strength
	operations center, and Network Solutions has one.

	I know what the response to this is -- "These folks are the InterNIC,
	they SHOULD be doing all these things!".  And I agree 100%.  The
	difference is they ARE doing these things now.  This is a direct
	result of the new management at the InterNIC.

	Also, a lot of folks have made derogatory comments about NSI going
	public.  Frankly, I think this is a good idea -- it will give them
	the financial depth to be able to support the InterNIC at a level
	that will benefit all of us.  Based on what I saw, and knowing the
	skills of Battista and Holtzman, NSI is sure to be one of the best
	stock deals of the '90s.

	[FWIW, I have no connection to the InterNIC what so ever -- like
	the rest of you, I rely on them for their domain registration services
	only.]

	So if your idea of the InterNIC is two engineers, two computers, and
	a half-dozen support personnel with an attitude, forget it -- it is
	not that way anymore.  Dave Holtzman wants to make the InterNIC a
	world-class support organization that puts reliability and customer
	support first.  From what I have seen, he is well on his way ...

	Dave Stoddard
	US Net Incorporated
	301-572-5926
	[email protected]