North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

RE: Max TNT ping thing

  • From: Ejay Hire
  • Date: Wed Aug 27 12:50:22 2003

Here is a summary of our experiences with the bug.

Last Thursday, A TNTs with years of uptime rebooted.  No cause was
apparent, and nothing relevant happened in the logs.  On Friday, It
happened to a different TNT.  This occurred with increasing frequency
over the weekend, and we didn't get a lot of sleep.  We tried using a
filter in the tnt to block port 135 and 4444 to no avail, and then tried
a filter to block ICMP in the tnt also to no avail.  Next, we removed
the tnt filters and tried rate-limiting ICMP to the TNT's.  That didn't
work.  Next we removed the rate-limit and applied the Cisco-supplied
anti-nachi route-map to the upstream interfaces facing the Tnt's.  This
significantly reduced the problem, but we were still rebooting every 12
hours or so.  Disabling route-caching on the TNT stopped the rebooting
problem, but we were seeing 40% packet loss on one of the TNTs.  (Note,
both TNT's have a Ds-3 of PRI's, and use the TNT-SL-E10-100 four port
Ethernet cards)  The packet loss was only affecting one TNT, and we
discovered that it was running 9.0.6 while the unaffected box was
running 9.0.9.  Upgrading the box to 9.0.9 fixed the packet loss issue.
We are currently up and haven't had any blips in 24 hours.  (knock on
wood.)

-Ejay

-----Original Message-----
From: Andy Walden [mailto:[email protected]] 
Sent: Wednesday, August 27, 2003 10:35 AM
To: Geo.
Cc: NANOG
Subject: Re: Max TNT ping thing


On Tue, 26 Aug 2003, Geo. wrote:

>
> Someone on this list had mentioned a network card for the Max TNT that
made
> it immune to the nachia worm ping issue.
>
> Is that the 4 port (3 ethernet, 1 fast ether) card or the single port
card
> with the dongle thing  or something else?

It turns out this was a bogus solution. Since the load was lower
afterwards, my tech thought it had been fixed. We tried limiting the
size
of the route cache as someone had recommended, as well as applying all
of
the filters without relief. This morning I had them just disable the
route
cache to see what happens. I will post the results. We did end up buying
a
support contract from Lucent after they said they had a fix and would
tell us what it was after we paid them. They just supplied the filter.
At
this point, they have exactly zero clue as to what to do next.


andy
--
PGP Key Available at http://www.tigerteam.net/andy/pgp