North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Cisco IOS Failure due to Virus

  • From: Stephen J. Wilcox
  • Date: Thu Sep 11 06:06:30 2003

Hi,
 we've seen this.. yuo need to make sure you filter the nachi worm 92 byte icmp
echo's on your interfaces and it will be fine. The problem seems to be input
buffers which use all the memory up for some reason.

Steve

On Wed, 10 Sep 2003, Richard J.Sears wrote:

> 
> Hey Everyone - 
> 
> We have two 7507 routers configured with dual RSP4s w/256MB RAM,
> VIP2-50s with 128/8MB RAM, Gig, POSIP OC3 and Fast Ethernet
> interfaces.
> 
> These routers have run flawlessly for over two years now. But about
> two weeks ago, all of a sudden we started having serious crashing
> problems with these two routers. The routers will lose bgp
> connectivity (one at a time) to our upstreams (configured on each
> router). First, we would see a keepalive not sent message, then a bgp
> hold timer expire, then the bgp peering session would go down. OSPF
> would start crashing, then we would see the memory error messages,
> then all interfaces would blink off-line. (Note - we are running the
> max memory we can on both the RSPs and the VIPs).
> 
> Within 1 minute, the exact same thing would happen to the other
> router. Often times we would have to reboot the router to get it to
> come back online. We would see the following errors and have to reboot
> multiple times to get the router back:
> 
> 
> %SYS-2-MALLOCFAIL: Memory allocation of 704 bytes failed from
> 0x60329F00, alignment 0
> Pool: Processor  Free: 92744  Cause: Memory fragmentation
> Alternate Pool: None  Free: 0  Cause: No Alternate pool
> -Process= "Pool Manager", ipl= 0, pid= 6
> -Traceback= 6038049C 60382200 60329F08 6038DEDC
> 
> %TCP-6-NOBUFF: TTY0, no buffer available
> -Process= "BGP Router", ipl= 0, pid= 132
> 
> %% Low on memory; try again later
> 
> GigabitEthernet1/1/0: keepalive not sent
> 
> 
> 
> We are running the latest S train IOS patched for the IPV4 issue -
> however downgrading to the code we had run for the previous year did
> not solve the problem, nor did replacing the RSPs, VIPs and interfaces
> with new cards. In addition, we have complied with the Cisco
> recommendations for mitigating the effects of the Nachi Worm.
> 
> http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_tech_note09186a00801b143a.shtml
> 
> We also shut down one of the routers totally and the other router
> still experienced the same issue.
> 
> None of these updates or fixes have solved the problem.
> 
> I am thinking it may have something to do with all the virus stuff
> running around (same thing was crashing my Lucent TNT's), but I cannot
> seem to get an answer from Cisco, nor can
> I find anyone seeing the same issues.
> 
> Hopefully someone here can shed some light on this problem.
> 
> Thanks in Advance
> 
> 
> Richard
> 
> I fly because it releases my mind 
> from the tyranny of petty things . . 
> 
> 
>