North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Global Crossing Contact / BGP and SONET interaction question

  • From: Forrest W Christian
  • Date: Wed Jul 26 00:56:13 2006

Two somewhat intertwined questions. I'll ask the second part first.

I buy transit from Global Crossing and another carrier on HDLC encapsulated DS3's.

Recently my BGP session has started flapping on the GX circuit... It looks something like this:

Jul 21 21:17:43.731 UTC: %BGP-3-NOTIFICATION: received from neighbor 6/6 (cease) 0 bytes
Jul 21 21:17:43.731 UTC: %BGP-5-ADJCHANGE: neighbor Down BGP Notification received
Jul 21 21:18:25.439 UTC: %BGP-5-ADJCHANGE: neighbor Up
Jul 21 21:29:52.315 UTC: %BGP-3-NOTIFICATION: received from neighbor 6/6 (cease) 0 bytes
Jul 21 21:29:52.315 UTC: %BGP-5-ADJCHANGE: neighbor Down BGP Notification received
Jul 21 21:30:38.511 UTC: %BGP-5-ADJCHANGE: neighbor Up
Jul 21 21:31:34.411 UTC: %BGP-3-NOTIFICATION: received from neighbor 6/6 (cease) 0 bytes
Jul 21 21:31:34.411 UTC: %BGP-5-ADJCHANGE: neighbor Down BGP Notification received
Jul 21 21:32:20.535 UTC: %BGP-5-ADJCHANGE: neighbor Up
Jul 21 21:32:52.547 UTC: %BGP-5-ADJCHANGE: neighbor Down Peer closed the session
Jul 21 21:33:32.703 UTC: %BGP-5-ADJCHANGE: neighbor Up

There are no other log entries during the periods when this occur. Unfortunately this causes enough prefix flaps that any prefixes which are preferred through GX are damped for like a half hour by certain providers as my BGP routes get added/withdrawn through the GX link.

GX claims (although I'm not sure they really know) that these are caused by SONET ring switches. I can believe this, since I haven't seen any real circuit flaps, and my understanding is that a SONET switch should generally be fast enough that you normally won't see the transition other than perhaps an error counter or two cranking up. However, it seems strange that I'm getting a 6/6 (cease) notification which I read as "configuration change" from their router. GX also seems to be at a loss to explain why my BGP is flapping - other than to point at the SONET switches.

I guess I'm trying to find out if someone on the list recognizes what this might be so I can perhaps help GX find and fix this. I'm also kinda curious as to whether or not typically a SONET ring switch event would actually propagate into a router in such a way that BGP would try to shut down the BGP sessions. I'm just having a hard time visualizing how a supposedly below-layer-two switch would cause bgp to reset in this manner. Not being a SONET expert even by any long stretch of the imagination leaves me with some holes here, but I thought the whole goal of SONET when used to provide DS3 circuits was to hide the ring switches as much as possible from the DS3 circuits - realizing that framing may be hard to preserve on a ring switch which would cause momentary loss of sync or similar - which usually shows up as an error instead of a interface flap.

And finally, does anyone have a contact within GX with a clue? So far I'm not sure I've talked to anyone who knows anything but how to spell BGP. I'd really like to talk to someone about the real cause of these flaps and try to resolve them so they don't reoccur.