|
North American Network Operators Group Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical Re: Erroneous BGP advertisement
We had this same problem with SprintLink on Friday 2/6. As one of the techs at their NOC put it, they had a `glitch' in one of their cards at sl-bb3-chi at 6 PM. They reset the card, but it was broadcasting bad addresses for our network. At first I figured there was a fiber cut somewhere and that's why our traffic dropped because I could still get to almost all of Sprint's network. But after a few hours when the routes should have already reconverged and we still couldn't get outside Sprint's network, I called them to report the problem (1-800-877-5045). They told me that a tech would get right back to me. Within 15 minutes I had a tech calling me back from the NOC to work out the problem. After spending an hour on the phone proving to him that I couldn't get outside of Sprint's network, he reset the interface on their router that's connected to us. This seemed to clear up the problem so I went to bed, only to be woken up Saturday morning by frantic 1st level techs who said that there were still problems. I called Sprint again and this time it took 3 hours and several phone calls to get a tech to even call me back. Once I had them on the phone again, I spent another 1/2 hour proving that there was a problem with me getting outside of their network. I remembered what had fixed the problem last time so I reset my BGP session with Sprint. This did not fix the problem. They got off the phone and I didn't hear from a tech again until 3 hours later and several phone calls to my Sprint Reps and subsequent supervisors (even to a Vice President). Finally at 3 PM they had mysteriously fixed the problem, but I had to reset my other BGP sessions because somehow my other providers had picked up bad info from Sprint's announcements (I don't even know if that's possible) and still had residual problems for several days with certain sites because of our flapping. Needless to say Sprint was quite apologetic, but our customers were isolated from a majority of the Net for 21 hours, our longest network down time ever. I don't deal directly with our Sprint Rep, but I had heard talk Saturday afternoon about compensation. If you would like more info, contact me privately. -Dean On Tue, 10 Feb 1998, Bruce Robertson wrote: > Currently, one half of our /18 is being erroneously advertised by > Sprint (207.228.0.0/19). This, naturally, is causing us no end of > trouble. Sprint, meanwhile, blames a third party. This has been > going on for over 24 hours. > > First, if you're reading this and you're the culprit, please stop > advertising 207.228.0.0/19. > > Second, I'm soliciting advice from others who have experienced this. > How did you get results from the culprits? Can you recommend an > attorney who is familiar with the issues, in case we decide to recover > our lost revenue stream? > > Thanks! > > -- > Bruce Robertson, President/CEO > Great Basin Internet Services, Inc. > +1-702-348-7299 fax: +1-702-348-9412 > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dean Morstad Spacestar Communications Systems Administrator A Division of Firmware of MN, Inc. [email protected] 9531 W. 78th St http://www.spacestar.net Eden Prairie, MN 55344 http://www.morstad.org 612.996.0000 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|