North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Software router state of the art

  • From: Justin Sharp
  • Date: Fri Jul 25 11:44:25 2008

Yes. We put in some Vyatta routers to extend our corporate network into another building as a temporary solution (the building had a very short lease, so our boss didn't want to spend any money on Juniper which is our usual net gear vendor). Consequently, we are still there.. go figure.

When we started w/ them, they were still using the XORP routing engine (and we haven't upgraded to the new platform yet). My experience wasn't terribly good. The first issue was a bad memory leak in the router manager process when VRRP hello times were set to 1 second. The first indication of something wrong is that our master router crashed, followed by his backup. Had to physically reboot the boxes to get them back online, which involved driving there as no one onsite had access to the cage at the office. All voice and data ran through these routers, basically rendering every employee useless until we got it back online. It wasn't a happy day. After that we had to monitor memory and do controlled reboots every month or so. We eventually convinced Vyatta of this memory leak and they were able to fix it, but that was a very frustrating process, and time consuming for us, which is why the next problems I describe, we have just found our own workarounds.

The next problem was a combination of a problem with the Vyatta and a problem w/ our IP phones. The Vyatta was sending garp's for the data vrrp address out the voice vlan (same 2 routers are default gate on both data and voice vlans). All of the workstations run through the phones (which sit tagged on voice vlan, and pass traffic from workstation untagged to data vlan). The phone, seeing the arp for the data vrrp address on its voice vlan, would send traffic to that address out the voice vlan, effectively taking that workstation off the net for anything other than local traffic. That was a bugger to figure out, and basically we solved it with an arptables rule on the vyatta's. That was the one advantage of using a Linux (debian) based router platform, was that we could load other 'unsupported' packages to solve problems like this.

The last thing is that OSPF never really converges correctly. You can view the OSPF database, and see which default the routers should converge to, but they do not. They will sit converged to one path for a while, and if you reboot the other router that generates default, they will reconverge to it for a while. This hasn't been a big enough problem for me to worry about it.

Last thing to say is, I haven't tried upgrading since Vyatta abandoned the XORP platform and moved to the Quagga platform, but I'm guessing (based on experience w/ Quagga) that they have a lot fewer of these quirks that I've described.

IMHO, YMMV, etc

--Justin

Tim Sanderson wrote:
Is anyone using Vyatta for routing? I sure would like to know about any experience with it in production.

http://www.vyatta.com/

--
Tim Sanderson, network administrator
[email protected]


-----Original Message----- From: randal k [mailto:[email protected]] Sent: Wednesday, July 23, 2008 1:46 PM To: Adrian Chadd Cc: [email protected] Subject: Re: Software router state of the art

That is a very interesting paper. Seriously, 7mpps with an
off-the-shelf Dell 2950? Even if it were -half- that throughput, for a
pure ethernet forwarding solution that is incredible. Shoot, buy a
handful of them as hot spares and still save a bundle.

Highly recommended reading, even if (like me) you're anti-commodity routing.

Cheers,
Randal

On Wed, Jul 23, 2008 at 10:17 AM, Adrian Chadd <[email protected]> wrote:
On Wed, Jul 23, 2008, Charles Wyble wrote:

This might be of interest:

http://nrg.cs.ucl.ac.uk/mjh/tmp/vrouter-perf.pdf
Various FreeBSD related guys are working on parallelising the forwarding
layer enough to use the multiple tx/rx queues in some chipsets such as the
Intel gig/10ge stuff.

1 mil pps has been broken that way, but it uses lots of cores to get there.
(8, I think?)

Linux apparently is/has headed down this path.