North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: shim6 @ NANOG (forwarded note from John Payne)

  • From: Kevin Day
  • Date: Wed Mar 01 02:54:27 2006

On Mar 1, 2006, at 12:47 AM, Joe Abley wrote:

  o a small to medium multi-homed tier-n isp
A small-to-medium, multi-homed, tier-n ISP can get PI space from their RIR, and don't need to worry about shim6 at all. Ditto larger ISPs, up to and including the largest.

If you include "Web hosting company" in your definition of ISP, that's not true. Unless you're providing connectivity to 200 or more networks, you can't get a /32. If all of your use is internal(fully managed hosting) or aren't selling leased lines or anything, you are not considered an LIR by the current IPv6 policies.

Even the proposed ARIN 2006-4 assignment policy for "end sites" doesn't help a lot of small to mid sized hosting companies. For that, to just get a /48, you need to already have a /19 or larger, and be using 80% of that. That's 6553 IPs being utilized. If you're running a managed hosting company (name based vhosts) and deploying 1 IP per web server, you're pretty huge before you've hit 6553 devices. Even assuming 20% of that is wasted, you're still talking about more than 5000 servers. 40 1U servers per rack, you need to have 125 racks of packed to the gills servers before you'd qualify for PI space. That excludes every definition I have of "small-to-medium" in the hosting arena.

You don't get PI space, and Shim6 is looking like your only alternative for multihoming.

Content providers have a different set of problems, since a server with N simultaneously-active clients, each with an average of M available locators needs to deal with N*M worth of state, which is presumably M times worse than the situation today.

For very large content providers, aggregating very large numbers of simultaneous clients through load balancers or other middleboxes, this is quite possibly not something that is going to be a simple matter of upgrading to a shim6-capable firmware release.

Yes, and content providers have other issues as well when it comes to IPv6 policy... I'm betting only the top 1 or 2 CDN/content providers out there qualify for a /32. Many content providers set up multiple non-interconnected POPs in different geographical locations. The only way this can be accomplished is by making separate announcements in each POP for each space. This means either being able to deaggregate, or to get a block for each POP. I don't know of *ANY* that are deploying 5000+ servers per POP.



Actually, I think the problem with shim6 is that there are far too few operators involved in designing it. This has evidently led to a widespread perception of an ivory tower with a moat around it.
I think the issue was... When I first heard of shim6, I thought "Oooh, that's really clever. A lot of small businesses/enterprises will use that, they don't need to deal with BGP, adding a new provider is just a drop in." Then when we got to deploying IPv6 the discovery of "Oh, wait, they expect EVERYONE who uses PA space to do this? That's not cool." was a negative reaction.

To gain real relevance it needs to be deployed; to be deployed, it needs to be embraced by enterprise operators and content providers.

If these operators dismiss it out of hand on principal, and refuse to actually find out whether the general approach is able to solve problems or not, then irrelevance does indeed seem inevitable. However, the only alternative on the table is a v6 swamp.

How about some actual technical complaints about shim6?
I'm just one guy, one ASN, and one content/hosting network. But I can tell you that to switch to using shim6 instead of BGP speaking would be a complete overhaul of how we do things.

Putting routing decisions in the control of servers we don't operate scares me. I wouldn't rely on 90% of our customers to get this right unless it was completely idiot proof. Even if it was, I don't see how we can trust that users aren't messing with things to "game the system" somehow.

We deal with long lived TCP sessions (hours/days). I don't see how routing updates can happen that won't result in a disconnect/ reconnect, which isn't acceptable. With current BGP technologies, if I need to move traffic off a transit port, I can do so without relying on all of our servers to know anything about it, the move is instant, and non-disruptive. Shim6 requires a keepalive to expire for the end nodes to realize something is broken, then re-negotiate the remaining routing decisions. With BGP, I can see if one of my transit links goes down directly, and compensate before users start getting impatient.

We have peering arrangements with about 120 ASNs. How do we mix BGP IPv6 peering and Shim6 for transit?

So far it looks like Shim6 is going to rely on DNS. The DNS caching issue is a real problem. We need changes to happen faster than DNS caching will allow.

Our network is complicated. We have a /21 that's split into 4 /23s. One for each non-interconnected POP. We only advertise the /23 for each POP out to transit, but we give peers access to our entire network wherever they peer with us and we pay to haul/tunnel it around. How do we even do this without PI space, let alone through shim6?

For quite the foreseeable future, we'd be running IPv4 and IPv6 at the same time, over the same transit connections. We'd have to TE our IPv6 bits completely differently than our IPv4 bits, even though we'd be billed for the aggregate usage of both. Automated tools for tweaking total usage per transit port is hard enough in BGP. Having to tweak both BGP and some external shim6 method of TE when the goal is a common aggregate number is going to be a very difficult issue.

Some of our applications are extremely sensitive to jitter/latency. We've spent ages tweaking route-maps manually (and through automated continual tweaking) to make sure we avoid any congested links. We also rely on BGP communities by our providers to give us some more information when it comes to route decisions. (If NSP A tells me through communities that they peer directly with someone, where NSP B is crossing the country, then hitting another NSP before the Origin ASN, we prefer NSP A). I don't see how information like this, or tweaking to that level is even possible with Shim6. BGP works well for applications like this because each network the traffic passes through can add its own hints (Communities, prepending, etc) to the route, that lots of us use.

We'd still be relying on PA space. No matter how great dhcp6 is, there will be significant renumbering pain when providers are changed. Static ACLs, firewall rules, etc. If you're including customer machines in the renumbering, many simply won't do it.

Putting the logic behind traffic engineering and routing decisions into thousands of boxes seems a step backwards from putting the decision on our border/edges. Many more places where things can break. If we want to do things in a non-standard way, every box has to support it. If there are refinements to Shim6 later, we're forced with either not using them, or forcing our customers to upgrade their OS.

How do we deal with "backup connections"? I.e. connections that are only used if all others are down. Right now we advertise only a supernet out to our "backup transit" provider, and the more specifics to our main providers. (Yes, I realize this isn't perfect, but it works fine for us.)



Please don't get me wrong, I think Shim6 is great for a lot of people. Being able to let ANYONE multihome with no impact on the world is great. BUT, there needs to be a fallback to the BGP/IPv4-ish way for people who need the "power user" set of tools, or there is going to be a huge pushback from a lot of groups when asked to switch to ipv6. This fallback has to be available to anyone who can justify the need, not just "anyone bigger than X size".


-- Kevin