North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Deploying IPv6 in a datacenter (Was: Awful quiet?)

  • From: Kevin Day
  • Date: Wed Dec 21 12:55:17 2005

On Dec 21, 2005, at 10:13 AM, Kevin Loch wrote:

Kevin Day wrote:

9) Once we started publishing AAAA records for a few sites, we started getting complaints from some users that they couldn't reach the sites.
It is possible that a broken 6to4 relay somewhere was causing problems.
Running your own local 6to4 relay (rfc3068) will improve performance and
reduce the chances of going through a broken one.

FWIW, I have been running some AAAA records for almost a year on some
revenue generating sites without any reachability complaints or
drop in traffic. I do run a local 6to4 relay though.
I will admit, the number of complaints was very small. But, I don't know how many it was affecting who didn't contact us, so it was decided it wasn't worth the risk with no perceived gain at this point.

I know this is still a hot topic and several proposals are being passed around to resolve some of these issues, but it seems like I *lose* functionality with IPv6 that I have with IPv4, mostly due to the "don't deaggregate your allocation" mantra, and how far the bar was raised to get PI space.
It sounds like you are an existing ISP in the ARIN reigon. If so then you qualify for a /32 allocation. Let us know if you have any problems getting one. For non-ISP's, the policy is still being worked out. See the ARIN ppml list for discussion. As for deaggregation, it is not
recommended because some filter (/48's) and some don't which results
in sub-optimal paths, but it can be done depending on what your
peers will accept.

Correct, we're in the ARIN region. We did qualify for a /32, but just barely, and only because of (hopeful) future growth and some new products we're launching. We do managed hosting for a small number of specialized clients, and act sort of as a content delivery network.

Not long ago we were a little smaller, and qualified for a /20 of our own in IPv4. /20 = 4096 addresses. We were able to pretty easily qualify for that between a few racks of servers, some IP based vhosting(necessary for a few applications), etc. In the IPv6 world, nothing we could do under that business model would qualify us for a direct allocation.

Back then we wouldn't have qualified for a /32 because we wouldn't have met the requirements. We wouldn't have met the proposed 2005-1 requirements for a /44 (we don't come close to 100,000 devices), and lose functionality if we're required to advertise it through a single aggregated address.

Just for the sake of argument though, let's say we didn't get a /32 and had to either get provider assigned space, or a /44 through the 2005-1 proposal.

1) We've got separate POPs in different cities, with no dedicated connectivity between them. They act as entirely independent networks. We don't even have the same transit providers in each city. Some content is replicated to each POP, so we can dump content directly on peers where they want it, or for redundancy. Some content is only available on one POP (dynamic sites, etc), so traffic destined for that POP can only go to that POP.

IPv4: We split our /20 into a /23 for each city, and announce only that.

IPv6: We have to announce the entire /44, and aren't allowed to deaggregate it. Where do we announce it? If I use transit provider Y only in city #1, and I announce to them our /44 even though I only really want a /48 worth coming there, I'm going to end up receiving traffic for city #2 at city #1. I can't even bounce it back off my router onto transit again, because I might see provider Y as the best path for it and it'll loop. Don't say we should ask for a /44 in each city, we don't need that much space in each location, and we're allocating space only to avoid deaggregating it, which is even more wasteful. :)

2) We use anycast to select the closest server on our network to a viewer, anycasted DNS for quicker lookups, and a few other anycast services.

IPv4: We have a /24 dedicated for anycast, and announce that block in each city.

IPv6: ??? I honestly have no idea how to do this with IPv6, and still announce only a single block, without effectively anycasting everything we do.

3) We're far past the size where we can renumber every time we change transit providers.

IPv4: We were able to qualify for PI allocations as small as a /22, because we were multihomed. a /20 if we weren't. This is within the range of feasibility of any small/medium hosting company to reach in a very short time.

IPv6: Even forgetting multihoming issues, we're beyond the size where we can do a renumbering without it becoming a serious ordeal. If we're forced to take provider assigned space, we're locked into that transit provider unless we want to leave them so badly that we're willing to go through the hassle. I can't even easily transition from one provider to another by having the new provider announce my PA space from my old provider temporarily, since my provider might be forced to announce everything as a single aggregate as well.


The allocation and advertisement policies work great for small end- sites (they get many subnets, and more addresses than they'll ever need), and great for large providers (they have no problem justifying the need for a /32, and are able to break things up if needed). If you fit somewhere between those two though, I don't think the policies really address our needs.

Small to medium sized hosting providers, content networks and other non-bandwidth-supplying companies generally aren't providing transit to others, so they can't get a /32. They ARE used to being able to deaggregate when necessary though, anycast as needed, multihome easily, and not go through renumbering hell every time you change providers. In the case of hosting companies, a renumbering is especially painful when you have to coordinate renumbering with thousands of customers who are going to see it as a hassle with no benefit.

I completely understand the need to reduce table growth, slow ASN allocations and not waste IPv6 space. But, (and I honestly don't mean this as an attack or flamebait, just a different perspective) it feels like a lot of the big providers are saying "We can't keep up with table growth, we need policies to stop this." Totally understandable. Policies got written to slow the table growth and ASN allocation, but for the most part they're painless for the big providers(easy allocations, and no loss of functionality compared to IPv4), and all the concessions are being made by those on the small- medium side, with very little benefit shown to them.

I don't mean to sound so resistant to change, but IPv6 is really a step backwards for people in our position if we're giving up so much just to get a bigger block of addresses.

-- Kevin

<waits for flame war to erupt> :)