|
North American Network Operators Group Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical Re: ATM (was Re: too many routes)
Cool, I love being talked down to by old guys. It's refreshing and doesn't happen nearly frequently enough. I'm almost at a loss to figure out where to begin with the scattershot flamefest you sent. Almost. Let's start here: > Lets see you allocate an ESF B8ZS Clear Channel T1 over > IP.... PDH is dead. POTS is only alive because you can emulate PDH still, and extracting a single DS0 from SDH is easy, and because the POTS user interface is well understood by a very large installed user base. I don't expect POTS as perceived by the end-user to change much over time. End-to-end POTS is already dying. Worldcom is making a big deal over relatively simple technology which shuffles fax traffic over the Internet. There goes alot of long-haul POTS right there. Deutsche Telekom is tight with Vocaltec and already has a tariff for voice-over-the-Internet. It's crusty in implementation because you end up dialling a local access number in DT land and talk to another dialler on the remote end which makes a more local phone call. However, there are neat plans for SS7 and neater plans for doing clever things with interpreting DTMF. There is a local distribution plant problem however there are a number of people working on aggregating up local access lines into VC11/VC12, dropping that into a POP-in-a-box at STM-16 and pulling out STM-16c to a big crunchy IP router. In this model POTS and historical telco voice and data schemes become services rather than infrastructure. However, emulating the incredibly ugly phone network is secondary to enabling the evolution of more and more complicated and interesting applications; it's also less cost-effective for the moment than running parallel networks but converting away from PDH (which, remember, is dead). BTW, your formatting sucks to the point that your note is unreadable and unquotable without fmt(1) or fill-paragraph. > Ahhh, tag switching, I am on that particular holy grail >as well..... You might want to examine my comments on the mpls list at some point. "Holy Grail"? No. That was Noel. I know we look alike and stuff, but he sees a great deal of promise in MPLS while I am somewhat sceptical about the implementation and utility in practice. >How many parallel paths have you ran on >layer 3 ? Ever watched the variability ? ( * shiver * ) >Now, tell me parallel paths on IP are smooth with todays >technology! Hum, not more than six hours ago I believe I was telling Alan Hannan about the various interim survival techniques in the migration path from 7k+SSP -> decent routers. I guess you must have me beat experientially. All I ever did was sit down with pst and tli and hack and slash at the 10.2-viktor caching scheme to try to get traffic to avoid moving over to the stabler of the two lines between ICM-DC and RENATER. Oh that and helping beat on CEF/DFIB packet-by-packet load balancing before my last retirement. So unfortunately I'm really not in a position to comment on today's technology or the variability of parallel paths with Cisco routers using any of the forwarding schemes from fast to cbus to route-cache to flow to fib. (To be honest I never really figured out wtf optimum was :) ). The reason that you see "strange" or at least "unsmooth" load balancing along parallel paths is that except with fib and slow switching cisco had always forwarded packets towards the same destination out the same interface, and load balancing was performed by assigning (upon a cache fault) particular destinations to particular interfaces. (Leon was invented to blow away cached entries so that over time prefixes would slosh about from one interface to another as they were re-demand-filled into the cache. Points if you know who Leon and Viktor are. Hint: they're both as dead as PDH.) With CEF these days you can load-balance on a per-packet basis. This has the side effect that you cannot guarantee that packets will remain in sequence if the one way delay across the load-balanced paths is off by more than about half a packet transmission time. However, you also get much more even link utilization and no ugly cache/uncache/recache at frequent intervals (which really sucks because unfortunately you have to push a packet through the slow path at every recache). So anyway, as I was saying, I'm ignorant about such matters. >Audio sounds great with lots of variability So if you aren't holding a full-duplex human-to-human conversation you introduce delay on the receiver side proportional to something like the 95th percentile and throw away outliers. If you're holding a full-duplex long-distance human-to-human conversation you can use POTS (which is dying but which will live on in emulation) and pay lots of money or you can use one of a number of rather clever VON member packages and pay alot less money but put up with little nagging problems. For local or toll-free stuff, to expect better price-performance from an end-user perspective now would require taking enormous doses of reality-altering drugs. > > You want bounded delay on some traffic profiles that > > approach having hard real time requirements. (Anything > > that has actual hard real time requirements has no > > business being on a statistically multiplexed network, no > > matter what the multiplexing fabric is). > > Such as voice? Why do you think SDM was created in the first place? > Or do you mean like a military application, 2ms to respond to a nuke.... > That is when channel priorities come into play. I wasn't around for the invention of statistical muxing, but I'm sure there are some people here who could clarify with first-hand knowledge (and I'll take email from you, thanks :) ). If it was created for doing voice, I'm going to be surprised, because none of the voice literature I've ever looked was anything but circuit-modeled with TD muxing of DS0s because that is how God would design a phone network. Um, ok, why is it my day for running into arguments about real time. Hmm... "Real time" events are those which must be responded to by a deadline, otherwise the value of the response decays. In most real time applications, the decay curve varies substantially with the value of a response dropping to zero after some amount of time. This is "soft real time". "Hard real time" is used when the decay curve is vertical, that is, if the deadline is passed the response to the event is worthless or worse. There are very few hard real time things out there. Anything that is truly in need of hard real time response should not be done on a statmuxed network or on a wide PDH network (especially not since PDH is dead in large part because the propagation delay is inconsistent and unpredictable thanks to bitstuffing) unless variance in propagation delay is less than the window for servicing the hard real time event. Soft real time things can be implemented across a wide variety of unpredictable media depending on the window available to service the real time events and the slope of the utility decay function. For instance, interactive voice and video have a number of milliseconds leeway before a human audience will notice lag. Inducing a delay to avoid missing the end of the optimal window for receiving in-sequence frames or blobs of compressed voice data is wise engineering, particularly if the induced delay is adjusted to avoid it itself leading to loss of data utility. > However, I have NEVER failed to get the bandwith > "promised" in our nets. Sure, the problem is with mixing TCP and other window-based congestion control schemes which rely on implicit feedback with a rate-based congestion control scheme, particularly when it relies on explicit feedback. The problem is exacerbated when the former overlaps the latter, such that only a part of the path between transmitter and receiver is congestion controlled by the same rate-based explicit feedback mechanism. What happens is that in the presence of transient congestion unless timing is very tightly synchronized (Van Jacobson has some really entertaining rants about this) the "outer loop" will react by either hovering around the equivalent of the CIR or by filling the pipe until the rate based mechanism induces queue drops. In easily observable pathological cases there is a stair step or vacillation effect resembling an old TCP sawtooth pattern rather than the much nicer patterns you get from a modern TCP with FT/FR/1321 stamps/SACK. In other words your goodput suffers dramatically. > But, doesn't that same thing happen when you over-run the receiving > router ????? Yes, and with OFRV's older equipment the lack of decent buffering (where decent output buffering is, per port, roughly the bandwidth x delay product across the network) was obvious as bandwidth * delay products increased. With this now fixed in modern equipment and WRED available, the implicit feedback is not so much dropped packets as delayed ACKs, which leads to a much nicer subtractive slow-down by the transmitter, rather than a multiplicative backing off. So, in other words, in a device properly designed to handle large TCP flows, you need quite a bit of buffering and benefit enormously from induced early drops. As a consequence, when the path between transmitter and receiver uses proper, modern routers, buffer overruns should never happen in the face of transient congestion. Unfortunately this is easily seen with many popular rate-based congestion-control schemes as they react to transient congestion. Finally another ABR demon is in the decay of the rate at which a VS is allowed to send traffic, which in the face of bursty traffic (as one tends to see with most TCP-based protocols) throttles goodput rather dramatically. Having to wait an RTT before an RM cell returns tends to produce unfortunate effects, and the patch around this is to try to adjust the scr contract to some decent but low value and assure that there is enough buffering to allow a VS's burst to wait to be serviced and hope that this doesn't worsen the bursty pattern by bunching up alot of data until an RM returns allowing the queue to drain suddenly. > Ahhh.. We await the completion, and proper interaction of RM, ILMI, > and OAM. > These will, (and in some cases already DO), provide that information > back to the router/tag switch. > Now do they use it well ????? > That is a different story.... The problem is that you need the source to slow transmission down, and the only mechanism to do that is to delay ACKs or induce packet drops. Even translating FECN/BECN into source quench or a drop close to the source is unhelpful since the data in flight will already lead to feedback which will slow down the source. The congestion control schemes are essentially fundamentally incompatible. > > Delay across any fabric of any decent size is largely > > determined by the speed of light. > > Where in the world does this come from in the industry. > Maybe I am wrong, but Guys, do the math. The typical run across the > North American Continent > is timed at about 70ms. This is NOT being limited by the speed of > light. That would be round-trip time. > Light can travel around the world 8 times in 1 second. This means it > can travel > once around the world (full trip) in ~ 120 ms. Milliseconds, not > micro.... > So, why does one trip across North america take 70ms... Light is slower in glass. > Hint, it is not the speed of light. Time is incurred encoding, decoding, > and routing. Kindly redo your calculation with a decent speed of light value. Unfortunately there is no vacuum between something in NYC and something in the SF Bay area. > BTW this (70ms median across the US) comes from a > predominantly ATM network. Actually, I am quoting > Pac-Bell. Oh now THERE's a reliable source. "Hi my name is Frank and ATM will Just Work. Hi my name is Warren and ATM is fantastic.". Ugh. (kent bait kent bait kent bait) > > Therefore, unless ABR > > is deliberately inducing queueing delays, there is no way > > your delay can be decreased when you send lots of traffic > > unless the ATM people have found a way to accelerate > > photons given enough pressure in the queues. > > > More available bandwidth = quicker transmission. > > Ie: at 1000kb/s available, how long does it take to transmit 1000kb ? 1 > second. > Now, at 2000kb/s available, how long does it take ? 1/2 second. > What were you saying ? At higher bandwidths bits are shorter not faster. Repeat that several times. Whether you are signalling at 300bps or at 293875983758917538924372589bps, the start of the first bit arrives at the same time. > Why do you think you have "centi"-second delays in the first place. Because photons and electrons are slow in glass and copper. > I would check yours, but I find time for a packet to > cross a router > backplane to be < 1ms, route > > determination in a traditional router can take up to > 20 ms (or more), > and slightly less than a 1 ms, > > if it is in cache. When I said cross a backplane, I > meant "From > hardware ingress to egress", ie to be > delivered. You are still stuck thinking of routers as things which demand-fill a cache by dropping a packet through a slow path. This was an artefact of OFRV's (mis)design, and the subject of many long and interesting rants by Dennis Ferguson on this list a couple of years ago. Modern routers simply don't do this, even the ones from OFRV. > traceroute to cesium.clock.org (140.174.97.8), 30 hops max, 40 byte > packets > 6 core2-fddi3-0.san-francisco.yourtransit.net (-.174.56.2) 567 ms > 154 ms > 292 ms > > >>>>>>>>>>>>>> Tell me this is a speed of light issue. > >>>>>>>>>>>>>> From the FDDI to the HSSI on the same router. This has nothing to do with the router's switching or route lookup mechanism. Router requirements allow routers to be selective in generating ICMP messages, and cisco's implementation on non-CEF routers will hand the task of generating ICMP time exceededs, port unreachables and echo replies to the main processor, which gets to the task as a low priority when it's good and ready. If the processor is doing anything else at the time you get rather long delays in replies, and if it's busy enough to start doing SPD you get nothing. This gets talked about quite frequently on the NANOG list. I suggest you investigate the archives. I'm sure Michael Dillon can point you at them. He's good at that. > > PING cesium.clock.org (140.174.97.8): 56 data bytes > 64 bytes from 140.174.97.8: icmp_seq=0 ttl=243 time=93 ms > 64 bytes from 140.174.97.8: icmp_seq=1 ttl=243 time=78 ms > 64 bytes from 140.174.97.8: icmp_seq=2 ttl=243 time=79 ms > 64 bytes from 140.174.97.8: icmp_seq=3 ttl=243 time=131 ms > 64 bytes from 140.174.97.8: icmp_seq=4 ttl=243 time=78 ms > 64 bytes from 140.174.97.8: icmp_seq=5 ttl=243 time=81 ms > 64 bytes from 140.174.97.8: icmp_seq=6 ttl=243 time=75 ms > 64 bytes from 140.174.97.8: icmp_seq=7 ttl=243 time=93 ms > > Nice and stable, huh. If this path were ATM switched (Dorian, I will > respond to you in another post) > it would have settled to a stable latency. There is extraordinary congestion in the path between your source and cesium.clock.org, and cesium is also rather busy being CPU bound on occasion. There is also a spread-spectrum radio link between where it lives (the land of Vicious Fishes) and "our" ISP (Toad House), and some of the equipment involved in bridging over that is flaky. If you were ATM switching over that link you would see the same last-hop variability because of that physical level instability. It works great for IP though, and I quite happily am typing this at an emacs thrown up onto my X display in Scandinavia across an SSH connection. > Flow switching does a route determination once per flow, after that > the packets are switched down a predetermined path "The Flow". Hence the > term "flow switching". This reduces the variability of > the entire flow. Um, no it doesn't. As with all demand-cached forwarding schemes you have to process a packet heavily when you have a cache miss. Darren Kerr did some really neat things to make it less disgusting than previous demand-cached switching schemes emanating out of OFRV, particularly with respect to gleaning lots of useful information out of the side-effects of a hand-tuned fast path that was designed to account for all the header processing one could expect. Flow switching does magic matching of related packets to cache entries which describe the disposition of the packet that in previous caching schemes could only be determined by processing individual packets to see if they matched various access lists and the like. It's principal neat feature is that less per-packet processing means more pps throughput. MPLS is conceptually related, btw. Flow switching does not improve queueing delays or speed up photons and electrons, however, nor does it worsen them, therefore the effect of flow switching on variability of normal traffic is nil. Flow switching has mechanisms cleverer than Leon the Cleaner to delete entries from the cache and consequently there are much reduced odds of a cache fault during a long-lived flow that is constantly sending at least occasional traffic. You may see this as reducing variability. I see it as fixing a openly-admitted design flaw. > However, I should also point out that much of your > argument is based in TCP. Most multimedia > (Voice/Audio/Video) content does not focus on TCP, but > UDP/Multicast. What does your slow start algorithm get > you then ? WRED and other admissions control schemes are being deployed that will penalize traffic that is out of profile, i.e., that doesn't behave like a reasonable TCP behaves. Most deployed streaming technologies have taken beatings from ISPs (EUNET, for example with CUSEEME, Vocaltec and Progressive with a wide range of ISPs) and have implemented congestion avoidance schemes that closely mimic TCP's, only in some cases there is no retransmission scheme. > PS MAC Layer Switching, and ATM switching are apples and oranges. > Although, one could be used to do the other. > (Told you Dorian) Huh? Sean. P.S.: You have some very entertaining and unique expansion of a number of acronyms in general and relating to ATM in particular. I'm curious how you expand "PDH". Personally I favour "Pretty Damn Historical", although other epithets come to mind. P.P.S.: It's dead.
|