|
North American Network Operators Group Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical keynotefootnote (it looks like i can't get away with avoiding this topic any longer)
have gotten too many comments/questions about my take on the keynote thing for which i posted a reference yesterday (emphasizing that i was neither endorsing nor dismissing it) ---------------------------------------- (THIS MESSAGE HAS NOTHING YOU SHOULD PUT NEAR YOUR ROUTER, and is Statistically Unlikely to change your behavior in the least so i would put it off till major downtime beyond your control if you bother reading at all) ---------------------------------------- i admit i was avoiding extending judgement (you all do it so well you don't need me) but apparently some of you think i should (extend) -- and since a lot of you turn out to be paying my salary, who am i to disobey (and since randy's already procmail'ed me out perhaps i'm not putting any more appendages at risk. heck i would assume half the list filters out anything with 'keynote' in subject by now maybe noone's reading this anyway yum yum) here goes (warning, go get coffee, it feels like the millionth time this has been said, sorry, blame the guys that asked for it): (you can also grep for 'Bottom Line' below, (but do it twice, i have 2, natch) and skip the rest. or if you already know the question, my answer's: No. there. you're done. you can owe me the 5 minutes i just saved you) the Keynote stuff as originally done would have been an interesting set of data and methodologies to Start with and help toward bridging a crevice-fast-becoming-a-canyon (i.e, the gap between Internet O and R) i reckon there's a strong contingent in the community that perceives the keynote stuff as just dumping landfill in that canyon won't argue frankly think it doesn't matter in the long term you all (ISPs) have survived worse; you'll survive keynote those making a fuss over it seem to me consistently missing the point which manages to get Blissfully Ignored: that the Keynote study got more attention (yes, controversial, so?) than any other study (or ISP, or vendor, or protocol) that i can remember (yes partly cuz boardwatch took the data and went inference-aerobic on us but keynote didn't exactly try to stop them; PR is PR when you're selling media, yum yum) oh my whatever could it mean is it possible that enough Internet users (and even some of you ISPs; you're so cute) are so desperate for a standard, professional, neutral metric/methodology for measuring performance/workload/qos (wth that is) that you'll give anything a shot you can find? ( gosh aren't you guys as bored of hearing this stuff as i am of saying it? ) but wait, it gets better: while i'm off gallavanting around the Internet on your tax dollars (don't worry i'm not getting many of them) trying to bring some folks together and start behaving like we actually have some common inter/cross-ISP geek-compatible verifiability goals, (trying to start CAIDA, extend NLANR, get OC3MON's out there, draw pictures to show how bad the macroscopic stuff continues to get, etc) -- these keynote/boardwatch folks actually went off and did Real Pinging to really sexy sounding FQDNs, and hired lots of folks to make flashy web pages/charts/moving.gifs, write fluffy white papers and sell them for lots of money to ISPs, who apparently have enough to buy (and fund someone to complain about) the paper but not to think about the underlying issues and do something constructive about it (or even contribute to those trying to) there was [is] a total Market Void there, guys; that it was filled is pretty econ 101-ish, ne? why is everyone continuing to act so flustered by it? shouldn't we have rather been surprised that (1) it didn't happen sooner, and/or (2) noone else has come in and tried to do something `better' or charge less what did we all think, that this market would be Different from Others? that we could just send a letter to some editor and say 'oh, no, you can't measure it that way, that's not stastically valid; you have to wait for us to figure out htf to do this' don't think so, guys. (and if you're expecting it to come from us research geeks then you're not paying us fast enuf, nor giving us enough data/access/warm.and.fuzzy.feelings/help) hey, come to think of it, wasn't the whole point of XIWT so you could do this yourself? as John said, XIWT invited this Keynote fellow to speak (i think the word Tracie used to me was 'mind-bending') as they say in their web pages, they apparently measure 1 or 2 isps in a city, then draw inferences about `the city's connectivity' to the Net. (``er, does the word Topology mean anything to you?'' ``uh, the market we're targeting wouldn't understand that stuff; it would only confuse them.'') cool. i wonder if that's how we sell nuclear weapons. yes, keynote did articulate some durable insights (`try to minimize the number of hops of data across the network' -- mmmm) oh and be sure not to miss their 'discoveries about the internet' http://www.keynote.com/measures/top10.html, neato-cool, with goodies like Keynote's measurements demonstrate that most of their performance problems occur out in the Internet's infrastructure somewhere between the web site and its users (whoa, that's really going out on a limb) .... CompuServe, CWIX, SAVVIS, and other less-well-known backbone providers currently offer some of the fastest Internet backbone services. We believe that's because their backbones are relatively underutilized compared to those of larger providers such as Sprint and InternetMCI. Large, capacious, well-engineered backbones (just like most of our freeways) can perform more slowly than others if they have to service more customers at once. (quick, everyone change over to the underutilized backbone! that will fix everything!) Internet brownouts...produce measurable performance degradation for all users. However, the Internet still continues to operate, albeit more sluggishly, while the accident damage is being repaired and traffic is re-routed around the accident scene. (even with all those contradicting Battered Gagging Provably suboptimal roadsigns -- 's amazin', isn't it) 9. Internet performance improves substantially during the day on holiday periods such as Christmas and Thanksgiving. We believe that's because users spend time on other activities instead of surfing the Net from home or office. (do you feel like you're reading your horoscope yet?) 10. Network engineers at backbone providers tend to focus on optimizing traffic flow within their own networks. They tend to de-emphasize or ignore connectivity and end-to-end response time to users on other networks. The Internet, however, is an interconnection of many backbones and private networks, with the result that users rarely access web sites that are directly connected to the same backbone that they are. ..... (it's them silly network engineers again what'r they thinkin'?) on the (waaaay) other hand, i got really tired of listening to people bash boardwatch for how inaccurate their ISP catalog thing was (``kc, c'mmmmoooooon, bad data is worse than none at all.'' pshaw. put up or shut up; the world's not waiting for you) without that database CAIDA couldn't have done http://www.caida.org/Tools/Mapnet/ which (at least java-enabled) folks really seem to get a kick out of. some of them providers, even. (no, no, of course not randy. let's not get crazy here.) and to do this, yes, i used jack's sub-accurate data. (he even gave me permission) yes, some of it's wrong yes, some backbones might not even be there (especially new or stealth ones) and backbone topology info's changing under our noses, so even if it's right today it's probably wrong tomorrow bummer. sounds like we can never get accurate data; we just shouldn't do anything at all. BZZZZZT. survey saaays: 0. guess what?! there's a button you can send in your correct data to CAIDA if you want to be able to see the Real Thing (yes, oob-authenticated). [kc points to water; pauses dramatically.] so yeah their work has accuracy problems. maybe even more than mine (good thing they get paid more; i think having problems is such Haarrrrd work -- ) -- so what. i'll dare/pay someone else to do better. oh, there's another thing you may wanna lose sleep over -- this city-framed measurement methodology thing (which measures at best an isp's site-specific web hosting capabilities, Not backbone performance (sorry jack)) might actually be more defensible (yeah yeah, i know, nowhere to go but up) if/as we move toward this gigapop model thing (if you have to ask you don't wanna know) but then they'd [someone'd] have to do a little more than measure a single machine at an ISP's web hosting facility if they wanted a chance at real insight so Bottom Line, ftr: No, i'm Not Endorsing the keynote or boardwatch backbone performance comparison studies and especially not the marketing spin/web pages. At All. ok? (or the netmedic stuff. or anything else i see out there. but i'm not quite unbiased.) on the other hand, there's a whole lotta people out there apparently willing to pay those guys a lot more than they're willing to pay me (well depends on their tax bracket i guess) so who am i to gripe. lots of people buy windows95 too (and lots gripe. with Absolutely Zero effect on that fellow's profit margin. shocking.) Bottomer Line: (ok we're low enuf to scrape barrel now) there's no free lunch if you want to be able to tell your customers how you're doing you're gonna have to be able to measure it which means you're gonna have to define what to measure (or pay someone you trust to do so) otherwise, fasten your seatbelts, i guarantee lots more low-flying misleading benchmarks from the i[nd|ll]ustrious Keynotes, Boardwatches and Netmedics of the world designing charming products and services to meet the needs of suits and corporate types with no clue (nor interest in) how the Internet works (i hate broken records i hate broken records i hate broken records more than smd hates analogies smd hates analogies smd hates analogies): unless you chaps mobilize, insights re how the Internet operates are going to continue to be defined by faulty methodologies. anything else requires serious OperationsResearchAgenda-driven Cooperation among ISPs and this Very Idea still seems to be slightly less appetizing to them than spinach is to 8 year olds (if it makes you feel any better i'm not planning on going anywhere, but then after this message who else would hire me) k
|