|
North American Network Operators Group Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical IP Provider Metrics (IPPM) WG/BOF
A number of people missed this the first time. To: [email protected] Subject: INCOMING: IP Provider Metrics Date: Wed, 15 Mar 1995 11:53:01 -0500 From: "Matt Mathis" <mathis> [... out of date adminstrivia deleted ...] Even though I am going to work hard to prevent it, this is likely to become quite political. My intent is to provide tools and methods (standard procedures) which will enable the market to reward providers who are true to their mission, what ever that may be. Although my historical point of view has always been in the operations community, I have clearly become "just a customer". Furthermore, there is a large pool of "experts" who think that they can do the job better than you. Therefor, it is crucial that current providers take an active role in this effort, if nothing else to offset the "experts". Comments are welcome. Thanks, - --MM-- Draft agenda of the "IP Provider Metrics" BOF 6:00 - 11:30 Wednesday, April 5, 1995 ============================================= Consider a possible WG. (15 minutes at opening) * - Is a new WG needed? * - Relationship to BMWG/GISD/others - Inventory of possible tasks - Requirements for IP Provider Metrics * - Define scope and priorities * - Chair issues, Editor/secretary * - Charter (* introduce at the open, but postpone detailed discussion until the close) General Requirements for IP provider metrics (15 min) IP bulk data transfer performance (technical issues, 30 min) IP routing stability and robustness (technical issues, 30 min) Consider a possible WG...., part 2 (30 min) - ---------------- Rough Draft Charter IP Provider Metrics (IPPM) The IPPM WG will develop a set of standard metrics that can be applied to the the quality, performance and reliability of an IP datagram service. These metrics will be designed such that they can be performed by the providers themselves, customers, potential customers or independent testing groups. The areas covered will include: I) Path Performance A) Bulk data transfer performance (ftp, etc) B) Interactive performance (Telnet, X11, etc) C) Real-time and multicast performance (delay statistics) II) Routing stability and robustness A) End to end availability B) Time to recover (e.g. switch to backup paths) C) Route stability (Spurious route transitions, etc) III) others TBD The IPPM WG will select or adapt existing tools and develop standard procedures for performing and documenting the measurements. - ---------------- Notes on the Charter o Services peripheral to basic IP, such as NOC/NIS services, DNS, etc. are expressly excluded because there is vehement anti-consensus among Internet providers. Prior efforts to standardize these areas have failed because they impinge on the different business models held by the providers. For example some providers include full, high quality DNS service bundled with their IP service. Others provide inexpensive DNS "starter" services, but expect most customers to eventually run their own servers. o It is also important that the language not be pejorative, to permit providers to strategicly balance their price and performance. We want to encourage all market positions including "Inexpensive, good enough service" as well as "The best possible service". o The milestones will be weighted to address I.A. (bulk performance) and possibly parts of II first. The other goals will be addressed as an ongoing effort. The scope will start very narrow, and will be extended only as we have assured closure on the initial tasks. o I have a prototype bulk transfer diagnostic that can safely and accurately measure IP bulk performance. It is a combination of traceroute and Reno TCP, and can provide traceroute like path information under exactly the conditions induced by TCP. See the discussion below. o The difficulty with routing stability metrics, is that the best ones require collusion with the Internet providers themselves. This clashes with the provider-to-provider trust model that is a necessary for healthy operation of the Internet as a whole. BGP can easily be used to collect engineering, business and other data on competitors. I believe that there exists a good technological solution based on the global collection and archiving of sanitized BGP logs. Clearly, there is a substantial piece of this technology that belongs with the RA and other global routing agents. However, there are also parts that belong to the community in the form of a working group. There needs to be a discussion of the possible role for IPPM in this process. o The IETF is not "poised" to address "standard measurement procedures". Is this a problem? o The final draft of the charter should be completed after the WG is convened. o IPPM should have an editor other than the chair. - ---------------- Comments on the bulk performance tool: I am working on a prototype diagnostic tool ("treno") designed to address bulk performance issues. It is derived from Windowed ping (Matt Mathis, INET'94, See ftp://ftp.psc.edu/pub/net_tools/WPing.ps), but designed to be safe for typical network administrators at user sites. It uses traceroute to emulate Reno TCP over any IP infrastructure. This approach has a number of strong advantages: - - Its bandwidth consumption is bound by the same congestion mechanism as TCP. (Although this is more than might be tolerated for ongoing monitoring). - - It measures the network under the same queue dynamics as inflicted by TCP. - - Differences in the end systems need not affect the results. - - Today it can identify the lowest performance provider in a long path if there is a suitable unix box near each interconnect. - - Someday it will be able to diagnose (to a specific hop) any path across the Internet. Treno is not yet completed. There are some open issues which can best be addressed as part of an IETF WG, with open and public input from all interested parties. In the end this will make it a stronger tool for supporting the growth of the Internet. Incomplete Tasks: - - Certify that treno emulates Idealized Reno TCP. Understand and evaluate the implications of differences between treno and Reno as it appears in the field. (IPPM and end2end research community) - - Develop testing methodology for "data sheet" measurements: How to select and specify endpoints, sampling schedule, etc for measuremets to be used on a "data sheet" or service level language in a contract. (IPPM and the network operators community) - - Document the need and utility for all router vendors to support full rate ICMP TTL exceeded messages. (IPPM and RREQ?) ------- End of Forwarded Message
|