Benchmarking Methodology (bmwg) ------------------------------- Charter Last Modified: 08-Jul-97 Current Status: Active Working Group Chair(s): Guy Almes Kevin Dubray Operations and Management Area Director(s): John Curran Michael O''Dell Operations and Management Area Advisor: John Curran Mailing Lists: General Discussion:bmwg@baynetworks.com To Subscribe: bmwg-request@baynetworks.com Archive: ftp://ndtl.harvard.edu/pub/bmwg/mailing.list Description of Working Group: The major goal of the Benchmarking Methodology Working Group is to make a series of recommendations concerning the measurement of the performance characteristics of various internetworking technologies; further, these recommendations may focus on the systems or services that are built from these technologies. Each recommendation will describe the class of equipment, system, or service being addressed; discuss the performance characteristics that are pertinent to that class; clearly identify a set of metrics that aid in the description of those characteristics; specify the methodologies required to collect said metrics; and lastly, present the requirements for the common, unambiguous reporting of benchmarking results. Because the demands of a class may vary from deployment to deployment, this Working Group will not attempt to define acceptance criteria or performance requirements. Currently, there are two distinct efforts underway in the BMWG. The first addresses the metrics and methodologies associated with benchmarking network interconnect devices. The second effort (IPPM) focuses on determining the practical benchmarks and procedures needed in gaining insight for users and providers of IP Internet services. An ongoing task is to provide a forum for the discussion and the advancement of measurements designed to provide insight on the operation internetworking technologies. Goals and Milestones: Done Expand the current Ethernet switch benchmarking methodology draft to define the metrics and methodologies particular to the general class of connectionless, LAN switches. Aug 97 Edit the LAN switch draft to reflect the input from BMWG. Issue a new version of document for comment. If appropriate, ascertain consensus on whether to recommend the draft for consideration as an RFC. Aug 97 Take controversial components of multicast draft to mailing list for discussion. Incorporate changes to draft and reissue appropriately. Aug 97 Incorporate BMWG input and continue to progress the Cell/Call Terminology Draft. Reissue draft as appropriate. Internet-Drafts: Posted Revised I-D Title ------ ------- ------------------------------------------ Aug 96 Sep 97 Benchmarking Terminology for LAN Switching Devices Nov 96 Mar 97 Terminology for Cell/Call Benchmarking Nov 96 Aug 97 Empirical Bulk Transfer Capacity Nov 96 Aug 97 Framework for IP Provider Metrics Nov 96 Aug 97 A One-way Delay Metric for IPPM Dec 96 Aug 97 Terminology for IP Multicast Benchmarking Mar 97 Aug 97 A Packet Loss Metric for IPPM Jul 97 New Benchmarking Terminology for Network Security Devices Request For Comments: RFC Stat Published Title ------- -- ---------- ----------------------------------------- RFC1242 Jul 91 Benchmarking Terminology for Network Interconnection Devices RFC1944 May 96 Benchmarking Methodology for Network Interconnect Devices