Internet Info 1997 December/Internet_Info_CD-ROM_Walnut_Creek_December_1997.iso/drafts/draft_ietf_q_t/draft-ietf-rtfm-new-traffic-flow-01.txt

Text File | 1997-03-26 | 22KB | 563 lines

Real Time Flow Measurement Working Group        S.W. Handelman
Internet-draft                                  IBM
                                                Hawthorne, NY USA

                                                N. Brownlee
                                                U of Auckland, NZ

                                                Greg Ruth
                                                GTE Laboratories, Inc
                                                Waltham, MA USA


                                                March 25, 1997
                                                expires
                                                September 25, 1997


Real Time Flow Measurement Working Group - New Attributes for
Traffic Flow Measurement


draft-ietf-rtfm-new-traffic-flow-01.txt


1. Status of this Memo

   This document is an Internet Draft.  Internet Drafts are working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet Drafts.

   Internet  Drafts  are  draft  documents  valid  for  a maximum of six
   months, and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet Drafts as reference
   material or to cite them other than as "work in progress".

   To learn the current status of any Internet Draft, please  check  the
   "1id-abstracts.txt" listing contained in the Internet Drafts shadow
   directories  on  ftp.is.co.za   (Africa),   nic.nordu.net   (Europe),
   munnari.oz.au  (Pacific  Rim),  ds.internic.net  (US  East Coast), or
   ftp.isi.edu (US West Coast).

   This memo provides information for the Internet community.  This memo
   does  not  specify an Internet standard of any kind.  Distribution of
   this memo is unlimited.







Handelman, Brownlee, Ruth                                       [Page 1]

Internet-Draft                                            March 25, 1997


2.1 Introduction

   The Real-time Traffic Flow Measurement (RTFM) working group has
   developed a system for measuring and reporting information about
   traffic flows in the Internet.  This document explores the definition
   of extensions to the flow measurements as currently defined in [1]
   and [5].  The new attributes described in this document will be
   useful for monitoring network performance and expand the scope of
   RTFM beyond traffic measurement. Performance attributes typically
   deal with throughput, packet loss, and delays. We will explore the
   methods in which RTFM can extract values from flows which measure
   these attributes. We will also look at capturing information on
   jitter and congestion control.

   The RTFM Working Group has defined the concept of a standardized
   meter which records flows from a traffic stream according to Rule
   Sets which are active in the meter[1].
    Implementations of this meter have been done by Nevil Brownlee in
   the University of Auckland, NZ, and Stephen Stibler and Sig Handelman
   at IBM in Hawthorne, NY, USA. The RTFM WG has also discussed the
   Meter Reader  Program whose job is to fetch the completed group of
   flows active in the Meter.

2.1.1  RTFM's Definition of Flows

   The RTFM Meter architecture views a flow as a set of packets between
   two end-points (as defined by their source and destination attribute
   values), and as BI-DIRECTIONAL (i.e. the meter effectively monitors
   two sub-flows, one in each direction).

   Reasons why RTFM flows are bi-directional:

   - We are interested in understanding the behavior of sessions between
   end-points.

   - We want to perform as much data reduction as possible, so as to
   reduce the amount of data to be retrieved from a remote meter.

   - The endpoint attribute values (the "Address" and "Type" ones) are
   the same for both directions; storing them in bi-directional flows
   reduces the meter's memory demands.

2.2 RTFM's Current Definition of  Flows and their Attributes

   Flows, as described in the "Architecture" I-D have the following
   properties:

   a. They occur between two endpoints, specified as sets of attribute



Handelman, Brownlee, Ruth                                       [Page 2]

Internet-Draft                                             March 25, 1997


   values in the meter's current rule set.  A flow is completely
   identified by its set of endpoint attribute values.

   b. Each flow may also have values for "computed" attributes (Class
   and Kind).  These are directly derived from the endpoint attribute
   values.

   c. A new flow comes into being when the a packet is seen which is not
   classified by the Rule Set into an existing flow. The meter records
   the time  when this new flow is created.

   d. Attribute values in (a), (b) and (c) are set when the meter sees
   the first packet for the flow, and are never changed.

   e. Each flow has a "LastTime" attribute, which indicates the time the
   meter last saw a packet for the flow.

   f. Each flow has two packet and byte counters, one for each flow
   direction (Forward and Backward).  These are updated as packets are
   observed by the meter.

   g. ALL the attributes have (more or less) the same meaning for a
   variety of protocols; IPX, AppleTalk, DECnet and CLNS as well as
   TCP/IP.

   Current flow attributes as described above, fit very well into the
   SNMP data model.  They are either static, or are continuously updated
   counters.  They are NEVER reset.  In this document they will be
   referred to as "old-style" attributes.

   It is easy to add further "old-style" attributes, since they don't
   require any new features in the architecture.  For example:

   - Count of the number of "lost" packets (determined by watching
   sequence number fields for packets in each direction; only available
   for protocols which have sequence numbers).

   - In the future, RTFM could coordinate directly with the Flow number
   from the IPv6 header.

   At the June, 1996 meeting of the RTFM WG, in Montreal, Canada, a
   proposal was put forth to extend the work of the group to produce an
   Internet Draft "New Attributes for Traffic Flow Measurement".  That
   proposal has brought forth this document. The goal of this work is to
   produce a simple set of abstractions, which can be easily implemented
   and at the same time enhance the value of RTFM meters. This document
   also defines a method for organizing the flow abstractions to
   preserve the existing RTFM flow table.



Handelman, Brownlee, Ruth                                       [Page 3]

Internet-Draft                                             March 25, 1997


   At the December, 1996 meeting of the RTFM WG and at a joint meeting
   of the RTFM and IPPM working groups the concepts of this document
   were discussed. The suggestions given at these discussions are
   included in this document.

   An addition to the main architecture document of RTFM is the use of
   High Watermarks, to set up Alerts when the value of a flow record
   variable  exceeds a watermark, e.g. the total byte count exceeds a
   preset amount, such as no user should send more than 2,000,000
   packets.  This is a generalization of the concept defined in RTFM to
   send Traps when a  Meter finds an exception condition in its own
   processing (The Architecture Document refers to running out of buffer
   space).

2.3 RTFM Flows, Integrated Services, IPPM and Research in Flows


   The concept of flows has been studied in various different contexts.
   For the purpose of extending RTFM, a starting point is the work of
   the Integrated Services WG. We will measure quantities that are often
   set by Integrated Services and configuration programs. We will look
   at the work of the Benchmarking - Internet Provider Performance
   Metrics Working Group, and also look at the work of Claffy, Braun and
   Polyzos. We will demonstrate how RTFM can compute throughput, packet
   loss, and delays from flows.

   An example of the use of capacity and performance information is
   found in "The Use of RSVP with IETF Integrated Services".  [2].
   RSVP's use of Integrated Services revolves around Token Bucket Rate,
   Token Bucket Size, Peak Data Rate, Minimum Policed Unit, Maximum
   Packet Size, and the Slack term. These are set by TSpec, ADspec and
   FLowspec (Integrated Services Keywords), and are used in
   configuration and operation of Integrated Services. RTFM could
   monitor explicitly Peak Data Rate,
    Minimum Policed Unit, Maximum Packet Size, and the Slack term. RTFM
   could infer details of the Token Bucket. We will develop measures to
   work with these service metrics.

   RTFM will work with several traffic measurements identified by IPPM
   [3]. There are three broad areas in which RTFM is useful for IPPM.

   1) RTFM could act as a passive device that can gather traffic and
   performance statistics at appropriate places in TCP/IP networks
   (servers or client locations).

   2) RTFM could give detailed analyses of IPPM test flows that pass
   through the Network segment that RTFM is monitoring.




Handelman, Brownlee, Ruth                                       [Page 4]

Internet-Draft                                            March 25, 1997


   3) RTFM could be used to identify most used paths in a network mesh,
   such that detailed IPPM work could be applied to the most used paths.

3. Flow Abstractions

   Performance attributes include throughput, packet loss, delays,
   jitter, and congestion analysis. RTFM will calculate these attributes
   in the form of extensions to the RTFM flow attributes according to
   three general classes:

   o 'packet traces' - collections of individual packets in a flow or a
   segment of a flow

   o 'aggregates' - statistics derived from the flow taken as a whole
   (e.g. mean rate, max packet size).

   o 'series'- sequences of attributes that depend on more than one
   packet (e.g. inter-arrival times)

   The following sections suggest implementations for each of these
   classes of extensions.

   As an introduction to flow abstractions one fact must be emphasized.
   Several of the measurements enumerated below can be implemented by a
   Meter Reader that is tied to the meter with instantaneous response,
   and very high bandwidth.  If the Meter Reader and Meter can be
   arranged in such a way, RTFM could collect Packet Traces with time
   stamps, and provide them to the Meter Reader for processing by the
   Meter Reader.

   A more useful alternative is to have the meter calculate some flow
   statistics locally. This allows a looser coupling between the meter
   and Meter Reader. RTFM will create an 'extended attribute' depending
   upon settings in the Rules table of RTFM. By default, RTFM will not
   create any extensions without explicit instructions in the Rule
   table.

   RTFM's traditional flows can be analyzed at two levels. The first is
   to analyze the Network traffic in terms of time, e.g. traffic load of
   a particular flow, to be called Network Flows. These flows can be
   looked at as an extension of the "old-style" flow attributes. The
   second, is to derive a value from the flow, e.g. analyzing packet
   sequence numbers and ACKS and estimating delay.  This second type
   will be called Derived Attributes.

3.1. Packet Traces





Handelman, Brownlee, Ruth                                      [Page 5]

Internet-Draft                                           March 25, 1997


   The simplest way of doing this in the meter would be to have a new
   attribute called, say, "PacketTrace."  This would be a table, with a
   column for each property of interest.  For example, one could have

   - Arrival time (TimeTicks from SysUptime, or microseconds from
   FirstTime for the flow).

   - Direction (Forward or Backward)

   - Sequence number (for protocols with sequence numbers)

   - Flags (for TCP at least).

   To add a row to the table, we only have to have a rule which PushPkts
   the PacketTrace attribute.


   To use this, one would write a rule set which selected out a small
   number of flows of interest, and PushPkted PacketTrace for each of
   them.  A MaxTraceRows default value of 2000 would be enough to allow
   a Meter Reader to read 1-second ping traces every 10 minutes or so.
   More realistically, a MaxTraceRows of 500 would be enough for one-
   minute pings, read once each hour.

3.2. Aggregate Attributes


   Performance aspects of flows are interesting  in the case of a flow
   between a server and client. RTFM could find the same data in TCP/IP
   and UDP flows, and can find additional data in TCP flows.  The
   performance data found by this method define the flow capacity used
   by the individual flow, as experienced in the locale of the RTFM
   meter.

   For both TCP/IP and UDP, RTFM's "old-style" flow attributes count the
   bytes/packets for packets which match the rule set for an individual
   flow.  In addition to these totals, RTFM could calculate Packet size
   and Bit rate statistics. Bit rate statistics point to the throughput
   of performance metrics.

   Packet size - RTFM's packet flows can be examined to determine the
   maximum packet size found in a flow. This will give the Network
   Operator an indication of the MTU being used in a flow. It will also
   give an indication of the sensitivity to loss of a flow, for losing
   large packets causes more data to be repeated.

   Bit rate  - The data could also be recorded as the maximum and
   minimum data rate of the flow, found over specific time periods



Handelman, Brownlee, Ruth                                       [Page 6]

Internet-Draft                                            March 25, 1997


   during the lifetime of a flow. Bit rate could be used to define the
   he throughput of a flow, and if the RTFM flow is defined to be the
   sum of all traffic in a network, one can find the throughput of the
   network.

   Note that aggregate attributes are a simple extension of the their
   values are never reset.  For example, an array of counters could hold
   a 'total bits observed' distribution.  The counters continue to
   increase, a meter reader will collect their values at regular
   intervals, and an analysis application will compute and display
   distributions of the data rate for each interval.  In this situation,
   the interval will be specified by the manager which controls the
   meter and meter reader.

3.3 Series Attributes


   The notion of series attributes, is to keep simple statistics that
   involve more than one packet. Methods to derive simple percentiles,
   means, and other statistics can be developed for each flow. The
   notation to construct such an attribute would be a command in the
   rule set, instructing the meter to compute the attribute. This is
   similar to the definition above of creating an aggregate attribute.

   Whereas aggregate attributes (see above) only require the meter to
   increment counters, series attributes require the meter to compute
   attribute values.  For example, if we want to produce a distribution
   of '10-second' forward data rates, the meter might compute this for
   each flow of interest as follows:  - maintain an array of counters to
   hold the flow's 10-second data rate distribution.

   - every 10 seconds, compute the 10-second octet count, and save a
   copy of the flow's forward octet counter.  To achieve this, the meter
   will have to keep a list of aggregate flows and the intervals at
   which they require processing.  It will require careful programming
   to achieve this, but provided the meter is not asked to do this for
   very large numbers of flows, it should not be too difficult!

   TCP and UDP

   Inter-arrival statistics - TCP and UDP. RTFM knows the time that it
   encounters each individual packet. Statistics can be kept to record
   the inter-arrival times of the packets, which would give an
   indication of the jitter found in the Flow.

   TCP Only - Packet loss  - RTFM can calculate packet loss performance
   metrics. This is an area for further study. TCP packets have byte
   sequence numbers and SYNS, FINS, and ACK's associated with them. RTFM



Handelman, Brownlee, Ruth                                        [Page 7]

Internet-Draft                                             March 25, 1997


   could track the sequence numbers in the flows, and calculate the
   packet loss occurring in a flow, and thus we can develop a metric of
   lost packets and useful traffic.

   Delay analysis -  TCP flows could be examined for the timing between
   Transmissions and ACKS and thus we can get some measure of delay of
   performance metrics . This assumes the forward and reverse packets
   are both visible to the meter. In the case of asymmetric flows, RTFM
   can be run on multiple paths, and with precise timing create packet
   traces, which can be compared at later times.

   Subflow analysis - TCP flows, e.g. a Web server's httpd flows
   actually contain many individual sub flows. Given, a well known Web
   Server WW, and a  client CC, RTFM would normally pick up an
   aggregation of all the flows of text, graphics, Java programs, etc.
   that are sent between WW and CC. By analyzing the Sequence numbers,
   RTFM could estimate when each subflow occurs, and thus maintain
   statistics about the subflows on a network.

   Congestion Analysis - In a TCP/IP flow we have information on the
   negotiation of Window sizes which are used by TCP/IP to control
   congestion. Well behaved flows  honor these requests and in the vast
   majority of cases the sender will slow down and thus decrease its
   rate of injecting packets into the congested network.  We will look
   for cases where flows do not honor these congestion control and are
   not slowing down. We will also look for flows which have the
   "precedence" fields turned on and thus are aggressively competing for
   network resources.

   3.4 Action on Exceptions

   The user of RTFM will have the ability to define Network and Derived
   flows, as having High Watermarks. The existence of abnormal service
   conditions, such as non-ending flow, a flow that exceeds a given
   limit in traffic (e.g. a flow that is exhausting the capacity of the
   line that carries it) causes an ALERT to be sent to the Meter Reader
   for forwarding to the Manager. Operations Support may define service
   situations in many different environments. This is an area for
   further discussion on Alert and Trap handling.

4. Packet Flow Table

   The architecture of RTFM has defined the structure of flows, and this
   draft does not change that structure. The flow table could have
   ancillary tables called "Packet Flow Tables", which would contain
   rows of values and or actions as defined under packet traces,
   aggregate attributes and series attributes. Each Packet Flow table
   would be marked with the number of its corresponding flow in the RTFM



Handelman, Brownlee, Ruth                                         [Page 8]

Internet-Draft                                               March 25, 1997


   flow table.  In order to identify the data in a Packet Flow Table,
   the value of the Rules Table Extension will be pushed into a string
   at the head of each row. For example, if a Packet Flow table entry
   has Bit Rates for a particular flow, the "BitRate" string would be
   found at the head of the row.

   A method of bundling the Packet Flow table and the packet data will
   be developed such that an SNMP manager can retrieve whole flow table
   entries, and whole Packet Flow Tables, with SNMP v2 Getbulk
   instructions. This will be accomplished by creating a flow attribute
   called FlowDataPackage. This will be an encoded sequence of all the
   objects such that the Getbulk could operate on the whole structure.

4.1 Note on Interchange between Meter and Meter Reader

   The above information on Getbulk could be superseded in the near
   future by the work of the RMONMIB Bulk Data Transfer.

5. Extensions to the Rules Table

   The Rules Table of "old-style" attributes will be extended for the
   new flow types. A list of actions, and Keywords, such as "BitRate"-
   for Bit Rate, "MaxPack", for Max Packet size will be developed and
   used to inform RTFM to collect a set of extended values for a
   particular flow (or set of flows).

6. Acknowledgments

   We thank Stephen Stibler of IBM for his comments on this draft.

7. Security Considerations

   Security considerations are not discussed in this memo.

8. Author's  Address:

   Sig Handelman
   IBM Research Division
   Hawthorne, NY
   Phone: [phone]
   E-mail: [e-mail]

   Nevil Brownlee
   The University of Auckland
   New Zealand
   Phone: [phone] x8941
   E-mail: [e-mail]




Handelman, Brownlee, Ruth                                         [Page 9]

Internet-Draft                                              March 25, 1997


   Greg Ruth
   GTE Laboratories
   Waltham, MA
   Phone: [phone]
   E-mail: grr1@gte,com

9. References:

   [1] Brownlee, N, Mills, C., Ruth, G.: "Traffic Flow Measurement:
   Architecture",  RFC 2063, 1997

   [2] Wroclawski, J., : "The Use of RSVP with IETF Integrated Services
   Internet" Draft,  October, 1996

   [3] Almes, G. et al: "Framework for IP Provider Metrics" Internet
   Draft. July 1996

   [4] Claffy, K., Braun, H-W, Polyzos, G. "A Parameterizable
   Methodology for Internet Traffic Flow Profiling," IEEE Journal on
   Selected Areas in Communications, Vol. 13, No. 8, October 1995.

   [5] Mills, C., Ruth, G.: "Internet Accounting Background," RFC 1272,
   1992




























Handelman, Brownlee, Ruth                                         [Page 10]