Allen Parrish
David Cordes
The University of Alabama
Department of Computer Science
Tuscaloosa, AL 35487
Tel: (205) 348-3749
Fax: (205) 348-0219
Email:
parrish@cs.ua.edu
David Hale
Joanne Hale
Shane Sharpe
The University of Alabama
Department of Management Science and Statistics
Tuscaloosa, AL 35487
Component-based software engineering is becoming an increasingly popular endeavor, in part because of the perceived gains in terms of software reuse. However, we feel that for component engineering to be successful in fully exploiting reuse opportunities, components must be independent of the processes used to derive them. That is, there should be standard canonical forms for software artifacts that do not carry with them the baggage associated with a particular development methodology or process. This is the type of effort associated with the rapidly emerging discipline of software architecture, and would make the notion of ``software component'' much more like the analogous ``hardware component'' concept from hardware engineering. In this paper, we outline some initial modest ideas with regard to capturing a process-independent notion of a component ``test case.'' Ultimately, the idea behind this work is to produce a complete conceptual framework for understanding test cases as objects. Such a framework would support the specification and reuse of test cases as part of a standardized component architecture.
Keywords: Software testing, conceptual models, test cases, test reuse
Working Groups: Component certification
One of the most important aspects of the field of software architecture is its impact on software reuse [4, 11]. Making software architecture as an explicit object of study is part of an evolution toward standard models of software components. The use of standard models of software components can support reuse, in that such components can be viewed without the baggage associated with either a particular design methodology or implementation language. Of course, certain design methodologies do lead to certain types of components, but many types of components may be identified that are common to many methodologies. If components are to be procured and reused in the context of different design methodologies, then the conceptual model of a component should be independent of the notation, terminology, etc. associated with a specific design methodology.
A number of recent papers in the area of software architecture support this and other worthwhile goals [1, 5, 9, 11, 12]. In this paper, we would like to extend this idea of ``methodology-independence'' to the notion of a test case. If testing is to be used as the approach to validating components, then optimally, one would like to associate test cases with specific components. Indeed, test cases which have been identified as part of a component's validation suite should be somehow linked to the component for its entire lifetime for regression testing purposes. Thus, someone wanting to reuse a particular component could obtain the test cases along with (or as part of) the component.
In order to support such a paradigm, a conceptual model is needed that promotes test cases as objects capable of being reused. Note that classical software testing frameworks lack much support for this idea. Classical software ``test plans'' tend to define the type of testing to be done in terms of process (e.g., ``all branches in the program must be executed'') or non-standard, semi-formal or informal descriptions. Our efforts in this paper are centered around providing formal templates that characterize test case structure in order to promote the conceptualization of test cases as objects.
Although this paper is not intended as a research paper, we wish to provide some insight into this line of research. Accordingly, in the rest of this paper, we present the beginnings of such a conceptual model for test cases. We identify two types of test cases for a restricted class of components (i.e., object-oriented components). Although this work is in its infancy, we feel that it is ripe for extension, and invite others to extend the model by identify other types of test cases for a larger, less restricted class of components.
We assume a restricted model of components, where a component is simply an object-oriented class. The component may or may not have some type of functional specification attached; the presence or absence of a specification makes no difference in our formalization of test cases. We assume the classical definitions of classes and objects from the object-oriented literature.
Regardless of whether a functional specification is present, we assume that all classes have a syntactic part called the definition. The definition contains two parts: an interface consisting of a list of operations that can be performed by instances of the class, and a body consisting of the implementation of the operations and the data attributes for an instance. The implementation of an operation is sometimes called a method, and invoking an operation with respect to a given object is sometimes referred to as sending a message to the object, which responds to the message by executing a specific method. Additionally, every object has state; this state may be characterized either by its history of method invocations or the current values of its attributes.
A method may have input parameters and output parameters. An input parameter is a parameter whose value is transmitted into the method. An output parameter is a parameter whose value is returned by the method. Input parameters may include both value and reference parameters (or IN and IN OUT parameters in Ada), as well as the ``implicit'' object parameter found in languages such as C++. Output parameters may include reference parameters (or IN OUT and OUT parameters in Ada), the implicit object parameter in C++-like languages (which may be changed by the method), or function return values. Some parameters may be both input and output parameters (e.g., IN OUT parameters in Ada).
Our formalization of test cases is based on how different types of methods are ordered during execution. To characterize the various types of methods, we use the classification scheme developed in [7] for data abstractions in general. In particular, a method may be either a constructor, modifier or observer. A constructor method returns a new object of the class from scratch (i.e., without that object as an input parameter). A modifier method modifies an existing class instance (i.e., it changes the state of one or more of the data attributes in the instance). An observer method inspects the object and returns a value characterizing one or more of its state attributes. In this classification system, it is assumed that observer methods do not cause side effects on the object state; thus, any methods producing side effects are either constructors or modifiers.
Also, to support our discussion, we refer periodically to the class under test (the CUT), as well as to one or more objects under test (referred to as OUTs). Testing involves invoking constructor, modifier and observer methods on one or more OUTs. Note that such OUTs serve as output parameters for CUT constructors, input parameters for CUT observers, and input/output parameters for CUT modifiers.
We have identified two distinct categories of test cases:
We first consider state inspection testing. Test cases of this form involve generating a ``result'' (by sending a sequence of messages to an OUT) and then examining this ``result'' for correctness. We have the following model for state inspection testing:
Thus, we are executing a constructor, followed possibly by
modifier(s), and terminating with observer(s). Let C be any
constructor, M be any modifier, and O be any observer. Then,
state inspection test cases may be characterized by the regular
expression 
. We claim that although there may be a
number of different ways to implement state-inspection-based test
cases, all such characterizations may be reduced to this form. A
``proof'' of this is found in [10]. Effectively therefore,
constitutes a template for defining the ``fundamental
form'' of state-inspection-based test cases.
We now consider state comparison testing. State comparison testing involves generating two object states and comparing them to determine whether or not they satisfy some predetermined relationship. For example, sending an Add message followed by a Remove message to a stack object s should result in an object equivalent to s. To conduct this state comparison test, one would generate s, send Add followed by Remove to s, and compare the final result with the original stack s. A second example might involve sending Depth to some stack s (yielding an integer object), and then verifying that the integer returned is one less than the integer object returned by Depth after sending Add to s. This type of testing is commonly done when using algebraic specifications [2, 3], or when using class invariants [6, 8].
Thus, in this model, two objects must be generated and compared. These objects may be either: (a) objects of the CUT or (b) objects returned by an observer method of the CUT. Case (a) corresponds to the case where testing a stack class might involve generating two stack objects and comparing them (e.g., verifying that New.Add.Add.Remove is equal to New.Add). In such a case, it is necessary to generate two objects of the CUT by executing a constructor followed by a sequence of modifiers (where each modifier takes the previous CUT object state as input and produces a new CUT object for consumption by the next modifier in the sequence). The resulting CUT objects can then be compared using a CUT comparison operator (e.g., an Equal method for the stack class).
Case (b) corresponds to the case where testing a stack class might involve comparing the Depths of two stacks and determining that the appropriate relationship is satisfied (e.g., verifying that New.Add.Depth is one less than New.Add.Add.Depth). In such a case, it is necessary to generate two integer objects. Each object is generated by executing a (stack) constructor, followed by a sequence of (stack) modifiers as above, followed by an observer (Depth).
More formally, the state comparison model can be described using a
triple ( 
, 
, comp), where each MS is a method
sequence that has a ``result'' that is compared using the comparison
operator comp. The MS sequences may be of two forms
corresponding to cases (a) and (b) above:
- indicating that comp is
comparing objects of the CUT or,
- indicating that comp is comparing the
objects returned by the modifier O at the end of
the sequence.
As with state inspection testing, the state comparison model is expressed in the most fundamental form possible. Formal arguments to this effect are found in [10].
The idea behind this work is to present a framework for thinking about
test cases as ``first-class objects,'' where each test case is capable
of being independently reused. In this paper, we have developed
conceptual models of two distinct types of component test cases.
State inspection test cases are of the form ; state
comparison test cases are of the form ( , ,
comp), where the MS method sequences are either of the form
or . This characterization presents an abstract
template for these two types of test cases. Such a template supports
a clear conceptual framework, and may ultimately be used to
standardize the structure of component test case documentation.
Just as many in the software architecture community have identified categories of components (e.g., clients, servers, filters, classes and objects), we have identified categories of component test cases. We do not claim that the universe as a whole is restricted to the categories that we have identified, but we do feel that we are on the right track with this direction of thought. We invite others to extend this model, or to identify alternative models for addressing the problems identified here.
Allen Parrish is an Associate Professor in the Department of Computer Science at The University of Alabama. His research interests are in software engineering, particularly in software testing, object-oriented systems, and software reuse. Dr. Parrish is also quite involved with computer science and software engineering education, and has received funding from the National Science Foundation, the Advanced Research Projects Agency and the Ada Joint Program Office to investigate new techniques in this regard.
David Cordes is also an Associate Professor in the Department of Computer Science at The University of Alabama, with research interests in software engineering. His particular interests are in requirements analysis, specification and testing of object-oriented systems. Additionally, Dr. Cordes is actively involved with the Foundation Coalition (FC), a seven-school partnership funded by the National Science Foundation to focus on the re-tooling of undergraduate engineering education. He is a member of the National Management Team for the FC, and is its (nationwide) Strategy Director for Dissemination.
David Hale is Director of the Enterprise Integration Laboratory in the College of Commerce at The University of Alabama. His research interests are in enterprise integration and modeling, collaborative human-computer problem-solving systems, database management system design, accounting information systems, and software maintenance. Dr. Hale's research has appeared in a number of research journals (including Management Information Systems Quarterly, Journal of Management Information Systems, Information and Management IEEE Systems, Man, and Cybernetics, and Journal of Software Maintenance), and has been funded by Texas Instruments, EDS, DEC, Data General, SEMATECH, and Mobil.
Joanne Hale is an Assistant Professor in the Management Information Systems program at The University of Alabama. Her research interests include decision support systems for crisis management, information system quality assurance, human-computer collaboration, strategic uses of information technology and software reuse.
Shane Sharpe is an Assistant Professor in the Management Information Systems program at The University of Alabama. His interests include enterprise modeling, workflow systems, requirements specification in software engineering, and software reuse and maintenance. His research has appeared in a number of publications including the International Journal of Human-Computer Studies, the Journal of Computer Information Systems, the Journal of Systems Management, and the Journal of Software Maintenance. Dr. Sharpe has work experience in the healthcare, pharmaceutical manufacturing and distribution, banking, and public utilities industries.