[ Up ] [ PISA ] [ Composition/Integration ] [ Domain Eng. ] [ Product Lines ] [ Certification ] [ Early Artifacts ] [ OO & DA ] [ Org. Principles ] [ Add. Info. ] [ References ]

The main objective of this working group was to examine issues in the broad area of component certification. The group quickly moved to redefine its charter to include the areas of verification and validation, which were understood to be distinct from the concept of certification. We then attempted to identify subgoals around which a specific consensus could be formed:

1. Characterization of various dimensions of reusable software that might need to be verified;
2. Identification of verification and validation considerations that are unique to reusable software;
3. Identification of economic considerations that drive verification and validation; and
4. Characterization of what certification should mean in the context of reusable software, and how certification is distinct from verification and validation.

Subgoal 1: Dimensions of Reusable Software Requiring Verification

The group identified five dimensions of reusable software where verification appears to be desirable:

• Functionality
• Timing
• Space
• Concurrent behavior
• Physical interactivity

The first four are self-explanatory; the idea of verifying physical interactivity has to do with verification of a software component that is embedded in some physical system (e.g., an avionics component that issues movement instructions to some flight surface). In such a case, it is necessary to verify that the output of the software really has the intended effect on the physical world. The group agreed that factoring in physical laws and properties makes software verification much more difficult than simply verifying that the software properly produces a set of discrete output values.

The group also felt that the above ordering roughly captures the state of both research and practice in terms of the maturity of the verification process in each of the five areas. Verifying functionality has received the most attention from both the research and the practice communities, while verifying physical interactivity has received the least amount of attention.

Subgoal 2: V&V Considerations Unique to Reusable Software

With respect to this subgoal, we developed several definitions and rules-of-thumb that seemed to apply. These are listed below in no particular order:

• Reusable components are developed and verified in a general context. Conventional software is developed and verified in a specific context.
• With both conventional and reusable software, the specific idea of verification means that the software is checked for correctness against some type of specification. This is the typical definition of verification found in many software engineering texts. On the other hand, validation is typically defined as checking the specification against customer requirements. This definition of validation, while appropriate for conventional software, does not apply cleanly to reusable components. An alternative formulation that might be more applicable is that validation of reusable components is between the specification and "alternative usages" of the component.
• Poor verification and validation of reusable components has a greater potential risk to clients than poor verification and validation of conventional software components, because of the potential for the reusable component to appear in a wide variety of systems.
• Adequate verification and validation of reusable components is more costly than single-use components, but the cost can be amortized over all instances in which a particular component is used.
• Verifications must be maintained against support system changes. For example, if a particular software component is to be reused in the context of several operating systems, the verification of that component must be shown to be independent of the operating system.

Subgoal 3: Economic Considerations

We identified four economic considerations that drive the verification and validation of reusable software:

• Span of application
• Criticality
• Marketability
• Lifetime

These considerations may impact the amount of verification and validation necessary for a particular component. The span of application is simply the number of components that depend on the component in question. If a large number of components rely on a given component, then the cost of repairing defects in the component may be higher than that of a component with a relatively narrow span of application. Thus, components with a large span of application can justify more V&V effort up front.

Criticality refers to the potential impact due to a fault in a reusable component. A more extensive V&V effort should be applied to a component if a fault in the component could result in a system causing physical harm to humans than to a component where humans cannot be injured. A similar argument can be made for a greater level of V&V if a failure in the component could result in failure of the systems to accomplish their intended mission, such as the failure of a communication module in remotely controlled sensors.

Marketability refers to the degree to which verification/validation sells a component. A component that is largely unverified tends to be regarded skeptically (i.e., the "not invented here" syndrome). Thus, V&V can definitely help to market a component.

Lifetime refers to the length of time that a component is likely to be in existence. Extensive verification cannot be justified for components with short lifetimes.

Subgoal 4: Certification Frameworks

Certification refers to the idea of associating a label with a component that characterizes the degree to which verification and validation has been performed on that component. The group agreed that although a large number of reuse certification frameworks were one-dimensional (e.g., frameworks with N distinct levels, where level K+1 is somehow "more certified" than level K), more multi-dimensional frameworks need to be developed. The Asset Certification Framework standard provides an approach for describing such multi-dimensional certification frameworks [ACF96].

Summary

Perhaps our most important "result" is in our characterization of the concepts of verification, validation and certification in the context of reuse:

• Verification: Does a reusable component meet its specification?
• Validation: Does the specification for a reusable component satisfy a wide range of alternative usages for that component? (This is an alternative to the classical formulation: Does the specification meet the customer requirements?)
• Certification: How much verification and validation have been performed on a particular component?

[ Up ] [ PISA ] [ Composition/Integration ] [ Domain Eng. ] [ Product Lines ] [ Certification ] [ Early Artifacts ] [ OO & DA ] [ Org. Principles ] [ Add. Info. ] [ References ]