Reuse Capability Assessment: Case Study Results

Ted Davis

Software Productivity Consortium SPC Building 2214 Rock Hill Road Herndon, Virginia 22070 Tel: (703) 742-7335, fax: (703) 742-7200 Email: davis@software.org

Abstract

This paper provides preliminary results of case studies conducted by the Software Productivity Consortium on the validity and usefulness of its Reuse Capability Model (RCM) and assessment process. The Consortium conducted the case studies with organizations who have successfully instituted software reuse. The objective was to evaluate the RCM against the experience of these organizations. Questions posed by the case studies and presented in this paper include: Does the RCM capture the factors critical to improving an organization’s reuse practice? Does the RCM assessment process accurately identify an organization’s reuse strengths and improvement opportunities? Is the RCM assessment process cost-effective?

Keywords: Reuse adoption, reuse capability, reuse process, assessment, case study

Workshop Goals: Promote discussion on issues critical to instituting software reuse in an organization.

Working Groups: Reuse management, organization, and economics; reuse maturity models; reuse process models

Copyright \begin{matrix}c \\ ◯ \end{matrix}1993, Software Productivity Consortium Services Corporation. All rights reserved. This material is based in part upon work sponsored by the Advanced Research Projects Agency under Grant # MDA972-92-J-1018. The content does not necessarily reflect the position or the policy of the U.S. Government, and no official endorsement should be inferred.

Produced by the Software Productivity Consortium under contract to the Virginia Center of Excellence for Software Reuse and Technology Transfer.

1 Background

In 1992 the Software Productivity Consortium released the first version of its Reuse Adoption Guidebook [1]. The guidebook provides a comprehensive approach to institutionalizing and improving an organization’s software reuse practice. This approach includes three major elements: the Reuse Adoption Process, Domain Assessment Guidelines, and Reuse Capability Model. The Reuse Adoption Process provides step-by-step guidance on initiating, planning, and implementing a program to institutionalize software reuse practices. The Domain Assessment Guidelines provide guidance on qualitatively estimating the potential for reuse in an organization’s business area. The Reuse Capability Model is a self-assessment and planning aid used to gain a comprehensive understanding of an organization’s process with respect to reuse and to establish priorities for improvement.

The Consortium developed the Reuse Adoption Guidebook by leveraging industry, government, and academic work in reuse, and through an evolutionary development approach including many internal and external expert reviews. In 1993 the Consortium is continuing to evolve its reuse technologies by applying them in:

• Reuse adoption case studies. Collaborative efforts to evaluate the reuse adoption technologies against the experience of organizations who have successfully instituted reuse.

• Reuse adoption pilot projects. Collaborative efforts with organizations seeking to initiate and implement a reuse program to improve their reuse practice.

The Consortium plans to conduct three case studies and three pilot projects in 1993. This paper presents some preliminary results of the three case studies on the RCM and its associated assessment process.

2 Position

Institutionalizing software reuse in an organization requires that many technical, organizational, economic, and cultural issues be addressed. As attention on reuse has increased in the past several years many issues have been raised. However, without a managed approach for addressing these issues, software engineers and managers will have difficulty in devising appropriate reuse strategies for their organizations, thereby increasing their risk of stalled, misguided, or failed reuse programs. The Reuse Adoption Process, Domain Assessment Guidelines, and Reuse Capability Model provide a managed approach to help engineers and managers address the many technical and nontechnical issues and implement reuse programs that reduce risk and increase the benefits the organizations realize from reuse.

The Consortium’s key concerns in developing the reuse adoption technologies are whether these technologies are valid and effective. The Reuse Adoption Case Study is one step the Consortium is taking to begin to establish and improve the validity and effectiveness of its reuse adoption technologies. This paper focuses on the case study results for the RCM and its associated assessment process. The specific questions addressed here include:

• Does the RCM capture the factors critical to improving an organization’s reuse practice?

• Does the RCM assessment process accurately identify an organization’s reuse strengths and improvement opportunities?

• Is the RCM assessment process cost-effective?

The following subsections characterize the case study participants, describe the case study approach, and present the preliminary case study results.

2.1 Case Study Participants

For the case studies, the Consortium sought organizations who have successfully institutionalized reuse. The motive for seeking these organizations was to get feedback on the reuse adoption technologies from people who have “been there.” Each of the case study organizations has been practicing reuse for at least five years in the development and/or maintenance of customer products. The case studies included organizations that developed systems for the Department of Defense (DoD) and organizations that developed commercial products; they included both small organizations (< 50 staff) and large organizations (> 1000 staff); and they included organizations in the embedded systems business as well as organizations in the information systems business.

Due to confidentiality agreements, specific attributes of the case study participants and specific data resulting from an organization’s assessment are not disclosed in this paper.

2.2 Case Study Approach

For the case studies, the Consortium took the approach of applying the RCM assessment process in conducting actual assessments of the organizations’ reuse capabilities. In addition, the Consortium also conducted structured interviews to understand how the organizations’ reuse practices evolved and administered post-assessment questionnaires to obtain additional feedback.

The assessment process is similar to a focus group format. A group of individuals from the organization review and discuss the factors in the RCM, then reach consensus on their strengths and improvement opportunities. Consortium staff acted as facilitators. The steps in the assessment process are as follows:

• Form the Team. A team of four to eight individuals is formed representing the major functions of the organization. Team members should be knowledgeable in the organization’s process, policies, procedures, and products.

• Train the Team. Facilitators provide a half-day training on the adoption process, models, and assessment for the assessment team.

• Develop Organizational Profile. The organizational profile is a rough sketch of the organization, its process, products, and business. It serves as a context for the assessment.

• Assess the Critical Success Factors. The RCM is defined by 21 critical success factors organized into four groups (application development, asset development, management, and process & technology). Each factor is defined in terms of one or more goals (see [1] for a complete description of the factors and goals). The team assesses their organization against each goal on:

– The extent the organization meets the specified goal on a scale of 1-not satisfied, 2partially satisfied, 3-fully satisfied.

– The expected impact on the organization’s reuse capability from fully satisfying the stated goal on a scale of 1-no positive impact, 2-low positive impact, 3-moderate positive impact, 4-high positive impact.

• Develop Findings. Based on the discussions resulting from the previous step, the group identifies its strengths and improvement opportunities through consensus.

2.3 Case Study Results

The results presented in this section are the statistical averages computed from the team member assessment responses and post-assessment questionnaires (25 team members from 3 case studies).

Figure 1 illustrates the computed average per critical success factor for the expected impact on the organization’s reuse capability from fully satisfying the associated goals. Nearly all of the factors were judged as having a moderate to high impact on an organizations reuse capability. This result is one indicator as to whether the RCM captures the factors critical to improving an organization’s reuse capability.

In addition, the post-assessment questionnaire asked the participants to rate the following statements on a scale of 1-strongly disagree, 2-disagree, 3-neutral, 4-agree, 5-strongly agree. The average response is indicated.

• The Reuse Capability Model covers the process issues most critical to effective software reuse. Average response = 4.3

• The reuse capability assessment accurately identified my organization’s reuse process strengths and improvement opportunities. Average response = 4.4

• The reuse capability assessment is a cost-effective technique for understanding an organization’s reuse process and planning improvements. Average response = 3.8

Two case study groups had also suggested the addition of a critical success factor addressing intergroup coordination and communication. Some participants rated the effectiveness somewhat lower because they believed they already had a good understanding of their process. However, they indicated the assessment would be more effective for organizations who are just beginning to implement a reuse program.

Although the Consortium does not view these results as proof of validity, it is encouraged that these results indicate its reuse adoption technologies are a step in the right direction. Further study will be conducted primarily through reuse adoption pilot projects. The lessons learned from the case studies and pilot projects will be worked into the next version of the Reuse Adoption Guidebook.

3 Comparison

Technologies similar to the Consortium’s reuse adoption technologies include the Central Archive for Reusable Defense Software (CARDS) Franchise Plan [2] and the Software Technology for Adaptable, Reliable Systems (STARS) Reuse Strategy Model (RSM) [3]. The CARDS Franchise Plan includes a series of handbooks designed to support the implementation of domain specific reuse in DoD software procurements. The Consortium’s reuse adoption technology is designed to support business organizations in general (DoD and commercial) and it is not tied to a specific reuse technology. The purpose of the STARS RSM is to identify areas in which organization objectives, policies, procedures, and process definitions can be applied in furthering a cost-effective reuse strategy. It is specifically designed to support the STARS vision of “doing reuse.” The RSM is primarily intended to support project planning for reuse, whereas the RCM is primarily intended to support capability improvement planning for product-line organizations.

References

[1] S. P. Consortium, “Reuse Adoption Guidebook,” Tech. Rep. SPC-92051-CMC, Software Productivity Consortium, November 1992.

[2] CARDS, “Franchise Plan, Central Archive for Reusable Defense Software,” Tech. Rep. STARS-AC-04116/000/00, Paramax Systems Corporation, March 1993.

[3] STARS, “Draft Reuse Maturity Model: Reuse Strategy Model Prototype,” Tech. Rep. D613-55159, Advanced Research Projects Agency, November 1992.

4 Biography

Ted Davis is the technical lead for the Software Productivity Consortium’s reuse adoption project and is responsible for developing the Reuse Capability Model and Reuse Adoption Process. Previously Ted has supported the development and validation of the Consortium’s Evolutionary Spiral Process for software development and the Synthesis process for reuse-oriented software production. Prior to that, Ted was an officer in the U.S. Air Force for eight years where he developed and acquired command and control systems. Ted has a M.S. in Computer Science from Purdue University and a M.S. in Systems Management from the University of Southern California.

Figure 1: Critical Success Factor Impact Data Davis T- 6