                       OVERVIEW OF U.S. AIR FORCE REPORT
                     Ada AND C++: A BUSINESS CASE ANALYSIS

Form G75-1191
AdaCPLUS.HLP

Ada Information Clearinghouse, 1-800-AdaIC-11, 1-703-685-1477

On July 9, 1991, the Air Force released to the public a report of a business 
case they conducted to determine under what circumstances a waiver to the DoD 
Ada requirement might be warranted for use of C++, particularly in DoD's 
Corporate Information Management (CIM) program.  The report is titled, "Ada 
and C++: A Business Case Analysis."  

Since its release the report has received a great deal of publicity in various 
newspapers and journals.  The information that follows was excerpted from 
remarks made by Mr. Lloyd K. Mosemann, II Deputy Assistant Secretary of the 
Air Force (Communications, Computers, and Logistics), at a press conference 
held July 9, 1991.

I.  Introduction

There has never been any intention to question DoD's commitment to Ada, but 
only to identify when waivers for C++ might be warranted.  This business case 
will support the development of DoD programming language policy for 
information systems and C3 systems.

I might say at the outset that language comparison is not merely a scientific 
issue: it evokes strong emotions as well, in that to a certain extent people 
adopt "favorite" languages for reasons other than purely dispassionate 
analysis, much as one might not be able to explain why he/she roots for the 
Chicago Cubs or drinks Coke rather than Pepsi.  The task is also rendered 
difficult because there are yet no well-established and standard methods for 
conducting such comparisons.  For these reasons, we endeavored to make our 
study as quantitative as possible, asking the best experts we could find to 
use a variety of methods that have historically been used for business 
analysis in such tasks.  We felt that by using a variety of methods and 
comparing their results, we would avoid the skewing that might result from the 
sole use of a single method.

In our business case, therefore, several different approaches were undertaken 
to identify, from a business perspective, when the life cycle cost 
effectiveness of C++ might be greater than that of Ada.

--    The first, conducted by the Institute for Defense Analyses (IDA), 
      examined quantitatively the availability of tools and training for the 
      two languages.

--    The second, conducted by the Software Engineering Institute, applied to 
      this problem a quantitative language selection methodology developed by 
      IBM for the Federal Aviation Administration (FAA).

--    The third, conducted by CTA, Inc., analyzed cost and cost analysis).

--    And the fourth, conducted by the TRW Corporation, applied a standard 
      cost model in depth to both languages for a typical information 
      systems/C3 project (micro analysis).

--    In addition, the Naval Postgraduate School (NPS) was asked to address 
      the overall policy issue of Ada, particularly in the context of emerging 
      fourth-generation language (4GL) software technology.

Each of the substudies reached the same conclusion: there are no compelling 
reasons to waive the Ada requirement to use C++.

The business case analysis was directed at information systems and C3 systems.  
However, there is no reason to believe the results would differ for computer 
programs embedded in weapons systems.

Let me now summarize for you the salient quantitative results of each study, 
and I think you will understand more fully how we arrived at our conclusion.

II.  Substudy Results.

A.   Tools, Environments, and Training: IDA Substudy.

The Institute for Defense Analyses (IDA) collected and analyzed information on 
the market availability of commercial- off-the-shelf products available from 
U.S. sources for Ada and C++ compilers, tools, education, and training.
 
The study provided a large quantity of demographic data. For example, there 
are 28 companies located in the U.S. that have Ada compilers with currently 
validated status; 18 vendors offer C++ compilers.  The Ada compiler vendors 
are more likely to have been in business five years or more.  Ada "validation" 
is more rigorous than that of other high order languages: only Ada is 
monitored and approved for conformity to a standard, without supersets or 
subsets, by a government-controlled process.  By contrast, no validation or 
even a standard of any kind exists for C++, although a standard by 1994 is 
expected.

Both languages are supported on PCs and workstations.  Ada is also supported 
on mainframes.  Ada, but not C++, has cross compilation systems.
 
Ada is supported with program engineering tools.  Compiler vendors provide a 
rich set.  Code generators exist for Ada but none so far for C++.  There is 
considerable variability among C++ products in language features supported and 
libraries provided.  

Ada is taught in 43 states at 223 universities and 13 DoD installations.  C++ 
is taught in four states at four universities and no DoD installations.  There 
are more Ada than C++ courses available.  The cost of training is about equal, 
but Ada course variety is wider.

B.   Faceted IBM Language Selection Methodology: SEI Substudy.

The Federal Aviation Administration (FAA) contracted with IBM in the mid-1980s 
to evaluate high order languages for use on its Advanced Automation System 
(AAS) Program.  In response, IBM developed a formal, quantitative faceted 
methodology comparing 48 language features (criteria) in six categories.  This 
IBM study concluded that use of Ada was "in the ultimate best interest of the 
AAS program and its goals, and that warrants coping with the temporary 
risks/problems that loom large in the near term in order to reap the 
significant benefits/payoffs over the long term."


Using this same methodology for each of the 48 criteria, the Software 
Engineering Institute (SEI) evaluated Ada and C++ for application in 
information systems/C3 systems.  The original FAA weighted scores for the six 
criteria categories were as shown in this matrix:

      Category       Max.  Ada    C   Pascal  JOVIAL  FORTRAN
                     Score

      Capability      16.7  6.1   9.6   10.4     7.6     3.9
      Efficiency      16.4  8.0  11.8   10.8    11.0    11.1
      Availability/   22.6 21.5  11.6   14.5    15.6    10.3
        Reliability
      Maintainability/17.4 14.0  10.2   12.2     6.8     8.3
        Extensibility
      Lifecycle cost  11.3  8.2   7.4    7.8     4.9     5.2
      Risk            15.6  8.8   8.9    7.6     9.6     8.2

      Total          100.0 76.6  59.5   63.3    55.5    47.0

The 1991 weighted scores for the six criteria categories were:

      Category             Max. Score  Ada   C++

      Capability               16.7   15.3   11.3
      Efficiency               16.4   10.7   10.9
      Availability/Reliability 22.6   19.1   12.6
      Maintainability/Extensibility17.413.6  11.4
      Lifecycle cost           11.3    8.4    8.0
      Risk                     15.6   11.7    9.8

      Total                   100.0   78.8   63.9

In 1985 Ada was considered considerably more capable than C. Today, the SEI 
study found there is still a significant difference between Ada and C++, C's 
successor.  The relative efficiency of Ada has improved markedly; Ada still 
scores significantly higher in availability/reliability; the Ada advantage in 
maintainability/extensibility persists; and from a position of parity in 1985, 
Ada has attained in 1991 a significant advantage over C++ in lowered risk.

An attachment lists numerous major Ada information systems/C3 systems.  It is 
not widely appreciated that such extensive use is now being made of Ada: in 
fact, the Ada 9X Project Office reports that the U.S. Ada market, excluding 
training, services, and government research/development, now exceeds $1 
billion.

C.   Macro Cost Analysis: CTA Substudy.

CTA compiled and compared available productivity and cost data of Ada and C++.  
Much of their data comes from Reifer Consultants' extensive database, one of 
the best, largest, and most current programming language cost databases now 
available.


Average productivity across the four domains for which data exists 
(environment/tools, telecommunications, test (with simulators) and other) for 
both Ada and C++ projects is shown in this matrix.  Note the productivity 
advantage for Ada: 

                        Productivity   Number of
                          (SLOC/MM)   Data Points
      *  Norm (all languages) 183         543
      *  Average (Ada)        210         153
      *  Average (C++)        187          23
      *  First project (Ada)  152          38
      *  First project (C++)  161           7

The C++ project data reflected information on 23 projects taken from seven 
firms who had been using C++, Unix, and object-oriented techniques for over 2 
years.  All projects were new developments.  Application size ranged from 25 
to 500 KSLOCs (thousand source lines of code).  Average size was about 100 
KSLOC.

The average costs across the four domains for both Ada and C++ projects are 
shown in this matrix.

                            Cost       Number of
                          ($/SLOC)    Data Points
      *  Cost (all languages)$ 70         543
      *  Average (Ada)         65         153
      *  Average (C++)         55          23

Typically, the Ada developments were performed in accordance with military 
standards and incorporated formal reviews, additional documentation, and 
additional engineering support activities such as formal quality assurance 
(QA) and configuration management (CM).  Most C++ projects are commercial and 
do not extensively incorporate such activities.  Additionally, on such 
projects developers are typically intimately involved with users, resulting in 
considerably less requirements engineering effort.  Consequently, applications 
on which C++ is used are inherently less costly, so that the reported 
productivity rates are favorably skewed toward C++.

The average error rates across the four domains for both Ada  and C++ projects 
were: 

                       Integration       FQT        Number of
                       Error Rates   Error Rates      Data
                     (Errors/KSLOC)(Errors/KSLOC)    Points
      *  Norm (all languages)33           3            543
      *  Average (Ada)     24             1            153
      *  Average (C++)     31             3            23

The integration error rates include all errors caught in test from start of 
integration testing until completion of software Formal Qualification Test 
(FQT).  The FQT error rate includes only those errors found during the FQT 
process. 

A so-called "transition state analysis" performed by Reifer's group indicates 
that 26 of the 38 firms within the Ada database had successfully made the 
changeover to effective use of Ada, while none of the 7 firms in the C++ 
database had made the transition.  Also, none of the 7 firms were fully using 
C++'s inheritance and other advanced features.

The standardization maturity of Ada was found by the CTA to be particularly 
important.  While Ada has a firm and well policed standard, allowing neither 
supersets nor subsets, it will be years before a stable C++ language 
specification is established.  New features are being considered for the 
latest standard C++ release.  Vendors are likely to offer their own enhanced 
versions of C++ compilers and CASE tools, complicating portability and reuse.

Finally, the original arguments for establishing a single programming language 
for military applications were found to remain.  Common training, tools, 
understanding, and standards simplify acquisition, support, and maintenance.  
The study concluded that after maturing for a decade, Ada's benefits have been 
proven for all application classes.  Ada projects have reported 15% higher 
productivity with increased quality and double the average size.  Normalizing 
these data to comparable size projects would result in an expected Ada 
productivity advantage of about 35%.  Ada should be the near term language of 
choice.  C++, the study felt, still needs significant maturing before it is a 
low risk solution for a large DoD application. 

D.   Micro Cost Analysis: TRW Substudy.

TRW performed a tradeoff analysis that generalized recent corporate cost 
analyses on a typical real-world information systems/C3 systems project.  
Their study defined a set of maximally independent criteria, judged each 
language with respect to those criteria, and then translated those judgments 
into cost impacts to emphasize the importance of each criterion from a 
lifecycle cost perspective.  Results were translated into perturbations of 
Boehm's Ada COCOMO cost model.


Rankings of the two languages based on this analysis are shown in this matrix 
(0 = no support; 5 = excellent support), followed by a total score, a weighted 
sum of the rankings based on weights determined by an expert panel:

                                               Ada     C++
Reliable S/W Engineering                       4.5     3.2
Maintainable S/W Engineering                   4.4     3.2
Reusable S/W Engineering                       4.1     3.8
Realtime S/W Engineering                       4.1     2.8
Portable S/W Engineering                       3.6     2.9
Runtime Performance                            3.0     3.6
Compile-Time Performance                       2.3     3.1
Multilingual Support                           3.1     2.4
OOD/Abstraction Support                        3.9     4.6
Program Support Environment                    4.1     2.1
Readability                                    4.4     2.9
Writeability                                   3.4     3.5
Large Scale S/W Engineering                    4.9     3.3
COTS S/W Integration                           2.8     3.6
Precedent Experience                           3.6     1.5
Popularity                                     2.8     4.0
Existing Skill Base                            3.0     1.8
Acceptance                                     2.5     3.3

TOTAL SCORE FOR MGT INFO SYSTEMS              1631    1324
    (Ada score is 23% higher)

TOTAL SCORE FOR C3 SYSTEMS                    1738    1401
    (Ada score is 24% higher)

The study concluded that both Ada and C++ represent improved vehicles for 
software engineering of higher quality products.  Currently, C++ was estimated 
to be approximately 3 years behind Ada in its maturity and tool support.  The 
case study used in this report (the Command Center Processing and Display 
System-- Replacement) demonstrated development cost advantages for Ada on the 
order of 35% and maintenance cost advantages for Ada on the order of 70% under 
today's technologies.  In the far term (1994+), the study felt, this Ada 
advantage might erode to approximately a 10% advantage in development costs 
and 30% in maintenance costs for a typical development intensive system.

The study listed the primary strengths of Ada as its support for realtime 
domains and large scale program development.  Its primary weaknesses are its 
compile-time and runtime efficiency.  The primary strengths of C++ listed were 
its support for better object oriented design, support for COTS integration, 
and its compile-time and runtime efficiency.  Its main weaknesses were 
identified as its support for reliability and large scale program development.  
In general, the study felt Ada's weaknesses to be solved by ever-increasing 
hardware performance and compiler technology advancement.  C++ weaknesses, on 
the other hand, remain to be solved by advances in its support environment.


E.   Ada Policy Issues: NPS Study.

Concurrently with the preparation of this Ada and C++ Business Case Analysis, 
the Naval Postgraduate School (NPS) reported on policy issues on the use of 
Ada for Management Information Systems.  Their report, an analysis of the need 
to see Ada in a total and evolving context, is an important vision statement 
leading from Ada as the primary third-generation language (3GL) to its 
conception as the basis for evolving to higher levels of productivity in 
so-called 3 1/2 GL and 4GL environments.

Rather than concentrating on programming language selection, the NPS report 
focuses on and argues for needed advances in software development technology.  
In particular, the Report contends, while traditional factors such as 
programming language selection, better training, and computer-assisted 
software engineering (CASE) tools can enhance productivity modestly, a 
fundamental change in the software development paradigm will be necessary to 
achieve an order of magnitude gain.  Such a gain is possible through use of 
4GLs, languages that will ultimately enable the developer to define the 
complete design of an application entirely in the 4GL's own high-level 
specification language.  The specification is then translated automatically by 
the 4GL into an executable program.  When accompanied by a productive 
development environment, an evolutionary implementation methodology, and well 
trained development teams, the report asserts, 4GLs can provide a tenfold gain 
in productivity.

An intermediate step cited by the report in the movement to 4GLs is 3 1/2 GL 
programming, a term referring to the extensive use of CASE tools coupled with 
a high level of code reuse.  The 3 1/2 GL approach requires a strong 
commitment to codifying and accrediting code modules, to the point where it 
becomes easier and more desirable to reuse code than to rewrite it.

Although experience with 4GLs has not yet been extensive (with existing 
experience limited largely to specific functional domains such as financial 
management and transaction processing), 4GLs are attractive for several 
reasons.  One is their robustness under change: changes made to the 
application, for whatever reason, are made at the specification level and then 
re- translated automatically into executable code.  Another is the facility 
with which they can be integrated into tightly knit and full-featured 
development environments.  For these reasons, the report strongly recommends 
that the DoD discourage use of traditional 3GL programming and take bold steps 
to incorporate the 4GL paradigm.

Finally, the report recommends that, given the importance of Ada to DoD 
software, greater effort and funding should be provided for the key Ada 
initiatives: the Ada Technology Improvement Program, Ada 9X, and Ada education 
initiatives.

Two issues on 3 1/2 GLs and 4GLs related to this business case were outside 
the scope of the NPS report.  The first of these is that, for the foreseeable 
future, state-of-the-art limitations will probably keep 4GLs from generating 
more than half the total code required by most applications.  In such cases, 
where a substantial amount of 3GL programming will be required to complete 
application development, use of a 3 1/2 GL approach, rather than a 4GL 
approach, is preferable.

Another issue outside the scope of the NPS Report was the evaluation of the 
relative merits of Ada and C++ as target (output) languages for 4GL 
application generators.  However, as section V.C of the NPS report points out, 
a "standard, stable target language portable to a variety of hardware 
platforms" with good software reuse and interface definition capabilities is 
appealing.  Although more study of the characteristics desired in 4GL target 
languages is warranted, the SEI and TRW substudies suggest no particular 
advantage of C++ over Ada in software reuse and interface definition, so there 
appears no reason to waive DoD's Ada requirement in favor of C++ as a target 
language for 4GLs.

3.  Conclusions.
 
All four substudies which specifically compared Ada and C++ (IDA, SEI, CTA, 
TRW) report a significant near term Ada advantage over C++ for all categories 
of systems.  This advantage could be eroded as C++ and its supporting 
environments mature over the next few years.  On the other hand, as aggressive 
overseas Ada initiatives stimulate even wider domestic Ada interest, as Ada 
tools/environments further mature, and when the Ada update (Ada 9X) is 
complete, the balance could tip further in Ada's favor.

Adding to the case for Ada is the fact that the Ada scoring so well in the 
business case was Ada's 1983 version, MIL-STD-1815A.  Just as C++ incorporates 
into C certain software engineering concepts already in Ada (e.g., modularity, 
strong typing, specification of interfaces), so an Ada update now underway 
will bring into Ada selected features now included in C++.  This update, known 
as the Ada 9X Project, is targeted for completion in 1993.  The product of 
extensive community involvement (including the C3 and MIS communities), Ada 9X 
will bring to Ada such improvements as decimal arithmetic, international 
character sets, improved input/output, support for calls between Ada and other 
languages, further representation specifications, and inheritance/polymorphism 
(popular features of C++).  The Ada 9X Project Office lists one of the goals 
of Ada 9X as "to provide all the flexibility of C++ with the safety, 
reliability, and understandability of Ada 83."

At the same time, Ada 9X has been designed so that neither existing Ada 
benefits nor performance will be lost.  For example, Ada 9X inheritance will 
be controlled so as not to reduce lifecycle supportability.  Some have 
criticized OOP features such as inheritance as potentially dangerous to DOD 
software mission goals (such as safety, reliability, and dependability).  
Bjarne Stroustrup himself, the originator of C++, has been quoted as follows: 
"C makes it easy for you to shoot yourself in the foot.  C++ makes that 
harder, but when you do, it blows away your whole leg."

In summary, it is not possible to make a credible case for the existence of 
classes of "more cost effective" C++ systems compared to Ada.  Business cost 
effectiveness data collected for this study are typified by the TRW study's 
conclusion that Ada provides development cost advantages on the order of 35% 
and maintenance cost advantages on the order of 70%.  In terms of full 
lifecycle costs, it will be some time before data exists which could justify a 
cost savings for C++.  Today, there is limited lifecycle data available for 
Ada and almost none for C++.

For the foreseeable future, then, this business case shows that there are more 
than enough reasons for the DoD to stick firmly with Ada, both for all high 
order language (3GL and 3 1/2 GL) development and for exclusive use as a 
target language of 4GL application generators in the large class of 
applications for which 3GL code must supplement generated code.

"Ada and C++: A Business Case Analysis" is available in hard copy form from 
Software Technology Support Center, OOALC/TISAC, Attn: Gary Peterson, Hill 
AFB, Utah 84056, DSN 458-7703, 801/777-7703.  It is available in electronic 
form in the ada-info directory on the AJPO machine on the Defense Data Network 
and on the AdaIC electronic bulletin board (703) 614-0215 or (301) 459-3865.


