NetNews Usenet Archive 1993 #1/NN_1993_1.iso/spool/comp/software/5270

Internet Message Format | 1993-01-08 | 21.3 KB

Path: sparky!uunet!auspex-gw!mbinesh
From: [e-mail] (Mac Binesh)
Newsgroups: comp.software-eng
Subject: GNU Code Coverage tool - Summary
Keywords: GNU Code Coverage Summary
Message-ID: <[e-mail]>
Date: 9 Jan 93 01:10:59 GMT
Sender: [e-mail]
Lines: 635
Nntp-Posting-Host: auspex.auspex.com

Here is a summary of all email that I have received so far regarding
GNU Code Coverage tool. Thanks to all for their responses.

The first email is from Brian Marick who is the Author of GNU Code Coverage
Test program and the second is the installation procedure.

Mac Binesh
Sr. Software Engineer
Auspex Systems

----------------------------Summary---------------------

From [e-mail] Wed Jan  6 10:45:41 1993
To: [e-mail]
Subject: Re: GNU code coverage tool
Newsgroups: comp.software-eng
Content-Length: 4840
X-Lines: 126
Status: RO

                            GCT 1.4BETA
                            Patchlevel 0
                            Nov 10, 1992

The Generic Coverage Tool (GCT) is a freeware coverage tool that
measures how thoroughly tests exercise C programs.  It can measure the
following:

- how often branches have been taken in either direction. (branch coverage)
- how often cases in switches have been taken.
- how often nested logical conditions have evaluated to true and false
  (multiple condition coverage)
- how often loops have iterated zero, one, and more than one times
  (loop coverage)
- how often the boundaries of relational operators have been probed
  (relational coverage)
- how often routines have been entered
- how often call sites have been exercised
- how often routines have had multiple threads of execution in them
- weak mutation coverage (a research technique).

If you use branch coverage, you will find the combination of branch,
multiple condition, loop, and relational operator coverage to be more
effective at little extra cost.  Routine and call coverage are useful
for system testing.  Race coverage is useful for system testing of
multiprocessor systems.

The tool has been widely used in production environments, including
the UNIX kernel.

MECHANISM

The tool takes C source code, adds instrumentation, then uses the
system C compiler (or a compiler of your choice) to produce an object
file.  Alternately, the instrumented source can be captured and moved
to another machine for compilation.

GCT is designed to work with existing makefiles (or other system build
mechanisms).  Most makefiles will require no changes.

For efficiency's sake, coverage information is stored in an internal
log.  The log is written on exit (or, in the case of an unending
program like the UNIX kernel, extracted from the running program).

Instrumented programs run slower than the original programs.  How much
slower depends on the program, how much of it is instrumented, and
what types of coverage are measured.  An instrumented GCT runs 44%
slower than normal.  (I instrument GCT to measure branch, loop,
multiple condition, and relational coverage.)

Reporting tools produce detailed reports on missing coverage, summary
reports for the whole program, and per-routine and per-file summaries.

NEW FEATURES

This is a list of changes from GCT 1.3:

- GCT now behaves like a true compiler. After instrumenting,
  it calls the C compiler.  The original source is never touched.
  The old behavior is retained as an option.
- GCT can insert gct_readlog and gct_writelog calls for you.
  You needn't edit the source. Original behavior is retained as an option.
- More flexible reporting tools (including per-file and
  per-routine reports from greport and gsummary)
- Better control over the interface to different C compilers.
- More flexibility in editing (suppressing, ignoring) mapfile entries.
- Previously unsupported ANSI C constructs now supported
- Optional locking of logfiles
- Numerous minor usability improvements


RETRIEVING GCT


GCT is available via anonymous FTP from cs.uiuc.edu.  The current
version is in the directory pub/testing/gct.files.  To retrieve GCT,
go to that directory and fetch one of these files:  ftp.ps
(postscript) or ftp.txt (formatted to be read online).  Follow those
instructions.


DEFICIENCIES

1.  The tool itself runs only on UNIX, although the instrumented code
may be compiled wherever you like.  (An earlier version of GCT also
ran on Apollo Aegis SR10; porting this version is probably trivial.)

2.  The tool is based on the GNU C compiler and still contains the
original code generation and optimization code.  This has two
practical disadvantages:

- The tool is large.  For the full installation (which includes the
source, compiled executables, a large test suite, and all the
documentation), you will need roughly 33,000 1K blocks.
(The tests - which are not required - take up about 17,000 blocks.)
The final executables will require roughly 3000 1K blocks.

- The original compiler is made portable through machine and system
configuration files.  These must still be used.  The distribution
includes configuration files for many different machines.  If there
isn't one for your machine, you may be able to use one for a similar
machine.


MAILING LIST

[e-mail] is a mailing list for GCT users.  It is used to
distribute bug reports, bug fixes, troubleshooting hints, proposed
major and minor revisions, documentation errata, and anything else of
interest to GCT users.  Mail to [e-mail] will get you
added to the list.

ADVERTISING

I provide GCT as a public service - and as a concrete demonstration of
my skills.  I provide testing tools, training, and consulting.
Contact me at:

Brian Marick
Testing Foundations
809 Balboa
Champaign, IL  61820
[phone]
[e-mail], testing![e-mail]

----------------------------------------------------------------------------

From [e-mail] Thu Jan  7 05:33:32 1993
To: [e-mail]
Subject: Re: Gnu code coverage tool
Newsgroups: comp.software-eng
Organization: Tampere University of Technology
Cc:
Content-Length: 15782
X-Lines: 479
Status: RO

In article <[e-mail]> you write:
>I am looking for any information or leads on how to
>get an evaluation copy of a GNU code coverage test
>tool that might exist out there.

Here is a GCT.README-file.

                        GCT 1.3 Patchlevel 2
                        July 19, 1992
INTRODUCTION

This file describes how to install the Generic Coverage Tool, GCT.

GCT measures how thoroughly tests exercise C programs.  It measures
branch coverage, loop coverage, multi-condition coverage, relational
operator coverage, and several other types of coverage.

GCT is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 1, or (at your option)
any later version.

GCT is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.


There is a GCT mailing list:  [e-mail].  To join the
list, send mail to [e-mail].

Please send comments about these installation instructions - or any
part of GCT - to me.  My address is given below.

Enjoy.

    Brian Marick


ADVERTISEMENT

In addition to this free software, I also offer services that you can
pay for.

1. GCT support and training.

2. On-site training courses in several testing topics.

3. Custom toolsmithing.

3. Consulting.

I plan major improvements to GCT: C++, better support for large,
rapidly changing systems, support for powerful and flexible data
analysis, support for reusable test components, porting to non-UNIX
systems, and so on.  Funding for future versions cheerfully accepted;
benefits to you include support, getting what you need when you need
it, and designs that closely match your needs.

For more information, contact

    Brian Marick
    Testing Foundations
    809 Balboa
    Champaign, Ill.  61820
    [phone]
    [e-mail]


INSTALLATION

GCT comes in three parts:
    The source.  You need this.
    The documentation.  You almost certainly want this.
    The test suite.  Since a small installation test suite is
        bundled with the source, you don't need this.
        However, it's wise to run the full test suite if you're
        installing on a new type of machine.

Read this whole section before starting.  If you're installing GCT
just for evaluation, to see if it's worth spending more time on, you
can skip the steps marked with (*).  After you go through the GCT
Tutorial, you can come back and finish the complete installation.

1. Make a directory for the source.  The name is unimportant; "gct" is
a good choice.  Change to that directory.

You will need roughly 30,000 1K blocks during the installation
process.  If you don't install the test suites, you'll save 16,000
blocks.  The documentation requires less than 1000 blocks.

The final executables will require roughly 5000 1K blocks.

2A.  (If you can fetch GCT via anonymous FTP.)

Log in to cs.uiuc.edu as user "anonymous".  Give your email address as
the password.  Change directory to /pub/testing/gct.VERSION, where
VERSION is the version of GCT you want.  Look at the directory, and
you will see these files:

    src.tar.Z     the source
    doc.tar.Z    the documentation
    tests.tar.Z    the tests

These are compressed tar files.  You may also see files like

    src.patch.1
    doc.patch.1

and so on.  These are bug fixes and minor enhancements that do not
justify an entirely new release.  See the file PATCHES in the
directory for a description of the patches.  See the patch(1) manpage
to find out how to apply patches.  (If there is no patch(1) on your
system, you can get it from various anonymous FTP repositories.)
Smaller patches can also be applied by hand.

Finally, you may see files like

    notes.machinename

These are files for the config.notes directory (see below) which
arrived after the distribution was made.

Fetch one or more of the compressed tar files.  Remember to transfer
in binary mode.

Type
    zcat src.tar.Z | tar xf -

This installs the source.  If you see error messages like

    src/: cannot create

and you're on System V, use "tar xof -" instead of "tar xf -".

If you fetched the documentation, type

    zcat doc.tar.Z | tar xf -

to install it. If you fetched the test suites, type

    zcat tests.tar.Z | tar xf -


You may also fetch and install the patches.  (Patches 1 and 2 have
already been installed.)

2B. (If you're installing GCT from tape)

Type

    tar xvf /dev/nrst1

This installs the source.  Replace "/dev/nrst1" with the appropriate
non-rewinding tape device on your machine.  If you see error messages
like

    src/: cannot create

and you're on System V, use "tar xof -" instead of "tar xf -".

If you want the documentation, type

    mt -f /dev/nrst1 fsf 1
    tar xvf /dev/nrst1

to retrieve the next file.  If you don't want the documentation, skip
past it with

    mt -f /dev/nrst1 fsf 2

or the equivalent on your machine.

If you want the test suites, type

    tar xvf /dev/rst1

Note that the rewinding tape device was used.  Otherwise,

    mt -f /dev/rst1 rewind

3. Look in the "config.notes" subdirectory of "src".  It describes
special-case installation procedures for particular systems, such as
the Apollo.

4. Change to the "src" directory and edit "Makefile".

4.1. Change the macro called "bindir" to be the directory where you
want the executables to go.

4.2. Change the macro called "libdir" to be the directory where you
want libraries and gct's C preprocessor to go.

4.3. Change the macro called "mandir" to be the directory where you
want manpages to go.  Also change "manext" to be the same as the
last letter of "mandir".

4.4. By default, the makefile installs the programs and manual pages using
"cp".  On BSD-derived systems, you may want to change the
"INSTALL" macro to "install -c".  On some System V systems,
you may want to use "cpset".

4.5. A useful GNU emacs utility, gedit.el, is by default installed in
$(libdir).  You may want to install it with other local emacs utilities.
Search for "gedit.el" in the Makefile and change what you find.

5. Configure the system for your machine type and operating system.
This is done by typing

    config.gcc TYPE

The types are listed in the file called INSTALL.  Common values
are "sun4-os4" for Sparc systems running SunOS 4.x, "sysV68" for
Motorola machines running V68R5 or V68R6, "sysV88" for Motorola
88K machines running R32, or "sysV88R4" for Motorola 88K machines
running System V Release 4.

If your system is not listed in INSTALL, try a similar system.  GCT's
configuration files are those of the GNU C compiler.  GCT actually
uses very little of that configuration information.  So, one
configuration file is often as good as another, if they both describe
roughly the same version of UNIX.  For example, the configuration file
for a VAX running BSD Unix can be used to produce a GCT for a Sun 3
running SunOS 3 (though there is also a sun3-os3 configuration file).

The "src/config.notes" files may help.  If they didn't, and you get
GCT working, tell me what the next release's config.notes should say.

6.  Type

    make

7 (*).  Head off a possible system-dependent problem that shows up
when processing functions that take a variable number of arguments.
These functions use the standard C macros va_start(), va_arg(), and
va_end(), defined in /usr/include/varargs.h.  These macros are
sometimes of the form

#ifdef DEFINITION_PREDEFINED_BY_THE_SYSTEM_SUPPLIED_C_COMPILER
#define va_arg(list, mode) ... something that works...
#else
#define va_arg(list, mode) ... something that doesn't work...
#endif

For example, on Sun 4 systems, va_arg() is defined incorrectly unless
__BUILTIN_VA_ARG_INCR is defined.

To find out if you have this problem, change to the varargs
subdirectory.  Then execute cc-var as follows:

    cc-var initial

If you see

    "Normal" function arguments (integers, doubles, and pointers) work.
    Structure arguments to functions work.
    Coverage results are correct.

continue to the next step.  Otherwise,
    1. Examine /usr/include/varargs.h to discover what the #defines are.
    2. Edit the src/tm.h file (this is a symbolic link to the config/tm-*.h
       file for your machine).
    3. Add the necessary #defines to CPP_PREDEFINES.
    4. Remake GCT.
    5. Mail what you did to me so I can include it in the standard
       distribution.  (I've already done this for sun4-os4.)

If <stdarg.h> exists on your system, the same fix probably applies.
We'll check that later in the installation.

8 (*).  If you retrieved the test suites, type

    cd testdir
    run standard

The tests take a long time to run.  (An hour is typical.) As each test
runs, it will log results.  If a test logs nothing but its name, it
succeeded. A portion of a successful log looks like this:

==  branch/
====  quest
====  if
====  switch
====  switch2
====  switch3

You may also run the weak mutation test suite by typing

    run weak

Note, however, that programs instrumented for weak mutation tend to be
a challenge to older C compilers, so you may see spurious failures.
The tests may also be in error; it's easy to write unportable weak
mutation tests.  Please report any failures you see, and I'll tell you
if they're a real problem.

9.  If you are satisfied with the results of the tests, return to the
"src" directory and type

    make install

To install the manpages, type

    make man_install

(The manpages are in the "doc" tar file, so this won't work if you
didn't extract that.)

10.  After installing, run the installation test suite.  It checks that
GCT was installed correctly and that it seems to be working.  Make
sure that the directory where you installed GCT is in your $PATH, then

    cd testdir
    run install

You should see

    ==  installed/
    ==  multiple-dir/
    ==  demo/
    ==== basic-gsummary
    ==== basic-greport
    ==== basic-all
    ==== greport-addition
    ==== gedit
    ==== gedit-gsummary
    ==== gedit-greport

11. To print the documentation, go to the "gct/doc" directory.  The
documents are in these directories:

tutorial    A Tutorial Introduction to GCT
user-guide    Generic Coverage Tool (GCT) User's Guide
trouble        GCT Troubleshooting
race        Using Race Coverage with GCT
weak-mutation    Using Weak Mutation Coverage with GCT
manpages    The manual pages (already installed)

You will want at least the first three documents.  Postscript files
are in each of the directories; they end in ".ps" (tutorial.ps, for
example).  Plain-text files end in "txt".  You can also format the
source, which is written in either the MS or ME nroff macro sets.
You'll probably have to change the file doc/include/Make.tmpl.

12 (*).  If users instrument programs containing functions with a
variable number of arguments, then compile them with the GNU C
compiler (gcc), they may run into problems.  By default, GCT looks for
<varargs.h> in /usr/include.  The definition of va_arg() in that file
may be incompatible with gcc, which has its own private copy of
<varargs.h>.  The solution is to tell GCT to use gcc's <varargs.h>.
This could be done with GCT's -I option (which works just like a C
compiler's -I option).  But that would result in too many makefiles
having the location of gcc's include files hardcoded into them.
They would break when gcc was moved, or a new version was installed,
or the makefile was moved to a new machine.

I'll describe below how you'll know if you have a problem.  If you do,
you have two possible solutions:

Choice 1:  Tell gcc users that they must warn GCT that they'll compile
with gcc by invoking GCT as

    gct -test-cc gcc

This causes GCT to look for include files in $(libdir)/gct-include/gcc
before looking in /usr/include.  ($(libdir) was defined in step 4
above.)  To arrange for the right files to be accessible through that
directory, type something like this:

# ln -s $(GCC_INCLUDE) $(libdir)/gct-include/gcc

replacing $(libdir) with whatever you chose in step 4 and $(GCC_INCLUDE)
with the appropriate directory on your machine.  On my development
machine, that could be either

    /local/lib/gnu/gcc-lib/sparc-sunos4.1/2.1/include     (gcc version 2.1)
or
    /local/lib/gnu/gcc-include                (gcc version 1.40)

(Note:  if your system doesn't support symbolic links, copy the
contents instead.)

Choice 2: If everyone who'll use GCT will also use gcc, you can make
gcc the default compiler.  Do this with

# ln -s $(GCC_INCLUDE) $(libdir)/gct-include/default

GCT always looks in gct-include/default before /usr/include.  It normally
doesn't exist, but now you've made it identical to gcc's private include directory.

Of course, anyone who uses cc will now likely be unable to instrument
programs that use <varargs.h>.  However, if you type

# ln -s /usr/include $(libdir)/gct-include/cc

cc users can instrument varargs-containing programs with

    gct -test-cc cc

12.1.  To determine if there's a problem on your machine, use
the 'run' script in the src/varargs directory.
Type

    run

and follow the instructions.  If all is well, you're finished with the
installation.  (NOTE:  Scripts like 'run' are hard to write well,
especially when you can't try them on a lot of machines.  Please tell
me if you have problems or suggestions.)

'run' tries these combinations:

- Instrumentation of code that uses <varargs.h>, using -test-cc if
  desired, followed by compilation by gcc.
- Instrumentation of code that uses <stdarg.h> (the ANSI replacement
  for <varargs.h>), using -test-cc if desired, followed by compilation by gcc.
- Instrumentation of code that uses <varargs.h>, using -test-cc if desired,
  followed by compilation by cc.
- Instrumentation of code that uses <stdarg.h>, using -test-cc if desired,
  followed by compilation by cc.  Note:  in most cases, cc is not an
  ANSI compiler, so expect this to fail.

You can experiment with changes to $(libdir)/gct-include to see their
effect.  Pick the configuration that will be most useful to your
users.  If you have compilers other than 'cc' and gcc, you can use the
run script to find out if they work, and perhaps set up links in
gct-include that will make them work.

12.2.  When you finish configuring GCT, run 'run' one last time.
It will produce a file called 'gct-varargs', which summarizes for
users what they need to know.

If there are cases that DON'T work (other than compiling a
<stdarg.h>-using program with cc, which usually would fail irrespective
of instrumentation),
    1.  Copy gct-varargs to $(libdir).
    2.  Make sure everyone can read it.
    3.  Type 'gct -test-varargs' to make sure it was installed correctly.

When GCT encounters a function with a variable-length argument list,
it prints a warning, but only if gct-varargs is installed.

INSTALLATION TROUBLESHOOTING

Installation problem after copying source

Symptoms:

The tar file was read and then the source was copied to another
directory or another machine.  The makefile fails with a message like
"Cannot load command 'bison'".

Solution:

Two C files, cexp.c and c-parse.tab.c, are created using the BISON
parser-generator.  The original source files are cexp.y and c-parse.y.
In the tar file, the C files are newer than the Y files, so that the
makefile doesn't attempt to recreate them.  If you copy the btool
sources around, the dates on the files will change, and the makefile
may attempt to recreate the C files from the Y files.  Unless you
have BISON on your system, the make will fail.  The solution is to
touch the C files after the copy:

    % cp -r * $destdir
    % cd $destdir
    % touch cexp.c c-parse.tab.c
    % make



- - - - - - - - -

    Tensu

--
Tero Ahtee * [e-mail]  *  Tampere  *  Finland  *  Europe
Login ergo sum.   <>   Workload is not killing - it is challenging.
COMMERCIAL : Why should you pay less for a better computer - buy IBM !