home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
OS/2 Professional
/
OS2PRO194.ISO
/
os2
/
prgramer
/
unix
/
doc
/
cvs13.doc
< prev
next >
Wrap
Text File
|
1993-01-17
|
53KB
|
1,057 lines
CCVVSS IIII::
PPaarraalllleelliizziinngg SSooffttwwaarree DDeevveellooppmmeenntt
_B_r_i_a_n _B_e_r_l_i_n_e_r
_P_r_i_s_m_a_, _I_n_c_.
_5_4_6_5 _M_a_r_k _D_a_b_l_i_n_g _B_l_v_d_.
_C_o_l_o_r_a_d_o _S_p_r_i_n_g_s_, _C_O _8_0_9_1_8
_b_e_r_l_i_n_e_r_@_p_r_i_s_m_a_._c_o_m
_A_B_S_T_R_A_C_T
The program described in this paper fills a
need in the UNIX community for a freely available
tool to manage software revision and release con-
trol in a multi-developer, multi-directory, multi-
group environment. This tool also addresses the
increasing need for tracking third-party vendor
source distributions while trying to maintain
local modifications to earlier releases.
11.. BBaacckkggrroouunndd
In large software development projects, it is usually
necessary for more than one software developer to be modify-
ing (usually different) modules of the code at the same
time. Some of these code modifications are done in an
experimental sense, at least until the code functions cor-
rectly, and some testing of the entire program is usually
necessary. Then, the modifications are returned to a master
source repository so that others in the project can enjoy
the new bug-fix or functionality. In order to manage such a
project, some sort of revision control system is necessary.
Specifically, UNIX1 kernel development is an excellent
example of the problems that an adequate revision control
system must address. The SunOS2 kernel is composed of over
a thousand files spread across a hierarchy of dozens of
directories.3 Pieces of the kernel must be edited by many
-----------
1 UNIX is a registered trademark of AT&T.
2 SunOS is a trademark of Sun Microsystems, Inc.
3 Yes, the SunOS 4.0 kernel is composed of over
-2-
software developers within an organization. While undesir-
able in theory, it is not uncommon to have two or more peo-
ple making modifications to the same file within the kernel
sources in order to facilitate a desired change. Existing
revision control systems like RCS [Tichy] or SCCS [Bell]
serialize file modifications by allowing only one developer
to have a writable copy of a particular file at any one
point in time. That developer is said to have ``locked''
the file for his exclusive use, and no other developer is
allowed to check out a writable copy of the file until the
locking developer has finished impeding others' productiv-
ity. Development pressures of productivity and deadlines
often force organizations to require that multiple develop-
ers be able to simultaneously edit copies of the same revi-
sion controlled file.
The necessity for multiple developers to modify the
same file concurrently questions the value of serialization-
based policies in traditional revision control. This paper
discusses the approach that Prisma took in adapting a stan-
dard revision control system, RCS, along with an existing
public-domain collection of shell scripts that sits atop RCS
and provides the basic conflict-resolution algorithms. The
resulting program, ccvvss, addresses not only the issue of con-
flict-resolution in a multi-developer open-editing environ-
ment, but also the issues of software release control and
vendor source support and integration.
22.. TThhee CCVVSS PPrrooggrraamm
ccvvss (Concurrent Versions System) is a front end to the
RCS revision control system which extends the notion of
revision control from a collection of files in a single
directory to a hierarchical collection of directories each
containing revision controlled files. Directories and files
in the ccvvss system can be combined together in many ways to
form a software release. ccvvss provides the functions neces-
sary to manage these software releases and to control the
concurrent editing of source files among multiple software
developers.
The six major features of ccvvss are listed below, and
will be described in more detail in the following sections:
1. Concurrent access and conflict-resolution algo-
rithms to guarantee that source changes are not
``lost.''
2. Support for tracking third-party vendor source
distributions while maintaining the local modifi-
cations made to those sources.
3. A flexible module database that provides a sym-
bolic mapping of names to components of a larger
software distribution. This symbolic mapping
-----------
a _t_h_o_u_s_a_n_d files!
provides for location independence within the
software release and, for example, allows one to
check out a copy of the ``diff'' program without
ever knowing that the sources to ``diff'' actually
reside in the ``bin/diff'' directory.
4. Configurable logging support allows all ``commit-
ted'' source file changes to be logged using an
arbitrary program to save the log messages in a
file, notesfile, or news database.
5. A software release can be symbolically tagged and
checked out at any time based on that tag. An
exact copy of a previous software release can be
checked out at any time, _r_e_g_a_r_d_l_e_s_s of whether
files or directories have been added/removed from
the ``current'' software release. As well, a
``date'' can be used to check out the _e_x_a_c_t ver-
sion of the software release as of the specified
date.
6. A ``patch'' format file [Wall] can be produced
between two software releases, even if the
releases span multiple directories.
The sources maintained by ccvvss are kept within a single
directory hierarchy known as the ``source repository.''
This ``source repository'' holds the actual RCS ``,v'' files
directly, as well as a special per-repository directory
(CVSROOT.adm) which contains a small number of administra-
tive files that describe the repository and how it can be
accessed. See Figure 1 for a picture of the ccvvss tree.
22..11.. SSooffttwwaarree CCoonnfflliicctt RReessoolluuttiioonn44
-----------
4 The basic conflict-resolution algorithms used
in the ccvvss program find their roots in the origi-
nal work done by Dick Grune at Vrije Universiteit
in Amsterdam and posted to ccoommpp..ssoouurrcceess..uunniixx in
the volume 6 release sometime in 1986. This orig-
inal version of ccvvss was a collection of shell
scripts that combined to form a front end to the
-4-
__________________________________________________
line from 4.112,9.200 to 5.550,8.887
line from 5.447,8.884 to 5.550,8.887 to 5.458,8.933
line from 4.112,9.200 to 4.550,8.950
line from 4.451,8.978 to 4.550,8.950 to 4.476,9.021
line from 4.112,9.200 to 3.737,8.887
line from 3.798,8.971 to 3.737,8.887 to 3.830,8.932
line from 3.612,8.762 to 4.737,8.137
line from 4.638,8.164 to 4.737,8.137 to 4.662,8.208
line from 3.612,8.762 to 3.737,8.137
line from 3.693,8.231 to 3.737,8.137 to 3.742,8.240
line from 3.612,8.762 to 2.612,8.200
line from 2.687,8.271 to 2.612,8.200 to 2.712,8.227
line from 2.362,9.262 to 2.737,8.950
line from 2.645,8.995 to 2.737,8.950 to 2.677,9.033
line from 2.362,9.262 to 1.925,8.950
line from 1.992,9.028 to 1.925,8.950 to 2.021,8.988
line from 3.362,9.762 to 4.050,9.387
line from 3.950,9.413 to 4.050,9.387 to 3.974,9.457
line from 3.362,9.762 to 2.487,9.387
line from 2.570,9.450 to 2.487,9.387 to 2.589,9.404
"newfs.c,v" at 4.487,8.043 ljust
"mkfs.c,v" at 3.487,8.043 ljust
"Makefile,v" at 2.237,8.043 ljust
"newfs" at 3.487,8.793 ljust
"halt.c,v" at 5.487,8.793 ljust
"Makefile,v" at 4.237,8.793 ljust
"modules,v" at 2.487,8.793 ljust
"loginfo,v" at 1.488,8.793 ljust
"etc" at 3.987,9.293 ljust
"CVSROOT.adm" at 1.988,9.293 ljust
"/src/master" at 2.987,9.793 ljust
__________________________________________________
FFiigguurree 11..
ccvvss Source Repository
ccvvss allows several software developers to edit personal
copies of a revision controlled file concurrently. The
revision number of each checked out file is maintained inde-
pendently for each user, and ccvvss forces the checked out file
to be current with the ``head'' revision before it can be
``committed'' as a permanent change. A checked out file is
brought up-to-date with the ``head'' revision using the
``update'' command of ccvvss. This command compares the
``head'' revision number with that of the user's file and
performs an RCS merge operation if they are not the same.
-----------
RCS programs.
The result of the merge is a file that contains the user's
modifications and those modifications that were ``commit-
ted'' after the user checked out his version of the file (as
well as a backup copy of the user's original file). ccvvss
points out any conflicts during the merge. It is the user's
responsibility to resolve these conflicts and to ``commit''
his/her changes when ready.
Although the ccvvss conflict-resolution algorithm was
defined in 1986, it is remarkably similar to the ``Copy-
Modify-Merge'' scenario included with NSE5 and described in
[Honda] and [Courington]. The following explanation from
[Honda] also applies to ccvvss:
Simply stated, a developer copies an object with-
out locking it, modifies the copy, and then merges
the modified copy with the original. This
paradigm allows developers to work in isolation
from one another since changes are made to copies
of objects. Because locks are not used, develop-
ment is not serialized and can proceed in paral-
lel. Developers, however, must merge objects
after the changes have been made. In particular,
a developer must resolve conflicts when the same
object has been modified by someone else.
In practice, Prisma has found that conflicts that occur
when the same object has been modified by someone else are
quite rare. When they do happen, the changes made by the
other developer are usually easily resolved. This practical
use has shown that the ``Copy-Modify-Merge'' paradigm is a
correct and useful one.
22..22.. TTrraacckkiinngg TThhiirrdd--PPaarrttyy SSoouurrccee DDiissttrriibbuuttiioonnss
Currently, a large amount of software is based on
source distributions from a third-party distributor. It is
often the case that local modifications are to be made to
this distribution, _a_n_d that the vendor's future releases
should be tracked. Rolling your local modifications forward
into the new vendor release is a time-consuming task, but
ccvvss can ease this burden somewhat. The cchheecckkiinn program of
ccvvss initially sets up a source repository by integrating the
source modules directly from the vendor's release, preserv-
ing the directory hierarchy of the vendor's distribution.
The branch support of RCS is used to build this vendor
release as a branch of the main RCS trunk. Figure 2 shows
how the ``head'' tracks a sample vendor branch when no local
modifications have been made to the file. Once this is
done, developers can check out files and make local changes
-----------
5 NSE is the Network Software Environment, a
product of Sun Microsystems, Inc.
-6-
__________________________________________________
ellipse at 3.237,6.763 wid 1.000 ht 0.500
dashwid = 0.050i
line dashed from 3.237,7.513 to 3.737,7.513 to 3.737,9.762 to 4.237,9.762
line from 4.138,9.737 to 4.237,9.762 to 4.138,9.787
line dashed from 2.237,8.262 to 3.237,8.262 to 3.237,7.013
line from 3.212,7.112 to 3.237,7.013 to 3.262,7.112
line from 3.737,6.763 to 4.237,6.763
line from 4.138,6.737 to 4.237,6.763 to 4.138,6.788
line from 2.237,6.763 to 2.737,6.763
line from 2.637,6.737 to 2.737,6.763 to 2.637,6.788
line from 1.738,6.013 to 1.738,6.513
line from 1.762,6.413 to 1.738,6.513 to 1.713,6.413
line from 1.238,7.013 to 2.237,7.013 to 2.237,6.513 to 1.238,6.513 to 1.238,7.013
line from 4.237,9.012 to 5.237,9.012 to 5.237,8.512 to 4.237,8.512 to 4.237,9.012
line from 4.237,8.012 to 5.237,8.012 to 5.237,7.513 to 4.237,7.513 to 4.237,8.012
line from 4.237,7.013 to 5.237,7.013 to 5.237,6.513 to 4.237,6.513 to 4.237,7.013
line from 4.737,7.013 to 4.737,7.513
line from 4.763,7.413 to 4.737,7.513 to 4.712,7.413
line from 4.737,8.012 to 4.737,8.512
line from 4.763,8.412 to 4.737,8.512 to 4.712,8.412
line from 4.237,10.012 to 5.237,10.012 to 5.237,9.512 to 4.237,9.512 to 4.237,10.012
line from 4.737,9.012 to 4.737,9.512
line from 4.763,9.412 to 4.737,9.512 to 4.712,9.412
line from 5.987,5.013 to 5.987,6.013 to 0.988,6.013 to 0.988,5.013 to 5.987,5.013
"
"'SunOS'" at 2.987,6.293 ljust
"1.1.1" at 3.050,6.793 ljust
"1.1" at 1.613,6.793 ljust
"1.1.1.1" at 4.487,6.793 ljust
"1.1.1.2" at 4.487,7.793 ljust
"1.1.1.3" at 4.487,8.793 ljust
"1.1.1.4" at 4.487,9.793 ljust
"'SunOS_4_0'" at 5.487,6.793 ljust
"'SunOS_4_0_1'" at 5.487,7.793 ljust
"'YAPT_5_5C'" at 5.487,8.793 ljust
"'SunOS_4_0_3'" at 5.487,9.793 ljust
"rcsfile.c,v" at 2.987,5.543 ljust
__________________________________________________
FFiigguurree 22..
ccvvss Vendor Branch Example
to the vendor's source distribution. These local changes
form a new branch to the tree which is then used as the
source for future check outs. Figure 3 shows how the
``head'' moves to the main RCS trunk when a local modifica-
tion is made.
When a new version of the vendor's source distribution
arrives, the cchheecckkiinn program adds the new and changed ven-
dor's files to the already existing source repository. For
files that have not been changed locally, the new file from
__________________________________________________
ellipse at 3.237,6.763 wid 1.000 ht 0.500
dashwid = 0.050i
line dashed from 2.800,9.075 to 1.738,9.075 to 1.738,8.012
line from 1.713,8.112 to 1.738,8.012 to 1.762,8.112
line from 1.738,7.013 to 1.738,7.513
line from 1.762,7.413 to 1.738,7.513 to 1.713,7.413
line from 1.238,8.012 to 2.237,8.012 to 2.237,7.513 to 1.238,7.513 to 1.238,8.012
line from 3.737,6.763 to 4.237,6.763
line from 4.138,6.737 to 4.237,6.763 to 4.138,6.788
line from 2.237,6.763 to 2.737,6.763
line from 2.637,6.737 to 2.737,6.763 to 2.637,6.788
line from 1.738,6.013 to 1.738,6.513
line from 1.762,6.413 to 1.738,6.513 to 1.713,6.413
line from 1.238,7.013 to 2.237,7.013 to 2.237,6.513 to 1.238,6.513 to 1.238,7.013
line from 4.237,9.012 to 5.237,9.012 to 5.237,8.512 to 4.237,8.512 to 4.237,9.012
line from 4.237,8.012 to 5.237,8.012 to 5.237,7.513 to 4.237,7.513 to 4.237,8.012
line from 4.237,7.013 to 5.237,7.013 to 5.237,6.513 to 4.237,6.513 to 4.237,7.013
line from 4.737,7.013 to 4.737,7.513
line from 4.763,7.413 to 4.737,7.513 to 4.712,7.413
line from 4.737,8.012 to 4.737,8.512
line from 4.763,8.412 to 4.737,8.512 to 4.712,8.412
line from 4.237,10.012 to 5.237,10.012 to 5.237,9.512 to 4.237,9.512 to 4.237,10.012
line from 4.737,9.012 to 4.737,9.512
line from 4.763,9.412 to 4.737,9.512 to 4.712,9.412
line from 5.987,5.013 to 5.987,6.013 to 0.988,6.013 to 0.988,5.013 to 5.987,5.013
"1.2" at 1.613,7.793 ljust
"
"'SunOS'" at 2.987,6.293 ljust
"1.1.1" at 3.050,6.793 ljust
"1.1" at 1.613,6.793 ljust
"1.1.1.1" at 4.487,6.793 ljust
"1.1.1.2" at 4.487,7.793 ljust
"1.1.1.3" at 4.487,8.793 ljust
"1.1.1.4" at 4.487,9.793 ljust
"'SunOS_4_0'" at 5.487,6.793 ljust
"'SunOS_4_0_1'" at 5.487,7.793 ljust
"'YAPT_5_5C'" at 5.487,8.793 ljust
"'SunOS_4_0_3'" at 5.487,9.793 ljust
"rcsfile.c,v" at 2.987,5.543 ljust
__________________________________________________
FFiigguurree 33..
ccvvss Local Modification to Vendor Branch
the vendor becomes the current ``head'' revision. For files
that have been modified locally, cchheecckkiinn warns that the file
must be merged with the new vendor release. The ccvvss
``join'' command is a useful tool that aids this process by
performing the necessary RCS merge, as is done above when
performing an ``update.''
-8-
There is also limited support for ``dual'' derivations
for source files. See Figure 4 for a sample dual-derived
file.
__________________________________________________
ellipse at 2.337,8.575 wid 0.700 ht 0.375
ellipse at 2.312,9.137 wid 0.700 ht 0.375
line from 1.225,9.012 to 1.225,9.363
line from 1.250,9.263 to 1.225,9.363 to 1.200,9.263
line from 0.875,9.725 to 1.600,9.725 to 1.600,9.363 to 0.875,9.363 to 0.875,9.725
line from 0.875,9.012 to 1.600,9.012 to 1.600,8.650 to 0.875,8.650 to 0.875,9.012
line from 4.050,10.200 to 4.775,10.200 to 4.775,9.850 to 4.050,9.850 to 4.050,10.200
line from 4.050,9.475 to 4.775,9.475 to 4.775,9.113 to 4.050,9.113 to 4.050,9.475
line from 4.050,8.762 to 4.775,8.762 to 4.775,8.400 to 4.050,8.400 to 4.050,8.762
line from 4.425,8.762 to 4.425,9.113
line from 4.450,9.013 to 4.425,9.113 to 4.400,9.013
line from 4.425,9.475 to 4.425,9.850
line from 4.450,9.750 to 4.425,9.850 to 4.400,9.750
line from 3.050,10.000 to 3.775,10.000 to 3.775,9.637 to 3.050,9.637 to 3.050,10.000
line from 3.050,9.312 to 3.775,9.312 to 3.775,8.950 to 3.050,8.950 to 3.050,9.312
line from 0.713,7.325 to 0.713,8.075 to 4.925,8.075 to 4.925,7.325 to 0.713,7.325
line from 1.238,8.075 to 1.238,8.637
line from 1.262,8.537 to 1.238,8.637 to 1.213,8.537
line from 1.613,8.825 to 1.975,8.575
line from 1.878,8.611 to 1.975,8.575 to 1.907,8.652
line from 2.675,8.575 to 4.050,8.575
line from 3.950,8.550 to 4.050,8.575 to 3.950,8.600
line from 2.675,9.137 to 3.050,9.137
line from 2.950,9.112 to 3.050,9.137 to 2.950,9.162
line from 3.425,9.325 to 3.425,9.637
line from 3.450,9.537 to 3.425,9.637 to 3.400,9.537
line from 1.613,8.825 to 1.925,9.137
line from 1.872,9.049 to 1.925,9.137 to 1.837,9.084
"'BSD'" at 2.138,9.481 ljust
"1.2" at 1.113,9.543 ljust
"1.1" at 1.125,8.831 ljust
"1.1.1.1" at 4.175,8.543 ljust
"1.1.1.2" at 4.175,9.281 ljust
"1.1.1.3" at 4.175,9.993 ljust
"1.1.2.2" at 3.175,9.793 ljust
"1.1.2.1" at 3.175,9.106 ljust
"rcsfile.c,v" at 2.425,7.706 ljust
"1.1.1" at 2.175,8.568 ljust
"'SunOS'" at 2.125,8.243 ljust
"1.1.2" at 2.163,9.131 ljust
__________________________________________________
FFiigguurree 44..
ccvvss Support For ``Dual'' Derivations
This example tracks the SunOS distribution but includes
major changes from Berkeley. These BSD files are saved
directly in the RCS file off a new branch.
22..33.. LLooccaattiioonn IInnddeeppeennddeenntt MMoodduullee DDaattaabbaassee
ccvvss contains support for a simple, yet powerful, ``mod-
ule'' database. For reasons of efficiency, this database is
stored in nnddbbmm(3) format. The module database is used to
apply names to collections of directories and files as a
matter of convenience for checking out pieces of a large
software distribution. The database records the physical
location of the sources as a form of information hiding,
allowing one to check out whole directory hierarchies or
individual files without regard for their actual location
within the global source distribution.
Consider the following small sample of a module
database, which must be tailored manually to each specific
source repository environment:
##kkeeyy [[--ooppttiioonn aarrgguummeenntt]] ddiirreeccttoorryy [[ffiilleess......]]
ddiiffff bbiinn//ddiiffff
lliibbcc lliibb//lliibbcc
ssyyss --oo ssyyss//ttoooollss//mmaakkee__lliinnkkss ssyyss
mmoodduulleess --ii mmkkmmoodduulleess CCVVSSRROOOOTT..aaddmm mmoodduulleess
kkeerrnneell --aa ssyyss llaanngg//aaddbb
ppss bbiinn MMaakkeeffiillee ppss..cc
The ``diff'' and ``libc'' modules refer to whole direc-
tory hierarchies that are extracted on check out. The
``sys'' module extracts the ``sys'' hierarchy, and runs the
``make_links'' program at the end of the check out process
(the _-_o option specifies a program to run on check_out). The
``modules'' module allows one to edit the module database
file and runs the ``mkmodules'' program on check_in to regen-
erate the nnddbbmm database that ccvvss uses. The ``kernel'' mod-
ule is an alias (as the _-_a option specifies) which causes
the remaining arguments after the _-_a to be interpreted
exactly as if they had been specified on the command line.
This is useful for objects that require shared pieces of
code from far away places to be compiled (as is the case
with the kernel debugger, kkaaddbb, which shares code with the
standard aaddbb debugger). The ``ps'' module shows that the
source for ``ps'' lives in the ``bin'' directory, but only
_M_a_k_e_f_i_l_e and _p_s_._c are required to build the object.
The module database at Prisma is now populated for the
entire UNIX distribution and thereby allows us to issue the
following convenient commands to check out components of the
UNIX distribution without regard for their actual location
within the master source repository:
eexxaammppllee%% ccvvss cchheecckkoouutt ddiiffff
eexxaammppllee%% ccvvss cchheecckkoouutt lliibbcc ppss
eexxaammppllee%% ccdd ddiiffff;; mmaakkee
In building the module database file, it is quite pos-
sible to have name conflicts within a global software dis-
tribution. For example, SunOS provides two ccaatt programs:
one for the standard environment, _/_b_i_n_/_c_a_t, and one for the
System V environment, _/_u_s_r_/_5_b_i_n_/_c_a_t. We resolved this
-10-
conflict by naming the standard ccaatt module ``cat'', and the
System V ccaatt module ``5cat''. Similar name modifications
must be applied to other conflicting names, as might be
found between a utility program and a library function,
though Prisma chose not to include individual library func-
tions within the module database at this time.
22..44.. CCoonnffiigguurraabbllee LLooggggiinngg SSuuppppoorrtt
The ccvvss ``commit'' command is used to make a permanent
change to the master source repository (where the RCS ``,v''
files live). Whenever a ``commit'' is done, the log message
for the change is carefully logged by an arbitrary program
(in a file, notesfile, news database, or mail). For exam-
ple, a collection of these updates can be used to produce
release notices. ccvvss can be configured to send log updates
through one or more filter programs, based on a regular
expression match on the directory that is being changed.
This allows multiple related or unrelated projects to exist
within a single ccvvss source repository tree, with each dif-
ferent project sending its ``commit'' reports to a unique
log device.
A sample logging configuration file might look as fol-
lows:
##rreeggeexx ffiilltteerr--pprrooggrraamm
DDEEFFAAUULLTT //uussrr//llooccaall//bbiinn//nnffppiippee --tt %%ss uuttiillss..uuppddaatteess
^^ddiiaagg //uussrr//llooccaall//bbiinn//nnffppiippee --tt %%ss ddiiaagg..uuppddaatteess
^^llooccaall //uussrr//llooccaall//bbiinn//nnffppiippee --tt %%ss llooccaall..uuppddaatteess
^^ppeerrff //uussrr//llooccaall//bbiinn//nnffppiippee --tt %%ss ppeerrff..uuppddaatteess
^^ssyyss //uussrr//llooccaall//bbiinn//nnffppiippee --tt %%ss kkeerrnneell..uuppddaatteess
This sample allows the diagnostics and performance
groups to share the same source repository with the kernel
and utilities groups. Changes that they make are sent
directly to their own notesfile [Essick] through the
``nfpipe'' program. A sufficiently simple title is substi-
tuted for the ``%s'' argument before the filter program is
executed. This logging configuration file is tailored manu-
ally to each specific source repository environment.
22..55.. TTaaggggeedd RReelleeaasseess aanndd DDaatteess
Any release can be given a symbolic tag name that is
stored directly in the RCS files. This tag can be used at
any time to get an exact copy of any previous release. With
equal ease, one can also extract an exact copy of the source
files as of any arbitrary date in the past as well. Thus,
all that's required to tag the current kernel, and to tag
the kernel as of the Fourth of July is:
eexxaammppllee%% ccvvss ttaagg TTEESSTT__KKEERRNNEELL kkeerrnneell
eexxaammppllee%% ccvvss ttaagg --DD ''JJuullyy 44'' PPAATTRRIIOOTTIICC__KKEERRNNEELL kkeerrnneell
The following command would retrieve an exact copy of the
test kernel at some later date:
eexxaammppllee%% ccvvss cchheecckkoouutt --ffpp --rrTTEESSTT__KKEERRNNEELL kkeerrnneell
The _-_f option causes only files that match the specified tag
to be extracted, while the _-_p option automatically prunes
empty directories. Consequently, directories added to the
kernel after the test kernel was tagged are not included in
the newly extracted copy of the test kernel.
The ccvvss date support has exactly the same interface as
that provided with RCS, however ccvvss must process the ``,v''
files directly due to the special handling required by the
vendor branch support. The standard RCS date handling only
processes one branch (or the main trunk) when checking out
based on a date specification. ccvvss must instead process the
current ``head'' branch and, if a match is not found, pro-
ceed to look for a match on the vendor branch. This, com-
bined with reasons of performance, is why ccvvss processes
revision (symbolic and numeric) and date specifications
directly from the ``,v'' files.
22..66.. BBuuiillddiinngg ````ppaattcchh'''' SSoouurrccee DDiissttrriibbuuttiioonnss
ccvvss can produce a ``patch'' format [Wall] output file
which can be used to bring a previously released software
distribution current with the newest release. This patch
file supports an entire directory hierarchy within a single
patch, as well as being able to add whole new files to the
previous release. One can combine symbolic revisions and
dates together to display changes in a very generic way:
eexxaammppllee%% ccvvss ppaattcchh --DD ''DDeecceemmbbeerr 11,, 11998888'' \\
--DD ''JJaannuuaarryy 11,, 11998899'' ssyyss
This example displays the kernel changes made in the month
of December, 1988. To release a patch file, for example, to
take the ccvvss distribution from version 1.0 to version 1.4
might be done as follows:
eexxaammppllee%% ccvvss ppaattcchh --rrCCVVSS__11__00 --rrCCVVSS__11__44 ccvvss
33.. CCVVSS EExxppeerriieennccee
33..11.. SSttaattiissttiiccss
A quick summary of the scale that ccvvss is addressing
today can be found in Table 1.
box center tab(:); c s c s c | c l | n . RReevviissiioonn CCoonnttrrooll
SSttaattiissttiiccss aatt PPrriissmmaa aass ooff 1111//1111//8899 _ How Many...:Total =
Files:17243 Directories:1005 Lines of code:3927255 Removed
files:131 Software developers:14 Software groups:6 Megabytes
of source:128
TTaabbllee 11..
ccvvss Statistics
Table 2 shows the history of files changed or added and the
number of source lines affected by the change at Prisma.
Only changes made to the kernel sources are included. The
large number of source file changes made in September are
the result of merging the SunOS 4.0.3 sources into the ker-
nel. This merge process is described in section 3.3.
33..22.. PPeerrffoorrmmaannccee
The performance of ccvvss is currently quite reasonable.
Little effort has been expended on tuning ccvvss, although
-12-
box center tab(:); c s s s s c s s s s c || c | c || c | c c
|| c | c || c | c l || n | n || n | n. PPrriissmmaa KKeerrnneell SSoouurrccee
FFiillee CChhaannggeess BByy MMoonntthh,, 11998888--11998899 _ Month:# Changed:# Lines:#
Added:# Lines :Files:Changed:Files:Added =
Dec:87:3619:68:9266 Jan:39:4324:0:0 Feb:73:1578:5:3550
Mar:99:5301:18:11461 Apr:112:7333:11:5759
May:138:5371:17:13986 Jun:65:2261:27:12875 Jul:34:2000:1:58
Aug:65:6378:8:4724 Sep:266:23410:113:39965 Oct:22:621:1:155
Total:1000:62196:269:101799
TTaabbllee 22..
ccvvss Usage History for the Kernel
performance related decisions were made during the ccvvss
design. For example, ccvvss parses the RCS ``,v'' files
directly instead of running an RCS process. This includes
following branches as well as integrating with the vendor
source branches and the main trunk when checking out files
based on a date.
Checking out the entire kernel source tree (1223
files/59 directories) currently takes 16 wall clock minutes
on a Sun-4/280. However, bringing the tree up-to-date with
the current kernel sources, once it has been checked out,
takes only 1.5 wall clock minutes. Updating the _c_o_m_p_l_e_t_e
128 MByte source tree under ccvvss control (17243 files/1005
directories) takes roughly 28 wall clock minutes and uti-
lizes one-third of the machine. For now this is entirely
acceptable; improvements on these numbers will possibly be
made in the future.
33..33.. TThhee SSuunnOOSS 44..00..33 MMeerrggee
The true test of the ccvvss vendor branch support came
with the arrival of the SunOS 4.0.3 source upgrade tape. As
described above, the cchheecckkiinn program was used to install the
new sources and the resulting output file listed the files
that had been locally modified, needing to be merged manu-
ally. For the kernel, there were 94 files in conflict. The
ccvvss ``join'' command was used on each of the 94 conflicting
files, and the remaining conflicts were resolved.
The ``join'' command performs an rrccssmmeerrggee operation.
This in turn uses _/_u_s_r_/_l_i_b_/_d_i_f_f_3 to produce a three-way diff
file. As it happens, the ddiiffff33 program has a hard-coded
limit of 200 source-file changes maximum. This proved to be
too small for a few of the kernel files that needed merging
by hand, due to the large number of local changes that
Prisma had made. The ddiiffff33 problem was solved by increasing
the hard-coded limit by an order of magnitude.
The SunOS 4.0.3 kernel source upgrade distribution con-
tained 346 files, 233 of which were modifications to previ-
ously released files, and 113 of which were newly added
files. cchheecckkiinn added the 113 new files to the source repos-
itory without intervention. Of the 233 modified files, 139
dropped in cleanly by cchheecckkiinn, since Prisma had not made any
local changes to them, and 94 required manual merging due to
local modifications. The 233 modified files consisted of
20,766 lines of differences. It took one developer two days
to manually merge the 94 files using the ``join'' command
and resolving conflicts manually. An additional day was
required for kernel debugging. The entire process of merg-
ing over 20,000 lines of differences was completed in less
than a week. This one time-savings alone was justification
enough for the ccvvss development effort; we expect to gain
even more when tracking future SunOS releases.
44.. FFuuttuurree EEnnhhaanncceemmeennttss aanndd CCuurrrreenntt BBuuggss
Since ccvvss was designed to be incomplete, for reasons of
design simplicity, there are naturally a good number of
enhancements that can be made to make it more useful. As
well, some nuisances exist in the current implementation.
o ccvvss does not currently ``remember'' who has a
checked out a copy of a module. As a result, it is
impossible to know who might be working on the same
module that you are. A simple-minded database that
is updated nightly would likely suffice.
o Signal processing, keyboard interrupt handling in
particular, is currently somewhat weak. This is due
to the heavy use of the ssyysstteemm(3) library function
to execute RCS programs like ccoo and ccii. It some-
times takes multiple interrupts to make ccvvss quit.
This can be fixed by using a home-grown ssyysstteemm()
replacement.
o Security of the source repository is currently not
dealt with directly. The usual UNIX approach of
user-group-other security permissions through the
file system is utilized, but nothing else. ccvvss
could likely be a set-group-id executable that
checks a protected database to verify user access
permissions for particular objects before allowing
any operations to affect those objects.
o With every checked-out directory, ccvvss maintains some
administrative files that record the current revi-
sion numbers of the checked-out files as well as the
location of the respective source repository. ccvvss
does not recover nicely at all if these administra-
tive files are removed.
o The source code for ccvvss has been tested extensively
on Sun-3 and Sun-4 systems, all running SunOS 4.0 or
later versions of the operating system. Since the
code has not yet been compiled under other plat-
forms, the overall portability of the code is still
questionable.
o As witnessed in the previous section, the ccvvss method
for tracking third party vendor source distributions
can work quite nicely. However, if the vendor
changes the directory structure or the file names
within the source distribution, ccvvss has no way of
matching the old release with the new one. It is
-14-
currently unclear as to how to solve this, though it
is certain to happen in practice.
55.. AAvvaaiillaabbiilliittyy
The ccvvss program sources can be found in a recent post-
ing to the ccoommpp..ssoouurrcceess..uunniixx newsgroup. It is also cur-
rently available via anonymous ftp from ``prisma.com''.
Copying rights for ccvvss will be covered by the GNU General
Public License.
66.. SSuummmmaarryy
Prisma has used ccvvss since December, 1988. It has
evolved to meet our specific needs of revision and release
control. We will make our code freely available so that
others can benefit from our work, and can enhance ccvvss to
meet broader needs yet.
Many of the other software release and revision control
systems, like the one described in [Glew], appear to use a
collection of tools that are geared toward specific environ-
ments -- one set of tools for the kernel, one set for
``generic'' software, one set for utilities, and one set for
kernel and utilities. Each of these tool sets apparently
handle some specific aspect of the problem uniquely. ccvvss
took a somewhat different approach. File sharing through
symbolic or hard links is not addressed; instead, the disk
space is simply burned since it is ``cheap.'' Support for
producing objects for multiple architectures is not
addressed; instead, a parallel checked-out source tree must
be used for each architecture, again wasting disk space to
simplify complexity and ease of use -- punting on this issue
allowed _M_a_k_e_f_i_l_es to remain unchanged, unlike the approach
taken in [Mahler], thereby maintaining closer compatibility
with the third-party vendor sources. ccvvss is essentially a
source-file server, making no assumptions or special han-
dling of the sources that it controls. To ccvvss:
A source is a source, of course, of course, unless
of course the source is Mr. Ed.6
Sources are maintained, saved, and retrievable at any time
based on symbolic or numeric revision or date in the past.
It is entirely up to ccvvss wrapper programs to provide for
release environments and such.
The major advantage of ccvvss over the many other similar
systems that have already been designed is the simplicity of
-----------
6 ccvvss, of course, does not really discriminate
against Mr. Ed.7
7 Yet.
ccvvss. ccvvss contains only three programs that do all the work
of release and revision control, and two manually-maintained
administrative files for each source repository. Of course,
the deciding factor of any tool is whether people use it,
and if they even _l_i_k_e to use it. At Prisma, ccvvss prevented
members of the kernel group from killing each other.
77.. AAcckknnoowwlleeddggeemmeennttss
Many thanks to Dick Grune at Vrije Universiteit in Ams-
terdam for his work on the original version of ccvvss and for
making it available to the world. Thanks to Jeff Polk of
Prisma for helping with the design of the module database,
vendor branch support, and for writing the cchheecckkiinn shell
script. Thanks also to the entire software group at Prisma
for taking the time to review the paper and correct my gram-
mar.
88.. RReeffeerreenncceess
[Bell] Bell Telephone Laboratories. ``Source Code Con-
trol System User's Guide.'' _U_N_I_X _S_y_s_t_e_m _I_I_I
_P_r_o_g_r_a_m_m_e_r_'_s _M_a_n_u_a_l, October 1981.
[Courington]
Courington, W. _T_h_e _N_e_t_w_o_r_k _S_o_f_t_w_a_r_e _E_n_v_i_r_o_n_-
_m_e_n_t, Sun Technical Report FE197-0, Sun
Microsystems Inc, February 1989.
[Essick] Essick, Raymond B. and Robert Bruce Kolstad.
_N_o_t_e_s_f_i_l_e _R_e_f_e_r_e_n_c_e _M_a_n_u_a_l, Department of Com-
puter Science Technical Report #1081, University
of Illinois at Urbana-Champaign, Urbana, Illi-
nois, 1982, p. 26.
[Glew] Glew, Andy. ``Boxes, Links, and Parallel Trees:
Elements of a Configuration Management System.''
_W_o_r_k_s_h_o_p _P_r_o_c_e_e_d_i_n_g_s _o_f _t_h_e _S_o_f_t_w_a_r_e _M_a_n_a_g_e_m_e_n_t
_C_o_n_f_e_r_e_n_c_e, USENIX, New Orleans, April 1989.
[Grune] Grune, Dick. Distributed the original shell
script version of ccvvss in the ccoommpp..ssoouurrcceess..uunniixx
volume 6 release in 1986.
[Honda] Honda, Masahiro and Terrence Miller. ``Software
Management Using a CASE Environment.'' _W_o_r_k_s_h_o_p
_P_r_o_c_e_e_d_i_n_g_s _o_f _t_h_e _S_o_f_t_w_a_r_e _M_a_n_a_g_e_m_e_n_t _C_o_n_f_e_r_-
_e_n_c_e, USENIX, New Orleans, April 1989.
[Mahler] Mahler, Alex and Andreas Lampen. ``An Inte-
grated Toolset for Engineering Software Configu-
rations.'' _P_r_o_c_e_e_d_i_n_g_s _o_f _t_h_e _A_C_M _S_I_G_-
_S_O_F_T_/_S_I_G_P_L_A_N _S_o_f_t_w_a_r_e _E_n_g_i_n_e_e_r_i_n_g _S_y_m_p_o_s_i_u_m _o_n
_P_r_a_c_t_i_c_a_l _S_o_f_t_w_a_r_e _D_e_v_e_l_o_p_m_e_n_t _E_n_v_i_r_o_n_m_e_n_t_s,
ACM, Boston, November 1988. Described is the
sshhaappee toolkit posted to the ccoommpp..ssoouurrcceess..uunniixx
newsgroup in the volume 19 release.
[Tichy] Tichy, Walter F. ``Design, Implementation, and
Evaluation of a Revision Control System.'' _P_r_o_-
_c_e_e_d_i_n_g_s _o_f _t_h_e _6_t_h _I_n_t_e_r_n_a_t_i_o_n_a_l _C_o_n_f_e_r_e_n_c_e _o_n
_S_o_f_t_w_a_r_e _E_n_g_i_n_e_e_r_i_n_g, IEEE, Tokyo, September
-16-
1982.
[Wall] Wall, Larry. The ppaattcchh program is an indispens-
able tool for applying a diff file to an origi-
nal. Can be found on uunet.uu.net in
~ftp/pub/patch.tar.
