Usenet 1994 January/usenetsourcesnewsgroupsinfomagicjanuary1994.iso/sources/std_unix/volume.29/text0086.txt

Text File | 1992-12-26 | 30.2 KB | 607 lines

Submitted-by: [e-mail]

``Standards are commercial and international politics dressed up as
technical arguments.''

I think POSIX is caving in under its own weight.  All of the hard
nuts-and-bolt work effort of defining a technical programming
specification is slowly being mired in the mud.  POSIX exists to ``...
support application portability at the source-code level.  It is
intended to be used by both application developers and system
implementors.'' (ISO/IEC IS 9945-1:1990, 1.1 Scope, p.1, lines:2-3).

It has been floundering for sometime in a mess of its own making. I
want to look at this mess, describing it and its historical context,
and offer up a few possibilities for solutions.  This article is long,
but there is a lot of context that needs to be understood to see
what's happening to an otherwise useful standards effort. The article
ends with a list of e-mail addresses to which you may wish to send any
questions and concerns. In fact, I encourage it, and hope that you'll
be convinced by the end of the article.

The Problem

There are two sets of people doing work in the POSIX working groups.
The first set sit in the individual working groups, distilling
historical practice and experience into a technical specification
``for application developers and systems implementors.''

The second set of people have typically been involved at the working
group level for quite some time. They are often chairs of the groups
or other officers.  These people have begun to have co-ordination
meetings and form steering committees outside the working group
structure.  All of the pieces of POSIX are related to one another, and
there is a genuine need to co-ordinate between the different groups of
heads-down-over-the-specification-technicians.  The bureaucracy has
grown because of need rather than desire to hold extra meetings.  Most
of the people involved can think of more enjoyable ways to spend their
time.

I wander these steering committees, sub-committees, and the hallways
of POSIX. It quickly became apparent to me that this is where the
politics that drives POSIX is most on display.  I was eventually
around long enough to get involved in some of these committees. (Fool
me.)

There has been a strange tension in these rooms for quite some time,
coupled with a terrible confusion and sense of apathy. This is not
noticable in the working groups themselves.  Heads down and oblivious
to the politics of POSIX, the working groups are buried in the
religious wars and politics of their own technical specification.

A couple of POSIX meetings back, it began. First in one steering
committee, then another, and another.  The group would hit a crisis
point, and throw up its hands.  Despite the fact that each room
contained people with a long history and knowledge of POSIX, they
would reach a point of apparent confusion as to how to co-ordinate
with another steering committee or sub-committee. (The running joke is
that we need a steering committee steering committee, but it really
isn't seriously contemplated.)

Finally, someone would suggest we need to define the problem. I
offered to go away and write it up. (More fool me.) Then the next
sub-committee meeting. The same process.  Tension, confusion, ``let's
define the problem.'' It started in the Project Management Committee.
I later saw it in the Steering Committee for Conformance Testing, then
the System Interface Co-ordination Committee. These are all really
fundamental sub-committees, with a lot of POSIX history in their
membership.

The co-ordination complexity is amazing. The major areas of POSIX
requiring co-ordination are the base documents themselves, their test
methods, and their structure with respect to language independent
specifications (LIS) and programming language bindings. (This
complexity has spawned profiles, about which I've yelled enough for
now.)

Steering committees were thought to be a way out of the mire. If we
just communicate with one another, the problems will all become
apparent, sort themselves out and go away.  But ultimately this falls
down. POSIX is too big. The steering committees have no authority to
impose their collective will.  POSIX is a volunteer effort. There are
no sticks and there are no carrots.

If it becomes too much trouble to build the standards, then the
volunteers will cease to arrive at the meetings. The POSIX standards
effort will fail. Or worse yet, they will continue to be defined by
fewer and fewer people with sound technical background and a proper
perspective on the subject. This will cast doubt on the good work
which has already been done.

Test Method Madness

To ensure that implementations of the POSIX.1 standard could somehow
be tested and certified in a uniform way, the POSIX.3 standard (Test
Methods) was created. This work was heavily supported and resourced by
the United States government, along with the testing agencies that were
supporting the actual testing requirements.

The POSIX.3 standard is not a bad thing. It defines a methodology by
which test methods and results of test cases written to these methods
can be uniformly described.

If you are creating a standard it's a useful tool to ask yourself
``how would I test this functionality or feature'' as you write the
specification.  It ensures you read and possibly re-write the
specification properly.  You may wish to deliberately not be complete
in the definition, but these areas in a standard specification should
be intentional.

This ``testing'' tool has even been proven.  Several working groups
have written test methods for their specifications, with some help
from people historically involved in the original POSIX.3 effort. Many
of these POSIX.3 members have formed the Steering Committee on
Conformance Testing (SCCT) that oversees how test methods are applied
and created in the working groups.  The SCCT has been too busy to
review these test methods in depth, but without judging whether the
new test methods are good or bad, the working groups that have gone to
the trouble of creating them have all felt that their base
specifications are better defined for the effort.  It seems that the
tool works!

Now for the problem.  Some time ago, the SCCT recommended to the
Sponsor Executive Committee (SEC) that all POSIX standards must have
associated test methods.  These test methods would be standards as
well. They convinced the SEC to make this a requirement.

Now, a standard cannot offically exit balloting without having a test
method specification that is also a standard.  This instantly sets up
a directly competing body of text to the original standard. This is
not a competing functional standard a la IEEE 802.n LAN standards.
This is a competing body of text. (Note: ALL discussions of formal
testing languages and formal specifications are red herrings here.
Anyone wishing to hear my three Canadian cents worth on the subject
can email me.)

Test methods standards will become the annointed specification for the
test suite to demonstrate conformance by organisations with the funds
or market presence to demand as much. Implementations can hit the
narrower mark of the test suite (embodying the standard test methods)
to naively certify rather than hit the standard itself. If you don't
realise the subtle and nasty differences that can appear, spend some
time with the POSIX.1 standard (IEEE Std 1003.1- 1990), and with its
newly declared standard test methods (IEEE Std 1003.3.1-1992).

And what happens when there are holes in the test methods?  Some
things cannot be tested.  The standard still has requirements on these
areas of behaviour, but they may not translate nicely. And there are
some places where the test methods simply aren't complete.  A
reasonably recent draft of the POSIX.3.1 test methods had test methods
for the POSIX.1 environment variables required by U.S. FIPS PUB 151-1,
(the U.S. government profile of POSIX.1,) but none for the other
environment variables. The international community might wish to take
note of this oversight on all LC_ environment variables, should the
POSIX.3.1 standard get to ISO. What other holes are there?

There is a terrible balloting problem. Balloting apathy or overload is
striking many places. The test methods documents are as big as the
standards they repeat. Fewer people care about the test methods,
they've seen the original specification and the job is done, right?
We run the terrible risk of passing bad test methods documents if these
documents are quickly processed through balloting groups whose members
have little time on their hands.  In the current commercial climate
for standards, this is dangerous.

Then, of course, there is the maintenance problem. All useful
standards have the same problem as all useful software. They need to
be maintained. It's just slower and more tedious.  One has added a
level of complexity to the administration of the interpretations.

POSIX.1 has the fun little contradiction that PATH_MAX is the length
of the pathname both explicitly including and excluding the
terminating null byte. An interpretation was requested, and came back
that it was an inconsistency and that both can be right.(2)

 (2) IEEE Std 1003.1-1988/INT, 1992 Edition, Interpretation
    Number: 15, p. 36.

Now what happens when someone requests an interpretation of a standard
with its test methods?

If the request is leveled against the base, what guarantees are there
that the test methods, i.e. a separate standard, will be kept
synchronized? If it's against an inconsistency between the base and
its test method standard, which one wins? If the PATH_MAX argument
holds, then both are correct.  Since one of them is implemented as a
test suite to demonstrate conformance, which one wins in the real
world?

Do test methods need to be standards? Who wins by forcing working
groups to completely re-specify their work as test methods? Testing is
expensive, but the market ultimately protects itself. What has been
done in the TCP/IP space? (If you don't think TCP/IP is a successful
widely implemented specification, stop reading now.) What about the C
language?  No one specified a set of test methods for the ANSI C
standard. People in the know wanted to see how to test the C standard,
and through a lot of hard work built the Plum-Hall test suite. The
U.S. government created a FIPS for C, and chose an available suite.
There were no test methods for this work. No added burden on the
volunteer standards community to respecify itself.

A great tool; But only a tool!

LIS - The Great Experiment

Language Independent Specification (LIS) is burden Number #2 on
working group members.  Two working groups have been operating in the
POSIX space for quite some time in programming languages other than C.
One is the POSIX.5 Ada Bindings group, which has re-cast the POSIX.1
standard into Ada, and is now working on POSIX.4 (Real-time
Extensions.)  The second is POSIX.9 which has similarly cast POSIX.1
into FORTRAN 77, and is now considering what to do with Fortran 90.
The two groups have finished their work. Two real standards exist
within the IEEE standards realm:

   - IEEE Std 1003.5-1992 (Ada Bindings to IEEE Std 1003.1-
     1990.)

   - IEEE Std 1003.9-1992 (F77 Bindings to IEEE Std 1003.1-
     1990.)

A small digression is required on ISO POSIX.  Along the way, IEEE
POSIX entered the international community and an ISO Working Group
(WG15) was created as its home in the Subcommittee on Programming
Languages (SC22). WG15 is not a standards development group per se, in
that it does no drafting of specifications. Its job is to review the
draft IEEE documents and make recommendations to the IEEE, through the
ANSI sponsored U.S. Technical Advisory Group (TAG) on POSIX, back to
the POSIX Sponsor Executive Committee.

Do not be fooled. There is a substantial overlap in the key personnel
of the IEEE working groups and people sitting in the WG15 meetings as
individual technical specialists from their respective national POSIX
standards groups.

ISO began trying to specify programming interface standards in
programming language independent ways, such that the functional
specification appears once, with multiple bindings. It seems expensive
to continually re-specify a standard from one language into a standard
in another language. There is the feeling that there is twice the work
effort, plus the co-ordination effort.

A different international group, WG11, is working at defining abstract
data types and such.  All programmatic interfaces could eventually be
described in some abstract functional way and each individual language
binding would just ``fall-out'' once the mapping from the abstract
types to program language types had been established. Because of early
experiments in specifying standards this way, language independence
was inflicted on POSIX as a requirement from WG15. POSIX the Guinea
Pig.  WG11 had never been faced with POSIX.

All this means every standard becomes two standards.  There is a book
describing the functional specification in abstract data types, and a
book specifying a mapping to a real programming language's syntax,
along with additional required semantics. Try re-reading each of the
last few paragraphs, and after each repeat, ``It is intended to be
used by both application developers and system implementors.''
Ideally, ISO WG members believed that the functional specification
would be a ``thick'' book, and that the language binding would be
``thin''.

The Ada group, POSIX.5, chose not to split their work. They argued it
was too late in their project and that a sufficiently mature POSIX.1
LIS did not exist. They further argued that they had to produce a
``thick'' language binding reproducing much of the semantic content of
the POSIX.1 book, re-cast into Ada-speak, in-line.  Programmer
usability was very high on their list of priorities. Think about that
for a minute.

I work in an environment where we regularly refer to the POSIX.1
standard.  We write code that needs to be portable to many non-Unix
based architectures that provide POSIX.1 interfaces. All of our many
copies of POSIX.1 are very dog- eared and marked up. We use our copies
daily. It is a useful book from which to program. It is not a
tutorial. It is a programmer's reference.

I recently had to go through the POSIX.5 and POSIX.9 standards. I am
not an Ada programmer, but still found the information I needed to
find, in an easily understandable form. The POSIX.5 group did their
job well. Yes, it is a thick binding repeating the semantic functional
material of POSIX.1. And yes, even though the POSIX.5 standard is
supposed to exactly mirror the POSIX.1 standard, I found a bug, (or at
least something about which to request an interpretation.) But I found
the information, clearly laid out; even the bug!

The POSIX.9 (FORTRAN 77) working group chose to attempt a thin
language binding to POSIX.1. They were very tight for resources and
they wanted to do the right thing with respect to the ISO WG15
requirements. Through no fault of their efforts, I found it to be a
difficult book to use, and I was a Fortran programmer in a previous
incarnation.

First, you immediately run into the two book issue. Look up the syntax
in POSIX.9 which immediately punts you to the semantics in POSIX.1. So
you jockey about two books in your lap, continually cross referencing.

Second, you continually switch frames of reference. In one book, there
is a solid real world line of language syntax; in the other book, a
description of that syntax's semantics in a different specification
language (C.)

In balloting the POSIX.1 Language Independent Specification (LIS), I
ran into the same problems. Two books, two frames of reference. At
least POSIX.1 Classic (IEEE Std 1003.1-1990 == ISO/IEC IS
9945.1:1990) stands as an existing reference against which to compare
these models. When we begin balloting drafts of API standards as LIS
and attendent bindings in at least one language, will we be able to
catch all the holes?

The IEEE paid to have the initial drafts of POSIX.1 LIS and its
C-binding (POSIX.16) produced. They couldn't get the work done any
other way. Paul Rabin worked long and hard to produce guidelines for
writing LIS and language bindings.  This work was done within the IEEE
POSIX realm, although Paul liaised closely with ISO WG11 and WG15. The
few IEEE POSIX working groups that have attempted partial or complete
drafts of their work using these guidelines, have immediately started
finding problems in their previous C language specific descriptions.
Just like test methods, prodding the text by attempting to re-cast it
into a different form made a better specification.

Again, one has to ask if this is a good way to define standards.  A
tool to test the specification, yes. The specification itself? One has
to assume that the standard has an audience, and that usability is an
important factor.  One should assume that the standard is based on
existing practice for the most part. That existing practice is in a
particular programming language for API type standards.  Those will be
the first people to come forward to develop the standard. (There has
to be a need to standardize.)

If others with a different programming language background
participate, this would be ideal. If the experience with the
functionality exists in more than one language, and they all want to
come to the table, this is even better. But we do not live in ideal
worlds. Specifying the functionality in a hard to use (2 document/2
context) format is error prone, especially when the document is being
balloted. Until formal methods become a common method of expression,
we are stuck with English descriptions, and the exacting programming
language syntax of the existing body of experience in that area of
functionality.

Language Independent Test Methods

Yes, you read the title correctly.  If the functionality can be
abstracted, described exactly, then bound in various programming
language syntaxes, so to can the test methods of that functionality.
Think about how you would test an Ada run-time implementation of
POSIX.1.

And each is a standard. So there is a base programming language
independent functional specification (LIS) standard, a programming
language binding standard, the LIS test methods standard, and the
language binding standard for those test methods. Balloting will kill
us. We will produce unusable junk if we continue.

Simple economics says we're doomed. The IEEE is being forced to pay up
into ANSI for its international standards efforts.  To cover the costs
of simply balloting the quantity of paper, the IEEE has been forced to
start charging $25 US to join balloting groups.  To cover the
international participation, they've considered raising this to $50 US.
That means it will cost the individual professional programming member
of the IEEE $200 dollars to join the balloting groups for a set of
standards that represent a simple piece of functionality in which they
are interested.

One might argue that a programmer will only join two balloting groups,
for the LIS and language binding. Because the test methods (LIS and
language binding) are a competing body of text, however, they will
need to check the test methods to confirm they are accurate.  Because
of government procurement policies here and abroad, the test methods
will be important!

An Architect's Nightmare

LIS, language bindings, LIS test methods, and their bindings. Now
imagine that we start amending the four standards at once. POSIX.6
(Security Extensions to POSIX.1 and POSIX.2) will amend POSIX.1 and
POSIX.2 somehow at some point in the not too distant future.  So will
POSIX.4 (Real-time Extensions), POSIX.8 (Transparent File Access), and
POSIX.12 (Sockets/XTI).

The original POSIX.6 document, which did contain all the information
they could put together on POSIX security has just needed to be split
SIX ways.

   - The API as an LIS, to amend POSIX.1/LIS,

   - The API as a C-binding, to amend POSIX.16,

   - The API test methods in LIS form, to amend POSIX.3.1,
     (which currently isn't in LIS form,)

   - The API test methods as a C-binding, to amend POSIX.3.1
     (in its current C form?)

   - The utilities, to amend POSIX.2,

   - The utility test methods, to amend POSIX.3.2.

Can't wait.

The Problem Revisited

If POSIX continues on its current course, one of two things will
happen.

ONE - They will succeed.  The useful standards which do exist will be
amended to an user unfriendly form. An ugly unusable set of standards
will eventually be born.  Because of the lack of use, they will fail.
People will not use them. It will be too easy to ignore them.
Programmers will not be able to rely on a certain portability model.
The vendors will continue to sell completely proprietary
implementations.

TWO - They will fail.  Under it's own weight, it will collapse. If not
with a bang, then with a slow sickening crunching sound. The people
with the knowledge will get tired, or lose support (as they obviously
aren't producing anything to show their management in recessionary
times.)  POSIX.1 will become unusable as it is amended and amended and
almost amended.  (``If we wait for another 6 months, we'll be able to
get all the wizzy features in POSIX.42....'')

ONE AND HALF - Life isn't this black or white. The ugly truth will lay
in the middle. We're talking about several thousands of pages of
functional specification.  We're talking several hundred people in
working groups, plus hundreds more in balloting groups, plus the
unsuspecting time-delayed purchasing public. The death will be long and
painful.  Senility will set in first.

Solutions!

Ok. Let's stop the gloom and doom. Let's take an optimistic pro-active
view!  What to do about the problems of POSIX?  Let's put it on a diet.

Remove the continued requirement on balloting the test methods as
standards.  The Steering Committee of Conformance testing would no
longer have a function. It's members could go do real work in the
POSIX.3 update effort, adding to a useful document which provides a
tool for testing the specifications developed in working groups.

These working groups would immediately cease worrying about developing
complete test methods documents. Those that cared, would when
occasionally confronted with ugly passages in their drafts have a
useful tool (POSIX.3) to use to try asking the question, ``how would I
test this?''

Ballot groups could concentrate on the real specification in front of
them. Repeat again: Bad test methods standards will be dangerous in
the marketplace.

Individual technical members in working groups could stop worrying
about completely re-specifying their document.  Possibly some that
cared, with the newly found time, might actually write some real
honest-to-god test cases. These would surface, instead of everyone
waiting to see which way the testing wind was going to blow by large
governmental agencies here and abroad. These test cases might even be
used, therefore useful.

Should these large governmental testing concerns wish to compare the
merits of test suites, they could require that they are documented,
and record results according to POSIX.3. Render unto the standards
community that which is the standards community's, and render unto the
marketplace that which is the marketplace's.

Who can act on this recommendation?  The IEEE POSIX Sponsor Executive
Committee can. They are made up of the working group chairs, the
steering committee chairs, and institutional representatives. There is
a list of these at the end of the article, with email addresses. Send
them e- mail. It really only takes a minute. It will save you a lot of
future grief to take the minute to ask questions NOW!

There is also a list of some important heads of delegation within the
ISO POSIX WG15. WG15 is considering forwarding IEEE test methods
documents as standards at the international level. Then we can all
live with any mistakes in the U.S. government procurement policies!
E-mail soon!  E-mail often!

Let's continue the POSIX diet. Programming Language Independent
Specifications should be stopped for the time being. The IEEE has put
forward an incredible good faith attempt.  The experiment should be
considered a success!  We have demonstrated that we don't yet know
enough about specifying API standards in this abstract way. We should
cease to hold up the working process.

Once the problem is better understood, and our methods of describing
things in an LIS improve, we can begin exporing the possibilites.
Notice that I didn't say retrofit or recast. I said explore the
possibilities. Until we actually add a few of the large amendments to
the base standard, changing its format midstream just opens things up
for abuse and error.  Let's do it a few times in languages that many
of us understand, ie. C, Fortran, Ada, before tackling the problem
with little understood methods, which have been untried at this scale.

What would happen? Working groups would spend less time trying to
re-cast their work (again!) into LIS. They would spend more time on
the real specification, making it usable ``for application developers
and systems implementors.''

When the existing working groups want to bind something in more than
one language, they arrange to attend one anothers' meetings, and they
work together. This sometimes takes the form of the complex strained
negotiations that are the consensus process. This process is already
in place in POSIX and has been for some time. It works.  The LIS has
not been required in producing the usable standards documents to date.

Who can act on this recommendation?  Once again, the IEEE POSIX
Sponsor Executive Committee can. This one is harder, however, as ISO
WG15 is also involved.

First, the SEC has to be willing to say ``no''. This is not a surly
uncooperative ``no''. A huge work effort has gone into the LIS
experiment. There is real experience in the IEEE POSIX projects with
this. The SEC can say ``no'' with confidence based on experience.  ISO
cannot claim the same experience. (If they could, they would have been
helping us a long time ago.)

Second, ISO WG15 has to be willing to say ``no''. Remember that there
is a sizable overlap in the small membership of WG15, and members of
the SEC. The IEEE POSIX working groups have many international members
who show up in the Canadian, UK, American, and German delegations.
Education is certainly not the problem here, however, communication
might be.

Other special working groups within ISO may be concerned with this
approach, but again the experience lies within the IEEE POSIX working
groups, which overlap with ISO WG15.  Other ISO concerns should be
acknowledged and put to rest.  Once again I say: E-mail soon! E-mail
often!

Ultimately, in a worst case scenario some level within ISO could
refuse to accept IEEE POSIX drafts for ISO ballotting.  I believe even
this case should not be of concern, based on the following examples:

   - ISO WG15 has not accepted the perfectly useful IEEE POSIX.5 for
     international standardization, since it did not fit the ISO
     requirements.  ISO WG9 (ISO Ada Working Group) has been very
     concerned by this action and is attempting to fast track the IEEE
     POSIX document.

   - A representative from AFNOR (France's National standards
     organization) voiced strong support for the IEEE POSIX groups to
     continue to bring forward the standards as LIS at the last ISO
     WG15 meeting.  He then immediately expressed grave concerns that
     POSIX.4 be brought forward as quickly as possible in its current
     C-based form to the Draft International Standard (DIS) state. You
     see the French government can procure against a DIS.

Ultimately, if the IEEE POSIX working groups do their job right and
produce useful and usable standards, the market will demand their use,
even if they have to be fast-tracked into the back door to make them
international standards for the international market place. Twisting
the standardization process away from defining detailed specifications
towards suiting procurement processes from organizations that are too
big to change is wrong!

POSIX has market momentum.  It will effect the way you do things. The
working groups have produced useful standards, but that is now in
jeopardy.  You can effect the process. If you can't get directly
involved, e-mail the appropriate people below and ask questions!
Explain your concerns!  Otherwise, you'll have to live with their
decisions.

Who Ya Gonna Call?

             Position          Name                    E-mail

                           IEEE Concerns

Chair SEC                   Jim Isaak           [e-mail]
Vice Chair Interpretations  Andrew Twigger      [e-mail]
Vice Chair Balloting        Lorraine Kevra      [e-mail]
Chair Steering Comm on
             Conf Testing   Roger Martin        [e-mail]
Chair Project Management
               Committee    Shane McCarron      [e-mail]
Chair POSIX.1               Paul Rabin          [e-mail]
Chair POSIX.2               Hal Jespersen       [e-mail]
Chair POSIX.3               Lowell Johnson      [e-mail]
Chair POSIX.4               Bill Corwin         [e-mail]
Chair POSIX.5               Jim Lonjers         [e-mail]
Chair POSIX.6               Ron Elliot          [e-mail]
Chair POSIX.7               Martin Kirk         [e-mail]
Chair POSIX.8               Jason Zions         [e-mail]
Chair POSIX.9               Michael Hannah      [e-mail]
Chair POSIX.12              Bob Durst           [e-mail]
USENIX Institutional Rep    Jeff Haemer         [e-mail]
EurOpen IR                  Stephen Walli       [e-mail]
Uniforum IR                 Ralph Barker        [e-mail]
DECUS IR                    Loren Buhle         [e-mail]
OSF IR                      John Morris         [e-mail]
Unix International IR       Shane McCarron      [e-mail]
X/Open IR                   Derek Kaufman       [e-mail]

                         WG15 Concerns

Convenor WG15               Jim Isaak           [e-mail]
US Head of Delegation       John Hill           [e-mail]
Canadian HoD                Arnie Powell        [e-mail]
UK HoD                      David Cannon        [e-mail]
German HoD                  Ron Elliot          [e-mail]
Dutch HoD                   Herman Wegenaar     (phone: [phone])
Japanese HoD                Yasushi Nakahara    [e-mail]
French HoD                  Jean-Michel Cornu   [e-mail]
Danish HoD                  Keld Simenson       [e-mail]


Volume-Number: Volume 29, Number 86