Usenet 1994 January/usenetsourcesnewsgroupsinfomagicjanuary1994.iso/sources/std_unix/v21/120

Internet Message Format | 1990-12-05 | 13KB

From [e-mail]  Thu Sep 20 14:21:46 1990
Received: from cs.utexas.edu by uunet.uu.net (5.61/1.14) with SMTP
    id AA11139; Thu, 20 Sep 90 14:21:46 -0400
Posted-Date: 20 Sep 90 18:01:17 GMT
Received: by cs.utexas.edu (5.64/1.76)
From: [e-mail] (Jeffrey S. Haemer)
Newsgroups: comp.std.unix
Subject: Standards Update, IEEE 1003.2: Shell and tools
Message-Id: <[e-mail]>
Sender: [e-mail]
Reply-To: [e-mail]
Organization: USENIX Standards Watchdog Committee
X-Submissions: [e-mail]
Date: 20 Sep 90 18:01:17 GMT
To: [e-mail]

Submitted-by: [e-mail] (Jeffrey S. Haemer)

           An Update on UNIX*-Related Standards Activities

                            September 1990

                 USENIX Standards Watchdog Committee

                   Jeffrey S. Haemer, Report Editor

IEEE 1003.2: Shell and tools

Randall Howard <[e-mail]> reports on the July 16-20 meeting in
Danvers, MA:

Background on POSIX.2

The POSIX.2 standard deals with the shell programming language and
utilities.  Currently, it is divided into two components:

   + POSIX.2, the base standard, deals with the basic shell
     programming language and a set of utilities required for
     application portability.  Application portability essentially
     means portability of shell scripts and thus excludes most
     features that might be considered interactive.  In an analogy to
     the ANSI C standard, the POSIX.2 shell command language is the
     counterpart to the C programming language, while the utilities
     play, roughly, the role of the C library.  In fact, because
     POSIX.2 provides an interface to most of the features (and
     possibly more) of POSIX.1, it might also be thought of as a
     particular language binding to the soon-to-be language
     independent version of that standard.  POSIX.2 also standardizes
     command-line and function interfaces related to certain POSIX.2
     utilities (e.g., popen(), regular expressions, etc.), as
     discussed in detail in the snitch report for the Snowbird
     meeting.  This part of POSIX.2, which was developed first, is
     also known as ``Dot 2 Classic.''

   + POSIX.2a, the User Portability Extension or UPE, is a supplement
     to the base POSIX.2 standard.  Not a stand-alone document, it
     will eventually be an optional chapter and a small number of
     other revisions to a future draft of that base document.  This
     approach allows the adoption of the UPE to trail Dot 2 Classic
     without delaying it.  The UPE standardizes commands, such as vi,
     that might not appear in shell scripts but are important enough
     that users must learn them on any real system.  It is essentially
     an interactive standard that attempts to reduce retraining costs
     caused by system-to-system variation.

__________

  * UNIXTM is a Registered Trademark of UNIX System Laboratories in
    the United States and other countries.

September 1990 Standards Update           IEEE 1003.2: Shell and tools


                - 2 -

     Some utilities have interactive as well as non-interactive
     features.  In such cases, the UPE defines extensions from the
     base POSIX.2 utility.  An example is the shell, for which the UPE
     defines job control, history, and aliases.  Features used both
     interactively and in scripts tend to be defined in the base
     standard.

Together, Dot 2 Classic and the UPE will make up the International
Standards Organization's IS 9945/2 -- the second volume of the
proposed ISO three-volume standard related to POSIX.

Status of POSIX.2 Balloting

Draft 10 of Dot 2 Classic was sent out during July in a recirculation
ballot.  Recirculation means that objections need only be considered
if they are existing unresolved objections or are based on new
material.  Other objections will be considered at the whim of the
Technical Editor.

Draft 10 is an imposing, if not intimidating, 780 pages, made even
denser by the decision to remove much white space in a (vain) attempt
to save paper.  Ballots are due by September 10.  Unfortunately, the
recirculation ballot materials arrived at my organization on August
17th, giving our group barely three weeks to review this massive
document.

The technical editors and others working behind the scenes (Hal
Jespersen, Don Cragun, and others) have done an admirable job of
diff-marking changes and producing personalized lists of unresolved
objections for each balloter.  In addition, all 96 pages of unresolved
objections are provided.  However, the amount of new material that has
never been reviewed and the major reorganization means that Draft 10
bears much less resemblance to Draft 9 than one might hope.  That,
combined with balloting on the UPE, has put many balloters -- myself
included -- in balloting overload.

If a recirculation simply means forming opinions on my (and other)
unresolved objections, then the time period is quite reasonable.
However, as I shall describe below, Draft 10 is so changed from the
previous drafts that it deserves to be read practically from cover to
cover, and the recirculation deadline does not provide adequate time
for that task.  The changes fall into a number of categories:

   + New Utilities: For example, a superset of the traditional od
     replaced the Draft 9 hexdump which was xd in Draft 8.  Pathchk
     and set -o noclobber have replaced create from Draft 9 and
     validfnam and mktemp from Draft 8.  Such examples demonstrate
     that Draft 10 is not mature and needs more consideration to
     achieve consensus.

September 1990 Standards Update           IEEE 1003.2: Shell and tools


                - 3 -

   + Expanded Material: Previous descriptions of such utilities as
     awk, sh, bc, etc., were neither sufficiently comprehensive nor
     sufficiently complete to be of the quality demanded of a
     standard.  In the latest draft, these descriptions have been
     fleshed out, and include much more detail on operator precedence,
     interactions, subtle semantics, and so on.  This is clearly a
     step in the right direction, but adds to the job of reviewing
     Draft 10.

   + Internationalization: While the localedef and locale utilities
     remain, they have changed substantially.  I personally support
     including these features, but am concerned that these are being
     designed during the balloting process which is, if anything,
     worse than design-by-committee.  Overall, balloting-group
     reaction to these utilities ranges from impassioned pleas for
     their removal to requests for greater functionality (complexity)
     to handle ever more arcane aspects of the internationalization
     problem.

   + Chapter 2: Chapter 2's front matter is substantially reorganized
     and more voluminous.  This chapter contains definitions, utility
     syntax information, requirements imported from POSIX.1, the
     definition of a locale, description of basic and extended regular
     expressions, etc..  Utility descriptions seem to be getting
     shorter, with more and more pointers to Chapter 2.  This is a
     good trend, as long as balloters adequately consider the
     chapter's technical contents.

Status of POSIX.2a Balloting

The first formal ballot on POSIX.2a UPE Draft 5 was due in the IEEE
offices by August 16th.  Unfortunately, the UPE is laced with
references to definitions and concepts largely defined in Chapter 2 of
Draft 10.  I did not receive my Draft 10 until after the UPE balloting
was due to be returned.  This hinders any attempt to review these two
documents as a single entity -- which is what they will eventually
become.

The UPE is starting to mature: it's converging.  The major controversy
is scope -- as it has been throughout the UPE's entire life.  This
draft aligns itself more closely to Dot-2-Classic in many ways, which
leads me to believe that combined review is essential to its
understanding.

A few utilities remain contentious:

   + nice, renice: These require underlying functionality absent from
     POSIX.1, although POSIX.4 has setscheduler(), which allows
     applications to set priority and scheduling algorithms.

September 1990 Standards Update           IEEE 1003.2: Shell and tools


                - 4 -

     Some working and balloting group members adamantly resist any
     attempt to add utilities that are not implementable on top of a
     bare POSIX.1.  Others view the UPE as addressing what users type,
     regardless of underlying implementation.  I am in the latter
     camp, not the least because other working groups, such as
     POSIX.4, have not yet standardized a utility interface, leaving a
     void which the much-maligned UPE group is most able to fill.  (It
     is telling that implementing df and ps is impossible using only
     POSIX.1 functions, yet there is little opposition to including
     either utility.

   + ps: The description for this utility was an interesting amalgam
     of two incompatible visions of how ps output should be
     formatted -- that in Draft 4 and that in Draft 5.  A correction
     should have been issued during balloting, so that balloters could
     concentrate on the real issues of what should be the scope of the
     ps utility.

   + patch: This utility differs from many others; its origins are in
     the public domain rather than in a traditional UNIX variants.  As
     a result, many people feel that patch is worthwhile, but not
     mature enough to standardize.

   + lint89: This utility is optional, largely because it is
     controversial for a number of reasons.  Obviously, the very name
     lint89 is painfully bureaucratic.  Furthermore, many feel that
     ANSI C makes it unnecessary; moreover, any remaining required
     functionality rightfully belongs as an additional option in the
     c89 (cc) utility.  Some point to existing practice.  But what is
     existing practice when the utility's name is lint89?  [Editor: On
     the other hand, it may prove indispensable in detecting
     portability problems in lex89- and yacc89-generated code.
     Parenthetically, Draft 10 calls these lex and yacc, but that must
     just be a temporary oversight; the utilities obligatorily have
     ANSI C input and output.  (One assumes we'll escape c89tags
     because ctags can be made to work with both flavors.)]

   + compress: The inclusion of this utility remains controversial
     because of the Unisys patent on the particular variable of
     Lempel-Ziv compression used by traditional implementations of
     this utility.  The working group appears to be divided on the
     subject of basing a standard on patented material -- no matter
     what the licensing fees are.  There is, however, general
     agreement that it is preferrable to remove compress entirely
     rather than ``invent'' some new compression algorithm.
     Therefore, it appears that a pax-like compromise, of having a
     single interface to a number of competing formats or algorithms,
     is not widely supported.  [Editor: see Andrew Hume's X3B11 report
     for another wrinkle on data compression.] Clearly, this issue
     will have to be resolved with further information from Unisys
     lawyers during the balloting process.

September 1990 Standards Update           IEEE 1003.2: Shell and tools


                - 5 -

Status of the Danvers Meeting

The Danvers working group dealt with neither Dot 2 Classic nor the
UPE.  Instead, at POSIX.3.2's request (that's the subgroup of Dot 3
producing test assertions for Dot 2), we met jointly to co-develop
test assertions for Dot 2 Classic.  This work is a consequence of the
SEC's recent decision requiring each POSIX working group to develop
its own test assertions and ballot them with the standard.  It also
stems from Dot 3's frustration over the (inadequate) way Dot 2
addressed testing.  For example, automated testing of lp, is
impossible; it can only be tested by a human test procedure.  Our
working group should have explored the implications of this before
subjecting POSIX.3 to that task.  (Some utilities can only be tested
manually, but the working group defining that utility should likely
put something to that effect in the Rationale or History of Decisions
Made to confirm to the testing people that they knew this.)

The three days of working with Dot 3 were a real learning experience
for our working group.  Nonetheless, many of us had our fill of test
assertions that week.

I'm also concerned that a three-day meeting cost my company nearly as
much as a five day meeting would have.  In the future, I would prefer
to see schedules that make productive use of the entire working week.

September 1990 Standards Update           IEEE 1003.2: Shell and tools

Volume-Number: Volume 21, Number 120