Usenet 1994 January/usenetsourcesnewsgroupsinfomagicjanuary1994.iso/sources/std_unix/volume.20/text0035.txt

Internet Message Format | 1990-08-02 | 12.7 KB

From:  <[e-mail]>

           An Update on UNIX*-Related Standards Activities

                              June 1990

                 USENIX Standards Watchdog Committee

                   Jeffrey S. Haemer, Report Editor

IEEE 1003.2: Shell and tools

Randall Howard <[e-mail]> reports on the April 23-27 meeting in
Salt Lake City, UT:

Background on POSIX.2

The POSIX.2 standard deals with the shell programming language and
utilities.  Currently, it is divided into two pieces:

   + POSIX.2, the base standard, deals with the basic shell
     programming language and a set of utilities required for
     application portability.  Application portability essentially
     means portability of shell scripts and thus excludes most
     features that might be considered interactive.  In an analogy to
     the ANSI C standard, the POSIX.2 shell command language is the
     counterpart to the C programming language, while the utilities
     play, roughly, the role of the C library.  POSIX.2 also
     standardizes command-line and function interfaces related to
     certain POSIX.2 utilities (e.g., popen, regular expressions,
     etc.) This document is also known as ``Dot 2 Classic.''

   + POSIX.2a, the User Portability Extension or UPE, is a supplement
     to the base POSIX.2 standard; it will eventually be an optional
     chapter of a future draft of the base document.  The UPE
     standardizes commands, such as screen editors, that might not
     appear in shell scripts but are important enough that users must
     learn them on any real system.  It is essentially an interactive
     standard that attempts to reduce retraining costs incurred by
     system-to-system variation.

     Some utilities, have interactive as well as non-interactive
     features.  In such cases, the UPE defines extensions from the
     base POSIX.2 utility.  An example is the shell, for which the UPE
     defines job control, history, and aliases.  Features used both
     interactively and in scripts tend to be defined in the base
     standard.

__________

  * UNIX is a registered trademark of AT&T in the U.S. and other
    countries.

June 1990 Standards Update                IEEE 1003.2: Shell and tools


                - 2 -

Together Dot 2 Classic and the UPE will make up the International
Standards Organization's IS 9945/2 - the second volume of the proposed
ISO four-volume standard related to POSIX.

In addition to providing current information about the activities of
the Working and Balloting Groups for POSIX.2, a special topic of focus
will be chosen for each report.  Therefore, the reader is referred to
earlier reports for information on such topics as the history of the
Shell Wars and the controversial scope of the UPE.  The next section
talks about the functions, rather than utilities, that are found with
POSIX.2.

The POSIX.2 API Functions

Perhaps it will come as a surprise to many readers that the POSIX
Shell and Utilities standard also contains specifications for about
fourteen new or extended C function bindings -- in effect, its own API
extending the POSIX.1 bindings -- as follows:

  confstr(), sysconf() The first function was created to provide
            string-valued, configuration-specific values such as the
            default setting of the PATH environment variable.  The
            second extends the POSIX.1 function of the same name with
            numeric-valued configuration information such as the
            largest scale value in the bc utility and the
            implementation's line length restriction.

  fnmatch() This functional interface implements the form of pattern
            matching used by file-name generation (glob) in the shell,
            case statements in the shell, and the -name option of the
            find utility.

  getopt()  This functional interface provides a standard utility
            argument parser that enforces the ``standard utility
            syntax'' guidelines and might be used to implement the
            getopts utility from POSIX.2.

  glob(), globfree() This set of functions does shell-style file-name
            generation and presumably calls the fnmatch() function.

  popen(), pclose() This pair of functions, which is a part of the
            standard I/O package on conventional UNIX systems,
            provides the ability to communicate through pipes to
            another process by executing a string in the POSIX.2 shell
            language.

  regexec(), regcomp() This set of routines provides support for both
            the Basic and Extended Regular Expressions defined in
            POSIX.2, including full internationalization support.

June 1990 Standards Update                IEEE 1003.2: Shell and tools


                - 3 -

  wordexp(), wordfree() This set of routines provides a mechanism for
            an application to use word expansion (parameter expansion)
            that is compatible with the POSIX.2 shell language.
            Although most implementations of this routine will
            normally call the shell, it is (at least conceptually)
            possible that the shell be implemented to call these
            routines for word expansion.

  system()  This ``classical'' function executes a command written in
            shell language.

All of these functions form part of an optional C binding to POSIX.2
and it is expected that the soon-to-be-released, draft version of the
NIST FIPS will make this ``optional'' functional interface mandatory
for US government procurements.  Other language-binding working
groups, such as those exploring Ada and FORTRAN, are presumably
encouraged to add their own optional bindings if they so wish.

Although the inclusion of these functions seems to be a little out of
place in a shell-and-tools standard, there is some rationale for this.
In fact, when POSIX consisted only of POSIX.1, the early attempts to
define system() and popen() made apparent the need to completely
specify the shell language in which the argument string to these
functions was written.  That, in turn, along with the desire to
standardize the classical UNIX utility set, led to the creation of
POSIX.2 as the first offshoot group in the POSIX family of standards.
From this beginning, the POSIX.1 sysconf() function was extended and
the confstr() function was created to provide an underlying
implementation for the getconf utility, which allowed shell-level
applications to query configuration-specific values such as maximum
line length of text files.  Once the beachhead of having functional
interfaces in POSIX.2 was established, the temptation to continually
add to this list has led to the current list as of Draft 9.

On the other hand, there are some very strong arguments against the
inclusion of these functions.  First, although the regular expression
functions will almost certainly be required to implement many POSIX.2
utilities such as ed, grep, awk, sed, etc., these functions stop short
of the complete support needed to implement some utilities.  For
example, the handling of error messages (as in a syntactically
incorrect regular expression) and the mechanisms of doing
substitutions (including & and \n support) are not addressed.  Because
of this most implementors will be required to have ``non-portable,''
proprietary extensions to their regular expression support to make a
``commercially-viable'' implementation.  The issue of where to ``draw
the line'' between inclusion and exclusion is a difficult one indeed.
Second, vendors and application writers may find it difficult, both
procedurally and from a licensing perspective, to have part of the
subroutine library come from a POSIX.1 developer and the other part
implemented by the POSIX.2 implementor.  For example, the implementor
of sysconf(), popen(), or system() might do a much better job if

June 1990 Standards Update                IEEE 1003.2: Shell and tools


                - 4 -

common source code and assumptions were possible between the POSIX.1
and POSIX.2 APIs.

Status of POSIX.2 Balloting

``Dot 2 Classic'' remains in its second round of balloting on Draft 9
with a new draft going to ballot in the June-to-July time frame,
according to Hal Jespersen the POSIX.2 Technical Editor.

During the Snowbird meeting, much of Monday was devoted to a
presentation on the status of the Dot 2 Classic Balloting resolution.
It is possible, and indeed likely, that Hal Jespersen will limit
balloting on Draft 10 to unresolved objections and new material.  If
this is the case, it most likely indicates (although he didn't
specifically say) that Hal has confidence that Draft 10 has a high
probability of achieving the requisite 75% affirmative vote.
Personally, I am not convinced that this is a likely event.  While
some decisions will be reversed (perhaps several times) before Draft
10, the following is a summary of issues and/or changes appearing in
Draft 10:

   + The internationalization utilities locale and particularly
     localedef are still controversial, particularly within AT&T.
     Because of the strong rationale for their existence it appears
     that they will remain in Draft 10, certainly with considerable
     amendment as the UniForum Technical Committee on
     Internationalization refines these newly developed utilities.
     This is just one case where the conflict between the role of
     standards to codify existing practice and the obvious holes in
     existing practice creates controversy.  Perhaps this issue will
     be resolved by a balloting referendum such as was used for uucp.

   + The Draft 10 shell will almost certainly strongly resemble that
     of Draft 9.  Most of the important controversies appear to be
     largely settled and most changes appear to be corrections and
     clarifications.

   + Most complex utilities, such as awk, shell, lex, yacc, etc., have
     undergone extensive reworking in response to ballot objections.
     Often a seemingly simple objection will cause large parts of the
     description to be rewritten in order to tighten it up with
     respect to completeness and clarity.  I believe that Hal
     Jespersen believes that most of these changes are uncontroversial
     and he has ensured this by circulating draft sections via E-mail
     to various ``experts.'' Certainly, many of these utilities
     desperately needed this clarification.

   + It appears that the newly-engineered hexdump utility is to be
     replaced by a (much simpler) reversion to od.  While od is the
     existing practice, the POSIX od will be a superset of the
     original one with most useful functionality in the new parts.  It

June 1990 Standards Update                IEEE 1003.2: Shell and tools


                - 5 -

     is not clear that hiding new invention under the same name is any
     less controversial than advertising its existence.

   + Of course, there will be innumerable other changes, obviously
     important to many, that cannot for reasons of space be covered
     here.

A mock ballot on Draft 4 of the UPE was sent to the working group
during February 1990 to allow ballot resolution to be the main focus
of the Salt Lake meeting this April.

Status of the New Orleans Meeting

Monday, the working group reviewed the current status of the balloting
on ``Dot 2 Classic''.  This has already been discussed in earlier in
this report.

The other four days were spent reviewing the 600 to 700 objections
produced by the mock balloting process for the UPE.  While the number
of objections seems low compared to the rate of objections for the
corresponding number of pages in Dot 2 Classic, this may simply be a
symptom of a general shortage of time and the lower number of people
(generally 15 to 20) in the UPE working group.  This lower number and
general lack of time, is a reflection of the fragmentation of the
entire POSIX process caused by a proliferation of working groups.

Most of the work during mock balloting was of the nature of cleaning
up incomplete or poorly worded textual descriptions.  Particularly
controversial issues were often left in the rationale for Draft 5.
Some controversial utilities were moved to an appendix based upon the
belief that they should be removed while still allowing the balloting
group one last chance to save them.  The lint89 was one such utility
whose raison d'etre was meager.  At best, the functionality probably
should be an option to c89 in the ``Dot 2 Classic'' document.  The
sdiff utility which was inadvertently omitted from Draft 4, is to be
included in Draft 5.

Altogether, it appears that Draft 5 is in a relatively healthy state
to survive the rigors of the balloting process.  None the less, I
expect that there will be a greater number of objections in the
balloting this summer than there were in the mock ballot.

June 1990 Standards Update                IEEE 1003.2: Shell and tools


Volume-Number: Volume 20, Number 35