Usenet 1994 January

home *** CD-ROM | disk | FTP | other *** search

/ Usenet 1994 January / usenetsourcesnewsgroupsinfomagicjanuary1994.iso / sources / std_unix / volume.20 / text0057.txt < prev next >

Wrap

Internet Message Format | 1990-08-02 | 27.4 KB

From: <jsh@usenix.org> An Update on UNIX*-Related Standards Activities June, 1990 USENIX Standards Watchdog Committee Jeffrey S. Haemer, Report Editor IEEE 1003.1: System services interface Paul Rabin <rabin@osf.org> reports on the April 23-27 meeting in Salt Lake City, UT: 1. Introduction The POSIX 1003.1 working group is the oldest POSIX group, responsible for specifying general-purpose operating system interfaces for portable applications. This group developed the original 1988 POSIX standard, and is now responsible for writing supplements and revisions to that standard. This work includes + corrections and clarifications to the 1988 POSIX standard + material that was too controversial to handle before + new interfaces requested by other POSIX working groups Like other working groups developing ``base standards,'' the 1003.1 working group is responsible for writing both C language and language-independent versions of the specifications that it develops. So far, the group has concentrated on the C language versions, but there is increasing pressure to make progress on the language- independent specifications. The working group recently completed a revision of the 1988 POSIX standard, and is currently working on a supplement to that revision. There has been a lot of turnover in the group since the 1988 POSIX standard was completed, but there are still a few old-timers to provide continuity. About 15 people attended the last two meetings. This seems to be a good size for getting work done. This is definitely a technical crowd; check your politics at the door. For more information about the group and how to participate, contact the chair, Donn Terry, at donn@hpfcla.fc.hp.com or hplabs!hpfcla!donn. __________ * UNIXTM is a Registered Trademark of UNIX System Laboratories in the United States and other countries. June, 1990 Standards Update IEEE 1003.1: System services interface - 2 - Send comments and proposals to the secretary, Keith Stuck, at keith@sp7040.uucp. I've made this report a bit more detailed than usual in order to give the technical details wider exposure. New proposals, and comments on any of the current active proposals or issues are welcome. 2. 1003.1a Status 1003.1a is the recently completed revision to the 1988 POSIX standard. No new interfaces or features were introduced, but the text was revised in several ways. The main reason for the revision was to prepare the text for balloting as an ISO standard, so the document had to be made to look like an ISO standard. This meant adding ISO boiler-plate, changing external document references to pretend that only standards exist, changing internal cross-references so that chapters are renamed sections, and sections are renamed clauses and subclauses, changing ``will'' to ``shall,'' etc., ad nauseam. While the working group was having fun with all that, they took the opportunity to do some cleaning up. They corrected errors, clarified unclear language, and changed the function synopses to use ANSI C prototypes. The group did make one normative change, which was to specify reserved namespaces. This will allow implementations and revisions to the standard to define extensions without breaking existing, conforming applications. It's messier than you might think. After four recirculation ballots, IEEE balloting was completed in April. Now it has to get through the ISO balloting process. See the recent snitch report on 1003.5 for a description of how IEEE and ISO balloting is synchronized, and what all of the acronyms mean. ISO has been balloting 1003.1a as ISO/IEC DIS 9945-1. After the first ISO ballot, JTC1 approved 9945-1 for promotion to full IS status. This approval was overruled by the ISO Central Secretariat on the grounds that the document format was still not satisfactory (still haven't caught all of those ``will''s). Rather than publish the current document and then immediately revise, ballot, and publish it again, it was decided to create a new DIS and to start a second round of ISO balloting. This will cause a delay in the publication of the international POSIX standard (and hence also the IEEE POSIX.1 standard). The U.S. Technical Advisory Group (TAG) is responsible for generating the U.S. ballot. Assuming that no normative changes are introduced by the ISO balloting process, the resulting document will be published by IEEE as IEEE Std 1003.1-1990. June, 1990 Standards Update IEEE 1003.1: System services interface - 3 - 3. 1003.1b Status Since 1003.1a is now out of IEEE's hands, the working group spent the Utah meeting working on 1003.1b, the first supplement to 1003.1a. This will include some corrections and clarifications that didn't make it into 1003.1a, but will mainly consist of new interfaces and features. 1003.1b has been in the works for several meetings, so the draft already contains a lot of material. The first day was devoted to revision of the draft, the rest of the week to considering new proposals. The previously announced schedule for 1003.1b specified the Utah meeting as the cutoff date for new proposals. Unfortunately, some expected proposals were not received, and some that were received were not ready for incorporation, so the cutoff was deferred until the next meeting, in Danvers, Massachusetts. Draft 2 of 1003.1b was distributed just before the meeting in Utah. Draft 3 should be available before the Danvers meeting. 1003.1b is expected to be approved sometime in early 1991, and to be published by IEEE as a separate supplement to the IEEE Std 1003.1-1990. 3.1 New Features in the Current Draft of 1003.1b Draft 2 of P1003.1b includes a new data interchange format, and new interface specifications for symbolic links, environment list access, and file-tree walking. These had been proposed and generally accepted at previous meetings. Many new issues were raised and discussed. 3.1.1 Symbolic_Links P1003.1b adds BSD symbolic links to the 1988 POSIX standard as a new required feature. New interfaces for symlink(), readlink(), and lstat() are specified, and the definition of pathname resolution is amended to include the handling of symbolic links. Implementations may optionally enforce a limit on the number of symbolic links that can be tolerated during the resolution of a single pathname, for instance to detect loops. The new symbol {_POSIX_SYMLOOP} is defined to be the minimum value of such a limit. A new error, [ELOOP], is returned if such a limit is exceeded. Symbolic links that are encountered in pathname prefixes are always resolved. Symbolic links named by the final component of a pathname will be resolved or not, depending on the particular interface. By default, such symbolic links will be resolved, unless specified otherwise. The interfaces that will not resolve symbolic links named by pathname arguments are: + readlink() If the pathname argument names a symbolic link, the contents of the link will be returned. + lstat() If the pathname argument names a symbolic link, a stat structure will be returned for the link itself. June, 1990 Standards Update IEEE 1003.1: System services interface - 4 - + unlink() If the pathname argument names a symbolic link, the link itself will be removed. + rmdir() If the pathname argument names a symbolic link, the link will not be followed and the call will fail. + open() Symbolic links are followed, unless both O_CREAT and O_EXCL are set. If both O_CREAT and O_EXCL are set, and the pathname argument names an existing symbolic link, the link will not be followed and the call will fail. + link() If the new pathname names a symbolic link, the link will not be followed and the call will fail. If the old pathname names a symbolic link, the link will be followed. This is the BSD behavior. SVR4.0 does not follow the link in this case, thus supporting hard links to symbolic links. The working group felt that the SVR4 behavior unnecessarily restricts implementations (for instance, those that do not implement symbolic links with inodes), and has much more complex semantics. + rename() If the old pathname names a symbolic link, the link will not be followed. Instead, the symbolic link itself will be renamed. If the new pathname names a symbolic link, it will not be followed. Instead, the symbolic link will be removed, and a new hard link will be created naming the file that was previously named by the old pathname. The 1988 POSIX standard specifies that if the new pathname names an existing file, rename() will fail if the new and old pathnames do not either both name directories or both name non-directory files. This rule needs to be expanded to include the case of the new pathname naming a symbolic link. Should the rename() call fail depending on whether or not the symbolic link named by the new pathname itself names a directory or a non-directory file? This will be resolved at the next meeting. Symbolic links are not required to have any attributes other than their file type and their contents. This is intended to provide the simplest semantics and to allow the greatest latitude for implementations. 3.1.2 Other_BSD_Interfaces P1003.1b also includes specifications for the following interfaces: June, 1990 Standards Update IEEE 1003.1: System services interface - 5 - + fchmod() + fchown() + fsync() + ftruncate() 3.1.3 Environment_List The ANSI-C standard defines the getenv() function to retrieve the value corresponding to a given name in a program's environment list, but does not specify the implementation or initialization of that list. The 1988 POSIX standard specified the traditional list implementation using the external variable environ, and specified the initialization of the list by the exec functions. In an attempt to extend the set of high-level interfaces to the environment list, and to pave the way for the possible eventual removal of environ, the working group has included putenv() and clearenv() interfaces in 1003.1b. Three problems have been identified with these high-level interfaces: + They cause static data to be shared between the application and the implementation. Neither the application nor the implementation can easily manage the storage for environment "name=value" strings. + They are not robust. The interactions between the high-level interfaces and access via environ is not specified. + They can't be easily extended to handle multiple lists. There is no way to copy a list, or to build a new list for execle() or execve(). The putenv() and clearenv() interfaces may be removed from 1003.1b at the next meeting if a revised proposal does not appear. 3.1.4 File_Tree_Walk The 1003.1 working group promised the 1003.2 group (Shell and Utilities) that a mechanism would be provided for walking an directory tree of unbounded depth using any given (non- zero) number of free file descriptors. The Berkeley folks have implemented a set of high-level interfaces defined by David Korn of Bell Labs, and proposed them for inclusion in 1003.1b. These interfaces support every option and search order required by the 1003.2 commands. The 1003.1 group wants a simpler interface suitable for typical application programs, so Keith Bostic will put the proposal on a ``weight-reducing diet'' and resubmit it for the next draft. The high-level file-tree walk interfaces can be implemented using only the existing 1003.1 interfaces. Since 1003.1 does not define a portable way to save and restore file position for a directory and cannot hold a file descriptor open for each directory level, the June, 1990 Standards Update IEEE 1003.1: System services interface - 6 - implementation must read and save all directory entries each time a new directory is visited. This requires only a single file descriptor (or whatever resource is used by by opendir()). If the underlying system does provide a way to save and restore file position for directories, the file-tree walk implementation can use it to reduce memory consumption. There was a discussion about whether it is possible (and preferable) to improve the low-level directory interfaces instead of adding new, high-level interfaces. Do the high-level interfaces really add new functionality for portable applications? Do they belong with the low-level operating system interfaces specified in 1003.1? Walking an unbounded file tree requires an unbounded number of directory file positions to be supported using a bounded number of file descriptors. Either seekdir() and telldir() are needed, or an unbounded number of opendir()'s must use a bounded number of file descriptors. The working group has already rejected seekdir() and telldir() because they cannot easily be supported on implementations that use a non-linear directory format. A prohibition of simple implementations of opendir() using file descriptors is also likely to be rejected. The underlying problem is that the orderedness of directory entries was implicit in the traditional implementations, but was not made fully explicit in 1003.1, partly out of a desire to permit alternate implementations (for instance, b-trees). As a result, orderedness must now be imposed by the application. On a non-linear directory implementation, if positioning is not supported, even opendir() must read in the whole directory. 3.1.5 Data-Interchange_Format The 1988 POSIX standard specified two data-interchange formats based on existing utilities. These define the data-stream encoding of a sequence of files, together with their pathnames and other attributes. The first format is based on tar and encodes files as a stream of 512-byte blocks. The second format is based on cpio and encodes files as an unblocked byte stream. The ISO POSIX group (JTC1/SC22/WG15) pointed out that both of these formats are incompatible with accepted international and U.S. standards. After some arm twisting, the 1003.1 working group agreed to devise a new data interchange format based on IS 1001: 1986, which is more or less equivalent to ANS X3.27-1987, the familiar ANSI labeled tape format. The current draft of 1003.1b includes the framework for the new specification, but a lot more work is needed. Previous meetings discussed alternate proposals. The topic has been strangely quiet lately, considering the confusion that may be expected when it goes to ballot. It wasn't discussed at the Utah meeting at all. June, 1990 Standards Update IEEE 1003.1: System services interface - 7 - 3.2 {_POSIX_PATH_MAX}: A Clarification The 1988 POSIX standard included two conflicting statements regarding {PATH_MAX} and {_POSIX_PATH_MAX}: one said that the null was included in the count; the other said that the null was excluded. Traditional implementations have included the trailing null; some new implementations have excluded the null. One alternative or the other had to be endorsed. The working group decided that {_POSIX_PATH_MAX} should not include the trailing null, since specifying this will not break currently conforming applications. 3.3 Headers and Name-Space Control Since 1003.1b is adding many new identifiers to the standard, there was discussion about whether new identifiers should be declared in new headers, or whether existing headers could be used, with new feature- test-macros to control visibility of the additional identifiers. It was agreed that although both headers and feature-test macros control identifier visibility, their functions are complementary. Headers are appropriately used to divide name-spaces horizontally, by functionality. Feature-test macros are appropriately used to divide name-spaces vertically, by specification level. With this understanding, the group decided that new identifiers will be declared in the ``right place.'' A new header will be created only if no existing header is functionally appropriate. A new feature-test macro will be specified by 1003.1b and subsequent revisions: _POSIX_1_SOURCE. This macro takes ordinal values, starting with 2 for 1003.1b, and will be incremented by 1 for every subsequent revision. If the value is 1, the effect will be the same as if _POSIX_SOURCE were defined. There are two changes here. The new name was used to indicate that the macro only controls the visibility of identifiers defined in POSIX.1. The usage was changed to allow the value to indicate the particular revision or supplement to the standard, rather than having to create a new macro each time. This should simplify the construction and maintenance of header files. 3.4 Requests Two requests were made by vendors trying to support POSIX behavior on non-UNIX file systems: + that {_POSIX_LINK_MAX} be reduced from 6 to 2 + that {_POSIX_PATH_MAX} be reduced from 255 to 252 June, 1990 Standards Update IEEE 1003.1: System services interface - 8 - Both requests were rejected. Either of these changes could have made existing conforming applications non-conforming. Even where the risk of breaking applications seemed small, the working group was reluctant to set a precedent without a pretty good rationale to protect them against similar requests in the future. 3.5 New Proposals Five proposals for new interfaces were submitted for inclusion in 1003.1b, many of which provoked lively discussion. Some were accepted, some were rejected, and others were deferred to allow a revised proposal to be submitted or to allow more time for consideration. 3.5.1 seteuid(),_setegid() Bob Lenk and Mike Karels proposed a set of changes to the way the effective user and group id's are handled, in order to provide better support for setuid/setgid programs. + Require that all implementations support the functionality of the saved user ID and saved group ID. These process attributes are set by the exec functions and by privileged calls to setuid() and setgid(). + Add seteuid() and setegid() as new functions that change only the effective user ID and effective group ID, respectively. A change is allowed if the proposed new user/group ID is the same as either the real user/group ID or the saved user/group ID. + Redefine the {_POSIX_SAVED_IDS} option to apply only to non- privileged calls to setuid() and setgid(). This proposal has general support in the working group, and will be included in the next draft of 1003.1b. The discussion of this proposal led to a general lament about how unclear the group model is in the 1988 POSIX standard, perhaps the result of a hasty marriage between the System V and BSD models. At the next meeting, the working group intends to add new text to P1003.1b to clarify the relation between the effective group ID and the supplementary group list. 3.5.2 Magnetic_Tape_Support The 1003.10 working group (Supercomputing Profiles) proposed new interfaces to support basic controller functions for magnetic tape drives, based on the ioctl() commands supported in 4.3BSD. Although support for these interfaces would be optional in 1003.1b, the working group decided that the functions should be further specified according to whether they are: + required for all types of tape drives; June, 1990 Standards Update IEEE 1003.1: System services interface - 9 - + required only for 9-track tape drives; + required only for cartridge tape drives; or + optional on all types of tape drives. The proposal needed further revision, but was generally supported by the working group. The submitted proposal also included interfaces for mounting labeled tape volumes. These were considered to be inappropriate for inclusion at this time and will be deferred until a later revision of the standard. 3.5.3 Checkpoint/Restart The 1003.10 working group also proposed new (optional) interfaces for checkpointing and restarting processes. This proposal is based on two existing implementations. The interfaces are intended to protect very-long-running applications from both scheduled shutdowns and unexpected failures of the system. The 1003.1 working group was not happy to have to deal with this and had lots of questions. Were programming interfaces for portable applications really needed, or was a command interface sufficient? How much state needed to be saved in the checkpoint? What if the processes depended on additional state information that was not in the checkpoint, such as file data or the states of other communicating processes or devices? In this case, the restart would only be successful if this additional state had not changed since the checkpoint. How could such changes be detected or prevented? What is the set of interfaces that an application can use and be sure that it can be checkpointed and restarted successfully, without dependencies on additional state? Should applications have a mechanism for checkpointing themselves, or for blocking an external request that they be checkpointed? Because a checkpoint/restart mechanism will have a major impact on implementations, and the requirements are not yet clear, the working group was unwilling to endorse the current proposal. A task force made up of representatives of the 1003.1 and 1003.10 working groups will be created to clarify the requirements and revise the proposal. This proposal is going to need a lot more discussion, so checkpoint restart interfaces will almost certainly not be included in P1003.1b, but they may be adopted in a subsequent revision. 3.5.4 Messaging The UniForum proposal for new messaging interfaces has been before the 1003.1 working group for a couple of meetings now. The proposed interfaces are intended as replacements for the message catalog interfaces specified in XPG3, and differ from those in several ways (since the discussion was fairly contentious, I'll try to be objective): June, 1990 Standards Update IEEE 1003.1: System services interface - 10 - + The XPG3 interfaces identify a message by the triple: <catalog name, set ID, msg ID>, where catalog name is a file name, and set ID and msg ID are integers. The UniForum interfaces identify a message by the triple: <locale name, domain name, message name>, where locale name, domain name, and message name are all strings. The locale for messages is specified by the new LC_MESSAGES category of the locale. Advocates of the UniForum proposal claim that string identifiers are easier to use and are more robust against errors during application development and maintenance. + In the XPG3 scheme, each message catalog is an ordinary file. Message catalogs must be specified by filename and explicitly opened before messages can be retrieved. The NLSPATH environment variable provides a search path for message catalogs that can be parameterized by (among other things) the language, territory, and codeset fields of the LANG environment variable. In the UniForum scheme, groups of messages are specified by an abstract ``domain.'' A default domain can be set to control message accesses, or the domain can be explicitly specified for an individual message access. Advocates of the UniForum proposal claim that the binding of message catalogs to files unnecessarily restricts implementations and imposes a more complex interface on application developers. + The XPG3 interface includes an additional string argument that is returned in case no message specified by <set ID, msg ID> can be retrieved from the message catalog. In the UniForum proposal, the message name itself is returned if the message cannot be found. Advocates of the UniForum proposal point out that the message name string makes a separate, ``default'' message string unnecessary. In addition, the UniForum proposal includes a specification of the message source file format that differs from the format specified in XPG3. + In the XPG3 format, message strings are implicitly delimited: at the beginning by the preceding message ID followed by a single space or tab character, and at the end by an unescaped newline. In the UniForum format, all strings, including domain names, message ID's, and message strings, are explicitly delimited by double quotes ("). Adjacent strings separated by white-space characters are concatenated. Advocates of the UniForum proposal claim that the new format provides better support for multi-line strings and for leading and trailing white-space characters in strings. + In the XPG3 format, the message ID and its corresponding message string are implicitly determined by their position on a source line. In the UniForum format, explicit directives are provided for message ID's and message strings. Advocates of the UniForum June, 1990 Standards Update IEEE 1003.1: System services interface - 11 - proposal claim that the new format is extensible. New attributes may be added to message entries, such as screen coordinates or font size. + The XPG3 format includes directives for deleting individual messages and sets of messages, and the associated gencat utility takes no switches. In the UniForum proposal, all deletion semantics are provided by switches on the associated gentext utility. There was much discussion of the interfaces; less about the source file format. The most divisive issue was whether string message ID's are preferable to numeric message ID's. Among those who felt that the new interfaces are better, there was disagreement about whether the advantages outweighed the cost of conversion for applications and implementations based on the XPG3 interfaces. The rationale accompanying the UniForum proposal described several ways to convert applications from the XPG3 interfaces to the proposed new interfaces. The working group asked X/Open to submit the XPG3 messaging interfaces as an alternate proposal, since they represent existing practice, and X/Open has agreed to do so. X/Open has said that they will follow POSIX if POSIX endorses a different interface. The decision regarding which, if any, messaging proposal to include in 1003.1b will be made at the POSIX meeting in Danvers. It's hard to predict the fate of this proposal. The UniForum proposal represents the consensus of one of the leading internationalization working groups and is reported to have some support within X/Open. On the other hand, the POSIX working groups are obliged to respect existing practice. Watch this space. 3.5.5 /dev/stdin,_/dev/fd/n,_etc. There was an unofficial proposal from members of the 1003.2 working group that open() be extended to recognize the special strings "/dev/stdin", "/dev/stdout", "/dev/stderr", and "/dev/fd/n", and return a new file descriptor dup()'ed from STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO, or file descriptor n, respectively. This proposal was intended to allow simplified command line parsing, by eliminating special casing for ``-'' and ``--'' arguments. The proposal was rejected after a short exploration of the possible semantics of these pathnames when used with link(), rename(), etc.. 4. Conclusion As you can see, there's a lot going on. Even though most of the attention has shifted to other working groups, the 1003.1 group is busy revising and extending the 1988 standard. The group is small now, by POSIX standards, but their work is as important as ever. June, 1990 Standards Update IEEE 1003.1: System services interface Volume-Number: Volume 20, Number 56