Usenet 1994 October

home *** CD-ROM | disk | FTP | other *** search

/ Usenet 1994 October / usenetsourcesnewsgroupsinfomagicoctober1994disk2.iso / std_unix / volume.18 / text0008.txt < prev next >

Wrap

Internet Message Format | 1990-03-18 | 37.4 KB

From: jsq@usenix.org (John S. Quarterman) Here is the White Paper on System Administration that USENIX sponsored for IEEE 1003.7. I am posting it now for three reasons: 1) It never has been posted. 2) It was one of the main reasons for the network orientation of 1003.7 that was mentioned prominently in the recent 1003.7 snitch report. 3) There is much talk of a possible White Paper for IEEE 1201. John S. Quarterman, Institutional Representative from USENIX to IEEE 1003. White Paper on System Administration for IEEE 1003.7 Susanne W. Smith Windsound Consulting John S. Quarterman USENIX Association ABSTRACT The new POSIX committee on System Administra- tion, IEEE 1003.7, is attempting to standardize an area in which there is little prior art, and no generally accepted solutions to many of the known problems. It is a large area, and one that inter- sects with other areas such as networking (IEEE 1003.8) and application programming (IEEE 1003.2). Some of the most applicable prior art was not designed for operating system administration, but for network administration. Perhaps most impor- tantly, there are two basic models for system administration. One must be chosen from the outset, and the choice will affect everything the committee does. The USENIX Association has coordinated the production of this White Paper to set forth the basic issues the committee must address, to recom- mend certain choices it will have to make, and to outline some of the existing solutions that must be considered. 1. Introduction The role of the systems administrator has evolved over the years. Where once an administrator was responsible for a single machine or machines from a single vendor there is now often a network of machines from different vendors. Both the homogeneous single machine case and the heterogene- ous networked case must be addressed by the systems adminis- tration committee in producing a standard. This paper July 17, 1989 - 2 - offers a description of systems administration, its com- ponent tasks, and its scope; it recommends a model upon which to build the standard; it presents an overview of some current systems administration practices; and it provides a reference list. 2. The Basic Model The most basic choice for a system administration stan- dard is between a single machine model and a model based on a network of machines. 2.1. A Single Machine The results of 1003.7 will be applied to many machines that are not connected to any other machines, except perhaps by some indirect technique such as UUCP. The standard must be applicable to such machines. For this purpose, it need only specify a command interface and detail what the com- mands are supposed to accomplish. However, there is a problem with basing the standard on a single machine as a model, because such a standard will not adapt well to a network of machines. The traditional methods used for administration of a single machine are not readily extended for a networked environment. For example maintaining user information on a single machine requires modifications to the /etc/passwd file. In a networked environment this further requires maintaining the con- sistency of this file across many machines. 2.2. A Network of Machines The number of machines connected to networks and the number of networks of computers have grown exponentially in the last several years. Many of us are accustomed to interacting with hundreds of computers on a local area net- work that is in turn connected to hundreds of thousands of other computers through wide area networks. 2.2.1. Remote Access Many machines do not even have local disks: files are kept on a central server, which is accessed over the net- work. There may be more than one server, and two machines may even act as servers for each other for different parts of their file system trees. 2.2.2. Distribution Databases may not have a single location. The mapping between login names and login IDs may be distributed among several machines. The whole database may be duplicated for redundancy. Parts of it may be kept in different places, July 17, 1989 - 3 - for local control. A tree structure may be used. 2.2.3. Heterogeneity Networked environments tend to have machines with many different hardware types and many different variants of operating systems. One machine may have /etc/passwd and another may use a distributed database. The possible param- eters to an operation may differ. Byte orders and word lengths vary. 2.3. Specifications A single interface specification is not sufficient for a networking model of system administration. Three things are needed: 2.3.1. Interface A specification of a programming interface is needed for a networked model, just as for a single machine model. Additional commands may be required for a networked model. But the specification of what the commands for the interface do has to be more complex for a networked model. 2.3.2. Database Because of differences among machines in a heterogene- ous network, such as varying byte orders, word lengths, and options supported, a generic specification of the informa- tion to be managed is needed. It is not practical to pro- vide specifications for every type of machine and software and translations between them, because the numbers of specifications needed would be very large. 2.3.3. Operations Given the interface specification of a command, and the database specification of the information it is to affect, a specification is also needed of how to communicate the necessary operation across the network. This should be done in a manner that is not specific to any of the underlying systems, but that can be translated into appropriate actions on any of them. 2.4. Network Management Standards These issues and this kind of model have been addressed for the purpose of managing networks. It is possible that the work can be adapted and extended for use by 1003.7. Two components, a management station and a management agent, work together to perform network management functions in the following two protocols. The management station monitors and controls network elements. Management agents perform July 17, 1989 - 4 - functions requested by the management station on the network element. 2.4.1. CMIP The Common Management Information Protocol is the emerging ISO standard for network management. It uses a MIB (Management Information Base) and defines operations to be performed on it over a network. 2.4.2. SNMP The Simple Network Management Protocol is in use now with TCP/IP on NSFNET. It addresses many of the basic net- work management problems and presents at least preliminary solutions to them. It proves the concept of a MIB with operations to manipulate it over a network. 2.4.3. ASN.1 Abstract Syntax Notation 1 is the ISO standard for encoding of information at the presentation layer of the seven layer ISO networking model. It is similar to Sun's XDR (External Data Representation) or Apollo's NIDL (Network Interface Definition Language) or NDR (Network Data Representation), but is more general than either. ``ASN.1 is useful for describing structures in a machine-independent fashion. Additionally, ASN.1 definitions can be written which convey to the human reader the semantics of the objects they define.''2 Both CMIP and SNMP are written in terms of ASN.1. 2.5. Scope The responsibilities of systems administrators vary widely among installations. In some environments the tasks of the systems administrator are defined as ``anything it takes to keep computing services available for the user com- munity.'' This definition could encompass everything from hardware diagnostics to network management. In some situa- tions the systems administrator may be responsible for user support and consulting. In other situations the tasks of the systems administrator could be rigidly defined to only include password file maintenance and backups. Because there is no commonly-accepted definition of the scope of system administration, the committee needs to define which specific areas are included as the functions of a systems administrator. Scope and definitions are also required parts of an IEEE standard. These should be addressed before commands and facilities are defined. The committee should consider previous work in network management. The OSI model for network management consists of five functional areas: configuration management, July 17, 1989 - 5 - performance management, fault management, accounting manage- ment, and security management. These functional areas map very well from network management to operating system management. 2.5.1. Configuration Management Configuration management in the network sense is defined as ``detection and control of the state of the net- work, both the logical and physical configurations of the network.''1 Configuration management in a systems adminis- tration context would refer to the management of the infor- mation which defines a machine's functions. Configuration information determines whether a machine is a file server or client, a timesharing service or single user, diskless or diskful. The configuration data identifies the location of other machines and services. 2.5.2. Performance Management Performance management could be defined as the collec- tion and analysis of information that determines a machine(s) performance. Examples could be disk throughput, service access times, or cpu utilization. 2.5.3. Fault Management Fault management is ``the detection, isolation, and correction of abnormal operations in the network.''[1] For systems administration this would be detection of a service's failure, notification of the user community of failure, and the initiation of a backup service. 2.5.4. Accounting Management Accounting management would be the management of the information required to determine the cost of using the sys- tem. This type of information is traditionally collected in units of disk storage blocks, cpu usage, and connect time. 2.5.5. Security Management Security management is composed of the functions required to regulate access to system resources. User authentication, server verification, and security logs are functions of security management. 2.6. Recommendations We strongly recommend the adoption of a network model. We also recommend that the committee focus on the entities to be managed and not the underlying transport protocol. July 17, 1989 - 6 - 2.6.1. Specifications Every command should be specified in terms of an inter- face, an information database, and operations to be per- formed over a network. Although the first of these alone would be sufficient in a single machine case, it is not ade- quate to a networked environment. A network model can be reduced to handle a single machine as a special case, but a single machine model cannot readily be expanded to support a networked environment. This is the main reason that a net- work model should be adopted instead of a single machine model. 2.6.2. Network Management The committee should examine the work done to date on SNMP and CMIP, and should follow the progress of the commit- tees that are producing those protocols. The 1003.7 MIB should be written in ASN.1. 3. Prior Art We present here some examples of areas in which there is prior art that the committee should consider. This is not an exhaustive list of either the areas to be covered or the prior art in a specific area. There are other such areas, and we encourage others to submit proposals to the committee outlining them. The examples are grouped according to the OSI model described above. Because system administration covers a broader area than network management the categories have been extended. Additional categories may be required to com- pletely include all system administration functions. 3.1. Configuration Management In addition to the description above configuration management could include user configuration information. This would include the information required to describe a user and their environment (i.e. the location of their home directory). This area could also include queueing systems. 3.1.1. /etc/passwd The simplest database of user information is /etc/passwd. It is a single file which contains information about each user. /etc/passwd contains a user's login name, user-id, group id, encrypted password, optional full name and additional information, home directory location, and program to be executed upon successful completion of the login process. User information is added, changed, or deleted by using the command vipw or one of many available shell scripts and programs. Access to the information is July 17, 1989 - 7 - controlled by file permissions. This scheme works well in a single machine environment. This method requires each machine to have an /etc/passwd file. As the number of machines on a network and the number of users increases, maintaining the file entries on each individual machine becomes an overwhelming, if not impossi- ble, task for the system administrator. Different methods have been proposed to handle the task of maintaining an /etc/passwd file on each machine in a network. 3.1.2. Yellow Pages Yellow Pages (yp) is a distributed network lookup ser- vice. The Yellow Pages provide configuration information for a group of machines called a domain. A machine requesting information is a yp client and the machine providing the information is a yp server. The information for a particular domain is a set of maps. Commonly the /etc/passwd and /etc/hosts files are replaced by yp maps. However, yp is indifferent to the type of data in the maps. A master flat file resides on a master server machine. Updates to the master file are made here. Dbm is used to transform the flat file into maps. The maps are then propagated to all slave server machines. The number of slave servers is dependent on network size and topology. A single machine may serve more than one domain. Once yp services are available (i.e. the maps have been made and the server machines configured) routines on the yp client machine must be modified to initiate yp requests rather than reading local files. Yp requests are remote procedure calls to a yp server. 3.1.3. Moira ``The purpose of Moira is to provide a single point of contact for authoritative information about resources and services in a distributed environment.''[3] Moira is used to store information about users, the location of network ser- vices, the information needed to create the configuration files for network servers, as well as other information. Updates to the database are made using an application inter- face which is based on curses. Validity checks are per- formed on data to be updated. Access to each object in the database is controlled by an access control list. Statistics are kept about who modified the object last. Network server configuration files are created from the Moira database and sent periodically to the appropriate servers. This eliminates the need to modify configuration files on individual machines. The Hesiod (see below) July 17, 1989 - 8 - database is also created from the information in the Moira database 3.1.4. Hesiod Hesiod provides a read only front end for user infor- mation and the location of network services. User informa- tion is extracted from the Moira database and formated into ASCII files in BIND-compatible resource record format. Modifications have been made to BIND to accept and process Hesiod type queries. Hesiod is used by the login process to acquire user information. Note however that Hesiod does not authenticate the user. Authentication is performed by Kerberos. Hesiod is also used by lpr to retrieve printer information tradition- ally stored in the /etc/printcap file. 3.1.5. Berkeley Print Spooling The Berkeley print spooling system was intended for use with network print services where printers are connected directly to the network or to the serial port of a host machine on the network. The command lpr is used to start the printing process. Line printer daemons (lpd) run on each machine in the network to control the spool area, queue, printing, and network transfers. Lpr looks up information for the requested printer in the /etc/printcap file. This file contains information about each printer, such as location, filters needed, header page format, etc. It determines what to do with this file from this information. The lpc command provides queue management functions. Lpc is used to restart and flush queues, abort jobs, and check the status of queues and printers. 3.1.6. MDQS - Multiple Device Queueing System MDQS provides for local printer support, remote printer support, local and remote batch job scheduling, conversion of troff to device specific format, and sending graphics data to plotters. MDQS consists of a queue management dae- mon, a general-purpose spooler, a set of device specific despooled-data processing slaves, and utilities for setting form types, disabling service, viewing queues, etc. A queue/device mapping table contains the queue name, device name, and the command to be executed as a slave pro- cess for the dequeued data. Remote printing and execution are handled by having slave processes which respool the data into the remote MDQS queues. The mapping table provides the flexibility for multiple devices to process from the July 17, 1989 - 9 - same queue or one device to process from multiple queues. If NFS (network file system) or some similar mechanism is used a single spooling area and daemon with control files can reside on one machine. This eliminates the need for respooling data into remote queues and the overhead of main- taining a local spooling area, daemon and control files. The remote devices simply process the queue from the remotely mounted file system. 3.2. Security Management Personal computers can be protected by making the machine physically secure. In a timesharing environment the operating system is used to protect one user from another. In a networked environment there are three approaches to prevent unauthorized access to network services: rely on the host to authenticate the user and then trust the host; require the host to prove its identity and then trust the host as to who the user is; make the user prove who they are for each network service. 3.2.1. Kerberos ``In an open network computing environment, a worksta- tion cannot be trusted to identify its users correctly to network services.''[4] Therefore Kerberos uses the third approach to system security; make the user prove their iden- tity for each network service. In order for a user to prove their identity, they must be authenticated by Kerberos, not the workstation they are using. Passwords are never sent over the network, but are used locally to decrypt the authentication message from Kerberos. To prevent unauthor- ized use the local workstation destroys the user's password after using it to decryt the initial Kerberos message. Once a user has been authenticated they have the keys to request various network services. Different applications can choose different levels of protection. The first is authentication at initiation but subsequent messages are just accepted if from the same network address. The second is where each message is authenticated but the contents of the message are not encrypted. The third level of security is private messages where each message is authenticated and encrypted. The Kerberos database contains a name, private key, and expiration date for each entity that will use Kerberos ser- vices. The master Kerberos database is kept and modified on one machine. Slave servers have read only versions of the database and provide read only types of services. Modifica- tion to the master database is accomplished by the adminis- tration server (KDBM server). There are two parts to this service, a client which will run on any machine in the net- work and a server that must run on the machine which houses July 17, 1989 - 10 - the master database. 3.3. Accounting Management Accounting is the recording and reporting of resource usage. This information can then be used to determine appropriate charges for a user. 3.3.1. Harvard Accounting System This system would track disk usage, cpu time, logins, connect time, printed pages, and budget on an account-by- account basis and charge the appropriate accounts. It was designed to run in a single machine environment. 3.4. Fault Management In order to restore service after a disk failure a sen- sible backup procedure needs to have been followed by the administrative staff. Basic commands to move data from one medium to another are described below. Tar and cpio file archiving and data interchange for- mats are the only backup formats specified in 1003.1. 3.4.1. System V Interface Definition (SVID) 3.4.1.1. volcopy The volcopy command will make a literal copy of a file system. Copies can be made to another disk location or to tape. 3.4.2. SVID & Berkeley 3.4.2.1. tar The tar command is used to create an archive file. Mul- tiple files can be saved to and restored from a single tar- file. The tarfile can reside on various physical media. tar will read from standard input and write to standard output so that it can be part of a pipeline. This feature can be used for moving directories. 3.4.2.2. cpio cpio copies a list of files to from a cpio archive file. Pathnames and status information are kept along with the files. 3.4.3. Berkeley dump/rdump/restore/rrestore The dump and rdump commands will copy all files in a filesystem to backup media. The restore and rrestore July 17, 1989 - 11 - commands will copy files stored via dump to a filesystem. Rdump and rrestore provide the same functionality as dump and restore over a network. Remote dump devices are speci- fied as a host-device combination. The dump command allows for different levels of back up. A level 0 dump copies every file in the filesystem. A level 5 dump would copy every file that has been modified since the last dump of a lower level. 3.5. Performance Management Performance management analyzes the output from system statistics to determine problem areas and develop solutions. 3.5.1. Berkeley Performance Monitoring Commands The following commands are executable directly on each machine to report local status. 3.5.1.1. vmstat The vmstat command provides information on the memory usage, process status, and disk utilization. 3.5.1.2. iostat The iostat command reports statistics related to I/O operations. Both terminal and disk I/O are included. 3.5.1.3. netstat The netstat command displays the contents of the network-related data structures. Information is provided about established connections and gateways. 4. Work in Progress 4.1. OSF RFT The Open Software Foundation will be issuing an Request for Technology (RFT) for Systems Administration software from the Munich office some time in August 1989. 4.2. FDDI A group is forming to determine which variables are appropriate for inclusion in the MIB for FDDI. 4.3. Network Management Language ``NML is seen as a canonical interface between the net- work management application programmer and the MIXP (Manage- ment Information Exchange Protocol).''5 It isolates the applications programmer from the specific MIXP being used. July 17, 1989 - 12 - Extending this to systems administration would enable the underlying protocol to be changed without the systems administrators programming environment to be changed. 5. Acknowledgements We would like to thank the following people for provid- ing information, support, and inspiration, Carolyn D. Coun- cill, John Lees, Jackie Carlson, Doug Gwyn, Keith Bostic, Clifford Neuman, Mark Ozur, Martin Schoffstall, Frank Cun- ningham, Paul Stutler, Ted Cook, and John Bossert. 6. Authors' Addresses Susanne W. Smith Windsound Consulting 6225 137th Place SW Edmonds, WA 98020 <smith@usenix.org> John S. Quarterman TIC 701 Brazos, Suite 500 Austin, TX 78701-3243 <jsq@usenix.org> References 1. Ben-Artzi, Amatzia, "The CMOT Network Management Archi- tecture," ConneXions, vol. 3, pp. 14-19, Advanced Com- puting Environments, Mountain View, California, March 1989. 2. McCloghrie, Keith and Marshall T. Ross, "Network Management of TCP/IP-based internets," ConneXions, vol. 3, pp. 3-9, Advanced Computing Environments, Mountain View, California, March 1989. 3. Rosenstein, Mark A., Daniel E. Geer, Jr., and Peter J. Levine, "The Athena Service Management System," USENIX Conference Proceedings, pp. 203-212, USENIX Associa- tion, Dallas, Texas, February 9-12, 1988. 4. Steiner, Jennifer G., Clifford Neuman, and Jeffrey I. Schiller, "Kerberos: An Authentication Service for Open Network Systems," USENIX Conference Proceedings, pp. 191-202, USENIX Association, Dallas, Texas, February 9-12, 1988. July 17, 1989 - 13 - 5. Warrier, Unni, "A Network Management Language," ConneX- ions, vol. 3, pp. 33-39, Advanced Computing Environ- ments, Mountain View, California, March 1989. Bibliography 1. System V Interface Definition, I, II, III, AT&T, 1986. 2. Networking on the Sun Workstation, Sun Microsystems, Inc., Mountain View, California, February 1986. 3. System Administration for the Sun Workstation, Sun Microsystems, Inc., Mountain View, California, February 1986. 4. USENIX Proceedings of the Large Installation Systems Administrators Workshop, USENIX Association, Philadel- phia, Pennsylvania, April 9-10, 1987. 5. USENIX Proceedings of the Large Installation Systems Administrators Workshop, USENIX Association, Monterey, California, November 17-18, 1988. 6. Arnold, Edward R. and Marc E. Nelson, "Automatic Unix Backup in a Mass-Storage Environment," USENIX Confer- ence Proceedings, pp. 131-136, USENIX Association, Dal- las, Texas, February 9-12, 1988. 7. Ben-Artzi, Amatzia, "The CMOT Network Management Archi- tecture," ConneXions, vol. 3, pp. 14-19, Advanced Com- puting Environments, Mountain View, California, March 1989. 8. DellaFera, C. Anthony, Mark W. Eichin, Robert S. French -, David C. Jedlinsky, John T. Kohl, and William E. Sommerfeld, "The Zephr Notification Service," USENIX Conference Proceedings, pp. 213-220, USENIX Associa- tion, Dallas, Texas, February 9-12, 1988. 9. Dyer, Stephen P., "The Hesiod Name Server," USENIX Conference Proceedings, pp. 183-190, USENIX Associa- tion, Dallas, Texas, February 9-12, 1988. 10. Eaton, Charles K., "Project Accounting on UNICOS," USENIX Conference Proceedings, pp. 163-170, USENIX Association, Dallas, Texas, February 9-12, 1988. 11. Epstein, Mark E., Curt Vandetta, and John Sechrest, "Asmodeus: A Daemon Servant for the System Administra- tor," USENIX Conference Proceedings, pp. 377-392, USENIX Association, San Francisco, California, June 20-24, 1988. July 17, 1989 - 14 - 12. Fiedler, David and Bruce H. Hunter, UNIX System Administration, Hayden Books, Indianapolis, Indiana, 1988. 13. Howard, John H., "An Overview of the Andrew File Sys- tem," USENIX Conference Proceedings, pp. 23-36, USENIX Association, Dallas, Texas, February 9-12, 1988. 14. Hume, Andrew, "An Incremental Backup System for UNIX," USENIX Conference Proceedings, pp. 61-72, USENIX Asso- ciation, San Francisco, California, June 20-24, 1988. 15. III, Douglas P. Kingston, A Tour Through the Multi- Device Queueing System, Ballistic Research Laboratory, Aberdeen Proving Grounds, Maryland, July 25, 1984. 16. Jatkowski, Paul, "PMON: Graphical Performance Monitor- ing Tool," USENIX Conference Proceedings, pp. 111-118, USENIX Association, Dallas, Texas, February 9-12, 1988. 17. Jones, Von, "System Administration Daemons," USENIX Conference Proceedings, pp. 137-144, USENIX Associa- tion, Dallas, Texas, February 9-12, 1988. 18. Krempel, Henry B. J. and John F. Fowler, "High- Performance Workstations in a Model University Environ- ment," Northeast Parallel Architectures Center Techni- cal Report, Syracuse University, Syracuse, New York, April 7, 1988. 19. McCloghrie, Keith and Marshall T. Ross, "Network Management of TCP/IP-based internets," ConneXions, vol. 3, pp. 3-9, Advanced Computing Environments, Mountain View, California, March 1989. 20. Partridge, Craig, "A UNIX Implementation of HEMS," USENIX Conference Proceedings, pp. 89-96, USENIX Asso- ciation, Dallas, Texas, February 9-12, 1988. 21. Pato, Joseph N., Elizabeth Martin, and Betsy Davis, "A User Account Registration System for a Large (Hetero- geneous) UNIX Network," USENIX Conference Proceedings, pp. 155-172, USENIX Association, Dallas, Texas, Febru- ary 9-12, 1988. 22. Peacock, Don and Mark Giuffrida, "Big Brother: A Net- work Services Expert," USENIX Conference Proceedings, pp. 393-398, USENIX Association, San Francisco, Cali- fornia, June 20-24, 1988. 23. Rosenstein, Mark A., Daniel E. Geer, Jr., and Peter J. Levine, "The Athena Service Management System," USENIX Conference Proceedings, pp. 203-212, USENIX Associa- tion, Dallas, Texas, February 9-12, 1988. July 17, 1989 - 15 - 24. Steiner, Jennifer G., Clifford Neuman, and Jeffrey I. Schiller, "Kerberos: An Authentication Service for Open Network Systems," USENIX Conference Proceedings, pp. 191-202, USENIX Association, Dallas, Texas, February 9-12, 1988. 25. Treese, G. Winfield, "Berkeley UNIX on 1000 Worksta- tions: Athena Changes to 4.3BSD," USENIX Conference Proceedings, pp. 175-182, USENIX Association, Dallas, Texas, February 9-12, 1988. 26. Warrier, Unni, "A Network Management Language," ConneX- ions, vol. 3, pp. 33-39, Advanced Computing Environ- ments, Mountain View, California, March 1989. 27. Yeong, Wengyik, Martin Lee Schoffstall, and Mark S. Fedor, "A UNIX Implementation of the Simple Network Management Protocol," USENIX Conference Proceedings, pp. 209-218, USENIX Association, San Diego, California, January 30 - February 3, 1989. July 17, 1989 Volume-Number: Volume 18, Number 9