Preface

How can he possibly be humble? He hasn't done anything yet.

This dissertation was motivated by the need to manage complex information. The wealth of the industrial economies is increasingly in the form of information, rather than of physical materials. In the United States, information services have grown relative to the rest of the economy since 1860, with a sharp acceleration after 1940 [53]. For a recent account of the implications of this evolution for the workforce, see Hunt [51]. A monograph surveying the underlying trends, and projecting their social and political implications, can be found in Prothero [79].

A similar evolution affects engineering. Engineering will increasingly be limited less by the properties of materials and more by the complexity of the tasks undertaken. This is because more powerful tools continually tilt the balance of difficulty further from the mechanics of construction and towards design and maintenance. As an indication of where engineering as a whole may be headed, consider the discipline most limited by complexity (and least by physical constraints), software engineering. The centrality of the complexity problem to software engineering is well-understood by its practitioners. According to Gelernter, ``If you're a software designer and you can't master and subdue monumental complexity, you're dead: your machines don't work'' [27] (p. 52). Similarly, Fredrick Brooks, in the 1995 update of his classic software engineering book The Mythical Man-Month [26], states that ``complexity is the business we are in, and complexity is what limits us.'' (p. 226; emphasis in the original.)

The frequent examples of complexity-related software failures are often summarized in the Association for Computing Machinery's Risks Digest [6]. In the May 1997 issue, for instance, we read of the 15-month delay in the opening of the ``world's most advanced air-traffic-control centre'' as a result of software bugs buried in 1.82 million lines of code [58].

One particularly intricate branch of software engineering, compiler construction, has evolved a mini-mythology to express the complexity problem. Successive editions of ``Compilers: Principles, Techniques and Tools'' [4] have featured a knight armed with the tools of compiler construction doing battle with a ``dragon of complexity''. It is a creature whose hot breath will become increasingly familiar to engineers of all disciplines as we progress into the information age. This dissertation is, in a sense, a joust with the dragon.

It is possible to cleave complexity into two related (and overlapping) areas of research. The first deals with the manipulation of information inside of coherent modules. This is in essence the study of algorithms, the domain of computer science. The other is the flow of information across the boundary of modules: the study of interfaces. As yet, there is no established academic setting for the study of interfaces. It incorporates parts of various disciplines, such as: computer science; education; electrical engineering; ergonomics/human factors; industrial engineering; mathematics; psychology; and technical communication. While the general definition of an interface includes the information boundary between any two or more systems, for instance two software or hardware modules, this dissertation is concerned only with the human-computer interface.

Comparing algorithms to interfaces, I suggest that interfaces are in the long run the harder problem. The reason is that complexity is related to uncertainty. The greatest uncertainty occurs where distinct systems come together (thus losing internal consistency): i.e., at the interface. It follows that the design of effective interfaces is central to confronting the complexity problem in engineering.

It is unfortunate, therefore, that we do not know more about how to design interfaces. The development of an interface for a new problem is guided either by general rules, which provide little precision for the problem at hand [92], or by usability testing [23], which has to be repeated for each new interface. At present, interface design is generally a trial-and-error process, with the consequent costs to development time and quality. As the tasks we need to perform become more complex, both the requirements placed on interfaces, and the difficulty of achieving these requirements, will increase.

The (perhaps unattainable) ideal would be to have a mature field of ``interface engineering'', which would allow one to systematically construct interfaces with predictable properties, in terms of their ability to convey information both directions across the human-computer boundary. To the extent that such a branch of engineering were achievable, it would augment the current techniques for building interfaces based on general rules, experience, intuition and usability testing.

Engineering (in our case, the hypothetical ``interface engineering'') does not strictly require, but does benefit from, an underlying scientific theory which provides a basis for predicting performance. Science in turn requires measures which can be used to accurately assess the outcome of experiments. A goal of this dissertation, therefore, was to investigate measures related to the goodness of interfaces. (For a brief survey of existing interface goodness measures, currently in a rudimentary state of development, see [78].)

This dissertation focuses on ``virtual interfaces'', a somewhat ill-defined term. I choose to define a ``virtual interface'' as one which has two properties: ``sensory immersion'' (meaning that the interface makes use of a large percentage of the human sensory bandwidth); and the ability to induce a sense of ``presence'', or of ``being in'', an environment implied by the interface. There are at least three reasons why the study of virtual interfaces is important. The first reason is that the sensory immersion which is characteristic of virtual interfaces has the potential to increase the human-computer bandwidth, an important point in view of the complexity problem. The second reason is that virtual interfaces will become increasingly common as technology advances. The third reason is that psychologically, virtual interfaces can be easier to study than traditional interfaces, in that by ``tricking'' the perceptual system virtual interfaces can evoke stronger psychological responses.

The fundamental advantage of virtual interfaces is that, by making better use of natural human sensory inputs, they have the potential to greatly increase the human-computer communication bandwidth. The fundamental disadvantage of virtual interfaces is that (precisely because they can make compelling use of the human perceptual system) virtual interfaces have the capability to cause unwanted side-effects. These side-effects include simulator sickness and postural instability.

Two crucial human factors problems limit the effective use of virtual interfaces. The first problem is the lack of robust measures for how good virtual interfaces are, without which it is difficult to build the knowledge needed for systematic and high-quality interface engineering. The second problem is side-effects. It is suggested here that useful light can be shed on both of these problems (and quite a few others) by a single, concise framework: the rest frame construct. The purpose of this dissertation is to introduce the rest frame construct and its applications.

This dissertation required the support and guidance of many people. I am pleased to acknowledge their efforts. I hope that the final result is worthy of their involvement.

I would like to thank the members of my supervisory committee: Dr. Thomas Furness (chair), Dr. Linda Brubaker, Dr. Earl Hunt, Dr. Kailash Kapur, Dr Gregory Miller and Dr. Donald Parker. Dr. Maxwell Wells informally played a similar advisory role, as well as administering the research grant under which most of this work was funded.

I owe additional thanks to Drs. Furness, Hunt, Parker and Wells for providing detailed comments on several drafts of this dissertation. The final version is greatly improved as a result of their involvement. Of course, I must take personal responsibility for remaining flaws.

My research was most influenced by Drs. Furness, Parker and Wells. Together, they provided a truly impressive expertise in perceptual psychology and the human factors of virtual environments. While the framework developed here is primarily of my own devising, it certainly would not have arisen without their insight and support. I owe a particular debt to Dr. Parker for introducing me to the literature on perception, for guidance on the design of the visual-inertial nulling experiments, and for helping to arrange for the loan of a rotating chair from the National Aeronautics and Space Administration (NASA).

While I was the first author for all experiments reported here except Experiment CogE1 in Appendix A, many of these experiments were conducted as collaborations. In particular, Dr. Hunter Hoffman played a very active role in the reported presence foreground occlusion experiments (AIIP1, AIIE1 and AIIE2) and Mark Draper was similarly involved in the ``low-end'' simulator sickness experiments (AIIIE1, AIIIP1 and AIIIE2). I learned a great deal from both and enjoyed their company.

Joris Groen, a visiting student from Leiden University in the Netherlands, was kind enough to administer Experiment AIIE2, which followed a double-blind procedure.

A short pilot study (AIIIP2) was conducted using a driving simulator at Hughes Research Laboratories (HRL). This study was done in collaboration with Dr. Wells, and with the assistance of HRL staff. I am particularly grateful to Dr. Peter Tinker of HRL for technical assistance, and to HRL for allowing use of this facility.

Another pilot study was carried out at the University of Washington's psychology department, with the aid of Dr. Hoffman, Jennifer McLean, and Dr. Geoffrey Loftus. While the study is not reported in this document it did help to guide later research. I thank them for their assistance.

Technical support was of course crucial to all of my dissertation research. I am particularly grateful for the software development of Paul Schwartz. Mr. Schwartz provided working software prototypes for many of the experiments described in this dissertation, which I subsequently enhanced as needed. He showed a remarkable patience, diligence and good humor throughout.

The visual-inertial nulling experiments of Chapter 4 and Appendix B were technically challenging, due both to the inherent difficulties of the design and due to repeated hardware failures. The electronics expertise of Robert Burstein was often of great use. Others who contributed to the hardware support of these experiments included Chris Airola, Arthur Gonzales, Jaswant Jabal and Herb Kramer.

I am grateful to Toni Emerson and her excellent assistants in the Human Interface Technology Laboratory (HITL) library. Good information is essential to research, and I feel very fortunate to have had access to a research library devoted to virtual interfaces.

This research was conducted at the HITL. While most of the HITL staff were not directly involved in this research, this dissertation would not have been possible without them. Rather than single out individuals, I would like to thank them all for their fine work.

While I am grateful for the assistance of many people from outside of the University of Washington, I would like to acknowledge two in particular. One is Dr. John Jahnke, who visited from Miami University in Ohio. Dr. Jahnke suggested having participants indicate the perceived inertial endpoints of their motion in the visual-inertial nulling experiments. This was a great improvement over the technique I had previously tried for measuring visual or inertial dominance. The second is Dr. Robert Patterson of Washington State University, for discussions of binocular rivalry (see Appendix E).

My appreciation also goes out to the numerous experimental participants who, with remarkable good humor, took part in a wide variety of odd experiments.

It is usual in dissertations to acknowledge the friends and family members who one to varying degrees deserted during the Ph.D. process, and who nevertheless provided their unflagging support. I have come to understand why this tradition arose. Of the many to whom my thanks are due, I mention in particular my fiancee, Rita Solon; my parents, John and Joyce Prothero; my brother, Jeff Prothero; my sister, Jacky Jeffery; and the members of my immediate ``chess circle'', Philipp and Vera Frenkel, Drayton Harrison, Ken Plesset and David Zick.

The experiments described in Chapter 4 made use of a rotating chair (Contraves Goerz Co. Direct Drive Rate Table Series 800 and a Neurokinetics, Inc. Motion Simulator Controller). This was graciously loaned to Dr. Parker by NASA.

The Division ProVision 100's used in Experiments AIIP1, AIIE1 and AIIE2 were acquired as the result of a grant from the U.S. West Foundation to a HITL educational project.

This research was supported in part by the Air Force Office of Scientific Research (Grant F49620-93-1-0339; Dr. John Tangney, contract monitor) and NASA Grants NAS 0-703, NAG5-4074 and 9-958, Basic.

Finally, I would like to thank the U.S. taxpayers who, whether or not they would approve, have funded the following research.

To the memory of my grandparents, and to Rita.