![[Previous Chapter]](/icons/button-back.gif){kind=icon}
![[Table of Contents]](/icons/button-contents.gif){kind=icon}
![[Next Chapter]](/icons/button-next.gif){kind=icon}
| Publications Page | HITL Home |
4.1 Overview of Experimentation
This chapter provides an empirical preface to the dissertation research discussed in Chapters 5 through 8. Early research is summarized, marking a progression from peripheral concepts to central focus. This chapter also presents is a detailed description of the research facility that was constructed to accomplish these and other similar research agendas. Lastly, an overview of the main dissertation research is provided.
The following studies illustrate the empirical journey towards the main research questions. Initial efforts primarily involved assessments of postural adaptation while later work focused on oculomotor adaptation. Some research on simulator sickness was also conducted. These experiments provided a foundation upon which to develop a better understanding of the issues surrounding adaptation, simulator sickness and oculomotor research.
This study arose from an incidental observation that I had regarding postural stability. I noticed that a fellow graduate student and I experienced more ataxia while performing the Sharpened Romberg stance (eyes closed) after prolonged sitting than after standing. A hypothesis was formed theorizing that postural control mechanisms adapt to extended periods of sitting, which results in temporary low-magnitude negative aftereffects when postural position is suddenly and radically changed.
The eyes-closed Sharpened Romberg is a very sensitive stance for identifying changes in postural stability (Hamilton, et al., 1989) and the Chattecx Balance Platform is well equipped to detect small postural changes (see Section 4.3.1 for a description of the platform). Therefore, to detect these hypothesized low-magnitude postural effects, the Sharpened Romberg stance was employed while the subject stood on the balance platform.
A pilot study of four subjects (counterbalanced, within-subjects design) provided an indication that the hypothesis may be supported. Subjects either sat or stood for 10 minutes prior to being tested on the Chattecx Platform (in the eyes-closed Sharpened Romberg position). The platform test recorded postural changes over a 10 second period. A paired t-test (assuming unequal variances) indicated a strong trend towards significance (p < 0.06) for position prior to balance test, with more ataxic effects occurring after 10 minutes of sitting vs. 10 minutes of standing (Figure 21).
A more rigorously controlled evaluation with 10 subjects failed to replicate the results of the pilot study, due in part to excessive variance in the data (the eyes-closed Sharpened Romberg stance is potentially a very sensitive metric for ataxia but it is also extremely variable). This experiment, however, was my first exposure to physiological adaptation research and as such it provided a good introduction to the complexities of the area.
A set of ataxia experiments was conducted with a fellow graduate student to determine the effects of HMD display transparency (completely occluded versus partially see-through) on ataxia, vection, and simulator sickness. A circular vection stimulus (in yaw) was presented to the subject though a Virtual i/O HMD (this HMD is described in Section 4.3.1). In one HMD condition, the subject could partially see the laboratory room superimposed with the virtual rotating scene when the subject viewed the display (the see-through display condition). In the occluded HMD condition, only the virtual vection scene could be seen. In both cases, the edges of the display were occluded to prevent peripheral viewing. It was hypothesized that the occluded condition would result in more ill-effects (in the form of ataxia and simulator sickness) due to a switching of internal reference frames (Prothero, 1998). A paper fully describing this research has been submitted for review in anticipation of publication (Prothero, Draper, Furness, Parker, & Wells, submitted). Below is a summary of the experimental details most relevant to this dissertation.
The first ataxia experiment (Ataxia #1) was my first experiment investigating simulator sickness. A total of 15 subjects were tested using a counterbalanced, within-subject, one-factor design. The factor manipulated was HMD transparency (see-through, occluded). The yaw vection stimulus rotated at 60 degree/sec so as to be maximally provocative for inducing both vection and sickness (Griffin, 1991). Kennedy�s Simulator Sickness Questionnaire (SSQ) was administered to assess sickness symptoms both pre- and post-exposure to each condition. Ataxia was measured during the 3 minute stimulus exposure while the subjects tried to maintain balance in the Sharpened-Romberg (eyes-open) position. Stance breaks, defined as the number of times subject broke the Sharpened Romberg stance, were used as the measure of ataxia. Stance breaks were recorded during the first and third minute of exposure, then totaled to provide an overall measure of ataxia for that condition.
One subject�s data were not analyzed due to his excessive difficulty in maintaining balance during the test; the results of the remaining 14 subjects are discussed here. Data were analyzed using a 2-tailed paired t-test (for stance measures) and a non-parametric, 2-tailed paired Wilcoxon (for SSQ data). The occluded display resulted in significantly more sickness symptoms (Figure 22) and more ataxia (Figure 23) than the see-through display. There was also an increase in stance breaks over the 3 minute exposure when pooled across display condition (Figure 24), which suggests that ataxia increased with exposure time.
This experiment provided an empirical introduction to the syndrome of simulator sickness. It also demonstrated that specific characteristics of a virtual interface could modulate sickness incidence as well as essential physiological control processes. Furthermore, the decrease in postural control over exposure time confirmed the importance of the visual motion cues in the maintenance of postural stability, even when those cues are at odds with the inertial cues. Visual-inertial stimulus rearrangements have definite physiological effects. The results of this experiment led to a more detailed follow-on effort, aptly named Ataxia #2.
Ataxia #2 addressed issues raised in Ataxia #1 and it added a pre- and post-exposure balance test (using the Chattecx system described in Section 4.3.1) to study ataxic aftereffects. Other changes included the addition of a visual search task to maintain subject attention within the virtual vection scene, a removal of head tilts performed by the subject during exposure, and increasing the exposure time from 3 to 4.5 minutes. A total of 21 subjects were tested in a counterbalanced, within-subject, one-factor design.
The data presented below were analyzed using the same statistical tests as in Ataxia #1. The results again showed a reduction in ataxia during exposure for the see-through display (p < 0.04) (Figure 25) and there was a slight trend toward less sickness reports for that same condition. The overall strength of the signal declined, however, most probably due to the removal of head tilts in this study. This probably also accounted for the failure of the sickness data to reach statistical significance. An interesting finding regarding aftereffects also occurred. Post-exposure ataxia did not differ significantly between display conditions (likely because of the low overall response magnitudes generated). When pooled, however, the post-exposure values were significantly worse than pre-exposure values (p < 0.04) (Figure 26). Thus, a 4.5 minute exposure to a virtual interface resulted in negative aftereffects of a fundamental physiological control system. This further solidified the assertion that virtual interfaces can alter essential physiologic processes, even after short duration exposures. However, it should be noted that this virtual interface was highly geared towards vection at the expense of naturalistic interaction (though some VR entertainment applications may strive to achieve this same effect).
4.2.4 Pilot Study on Oculomotor Adaptation
Given that virtual interfaces could effect postural control both during and after exposure, a pilot study was conducted to determine if these interfaces could also alter oculomotor response. The hypothesis was that short-term exposure to a immersive virtual interface could result in altered oculomotor responses associated with the VOR and eye-head gaze shifts.
This experiment was conducted at the Jules Stein Eye Institute on the UCLA campus. The reason for the remote test site was to utilize a magnetic search coil eye tracking system. The search coil technique is a high resolution (spatially and temporally) system for measuring eye and head movements.
The virtual interface consisted of a Virtual i/O HMD, its associated head position tracking system, and VE software. The HMD was full color, bi-ocular, and collimated to approximately 3.35 m. with a DFOV of approximately 19 degrees vertical by 25 degrees horizontal (see Section 4.3.1 for a more detailed description of the HMD). The head tracker, sold by Virtual i/O along with the HMD, utilized a compass (for yaw movements) and liquid-based tilt sensors (for pitch and roll movements) in order to track head rotations at a 250 Hz update rate (head translations were not tracked). The VE was a virtual reality game provided with the HMD called �Ascent�. This game, which ran on a Pentium PC, required many yaw head rotations by the subject in order to successfully advance.
Due to equipment malfunction, data were collected from only one subject. The subject�s VOR was measured before and after a 12-minute exposure to the immersive virtual interface. Results indicate that both gain and phase of the VOR changed from baseline levels (Figure 27). This suggests that oculomotor adaptations may occur from using an immersive virtual interface in a naturalistic fashion. Interpretation of these results is restricted given that only one subject was run. However, this study provided a basis for more rigorously exploring oculomotor adaptation to virtual interfaces.
4.2.5 Missteps, Roadblocks, and Should-Have-Workeds
Lest it be presumed based upon the orderly nature of this presentation that early research proceeded smoothly and quickly from step to step, it should be made clear that the �steps� appear orderly because the �trips� were omitted from detailed discussion. For instance, the selection of an appropriate eye tracking system was an adventure to say the least. From early troubles in getting an antiquated BioMuse to track eye movements, to trouble with a malfunctioning EOG eye tracking system (Figure 28), to continually jerking the head of a dissertation committee member wearing an inexpensive VOG system as I attempted to collect pilot VOR data without use of sinusoidal oscillations (the technique, which relied on corrective saccades, is shown in Figure 29, though all we ever obtained were sore necks), to the mistake of trying an ISCAN beta-version VOG camera system (which we returned after two months of strange behavior), the road was indeed a circuitous one. Also not included are the numerous failed simulation/optimization attempts, pilot study miscues, homemade VOR analysis programs, manual de-saccading techniques that gave horrendous results, a mid-stream HMD switch, multiple lab reconfigurations, etc. But with each failure and misstep came increased understanding and experience which resulted in an improved dissertation program overall.
4.2.6 Summary of Preliminary Research
Early research advanced this dissertation in two ways: experience and relevant findings. First, these studies provided an empirical introduction to the topics of human physiological adaptation, oculomotor research, and simulator sickness. Knowledge was gained, issues were raised, and lessons learned. Second, results from this research provided logical stepping stones towards the major questions asked in this research. Physiological recalibration processes (in the form of postural stability) were observed during and after exposure to a virtual interface. In addition, modifications to the virtual interface modulated physiological response characteristics. Finally, VOR changes were observed in one subject after a fairly short exposure to a commercially available immersive virtual interface.
Prior to initiation of a research agenda into the physiologic effects of virtual interfaces, a facility had to be constructed with the specialized capability to explore these issues. With the full backing and support from the HITL Director, a physiologic mini-laboratory was assembled at the HITL. This lab, titled the VR Effects Lab, has the capability to research a range of oculomotor, postural stability, and psychophysical responses to virtual environments. Below is a detailing of the lab�s equipment along with its current configuration.
Visual image: WARP TV software provides a low latency virtual image in response to head movements. A 360-degree cylindrical image is pre-computed and pre-rendered into memory (RAM) at program initiation (Figure 30). When a subject moves his/her head, the portion of the image that corresponds to the new head position is drawn on the display directly from RAM (shown as the square insert in Figure 30) via a memory address computed from head position sensor data. The displayed image does not have to be rendered in real time, reducing total system latency. The measured update rate is 65 fps in isolation and approximately 45 fps when integrated with the head tracker. Only rotations (pitch, yaw, and roll) are registered; linear translation through the VE is not an option with the current configuration. WARP TV has a selectable time-delay buffer so that a desired time delay can be directly input into the system. The minimum system time delay (from head tracker movement to visual scene update) is 48 ms and the maximum attainable is approximately 500 ms.
WARP TV runs on a Pentium 166MHz PC with an ATI Technologies Mach 64 Pro Turbo graphic accelerator with 4 MB of video memory. The PC has 16 MB of RAM and a PCI bus.
HMD: A Virtual i/O HMD was used as the virtual interface display (shown in Figure 31 with the eye tracker mounted) for these experiments. This HMD has 2 full-color, 1.78 cm active matrix liquid crystal displays (each with 180,000 addressable pixels) and the unit accepts VGA input at 60 or 70 Hz (Real Time Graphics, 1995). Angular resolution is 6.84 arc min/color pixel. This HMD has a 25 deg horizontal by 19 deg vertical DFOV, with 100% overlap between the two eyes (the virtual images were not presented in stereo). The Virtual i/O HMD has a fixed focus at 3.35 m in order to minimize eye-strain. Though the HMD can be worn with glasses, it was not an option in these experiments because the spectacle lenses would interfere with the attached infrared eye-tracking system. The weight of the HMD is approximately 228 grams.
Head Position Tracking: An InterSense IS-300 system is used to track head movements. �sourceless� technology (a miniature, solid-state, drift-corrected inertial measurement unit with angular rate sensor, gravitometer, and compass), the system has essentially unlimited range. It also minimizes �jitter� in the visual scene and is scarcely impacted by electromagnetic interference. The sensor unit�s dimensions are 2.54 x 3.3 x 3.0 cm and it weighs 60 grams. When used in the current configuration, the system updates at 250 Hz (system latency of approximately 4 ms) with an angular resolution of 0.02 deg RMS, a dynamic accuracy of 1 deg RMS and a static accuracy of 3 deg RMS. The tracker was mounted directly above and centered on the subjects head via an aluminum bar attached to the HMD (Figure 32).
The InterSense system was replaced the Shooting Star ADL-1 (a mechanical head tracking system based on high precision potentiometers) because it performed as well without the safety implications of a mechanical linkage coupled to the subject�s head. However, the ADL-1 remained as a back-up unit and was useful for calibration purposes (see Appendix A). Therefore, its specifications are included for completeness. The ADL-1 provides good resolution (0.15 to 0.30 deg), repeatability (less than 2.5 mm) and low system latency (3 to 5 ms). The ADL-1 has a limited working volume due to the mechanical linkages (half cylinder, approximately 1 m diameter, 0.5 m high).
Eye Position Tracking System: An ISCAN video-oculography (VOG) system is used to measure eye position relative to the subject�s head (Figure 33). This system utilizes a high-speed CCD camera, a real-time digital image processor, and an IR LED emitter. The camera and emitter are mounted on the Virtual i/O HMD (Figure 32). The camera records a video image of the left eye as reflected through the HMD optics and off a dichroic mirror. The IR LED provides infrared illumination (not coaxial with the eye imaging camera) so that the pupil and a corneal reflection can be identified in software. The system uses �dark pupil� processing, with the pupil acting like a sink and the surrounding area of the eye reflecting the IR back towards the camera. The IR illumination also allows for the unit to be used in complete darkness. Software detects and tracks the pupil centroid as well as a corneal reflection even in the presence of shadows and/or eyelashes in the image. The system has a selectable update rate of 60, 120, 180, or 240 Hz and an average horizontal resolution of 0.5 deg. For these experiments, the ISCAN system update rate was fixed at 180 Hz. The ISCAN system is controlled via a Pentium P-166 PC with 32 MB of RAM.
Rotating Chair: A rotating chair (Contraves Direct-Drive Rate Table; Series 800; Figure 34) was obtained via a loan from NASA (through Dr. D.E. Parker). A Neurokinetics Servo Controller and Stanford Research Systems function generator (Model DS-345) are used to control the driving signal to the chair. For these experiments, the chair was always commanded to perform sinusoidal oscillations between 0.2 and 0.8 Hz at no more than 50 deg/s peak velocity.
Balance Platform: A Chattecx Balance Platform is used to assess balance (Figure 35). The subject stands with a foot on a each of two force-sensing plates that rest on the platform base. The force plates detect shifts in the subject�s center of gravity over a 10 or 25 s test. The main measure of a subject�s balance is an �index of dispersion�, which is the standard deviation of the subject�s center of gravity (in cm) over the period of the test. The platform has an update rate of 100 Hz. The experimenter controls the balance test via an integrated PC (286 Intel processor) running Chattecx DOS-based software. Subjects can stand in many different stances, including the sensitive Sharpened Romberg stance, simply by moving the two force-sensing plates. In addition, the base of the platform can be made to move in sinusoidal or impulse fashion in order to explore dynamic postural control. There are safety rails around the platform base and an optional harness can be installed for further protection. This platform is also on loan from NASA (through Dr. D.E. Parker).
Acquisition/Analysis software: NI-DAQ software and data acquisition cards from National Instruments control transmission of data from the two Pentiums to a Macintosh computer for storage. MacEyeball software, developed by researchers at UCLA, controls test file configuration, storage, and analysis.
There are two separate MacEyeball programs, one for data acquisition (MAQ) and one for data analysis (MAP). The MAQ allows the experimenter to set the parameters of a VOR test (including experimenter comments), calibrate eye and head position data, determine the sampling rate for data collection, and initiate VOR testing. After VOR test completion, MAQ saves the collected eye and head position data in a proprietary format prior to the beginning of a new test trial. The MAQ needed to be slightly modified in order to function correctly using the equipment configuration in the VR Effects Lab. Some extraneous functionality was removed and the program was modified to read from the appropriate A/D DAQ card.
The MAP reads files stored by MAQ and performs VOR analysis using three methods: XY Analysis, Fourier, and Varant. However, only the latter two methods were used by this research. The specially coded MAQ files contain the relevant information needed for automated VOR analysis including sampling rate and testing frequency. A detailed description of the overall capabilities of MAP can be found in Demer, et al., (1989) and Demer (1992). A summary of its operation is described in Section 5.6.1.
The MAP had to be altered to better link with the capabilities of the specific equipment used to collect eye and head position information. Specifically, the saccadic removal subroutine and the low-pass frequency cut-off were adjusted after many simulations to achieve optimal performance.
Computers: Two PCs (Pentium P166 MHz processors) and one Macintosh IIfx computer exist in the lab. The WARP Pentium (which controls the virtual image) has 16MB of RAM and an ATI Mach 64 graphics accelerator card while the VOG Pentium (which controls the ISCAN system) has 32 MB of RAM. The Macintosh (used for data acquisition and storage) has 32 MB RAM and includes a ZIP Drive and CD ROM player.
4.3.2 VR Effects Laboratory Configuration
The physical layout of the VR Effects Lab is described first, followed by the specific equipment configuration utilized for these experiments.
4.3.2.1 Physical Layout
The physical layout of the VR Effects Lab is shown in Figure 36. There are three areas: an experimenter control shed, a subject area, and a reception area.
The experimenter shed contains all the equipment that needs to be operated during a VE exposure and VOR test. The experimenter is enclosed in the shed with the necessary equipment, and the shed is made light tight with respect to the other two areas. Equipment in the shed includes the Macintosh data acquisition computer, the WARP Pentium PC, the servo-controller, the function generator, and a CRT displaying the subject�s eye image as seen through the ISCAN camera. Using this equipment, the experimenter can monitor what the subject sees in the VE (using the WARP PC monitor), change VEs, monitor eye image quality, start, stop and alter chair oscillations, and control data collection for during each test trial (see Figure 37 for an internal view of the shed). The experimenter communicates with the subject using the sophisticated method of talking loud enough to be heard from within the shed.
The subject remains in the subject area for the entire VE exposure and VOR testing periods. This area contains the rotating chair, the HMD/VOG unit, the head tracker, two fans for heat dissipation (one of which is attached to the chair for masking of auditory cues), an area light, and a mini flashlight. A research assistant also remains in the subject area to help conduct testing. Though the room was light-tight with regards to the exterior and the shed was light-tight with regards to the reception area, there was still a slight possibility that small stray light sources could exist. Therefore, the subject area was further made light-tight by a wall of drawn occluding blinds which isolated this area from the reception area.
The reception area is used for subject briefings, questionnaire completion, VOG set-up and calibration, and balance testing. The Chattecx Balance Platform is located in this area as is the VOG computer. The eye calibration markers are located just above the experimenter shed, approximately 4.1 m from the subject.
4.3.2.2 Equipment Configuration
For these dissertation experiments, the equipment was configured as depicted in Figure 38. The virtual scene was generated on the WARP PC and presented to the subject via the Virtual i/O HMD. Head position changes were tracked by the InterSense system which sent this information via serial connection to the WARP Pentium PC for image updates. Since the entire 360 degree image had already been pre-rendered and stored in RAM, the minimum system time was very small (approximately 48 ms). Changes to the GFOV (which influences image scale) and changes to system time delay could be made using the WARP computer.
During VOR testing, the subject was sinusoidally oscillated on the rotating chair in the dark. The ISCAN head-mounted camera recorded video images of the eye which were sent to the VOG PC for processing of eye position data. The InterSense tracker provided head position data via the WARP PC. Both head and eye data were automatically converted from digital to analog input using digital-to-analog cards (National Instruments LAB PC+ DAQ Cards) residing in each PC and the resulting analog signals were sent via cables to the data acquisition computer. Upon entering this computer, the data were re-digitized (National Instruments Lab NB DAQ Card). MAQ software synchronized and stored the combined eye-head data file along with the necessary test configuration information for later analysis. MAP software was used to analyze the data (including filtering, digital differentiation, fast component removal and calculation of best curve fit) to obtain VOR gain and phase metrics.
This section summarizes the calibrations performed for this research. First, head and eye tracking system calibrations are discussed, followed by GFOV calibration. System time delay calibration is discussed in Appendix A.
4.3.3.1 Head and Eye Tracking Calibrations
The InterSense head position sensor was calibrated both statically and dynamically. The static tests consisted of reading raw tracker angular position outputs in response to sensor placement at predefined angles. Several trials were accomplished at each angle to verify acceptable repeatability. Dynamic accuracy in the yaw direction was verified by oscillating the sensor on the rotating chair at predefined amplitudes. In addition, amplitude data were recorded from the ADL-1 mechanical tracker as a cross-check. The InterSense tracker was found to perform according to its specifications with no recalibrations required.
Horizontal eye position was calibrated for each subject at the beginning and end of each experimental session. Three markers (center, 10 deg left, 10 deg right) were placed at approximate eye-height on a wall 4.1 m away from the subject. The exact separation of these markers was determined first by a small laser pen positioned on the chair (at approximate eye-height; see Figure 39) and later confirmed through geometry. Eye calibration values were determined by having the subject fixate on each of the three markers in succession while the output of the eye tracker was recorded.
4.3.3.2 GFOV and System Time Delay Calibration
Though GFOV already existed as an adjustable parameter of WARP software, angle accuracy needed to be verified. GFOV was calibrated by using a virtual compass. A model of a virtual compass was developed such that the station point (i.e., observer location in virtual space) was at the center of the compass. Markers identified each five-degree increment of the compass (in yaw only). When rendered using WARP software, the number of marker segments spanning the horizontal DFOV of the HMD was an empirical estimate of the GFOV. Figure 40 illustrates this procedure for the setting GFOV = 25 deg (five post segments, each five degrees separation). Note that this figure represents an image scale of 1.0X because the HMD horizontal DFOV was also 25 deg. This calibration procedure verified the accuracy of Warp TV�S GFOV settings.
Calibration of system time delay was a more complex undertaking. Therefore, a description of this process is presented in Appendix A.
4.4 Research Overview
The following research (Chapters 5 through 8) investigated the physiologic effects of virtual interfaces through a detailed study of the VOR. Four studies were conducted to address the five main objectives presented in Chapter 1 (including the two hypotheses described in Chapter 3).
The first experiment (Image Scale Experiment: Chapter 5) investigated VE image scale changes caused by GFOV/DFOV inequalities in order to determine if the resulting magnification or minification of the VE could drive VOR gain changes. GFOV is often misunderstood or ignored when generating VEs, potentially resulting in an incorrect setting and subsequent stimulus rearrangement being generated. This experiment comes the closest to matching the stimuli used in previous VOR adaptation research, since the visual manifestation of a GFOV/DFOV inequality is similar to that obtained using telescopic spectacles. Therefore, it was the most straightforward way to directly assess the effects of virtual interfaces on VOR recalibration mechanisms. Simulator sickness data also were collected for independent analysis and to correlate with any occurring VOR gain changes.
The second experiment (Time Delay Experiment: Chapter 6) explored the effect of system time delay on VOR gain and phase adaptation as well as on simulator sickness. Time delays are inherent in current virtual interfaces and they are one of the major sources of stimulus rearrangements regarding self-motion. As detailed in Chapter 3, system time delays create a variable VOR phase change demand. Since there have also been many claims circulating in the literature that time delays cause simulator sickness, these data were also collected.
Having demonstrated statistically significant VOR adaptation in the first two experiments, the third experiment (Longitudinal Experiment: Chapter 7) was an investigation into the time-course of adaptation to virtual interfaces. Two subjects had their VOR tested at 0, 10, 20 and 30 min of a 30 min VE exposure. These data provided insight into the influence of exposure time on VOR gain changes.
The fourth experiment (Step Experiment: Chapter 8) explored an underlying premise behind a potential technique for increasing the VOR gain of patients with chronically reduced vestibular function. The technique, proposed by Viirre (1996), utilizes virtual interfaces to incrementally increase the VOR gain change demand over a series of steps instead of a single large gain-change demand. This experiment investigated the relative benefits of incremental vs. single step gain changes on VOR adaptation.
Rather than passively expose the subject to a VE via forced rotation, these four experiments attempted to determine the physiologic effects of virtual interfaces during active, unrestricted, head-coupled interaction with the VE. This method was chosen to specifically address the applied health and safety questions of virtual interfaces.