Foreground Occlusions and Binocular Rivalry

The below literature review and pilot study is labelled ``BRP1'' in Section 3.5.

This section differs from the rest of the dissertation in that it is not directly related to presence or to the RFC. It is included because of its practical importance, and because it points to another important use for foreground occlusions in HMD's, besides increasing the sense of presence (see Section 3.3.4 and Chapter 5). The discussion is primarily based on research by others and on theoretical arguments. Only informal new data is presented here.

HMD's typically provide a separate screen for each eye. In the simplest case, there is a total overlap between the images presented in the two screens. That is, each screen shows the same scene, although from a slightly different angle corresponding to the distance between the two eyes.

However, there is a strong temptation in HMD design to only partially overlap the images from the two eyes. In this configuration, only the central 40 or so of the FOV is shared between the two displays. Outside of this region, each eye sees a peripheral region which the other eye can not see.

The advantage of partial overlap is that it allows a wider FOV to be seen with the same equipment. Furthermore, since binocular vision is only effective in the central region where both eyes can converge easily, there should in principle be little perceptual cost to partial overlap.

The disadvantage of partial overlap is an unfortunate binocular rivalry effect [66,56,35]. The shared binocular region is flanked on both sides by monocular regions. At each boundary between the monocular and binocular regions, one eye sees the nasal (inner) edge of a screen, and the other eye sees the monocular continuation of the scene. Thus, the two eyes provide different information about the same point in the visual field.

This presents the brain with a conflict: what is ``really'' out there? The edge of a screen (as indicated by one eye) or the scene displayed to the other eye? The conflict tends to be resolved in favor of the edge of the screen. One perceives two dark bands, corresponding to the nasal edge of each screen, in the shared visual field. When the boundaries of the screen are circular this effect is known as ``luning''.

Why is the conflict resolved in favor of the edge of the screen? Why does one not instead perceive a continuous scene, with the display edges suppressed? This question is usually addressed in terms of contrast gradients. Levelt [59], for instance, did an extensive and quantitative study of binocular rivalry. In general, if conflicting images are presented to the two eyes, the one with the higher contrast gradient is the one perceived (weighted, to some degree, by eye dominance). In the case of an HMD, the edge of the screen is usually dark black, whereas the screen itself is brightly illuminated. This creates a very strong contrast, which perceptually dominates the continuous scene viewed in the other eye. Consequently, the edge of the scene is perceived, resulting in the luning effect.

A method to ameliorate the luning effect is therefore to reduce the contrast gradient at the edge of a screen. Haseltine [35] recommends accomplishing this by

placing an aperture stop near the front surface of each eyepiece of an HMD so that the binocular boundaries of the left and right fields of view are substantially out of focus. The aperture stop is out of focus, and thus relatively inconspicuous, because the observer is focusing on distant virtual images, and also because the aperture stop may be closer to the eye than the shortest distance at which the eye is able to focus.

Lighting the boundary of the screen, so that the screen fades off slowly into blackness, might also be considered.

It is certainly true that contrast gradients play a role in binocular rivalry, and hence that reducing contrast gradients at the edge of the screen should reduce the luning effect. However, a different approach to the luning problem is suggested by thinking of binocular rivalry as a high-level phenomenon, associated with competing perceptual interpretations, rather than as a low-level phenomenon associated with competing monocular stimuli.

Logothetis et al. [61] mention that ``binocular rivalry is thought to reflect competition between monocular neurons within the primary visual cortex. However, neurons whose activity correlates with perception during rivalry are found mainly in higher cortical areas, and respond to input from both eyes. Thus rivalry may involve competition between alternative perceptual interpretations at a higher level of analysis.'' They describe an experiment in which alternating images compete independently of the eye from which they are detected.

Similarly, Shimojo and Nakayama [95] note that real world occlusions result in unpaired regions: the corresponding parts of the two eyes receive different images in the region of the occlusion. ``The authors report a demonstration and experiments to show that opto- geometrically `valid' unpaired regions are seen as continuous with the rear plane and escape interocular suppression, whereas `invalid' unpaired regions are perceived as closer and are suppressed vigorously. An additional experiment indicates that the results cannot be understood in terms of correspondence solving, but require neural mechanisms that embody real-world occlusion constraints.''

These observations suggest that the luning effect does not arise from conflicting binocular cues per se, but rather from conflicting perceptual interpretations. (This bears a resemblance to the interpretation of motion sickness put forward in Section 3.3.5.) From this point-of-view, the fundamental problem is not that there is a sharp contrast at the nasal screen boundary which does not correspond to the scene in the other eye. The problem is that the perceptual system can not find a consistent interpretation of these disparate stimuli.

One might summarize the problem as follows.

1.

Inconsistent images are provided to paired regions of the two eyes. (Screen boundary in one eye, visual scene in the other.)

2.

The perceptual system has no consistent interpretation which explains this discrepancy. Consequently, the problem can not be resolved smoothly.

3.

The more compelling stimulus should ``win'' and be placed in the perceived visual field, overwhelming the less compelling stimulus.

4.

The more compelling image is the screen boundary. (That is, the screen boundary has a higher contrast gradient. Possibly it would be useful to formulate this information-theoretically.)

5.

Consequently, the screen boundary is placed in the perceived visual field, resulting in the luning effect.

In keeping with Shimojo and Nakayama [95], one might expect that providing the nervous system with an ecologically valid interpretation of why the inconsistency occurs would remove the luning effect. An ecologically valid interpretation can be provided simply by supplying a foreground occlusion mounted close to each eye individually which blocks the view of the nasal screen boundary. The screen boundary is in effect moved close to each eye. Since the boundary is close to each eye, it would not be expected to be visible by the other eye. Consequently, there is an ecologically valid interpretation concerning why the boundary is visible in one eye but not in the other. One might therefore expect the luning effect to be reduced or removed entirely.

Given an HMD with partial overlap, this hypothesis can be tested quite easily. My own observations were made with a Division dVisor (see Section 3.7.3) and the black cap of a Write Brothers Papermate pen (medium point). Holding the cap of the pen under the HMD close to the eye, in such a way as to block the nasal edge of the screen, greatly reduces or eliminates the luning effect.

The color of the pen cap was chosen to match the black edge of the screen (and hence the contrast gradient). However, the spatial frequency could not be controlled. That is, a pen cap close to the eye is blurrier than the more distant edge of the screen, and consequently the effectiveness of the pen cap could be attributed to a spatial frequency effect, rather than to the ecological argument.

It is an open question, so far as I know, but I find the ecological interpretation more plausible than the spatial frequency interpretation. The ecological interpretation addresses a real problem which the nervous system must solve (creating a single consistent perceived visual field from binocular input). Furthermore, the ecological argument is consistent with recent findings, described above, that binocular rivalry is a fairly high-level phenomenon.

From the point-of-view of practical HMD design, however, it does not much matter whether one favors the spatial frequency or the ecological interpretation. In addition to presence considerations, described in Chapter 5, the luning effect provides a second reason for mounting foreground occlusions in HMD's.