Most of the medical AR applications reported to date involve structural information to assist surgical procedures. Robb et al. report a system for use in surgery planning, rehearsal and performance [5]. Grimson et al. have created a system to guide neurosurgical procedures that mixes video from a patient with MRI imagery [6]. Pieper et al. have created a real-time system to assist arthroscopy [7]. Non-surgical structural visualization projects include the work of Slate et al. who have prototyped a system that superimposes real-time fetal ultrasound images on the mother's abdomen [8, 9], and Chihara et al. whose system projects live echocardiographic images on a patient's chest [10].
Non-structural medical information has been much less explored so far. Block et al. describe a monocular heads-up system that displays anesthesia machine information that is conventionally seen in on a computer monitor [11]. Horvitz et al. have shown an early prototype of a hands-free medical computing environment for use by emergency personnel; this system presents users information via see-through heads-up displays, but the information itself is conventional windows-based data [12]. The use of AR for non-structural information is much further advanced in non-medical domains. Feiner et al. have developed a variety of interfaces for maintenance and repair tasks [13, 14]. Other applications include aviation design and manufacturing [15, 16].
Our work reported here differs from the above efforts in that we are modeling waveform data, not structural data. We are rendering information which is not registered with any "real" object in the environment-a considerably easier challenge that "true" AR. Finally, our current focus is on individual interface components, not the larger system issues.
The other stream of related work is the long-standing effort to design new representations for medical information. Several groups have developed new graphical techniques to portray laboratory information [17-19]. Many groups are working on 3D reconstruction techniques for MRI images [20] and echocardiography [20, 21], and Jensch et al. report a novel technique that combines angiographic, nuclear scans and PET imaging into a single 3D display [22]. A number of groups have reported with 3D visualization techniques for intraoperative cardiac electric fields obtained through application of large numbers of epicardial and endocardial electrodes [23-27]. Hulin et al. describe a novel animated "billowing sheet" representation of ECG data in which voltages are mapped distinctively to elevations above a reference plane [28, 29]. New information representations have generally received cool receptions in the medical domain, partly because of practitioners' psychological inertia, and partly because effective designs are extremely difficult to create and validate [30-33]. Our work is similar to these other efforts in that we are trying to apply new technologies to old information tasks, but we are working in an AR/VR medium.