DirectX Media for Animation Media Types

Previous

TOC

Index

Next

Media Types

This document begins with an overview that first explains the goals of DirectX™ Media for Animation (which will, from now on, be called Direct Animation) and then shows how it integrates and coordinates different media types. There is also a discussion of the ways other systems have traditionally handled different media types and of how Direct Animation fits into these traditions. A section on both 2 and 3D coordinate systems follows. The document then explains, in detail, each of the media types that are available and the operations that can be performed on them.

Introduction

• Three of the primary technical goals of Direct Animation are:

1. Unifying and integrating different media types into a cohesive whole. These media types include:
   • 2D and 3D geometry.
   • Images and sprites.
   • Video.
   • Sound.
2. Abstracting the notions of image, geometry, and sound sufficiently to allow powerful and compact representations of constructs and combinations of these media types.
3. Providing support for a behavior/time and event model that is applied across the different media forms. Incorporating time in a general media system has several advantages. They include de-coupling the modeling and specification of animate content from specific frame generation, as well as the support of inherently time-varying media types, such as audio and video.

These different media types have traditionally been considered distinct categories that have been developed by separate communities of experts. Examples of the ways these media types have traditionally been approached include:

• Image-based systems. An example of an image-based system is Macromedia's Director. This system handles, for example, motion GIF and sprite animations used in 2D games and CD-ROM titles. The entities and their positioning are expressed in terms of pixels. Because pixel-blitting is fast, an advantage to this approach is good performance. A drawback is that the content has limited interactivity and is of fixed-pixel dimensions.
• 3D graphics systems. Examples of these systems include Microsoft's D3D-RM, Silicon Graphics' OpenInventor, and VRML2. In these systems, the tradition is to use hierarchical scene—graphs with deep nesting to represent geometry, attribution, and part-assemblies. Scene—graphs have associated with them a traversal and processing model for rendering into images. Advantages to this approach include flexibility and the level of interactivity that is afforded. A disadvantage is that images video, and sound are incorporated as afterthoughts and treated as degenerate cases of 3D. The power that these data types can provide is ignored.
• Audio/Video systems. Examples of these systems include Apple'sQuicktime and Microsoft's Active Movie. Here, the tradition is to set up dataflow graphs that represent media sources, filtering, and presentation nodes. The entities are inherently time-varying and typically recorded from nature. One advantage to this media form is that it is so vivid because nature itself is so rich and complex.

Microsoft's Direct Animation includes elements from these different traditions and presents a comprehensive and coherent mixed media solution. It supports:

• Importing images as well as translating and displaying them with pixel-positioning and pixel-to-pixel mapping to the screen.
• Representing hierarchical 3D part assemblies and rendering them with traditional lighting, shading and camera models.
• Streaming of audio and video either from files or, eventually, from the network.

Interesting combinations of these media types include temporal and spatial coordination.

Direct Animation provides its support for highly interactive, richly animate, and mixed-media content, through a declarative, high-level API that emphasizes ease of construction. In addition, this declarative approach is amenable to efficient implementations that approach the performance level of low-level APIs, which tend to be much harder to use.

The capabilities of Direct Animation open the door for rich interactive animation that can draw upon both the flexibility of synthetic media (such as 3D graphics) and the vividness of recorded media (such as audio and video recorded from nature). Examples include navigating a 3D space to inspect either a collection of museum items or advertised products in a shopping display. Spatial navigation can be implemented, using position and orientation for indexing into the different items that are represented as images and video clips.

The goal of this section is to describe the abstract media types and coordinate systems used to represent image, geometry, sound, and their interrelationships.