About the Multimedia Streaming Architecture

This article describes the architecture of multimedia streaming and how software developers typically use streams in their tools and applications. It also covers the advantages of streaming as a base for data transfer and how to address multimedia programming issues, such as data transfer, performance optimization, and time stamping in streaming applications. Programmers who want to use multimedia streaming should be familiar with COM programming concepts.

Contents of this article:

• Advantages of Multimedia Streaming
• The Object Hierarchy
• Creating Multimedia Stream Objects and Stream Samples
• Using Multimedia Streams in Applications
• Sharing Data Between Streams

Advantages of Multimedia Streaming

When developers use multimedia streaming in their applications, it greatly reduces the amount of format-specific programming needed. Typically, an application that must obtain media data from a file or hardware source must know everything about the data format and the hardware device. The application must handle the connection, transfer of data, any necessary data conversion, and the actual data rendering or file storage. Because each format and device is slightly different, this process is often complex and cumbersome. Multimedia streaming, on the other hand, automatically negotiates the transfer and conversion of data from the source to the application. The streaming interfaces provide a uniform and predictable method of data access and control, which makes it easy for an application to play back the data, regardless of its original source or format.

The following steps show how to implement streaming, from hardware device to rendered playback.

1. A source of video data, such as DirectShow, exposes the streaming interfaces.
2. The application developer uses the Multimedia Streaming interfaces to handle data format conversion.
3. The application developer uses the Microsoft® DirectDraw® interfaces to render the resulting data.

The specification for multimedia streams comprises several interfaces; each interface includes methods that control a certain aspect of the streaming process or handle a certain type of data. See List of Multimedia Streaming Interfaces for additional information.

The Object Hierarchy

The following diagram shows the basic object hierarchy used in multimedia streaming.

There are three basic object types defined in the multimedia streaming architecture:

1. A multimedia stream, which supports the IMultiMediaStream interface.
2. Media streams support the IMediaStream interface and are data specific. Every multimedia stream contains one or more of these media streams.
3. Stream samples support the IStreamSample interface and are created by a media stream. These objects represent a basic unit of work for the stream.

Creating Multimedia Stream Objects and Stream Samples

Objects that support the IMultiMediaStream interface are the basic containers for multimedia data streams. The IMultiMediaStream interface includes methods that enumerate the object's data streams; these streams are typically video and audio data, but can include data of any format, such as closed-captioning, plain text, or SMTPE timecode. The IMultiMediaStream interface is a generic container, however; developers can create other versions of the interface that support specific data formats. Objects that implement the IAMMultiMediaStream interface, for example, can enumerate and control streams of any Microsoft® DirectShow™ data format. Because individual data streams are format specific, they support at least two different interfaces: one generic and one data-specific. Every stream supports the IMediaStream interface, which provides methods to retrieve its format and a pointer to the stream itself. The IDirectDrawMediaStream interface, on the other hand, has methods that deal specifically with rendering video data. Any interface derived from IMultiMediaStream also supports the creation of stream samples, the basic units of streaming data.

A multimedia sample is a reference to an object containing the media data. For a video image this is a DirectDraw surface. The sample's exact content varies, depending on the type of media (sound, text, and so on). Because a sample is only a reference to the data object, any number of stream samples can refer to the same object. The IStreamSample interface provides methods that get and set a sample's characteristics, such as its start and stop time, status, and stream association. The IStreamSample::Update method refreshes the sample's data in the case of readable streams. For writable streams, it will write the sample's data to the stream. Typically, you use the Update method in a loop that renders, transfers, or stores streaming data. See Use Multimedia Streaming in DirectShow Applications for a practical example of this method in source code.

Using Multimedia Streams in Applications

The Multimedia Streaming interfaces greatly simplify the process of manipulating multimedia data by removing the dependency on specific characteristics of the hardware or software source and providing support for all DirectX media formats. Streams abstract the data to a very high level; applications can even move data from one stream to another without knowing anything about the data's format.

The process of creating a multimedia stream and linking it to a media source is:

1. Create the multimedia stream. The method of creation and initialization of the stream is architecture specific. DirectShow supports the IAMMultiMediaStream interface which is used to initialize the stream. Other inproc server implementations of IMultiMediaStream will be created and initialized using different mechanisms.
2. After the multimedia stream object is initialized, the application will use QueryInterface to retrieve the IMultiMediaStream interface for the object. Use this interface to determine the stream's properties and enumerate the streams themselves. You can retrieve a specific stream by calling the IMultiMediaStream::GetMediaStream method with a specific purpose ID. MSPID_PrimaryVideo and MSPID_PrimaryAudio, which represent the primary video and audio streams, are the most commonly used purpose IDs.
3. Call IUnknown::QueryInterface for an interface specific to the stream's media type. If you want to render a video stream, for example, retrieve its IDirectDrawMediaStream interface. Media-specific interfaces define additional methods necessary for taking full advantage of a format's capabilities.
4. Create one or more samples from the stream data. Every media stream supports the IMediaStream::CreateSharedSample method for sample creation. The resulting sample supports the IStreamSample interface, which provides control over the sample and its characteristics. Typically, the media stream supports a format-specific method of sample creation that is more powerful than the aforementioned IStreamSample methods. IDirectDrawMediaStream, for example, can create samples attached to a desired DirectDraw surface and clipping rectangle. In some situations, however, you must handle data without knowing about its data format, such as when constructing a cutlist. If you want to stream data independent of its format, use the IMediaStream::CreateSharedSample method to create the data samples.
5. After creating all desired stream samples, start the stream by calling the IMultiMediaStream::SetState method and pass in the STREAMSTATE_RUN flag as its parameter.
6. Call IStreamSample::Update to update the stream sample. When the IStreamSample::Update method exits, you can access the sample's data. The code sample in Use Multimedia Streaming in DirectShow Applications uses the IStreamSample::Update method in a loop to render a stream of video data. If you want a trigger a specific event or function call when the update returns, pass the appropriate pointers to the IStreamSample::Update method.

For more information on the Multimedia Streaming interfaces, see the reference documentation included with the DirectX media SDK.

Sharing Data Between Streams

Processing multimedia data typically requires a great deal of system resources; therefore, you should avoid copying data whenever possible. The streaming architecture supports shared stream samples, a mechanism that moves data from one stream to another without copying it. This buffer allows the efficient transportation of data between two streams even if the destination stream doesn't specifically support the underlying data format.

For example, assume that you have a multimedia stream with three data streams: video and audio, and URL data time-stamped to match the video content. You want to write an application that adds a copyright notice on every video frame and writes the data to another stream for storage, but your application doesn't understand any data formats except the video stream. For the video stream, you create a sample attached to the desired DirectDraw surface. You can then create an output stream by calling either the IDirectDrawMediaStream::CreateSample method with that pointer to the same surface, or IMediaStream::CreateSharedSample. In both cases, the input and output streams share the DirectDraw surface. Because you understand the video format, you can access this surface as needed.

To retrieve the other source stream pointers (audio and URL), enumerate the source container stream and grab pointers to the nonvideo streams. Each of these source streams has an associated output stream in the output stream container. Retrieve these output pointers by calling the IMultiMediaStream::GetMediaStream method on the output container with each of the source stream pointers. The following steps describe this process.

1. Call IMultiMediaStream::EnumMediaStreams to retrieve a pointer to a source stream. Make sure that it's not the video stream, because your application already understands its format.
2. Call IMultiMediaStream::GetMediaStream on the output container stream with the pointer from step 1. This returns a pointer to the desired output stream.
3. Call AllocateSample on the source stream.
4. Call CreateSharedSample on the output stream.
5. Call Update on the source stream to read the data.
6. Call Update on the output stream to write the data.

Repeat these steps for each stream whose format you don't support. When both samples finish updating, the output stream has all data from the source stream and you are done.

© 1997 Microsoft Corporation. All rights reserved. Terms of Use.