This article describes the architecture of multimedia streaming and how software developers typically use streams in their tools and applications. It also covers the advantages of streaming as a base for data transfer and how to address multimedia programming issues, such as data transfer, performance optimization, and time stamping in streaming applications. Programmers who want to use multimedia streaming should be familiar with COM programming concepts.
Contents of this article:
Advantages of Multimedia Streaming
When developers use multimedia streaming in their applications, it greatly reduces the amount of format-specific programming needed. Typically, an application that must obtain media data from a file or hardware source must know everything about the data format and the hardware device. The application must handle the connection, transfer of data, any necessary data conversion, and the actual data rendering or file storage. Because each format and device is slightly different, this process is often complex and cumbersome. Multimedia streaming, on the other hand, automatically negotiates the transfer and conversion of data from the source to the application. The streaming interfaces provide a uniform and predictable method of data access and control, which makes it easy for an application to play back the data, regardless of its original source or format.
The following steps show how to implement streaming, from hardware device to rendered playback.
The specification for multimedia streams comprises several interfaces; each interface includes methods that control a certain aspect of the streaming process or handle a certain type of data. See List of Multimedia Streaming Interfaces for additional information.
The Object Hierarchy
The following diagram shows the basic object hierarchy used in multimedia streaming.
There are three basic object types defined in the multimedia streaming architecture:
Creating Multimedia Stream Objects and Stream Samples
Objects that support the IMultiMediaStream interface are the basic containers for multimedia data streams. The IMultiMediaStream interface includes methods that enumerate the object's data streams; these streams are typically video and audio data, but can include data of any format, such as closed-captioning, plain text, or SMTPE timecode. The IMultiMediaStream interface is a generic container, however; developers can create other versions of the interface that support specific data formats. Objects that implement the IAMMultiMediaStream interface, for example, can enumerate and control streams of any Microsoft® DirectShow™ data format. Because individual data streams are format specific, they support at least two different interfaces: one generic and one data-specific. Every stream supports the IMediaStream interface, which provides methods to retrieve its format and a pointer to the stream itself. The IDirectDrawMediaStream interface, on the other hand, has methods that deal specifically with rendering video data. Any interface derived from IMultiMediaStream also supports the creation of stream samples, the basic units of streaming data.
A multimedia sample is a reference to an object containing the media data. For a video image this is a DirectDraw surface. The sample's exact content varies, depending on the type of media (sound, text, and so on). Because a sample is only a reference to the data object, any number of stream samples can refer to the same object. The IStreamSample interface provides methods that get and set a sample's characteristics, such as its start and stop time, status, and stream association. The IStreamSample::Update method refreshes the sample's data in the case of readable streams. For writable streams, it will write the sample's data to the stream. Typically, you use the Update method in a loop that renders, transfers, or stores streaming data. See Use Multimedia Streaming in DirectShow Applications for a practical example of this method in source code.
Using Multimedia Streams in Applications
The Multimedia Streaming interfaces greatly simplify the process of manipulating multimedia data by removing the dependency on specific characteristics of the hardware or software source and providing support for all DirectX media formats. Streams abstract the data to a very high level; applications can even move data from one stream to another without knowing anything about the data's format.
The process of creating a multimedia stream and linking it to a media source is:
For more information on the Multimedia Streaming interfaces, see the reference documentation included with the DirectX media SDK.
Sharing Data Between Streams
Processing multimedia data typically requires a great deal of system resources; therefore, you should avoid copying data whenever possible. The streaming architecture supports shared stream samples, a mechanism that moves data from one stream to another without copying it. This buffer allows the efficient transportation of data between two streams even if the destination stream doesn't specifically support the underlying data format.
For example, assume that you have a multimedia stream with three data streams: video and audio, and URL data time-stamped to match the video content. You want to write an application that adds a copyright notice on every video frame and writes the data to another stream for storage, but your application doesn't understand any data formats except the video stream. For the video stream, you create a sample attached to the desired DirectDraw surface. You can then create an output stream by calling either the IDirectDrawMediaStream::CreateSample method with that pointer to the same surface, or IMediaStream::CreateSharedSample. In both cases, the input and output streams share the DirectDraw surface. Because you understand the video format, you can access this surface as needed.
To retrieve the other source stream pointers (audio and URL), enumerate the source container stream and grab pointers to the nonvideo streams. Each of these source streams has an associated output stream in the output stream container. Retrieve these output pointers by calling the IMultiMediaStream::GetMediaStream method on the output container with each of the source stream pointers. The following steps describe this process.
Repeat these steps for each stream whose format you don't support. When both samples finish updating, the output stream has all data from the source stream and you are done.
© 1997 Microsoft Corporation. All rights reserved. Terms of Use.