This article examines the types of data, including samples, events, and notifications, that move through a filter graph, including where this data and information originates, where it is routed, and the protocols that must be followed for data to flow correctly.
Contents of this article:
How Data Moves in a Filter Graph
Data flow in the filter graph can be viewed by examining the paths through which it flows, and also by examining the protocols that are used within those paths. Data flows primarily in the following paths.
This article describes each of these data paths. Data movement in a filter graph is enabled by implementation of the following Microsoft® DirectShow™ filter graph protocols.
Media Sample Data Flow
DirectShow filters pass media data downstream, that is, from the output pin of one filter to the input pin of the next filter. The flow and control of the data is effected by the interfaces on those pins and the filters themselves. The filters serialize data streaming activity; all data streaming calls for a given pin are explicitly serialized and usually originate from a single thread.
Data starts at a source filter and ends at a renderer filter. The source filter can either push the data down the graph (that is, originate the thread and send data to the IMemInputPin::Receive method of the downstream pin), or implement the IAsyncReader interface and let the downstream filter originate the thread, pull the data from the source filter, and send it downstream. For a description of how this latter case differs from other protocols, see IAsyncReader Transport.
Every filter should accept and process data received by its input pins, with the following exceptions.
There can be other conditions for a filter to reject data as well. For instance, a transform filter would reject data at its input pin if its output pin was not connected.
Media samples are data objects that support the IMediaSample interface. They are usually obtained from an allocator, which is most likely represented by an object supporting the IMemAllocator interface. The two connected pins of adjacent filters agree on a common method of exchanging data, called a transport. Many of the base classes for the DirectShow class library are used to implement objects that support the local memory transport.
In the local memory transport, the input pin for a connection supports the IMemInputPin interface. An output pin can determine that it can use the local memory transport if a call to the IUnknown::QueryInterface method on the input pin to request the IMemInputPin interface succeeds. For this transport, data is passed from the output pin of one filter to the input pin of an adjacent filter in media samples. During connection, the output pin and input pin agree on the connection's allocator object, which is used to create the media samples.
Filters must follow protocols to pass and receive media samples. The connected pins must agree on the allocator to be used, must have a means of passing the data, and must follow the correct procedure for holding on to a sample or releasing it back to the sender.
For the local memory transport, an output pin passes a media sample to the input pin it is connected to by calling the input pin's IMemInputPin::Receive or IMemInputPin::ReceiveMultiple methods, depending on whether it is passing single or multiple samples. Before it can pass this data, however, the output pin must obtain a media sample. The IMemInputPin interface on the input pin provides an IMemAllocator object when requested to provide an allocator. If the output pin is not using its own allocator, or one provided to it from further upstream, it calls the IMemAllocator::GetBuffer method on the input pin to retrieve one.
The input pin can either process the data right away or save it for later processing. In the latter case, it must call the IUnknown::AddRef method on the media sample to prevent the sample from being returned to the allocator. When the output pin has called the input pin's IMemInputPin::Receive method, it must call the IUnknown::Release method to free the sample. If the input pin did not save the sample by calling IUnknown::AddRef, the sample is immediately returned to the allocator.
The output pin can decide not to pass the media sample on to the input, in which case it can just call the sample's IUnknown::Release method without calling the input pin's IMemInputPin::Receive method.
Control Information Data
There are two types of control information which are passed downstream filter to filter:
End-Of-Stream Notifications
It is important for filters to indicate when there will be no more data in the current set of data. A filter does this by sending an end-of-stream notification to the next filter, which is accomplished by the output pin calling the IPin::EndOfStream method on the downstream filter's input pin.
When a source filter (an originator of data) reaches the end of its data, it calls the IPin::EndOfStream method on all pins connected to its output pins. This mechanism is propagated down the filter graph so that each filter that processes its EndOfStream method in turn calls EndOfStream on the pins connected to its output pins. When the notification reaches the end of the line in the filter graph, the renderer passes an EC_COMPLETE notification to the filter graph manager. The filter graph manager counts the EC_COMPLETE notifications it receives and when all renderer filters have completed, passes the notification to the application. The filter graph manager counts rendered streams by counting the number of filters (not pins) that support IMediaSeeking or IMediaPosition and have a rendered input pin. A rendered input pin is a pin with no corresponding outputs, which can be determined with IPin::QueryInternalConnections. input pins. A renderer input pin returns zero pins when its IPin::QueryInternalConnections is called. Note that the filter, not the pins, support IMediaSeeking in this case.
Although source filters usually originate the end-of-stream notification, it is also possible for other filters to detect this condition and generate the notification downstream. Most notably, this applies to parser filters that connect to asynchronous reader filters (filters implementing the IAsyncReader interface).
For example, the MPEG parser (in the MPEG splitter filter) can detect the end of the stream and when it does, return S_FALSE from the Receive method, which signals the upstream filter to stop sending data until a seek occurs or the filter graph is stopped. In this case, the upstream filter does not need to call EndOfStream. Instead, EndOfStream should be called by the filter detecting the end-of-stream condition (the downstream splitter or parser) before returning from Receive or ReceiveMultiple.
Note that EndOfStream should be serialized with data passed in the stream. It is a single piece of information that must be passed after all the other data in the stream.
Flushing
In the DirectShow filter graph architecture, flushing is a two-stage process. Flushing is not usually initiated as part of normal data flow, but rather as a result of a control action from the filter graph manager. As such, it is an asynchronous event which requires flushing of old data followed by a resynchronization and pushing of new data.
In a flushing operation, first IPin::BeginFlush is called by the source filter on all input pins connected to its output pins. This call is propagated down the graph by all filters to the rendering filter or filters. BeginFlush should flush any pending EndOfStream calls or EC_COMPLETE notifications. After BeginFlush has been called, an input pin should reject all data until its IPin::EndFlush method has been called (this includes end-of-stream notifications). It should also free any connected pins waiting for any of its resources. In the case of the local memory transport, this means that every filter should free any filter waiting for a sample from its allocator. This is usually done by calling IMemAllocator::Decommit on the allocator.
After a filter has called BeginFlush on the pins connected to its output pins, and when it can guarantee that all processing of samples by its pins is complete and no more samples will be processed, it should call EndFlush. For source filters this means shutting down data generation, then calling EndFlush on the pins connected to its output pins. For other filters it means waiting for an EndFlush call (which guarantees that no more samples will be sent) and then waiting for any queues it maintains itself to empty. Because calls can block on downstream filters for the local memory transport model, it is important to wait for queues to empty when EndFlush is called, rather than trying to do so when BeginFlush is called.
Event Notifications
Notification data goes from filters to the filter graph manager and can be passed on to the application. The EC_COMPLETE notification, which is sent from renderers at the end of a data stream, has already been mentioned.
The filter graph manager should not be notified of EC_COMPLETE until a Run command is issued. A renderer filter that has EndOfStream called on its input pin while in a paused state must not notify the filter graph manager until its IMediaFilter::Run method is called. Stop and EndFlush calls cancel any such deferred notifications and allow more data to be subsequently processed by the pin. After notifying the filter graph manager once with EC_COMPLETE, the renderer must not generate another EC_COMPLETE notification before processing a Stop or EndFlush method.
If a running filter graph is paused while at the end of its stream and IMediaFilter::Run is subsequently called, renderers should notify the filter graph manager with EC_COMPLETE again.
Besides EC_COMPLETE, there are many other event notifications, many of which are sent by specialized filters, such as the renderer, to communicate with a host application. Error notifications are another class of notifications that are also sent from filters to the filter graph manager.
The convention for DirectShow filters is that when a filter detects an error, it passes a notification to the filter graph manager by calling the IMediaEventSink::Notify method. Errors in processing data can generate several error events, including the following:
If processing can no longer continue, the filter graph manager should be notified with EC_STREAM_ERROR_STOPPED and the appropriate convention for the particular transport should be used to notify the upstream output pin. In the case of the local memory transport, this involves returning an error value from IMemInputPin::Receive. In addition to notifying the filter graph manager of the error, a filter should also either call EndOfStream on all the pins connected to its output pins or, if it is a renderer, also notify the filter graph manager with EC_COMPLETE. This ensures the play will complete gracefully.
Errors of this type can be caused by encountering events such as being out of memory or other resource problems. Or they might be caused by other events such as a failure to obtain a buffer when trying to pass data downstream.
On the other hand, when an error occurs but processing can still continue, EC_STREAM_ERROR_STILLPLAYING should be sent to the filter graph manager. In this case, the upstream output pin should not be notified. Specifically, for the local memory transport, the input pin's IMemInputPin::Receive method should return NOERROR when this type of error occurs.
Filter Graph Control Data
Control data originates at the application and is passed to the filter graph manager. At the COM level, this is handled by filter graph manager interfaces in the Control.odl type definition library. Examples of control data are calls to the IMediaControl interfaces, such as IMediaControl::Run, IMediaControl::Pause, and IMediaControl::Stop. The IMediaPosition and IMediaSeeking interfaces provide a means of moving forward or backward in a media stream.
The most important thing to understand about the flow of control data is that it should always pass through the filter graph manager first. This is because there is usually an order that must be followed in controlling the filters in the filter graph to make sure filters are called in the correct order and with regard for internal filter graph states.
Quality Control Data
The DirectShow stream architecture provides a means of gracefully adapting to load differences in the media stream so that rendering of the data is maintained at the highest possible resolution. The IQualityControl interface is used to send quality-control notification messages from a renderer filter either upstream, to be acted on eventually by some filter in the graph, or directly to a designated location. For example, a renderer that is getting too many frames to process can try to get an upstream filter to cut down on the number of frames it is sending. This is usually more efficient than simply dropping frames at the renderer. (A video decompressor filter uses many CPU cycles to decompress frames, so it is better to discard samples before processing them rather than after processing them.) Likewise, when the renderer filter can handle more data, it sends notifications to increase the number of samples.
Quality-control messages are passed upstream by default; if a filter has no registered quality sink, the default action passes the message to the IQualityControl interface of the connected output pin. Internally, the output pin passes the quality-control message to its input pin, if it has one, and the message travels upstream until it reaches a filter that can affect the data stream quality in the requested manner. DirectShow handles this mechanism automatically in the transform base classes.
If a filter can handle the quality notification (by increasing or decreasing the flow) and if it is not appropriate for filters further upstream to take any action, that filter will act on the notification and not pass it on. A filter must pass the quality-control message on if it does not act on the message. It can also pass it on even if it does act on the message. Silently accepting the message without acting on it or passing it on is considered bad behavior, and might damage the performance of the filter graph as a whole.
A quality sink is a feature implemented by the IQualityControl::SetSink method. When this method is called, the filter is instructed not to send messages upstream, but rather to send them to the object passed to the SetSink method. Typically, this object would be a component called a quality-control manager. Such a component would set itself as the sink for all the filters to send the quality-control messages rather than anywhere else. It would then determine whether to route the messages upstream or to take some other action, such as cutting back the video stream to avoid breaking the audio. A quality-control manager can be implemented by using the IQualityControl interface and should be anticipated when writing filters.
Serializing Data
A filter usually has to synchronize two contexts: filter state and data flow. Usually a filter will create a critical section for each context.
The data flow critical section is held during streaming operations. For example, for the local memory transport, this critical section should be held while processing the following methods.
The filter state critical section is held while processing state changes when the following IBaseFilter methods are called.
It is also held during BeginFlush and EndFlush streaming control operations.
During Stop and EndFlush calls, the stream state must be synchronized with the filter state. An example of how to do this is in the base class CTransformFilter. In the case of implementing the Stop method for the local memory transport, for example, the stream must be "released" to avoid deadlocks by decommitting the input allocator pin. This is not required to process EndFlush, because this will have already been done in BeginFlush processing. Once the stream is released, the data flow critical section (as implemented in Receive) can also be locked to synchronize with the stream state.
Note that because Stop requires access to the filter state before synchronizing with the data flow component, these two critical sections must be different.
A filter should not, in general, have its filter state critical section locked while calling methods on other filters. The filter graph synchronizes graph-wide operations such as setting the current position.
IAsyncReader Transport
For source parsing filters, the IAsyncReader interface helps implement a "pull" data flow model, as opposed to the "push" model, in which a thread in the source filter pushes data downstream. The parsing filter is connected downstream to the filter whose pin implements IAsyncReader. In this case, the downstream parsing filter initiates data transfer by calling IAsyncReader methods such as SyncReadAligned. The parsing filter, in this case, creates the thread, pulls data from the source, and pushes it downstream.
Because all data flow activity in this transport is initiated by the downstream filter, several of the protocols mentioned previously operate in reverse. For example:
© 1997 Microsoft Corporation. All rights reserved. Terms of Use.