Microsoft DirectX 8.0

Data Flow for Filter Developers

This article describes in detail how data moves through the filter graph. It is intended for developers who are writing their own custom filters. For a general introduction to how Microsoft® DirectShow® handles data flow, see Data Flow in the Filter Graph.

This article focuses on the local memory transport, using the IMemInputPin and IMemAllocator interfaces. It contains the following sections:

Push versus Pull

The role of a source filter is to introduce data into the graph. It can follow one of two models, the push model or the pull model.

The push model is typical of live sources, such as video cameras. The pull model is typical of file readers, where the downstream filter is a file parser, such as the AVI Splitter filter.

Generating New Samples

This section describes how source filters generate media samples.

Push Model

In the push model, the source filter initiates the process, as follows:

  1. The source filter calls IMemAllocator::GetBuffer to retrieve an empty media sample.
  2. The source filter fills the media sample with data. How this occurs depends entirely on the nature of the source.
  3. The source filter calls IMemInputPin::Receive on the downstream input pin, passing it a pointer to the sample's IMediaSample interface.
  4. The downstream filter can either process the sample before returning from Receive, or else hold the sample and process it afterward. If the downstream filter holds the sample, it calls IUnknown::AddRef on the sample.
  5. The source filter calls IUnknown::Release on the sample.

At this point, the downstream filter might hold a reference count on the sample, so the source filter cannot simply re-use the sample. To deliver the next sample, it must call IMemAlloctor::GetBuffer again, as described in step 1.

Note  To deliver multiple samples, in step 3 the source filter could call IMemInputPin::ReceiveMultiple instead.

Pull Model

In the pull model, the parser filter requests data from the source filter. The parser filter uses the IAsyncReader interface on the source filter's output pin, as follows:

  1. The parser filter calls IMemAllocator::GetBuffer to retrieve an empty media sample.
  2. It calls IAsyncReader::Request to request data from the source filter.
  3. While the source filter is retrieving the data, the parser calls IAsyncReader::WaitForNext. This method waits until the request from step 2 is completed.
  4. The parser filter processes the data and delivers it downstream.

Steps 2–3 perform an asynchronous read operation. The parser can request a synchronous read operation instead, using the IAsyncReader::SyncRead or IAsyncReader::SyncReadAligned method.

Processing a Sample

When a filter's input pin receives a sample, the filter processes the data in the sample and then delivers the sample to the next downstream filter. As the previous section described, the filter can do all of its processing while inside the Receive method, or it can hold the sample and process it afterward.

Within the call to Receive, the input pin has the option of blocking the upstream filter's calling thread. You can query the input pin's behavior by calling the IMemInputPin::ReceiveCanBlock method. If the return value is S_OK, the pin might block on a call to Receive. If the return value is S_FALSE, the pin will never block on calls to Receive. Based on the return value, the upstream filter might use a separate worker thread to deliver samples.

Some filters process samples in place, without copying any data. Other filters copy the data before processing it. Obviously, it is better to process samples in place whenever possible, to avoid the overhead of unneeded copy operations.

The input pin can reject samples by returning an error code or S_FALSE in its Receive method. The value S_FALSE indicates that the pin is flushing data (see Flushing). An error code indicates some other problem. For example, if the filter is stopped, the return value is VFW_E_WRONG_STATE. If a pin rejects a sample, the filter calls the IPin::EndOfStream method on the input pins connected to it. Also, if an error occurred, the filter sends an EC_ERRORABORT event to the filter graph manager. After a pin has rejected a sample, the upstream filter stops sending data. It can start again after the flush completes, or when the graph is stopped and restarted.

The filter must serialize all of its Receive calls to a given input pin.

New Segment

At the beginning of each new stream, the source filter (or a parser filter) must call IPin::NewSegment. This method specifies the start and stop times of the stream, relative to the original source, and the playback rate. The filter calls NewSegment when streaming starts, and after a seek operation. Stream time resets to zero after each new segment, and samples delivered after the new segment are time-stamped starting from zero.

Segment information enables certain filters to process samples correctly. For example, in a format such as MPEG-1, a filter might need a key frame beyond the stop time, in order to construct the intermediate frames. The segment information informs the filter of the actual stop time. Regardless of whether the filter uses the segment information or not, it passes the NewSegment call downstream. The NewSegment calls must be serialized with other streaming calls, such as Receive.

End of Stream

When a source filter has no more data to send, it calls the IPin::EndOfStream method on the downstream input pin. The downstream filter propagates the call to the next filter. Eventually the EndOfStream call reaches the renderer filter. The renderer filter sends an EC_COMPLETE event to the filter graph manager.

The filter graph manager does not immediately forward the EC_COMPLETE notification to the application. Instead, it waits until all the streams signal EC_COMPLETE. The application does not receive an EC_COMPLETE notice until all the streams have completed. To determine the number of streams in the graph, the filter graph manager counts the number of filters that support IMediaSeeking or IMediaPosition and have a rendered input pin. A rendered input pin is an input pin with no corresponding outputs. The IPin::QueryInternalConnections method returns zero for a rendered input pin. As another option, a filter can implement the IAMFilterMiscFlags interface and return the AM_FILTER_MISC_FLAGS_IS_RENDERER flag.

A filter must serialize EndOfStream calls with other streaming calls, such as Receive.

Flushing

Flushing is the process of discarding all the pending samples in the graph. Flushing enables the graph to be more responsive when events alter the normal data flow. For example, in a seek operation, old data is flushed from the graph before new data is introduced. If the graph contains multiple streams, it is possible to flush individual streams separately.

Flushing is a two-stage process:

When an input pin's BeginFlush method is called, the filter performs the following actions:

  1. Calls BeginFlush on downstream input pins.
  2. Rejects any further calls that stream data, including Receive and EndOfStream.
  3. Unblocks any upstream filters that are blocked waiting for a sample from the filter's allocator. Filters often decommit their allocators for this purpose.
  4. Exits from any waits that block streaming. For example, renderer filters block when paused. They also block when they are waiting to render a sample at the correct stream time. The filter must unblock, so that samples queued upstream can be delivered and rejected. This step ensures that all the filters in the streaming chain are eventually unblocked.

When the EndFlush method is called, the filter performs the following actions:

  1. Waits for all queued samples to be discarded.
  2. Frees any buffered data. This step can sometimes be performed in the BeginFlush method. However, BeginFlush is not synchronized with the streaming thread. The filter must not process or buffer any more data between the call to BeginFlush and the call to EndFlush.
  3. Clears any pending EC_COMPLETE notifications.
  4. Calls EndFlush downstream.

At this point, the filter can accept samples again. All samples are guaranteed to be more recent than the flush.

In the pull model, the parser filter initiates flushing rather than the source filter. The sequence of events is the same, except that the parser also calls BeginFlush and EndFlush on the source filter's output pin.

Threads and Critical Sections

In any DirectShow application, at least two important threads are running. The first is the application thread. This thread initiates changes in the state of the filter graph. State changes include running, pausing, and stopping the graph; connecting and disconnecting pins; adding filters to the graph; and removing filters from the graph. The second important thread is the streaming thread, in which pins receive and deliver samples. A separate streaming thread is necessary so that the application thread can block (for example, while the application interacts with the user) without interrupting the data flow.

Filters can also create threads. For example, in its IMemInputPin::Receive method, a filter can return immediately and process the received samples on a separate thread. Some filters also use a separate thread to queue samples for delivery.

Because filter operations are multithreaded, it is crucial that a filter serialize certain operations. Otherwise, race conditions can result. For example, consider the following code, which checks if there is a reference clock and gets the current time:

if (m_pClock != NULL)  // Is there a reference clock?
{
    m_pClock->GetTime(&rtTimeNow);
    // More code, not shown...
}

Suppose that the streaming thread executes this code, and that the code is not protected. Between the if statement and the following line, the application thread might call IMediaFilter::SetSyncSource and set the reference clock to NULL. As a result, the streaming thread dereferences the NULL pointer. An exception occurs, and the application crashes.

To prevent race conditions, filters use two critical sections. The first is the streaming lock. It serializes streaming operations. The second is the filter lock. It serializes state changes, and protects the integrity of the state information.

A filter holds the streaming lock when it processes the following method calls:

A filter holds the filter lock when it processes the following types of calls:

The preceding list is not meant to be exhaustive. When you implement a filter, you must consider which methods change the filter state, and which methods perform streaming operations.

Critical sections create the potential for deadlocks. To avoid deadlocks, streaming methods should never hold the filter lock. The reason for this rule is that some streaming operations cannot complete until the filter graph finishes a state transition. For example, renderers block Receive calls while paused. If the upstream filter holds the filter lock when it calls Receive, it can create a circular wait condition. The streaming thread is waiting for the graph to switch out of paused state, but that state transition can never occur, because the application thread waits to obtain the filter lock.

The following diagram illustrates this deadlock.

Filter Deadlock

In the Stop and EndFlush methods, streaming operations must be synchronized with the filter state. Otherwise, the filter might continue to deliver old samples after the state transition. In the Stop method, care must be taken to avoid deadlock. First hold the filter lock and decommit the input pin's allocator. Then hold the streaming lock. Decomitting the allocator guarantees that calls to Receive will fail, and thus not block. (See CTransformFilter::Stop for an implementation.) In a flush operation, the BeginFlush method starts rejecting samples. The EndFlush method holds the streaming lock.