VMR7 contains all the necessary functionality to manage several filter graphs; this task can be accomplished by providing VMR7 with custom allocator-presenter (as shown in sample VMRMulti, see Samples\cpp\DirectShow\VMR\VMRMulti). However, introduction of VMR9 essentially simplifies this task:
Development of custom (user provided) Allocator-Presenter for the renderer has not changed much since VMR7, but there are several differences. Below is the “how to” scenario, it is implemented in the sample VMR9Allocator (See Samples\cpp\DirectShow\VMR9\VMR9Allocator). First, developer of the client application has to implement custom allocator-presenter (A/P), an object derived from IVMRSurfaceAllocator9 and IVMRImagePresenter9. See DirectX 9.0 SDK help for more information on custom allocator-presenter (see “Supplying a Custom Allocator-Presenter for VMR9”). Figure 1 shows relationship between VMR9 filter and its Allocator-Presenter.
structure. Usually this method performs
the following tasks:
Create Direct3D9 object (or acquire it
from some other part of the application).
Create Direct3D9 device (or acquire it
from some other part of the application).
Allocate Direct3D9 surfaces according to
VMR9AllocationInfo structure. In most cases, it is advised to use helper
function
IVMRSurfaceAllocatorNotify9::AllocateSurfaceHelper
rather than directly allocate surfaces with methods of IDirect3Ddevice9.Create all the other Direct3D9 objects
associated with this custom A/P (if applicable)
When client application supplies a custom A/P for VMR9, there are several rules to be followed: custom A/P must be thread-safe; when advising custom A/P, filter graph must be stopped and VMR9 filter must be in the rendeless mode; and all its inputs pins must be disconnected. Below is the typical scenario of using custom A/P.
1. Create instance of the filter graph (CLSID_FilterGraph).
2. Create instance of Video Mixing Renderer 9 (CLSID_VideoMixingRender9) and add it to the filter graph.
3. Set VMR9 to the Renderless mode (see IVMRFilterConfig9::SetRenderingMode) .
4. Query VMR9 filter for interface IVMRSurfaceAllocatorNotify9.
5. Create instance of custom allocator-presenter.
6. Associate your filter graph with some cookie (any DWORD_PTR value that uniquely identifies the filter graph).
7. Advice custom A/P to the VMR9 (see IVMRSurfaceAllocatorNotify9::AdviseSurfaceAllocator).
8. Inform custom A/P about the allocator notifier (see IVMRSurfaceAllocator9::AdviseNotify).
9. Build the rest of the graph, render, and run.
10. When destroying the filter graph, stop the graph, disconnect all the pins of VMR9 filter and advise it with NULL surface allocator
To provide multi-graph
functionality to the scenario above, few modifications are needed. Custom A/P
should contain references to all the connected filter graphs. Each filter graph
is uniquely identified with a cookie (parameter DWORD_PTR dwUserID in methods of IVMRSurfaceAllocator9 and IVMRImagePresenter9). Since several filter graphs are
asynchronous to each other, it is advised to perform rendering in a separate
thread with some fixed user-defined frame rate. In this case, method IVMRImagePresenter9::PresentImage just copies images from VMR9
filters to some private textures. This causes some performance drop, but leaves
filter graphs independent on each other. It is advised to set the frame rate to
the highest frame rate of all connected video streams to prevent frame
dropping. At last, it would be
logical if such custom allocator-presenter have methods Attach
and Detach, to connect and disconnect involved filter
graphs. Figure 2 illustrates relationship between connecting filter graphs and
custom A/P.
DirectX 9.0 SDK samples include sources for MultiVMR9.dll (see Samples\cpp\DirectShow\VMR9\MultiVMR9). This helper library demonstrates how to implement a user-provided (customized) allocator-presenter for the Video Mixing Renderer 9 in a multigraph environment. It seemed logical to extract several abstraction layers that could be overriden by client applications:
· Wizard, an object that implements actual allocation-presenting and contains references to all connected filter graphs. The wizard is responsible for interaction with client application as well as connecting and disconnecting graphs. Object is implemented by IMultiVMR9Wizard interface.
· Rendering Engine, an object that contains all Direct3D data and performs actual rendering asynchronously from calls to IVMRImagePresenter9::PresentImage. Object is implemented by IMultiVMR9RenderEngine interface.
· Mixer Control, an object that performs scene composition and defines geometry of Direct3D objects. By default, this object acts as IVMRMixerControl9 in a single graph (see sample MultiPlayer, Samples\cpp\DirectShow\VMR9\MultiVMR9\MultiPlayer), but can be overriden to perform any sort of 3D transformation (as in the sample GamePlayer, Samples\cpp\DirectShow\VMR9\MultiVMR9\GamePlayer). Object is implemented by IMultiVMR9MixerControl interface.
· User Interface Layer, an object that is responsible for interaction with user in video playback window. User Interface Layer is always drawn over the scene rendered by mixer control. Message processing function of the video window sends all the appropriate messages to this object. Object is implemented by IMultiVMR9UILayer interface. By default, MultiVMR9.dll does not use this object, so if client application needs user interaction, it must provide IMultiVMR9UILayer object.
Figure 3 illustrates structure of MultiVMR9.dll.
See Samples\cpp\DirectShow\VMR9\MultiVMR9\DLL\MultiVMR9.idl
Objects exposing this interface are responsible for managing several VMRs with single rendering environment. Object that implements this interface must also implement IVMRSurfaceAllocator9 and IVMRImagePresenter9. Otherwise method Initialize should return E_NOTIMPL. CLSID_MultiVMR9Wizard is the default implementation of this interface.
Initialize |
Configure and initialize internal structures. This method must be called before using the wizard; otherwise consequent calls to the wizard will return VFW_E_WRONG_STATE. Also, use this method to supply customized Rendering Engine. |
Terminate |
Releases all internal resources of the wizard; disconnects all connected filter graphs. This method must be called before destroying the wizard. |
Attach |
Attaches filter graph to the wizard |
Detach |
Detaches filter graph from the wizard |
VerifyID |
Verifies if there is any attached filter graph with specified “UserID” cookie |
GetGraph |
Returns pointer to IFilterGraph of the connected filter graph identified by “UserID” cookie |
GetRenderEngine |
Returns pointer to IMultiVMR9RenderEngine actively used by the wizard |
GetMixerControl |
Returns pointer to IMultiVMR9MixerControl actively used by the wizard |
GetTexture |
Returns pointer to IDirect3DTexture9 that contains “ready-to-render” image from the filter graph identified by “UserID” cookie |
GetVideoSize |
Returns native image size of the video stream from the filter graph identified by “UserID” cookie |
Objects exposing this interface
perfom actual rendering of video streams to a single Direct3D device. While IMultiVMR9Wizard provides surface management requested by
VMR9, IMultiVMR9RenderEngine performs actual rendering
of the image to the screen, asynchronously from VMR9 calls. Rendering engine
manages two objects: MultiVMR9MixerControl
and MultiVMR9UILayer. By default, UI layer is not
implemented, and the default mixer control acts similar to IVMRMixerControl9 in a single graph. CLSID_MultiVMR9RenderEngine
is the default implementation of this interface.
Initialize |
This method must be called before connecting “this” rendering engine to the wizard. Client application has to call this method only if it supplies customized rendering engine (in IMultiVMR9Wizard::Initialize). Otherwise, wizard creates and initializes default instance of this object. Also, this method can be used to supply customized Mixer Control and User Interface Layer to the rendering engine. |
Terminate |
Releases all internal resources of the rendering engine. Client application must call this method only if it supplies customized rendering engine. Client applications must call this method after destroying the wizard. |
Render |
A call-back function invoked from a separate thread with frequency specified by IMultiVMR9RenderEngine::SetFrameRate. Method performs actual rendering of the scene in several steps: scene initialization (such as IDirect3Ddevice9::BeginScene); composition of the mixer objects (IMultiVMR9MixerControl::Compose); rendering of the mixer objects (IMultiVMR9MixerControl::Render); composition of the UI layer if it is present (IMultiVMR9UILayer::Compose); rendering of the UI Layer (IMultiVMR9UILayer::Render), and scene completion (IDirect3Ddevice9::EndScene and IDirect3Ddevice9::Present). It also updates actual frame rate. |
GetUILayer |
Returns pointer to IMultiVMR9UILayer actively used by the render engine |
SetFrameRate |
Sets desired frames-per-second rate, multiplied by 100 |
GetFrameRate |
Returns actual frame rate (multiplied by 100) from the last rendering |
GetFrameRateAvg |
Returns actual frame rate (multiplied by 100) averaged over the lifetime of the rendering object |
GetMixingPrefs |
Returns mixing preferences flags originally set in the method Initialize. By default, mixing preferences flags are ignored |
SetWizardOwner |
Wizard calls this method to inform redering engine that it is the owner of the object. Also, when wizard is terminating, it sets the owner of its child rendering engine to NULL. |
GetWizardOwner |
Returns pointer to IMultiVMR9Wizard, the parent wizard object that currently owns this rendering engine |
GetMixerControl |
Returns pointer to IMultiVMR9MixerControl actively used by the render engine |
Get3DDevice |
Returns pointer to IDirect3DDevice9 actively used for rendering |
GetVideoWindow |
Returns handle to the video window. |
Object that implements this interface, performs scene composition and defines geometry of Direct3D primitives. By default, this interface is similar to IVMRMixerControl9 interface of a single graph. Provide custom implementation of this interface to add animation, special effects, or to inject video into 3D environent. CLSID_MultiVMR9MixerControl is the default implementation of this interface.
Initialize |
This method must be called before connecting “this” mixer control to the rendering engine. Client application has to call this method only if it supplies customized mixer control (in IMultiVMR9RenderEngine::Initialize). Otherwise, rendering engine creates and initializes default instance of this object. |
SetRenderEngineOwner |
Rendering engine calls this method to inform mixer control that it is the owner of the object. Also, when rendering engine is terminating, it sets the owner of its child mixer control to NULL. |
GetRenderEngineOwner |
Returns pointer to IMultiVMR9RenderEngine, the parent rendering engine object that currently owns this mixer control |
Compose |
This method is called from IMultiVMR9RenderEngine::Render. Method performs calculation of primitives’ coordinates that provides animation, special effects, etc. |
Render |
This method is called from IMultiVMR9RenderEngine::Render. Method performs actual rendering of primitives. Please note that this method must not call IDirect3Ddevice9::BeginScene or IDirect3Ddevice9::EndScene, because it is done inside rendering engine. |
AddVideoSource |
Wizard calls this method when it attaches a new filter graph. Upon receiving this call, mixer control must allocate objects (primitives) associated with new video source. |
DeleteVideoSource |
Wizard calls this method when it detaches some filter graph. Upon receiving this call, mixer control must release objects (primitives) associated with this video source. |
GetIdealOutputRect |
Returns “ideal” location of the video image, that is, normalized rectangle that covers maximum space of the render target and maintains original aspect ratio of the image. Please note that normalized rectangle is calculated with respect to the render target size (rather than the client size of the video window). Use IMultiVMR9RenderEngine::Get3DDevice and IDirect3Ddevice9::GetRenderTarger to transform the “render target” space into the “client area of the window” space. |
The following methods are implemented in the default instance of IMultiVMR9MixerControl and basically repeat functionality of IVMRMixerControl9. The only difference is that cookie “UserID” is used instead of the number of input pin. When these methods do not make sense in the application-specific implementation of IMultiVMR9MixerControl, just make them returning S_OK.
GetOutputRect |
SetOutputRect |
GetZOrder |
SetZOrder |
GetAlpha |
SetAlpha |
GetBackgroundColor |
SetBackgroundColor |
Object that exposes this interface, represents UI- related Direct3D primitives (such as buttons, menus, sliders etc.) as well as user input to the video window (Win32 messages). User Interface layer is always drawn over the scene rendered by IMultiVMR9MixerControl. By default, rendering engine does not use this object, so if client application implies UI interaction from the video window, it should provide implementation of IMultiVMR9UILayer and send it to IMultiVMR9RenderEngine::Initialize.
Initialize |
This method can be called to initialize internal objects before connecting “this” UI layer to the rendering engine. |
ProcessMessage |
This method is called by the video window message processing thread. Method can modify state of the UI Layer and send commands to client application, rendering engine, or the wizard. |
Render |
This method is called from IMultiVMR9RenderEngine::Render. Method performs actual rendering of the primitives of UI layer. Please note that this method must not call IDirect3Ddevice9::BeginScene or IDirect3Ddevice9::EndScene, because it is done inside rendering engine |
SetRenderEngineOwner |
Rendering engine calls this method to inform UI Layer that it is the owner of the object. Also, when rendering engine is terminating, it sets the owner of its child UI Layer to NULL. |
GetRenderEngineOwner |
Returns pointer to IMultiVMR9RenderEngine, the parent rendering engine object that currently owns this UI Layer |
Samples\cpp\DirectShow\VMR9\MultiVMR9\MultiPlayer.
This MFC-based sample implements default functionality of MultiVMR9.dll, so make sure that MultiVMR9.dll is built and registered with regsvr32.exe. MultiPlayer looks much like the property page of the VMR9 in the mixing mode; the only evident difference is that this application supports several filter graphs at once. Application starts with configuration dialog.
1. Controls related to the application:
2. Controls related to the selected item of the combo box:
Notes: Application does NOT support multiple monitors. Desktop must be set to the 32 bit mode.
For other details, see the source code.
Samples\cpp\DirectShow\VMR9\MultiVMR9\GamePlayer.
This sample illustrates how to integrate video powered by VMR9 into (some imaginary 3D game) Direct3D environment. Application uses the same MultiVMR9.dll, but overrides IMultiVMR9MixerControl and IMultiVMR9UILayer. Before building this sample, make sure that MultiVMR9.dll is built and registered with regsvr32.exe. Please note that slow machines and older video cards can demonstrate significant performance drop. Machines like Pentium-III 500MHz and up with videocards no older that 2 years would be enough.
Usage
1. Run GamePlayer
2. Select a few (3-5) mpeg1 or AVI movies and hit "start".
3. New window should appear; a girl walking along the wall.
4. In a few seconds, girl starts passing videos on the wall.
5. To focus on the highlighted movie, hit the yellow button in the lower right corner.
6. To stop the presentation, go back to initial dialog and hit "stop" and "exit".
For other details, see the source code.