Microsoft DirectX 8.0 (Visual Basic)

How the Stencil Buffer Works

Microsoft® Direct3D® performs a test on the contents of the stencil buffer on a pixel-by-pixel basis. For each pixel in the target surface, it performs a test using the corresponding value in the stencil buffer, a stencil reference value, and a stencil mask value. If the test passes, Direct3D performs an action. The test is performed using the following steps.

  1. Perform a bitwise And operation of the stencil reference value with the stencil mask.
  2. Perform a bitwise And operation of the stencil buffer value for the current pixel with the stencil mask.
  3. Compare the result of step 1 to the result of step 2, using the comparison function.

These steps are shown in the following code example.

(StencilRef And StencilMask) CompFunc (StencilBufferValue And StencilMask)

StencilBufferValue is the contents of the stencil buffer for the current pixel. StencilMask represents the value of the stencil mask, and StencilRef represents the stencil reference value. CompFunc is the comparison function.

The current pixel is written to the target surface if the stencil test passes, and is ignored otherwise. The default comparison behavior is to write the pixel, no matter how each bitwise operation turns out (D3DCMP_ALWAYS). You can change this behavior by changing the value of the D3DRS_STENCILFUNC render state, passing a member of the CONST_D3DCMPFUNC enumeration to identify the desired comparison function.