Testing With 3DMark
This page contains important information that is required for reliable testing with 3DMark2000. Please read this page through very carefully.
Official 3DMark2000 Testing Guidelines
In order to get accurate and repeatable results, please make sure you have done the following:
All benchmark tests run with VSync disabled to eliminate the refresh rate dependencies along different test environments. However, older display card drivers might not support this feature. Take care that you observe whether VSync is disabled or not.
Official Testing Requirements are:
- Compare 3D accelerators using the same CPU and CPU Optimization setting on all accelerators (unless you are also comparing an Hardware T&L accelerator).
- Compare 3D accelerators using the the same resolution, color depth, texture resolution, z-buffer format, buffering mode and refresh rate settings.
- Compare CPUs using the same system configuration (where possible). If you would like to obtain a CPU score please make sure you select the CPU software DLL from display settings.
- Stop all other programs. Under the 32-bit Windows operating systems, other tasks can preemptively run during the tests, thus affecting the results. You will want to eliminate as many of these types of tasks as possible, such as PerfMon or System Agent or other tasks from your StartUp folder. This action also maximizes your available memory.
- Disable any network connections and file sharing. Background network activity may affect results. Restart the computer before running the benchmark.
- Run the tests at least three times to verify that the results are reproducible. Follow a standard testing procedure each time you run the tests.
Additional Recommendations for Testing:
- We recommend using Triple Buffering even when VSync is off.
- We recommend that you use the same monitor and monitor settings for all tested 3D accelerators.
- We recommend that you test 3D accelerators on a "clean" system only. You should use a program like "Norton Ghost" (http://www.ghost.com) to restore a clean system for every 3D accelerator test.
- We recommend testing CPUs with the fastest available 3D accelerator, in the lowest possible resolution.
These measures help to ensure that your results are consistent.
3DMARK2000 BENCHMARK RESULTS ARE INCOMPATIBLE WITH ANY PREVIOUS VERSIONS OF 3DMARK. YOU SHOULD NOT COMPARE THE RESULTS OF DIFFERENT VERSIONS AGAINST EACH OTHER.
Testing Methodology
Overview
Testing methodology has changed from earlier 3DMark versions to ensure that all latest advances in hardware are tested properly.
3DMark tries to disable VSync automatically when benchmarking. This may not be supported by all hardware or drivers.
Please note that 3DMark tests run as time based, not fixed frame based (with the exception of CPU Speed test). The goal is to measure real game application performance and allow the CPU and display card work concurrenctly.
General Test Methodology
1. Initialize scene
2. Start timer
3. Render as many frames as possible within n seconds
4. When n seconds is reached, lock the frame buffer to make sure that all frames are rendered
5. Stop timer
6. Unlock frame buffer
7. Calculate result based on "NumberOfFramesRendered / Time"
8. De-initialize scene
CPU Speed Test Methodology
This method is used in the CPU Test only. The CPU Speed test renders always the same fixed frames.
2. Start timer 3. Render fixed frames 4. When all frames have been rendered, lock the frame buffer to make sure that all frames are rendered
5. Stop timer
7. Unlock frame buffer
3. Calculate result based on FixedNumberOfFrames / Time
4. De-initialize scene
All tests use DirectX 7 functions. Local vertex buffers and guardband clipping is also used throughout the benchmark.
Refresh Rate and Frame Buffering Modes
What is Refresh Rate and VSync
Refresh Rate indicates how many times per second the image on your monitor is drawn. It is not directly related to how many times per second the content of the screen changes. Normal refresh rate is for example 75Hz.
Vertical Retrace or VSync is a signal that occurs once per screen refresh, when the electron beam on the monitor starts drawing a new frame from the top-left corner of the screen. Usually all programs wait for Vertical Retrace to occur and then flip front and back buffers. This eliminates any "tearing" effects on the screen and results in a steady image being shown. If VSync is disabled, the programs do not wait for the electron beam to finish updating the previous frame but can change in the middle of the refresh, as soon as the back buffer image has been rendered completely.
Frame Buffering Modes Explained
Frame Buffer is an area of video memory that is used by the 3D accelerator to store the pixels (image) rendered. Frame buffer size depends on the used resolution and color depth. For example in 800 x 600 resolution, 16 bit color mode one frame buffer takes 800*600*2 (16 bits = 2 bytes) of memory, that is 960,000 bytes. Front Buffer is the frame buffer currently displayed on screen. Back Buffer is a non-visible frame buffer.
Single Buffering uses only one front buffer. It is displayed on screen at all times, while also rendering to it simultaneously. Therefore the image "flickers" as it is getting updated. Single buffering is not used by real world applications because of the flickering issues, but if it is used, the results are not affected by refresh rate or vsync setting. 3DMark detects VSync to be off when single buffering is used.
Double Buffering uses two frame buffers, one front buffer and one back buffer. The 3D accelerator renders always to the back buffer, and after the render is complete, the 3D accelerator waits for VSync, after which it swaps front and back buffers and starts rendering again. Double buffering is used by most games by default.
Triple Buffering [default] uses one front buffer and two back buffers. The 3D accelerator renders to the 1st back buffer, and immediately after completing it starts with the second back buffer. Triple buffering is gaining popularity in games as it usually makes gameplay smoother.
Quad Buffering does not exist as it is not practical. In a fast-paced interactive game, the time when your actions would be visible on screen could be as high as only after 4 drawn frames. With 30 fps, this would mean a 0.13 second delay in user's actions. A man can get fragged in less time!
Effects of Different Buffering Modes
With Double and Triple buffering, the maximum attainable frame rate is the refresh rate unless VSync is disabled (off). Single buffering behaves always just like VSync would be off and has no maximum frame rate limits. In other words, the refresh rate limits the maximum frame rate performance - the frame rate can not go over the refresh rate if VSync is enabled.
Using triple buffer is a good thing as it usually improves game frame rates. Also, because of various technical reasons, triple buffering can actually be faster than single buffering in some cases.
Other Notable Facts
Different 3D accelerators may have different internal (hardware or driver level) implementations of buffering modes, refresh rate and VSync on/off. This means that one accelerator may, in comparison, perform better using a certain setting, but perform poorly with other settings.
Even with VSync off, comparing two accelerators, one using for example 60Hz refresh rate and the other 120Hz, may bias the performance toward the accelerator with 60Hz refresh rate. The lower refresh rate consumes less internal memory bandwith on the 3D accelerator, leaving more bandwidth for rendering purposes. Theoretically best results should be achieved with VSync off, using 60Hz refresh rate. For absolute accuracy, remember to use the same VSync setting and refresh rate on all accelerators.
