Guidelines for reliable testing

For consistent and reliable testing, there are certain procedures that should be followed as good practice. The following suggestions are not necessary in order to run 3DMark03 but they will help to ensure that any analysis you make of the test results is more meaningful.

3DMark03 testing suggestions:

During all tests, 3DMark03 will instruct the graphics adapter drivers to disable Vsync. However, some driver revisions and older pieces of hardware will ignore such instructions. Futuremark therefore recommends that users make additional checks to guarantee that Vsync is disabled.

For professional level testing, the following suggestions will help to ensure a greater level of consistency:

NOTE: No benchmark results from 3DMark03 can be compared to those obtained from any previous version of 3DMark, as each one uses a completely different rendering system and test procedure to obtain the final results.


Non-comparable results and cheating

During the lifetime of the 3DMark benchmark series, eager users and even professionals have scewed the 3DMark results, in order to increase the performance measurement of their systems or the hardware products they represent. This kind of dishonest results may not be published or quoted without pointing out what kind of measures where used for increasing the results. There are a large number of known ways to "cheat" in 3DMark, but we will only list a few obvious ways to give some examples of what kind of "tweaks" produce results that are not comparable with default 3DMark results.

Requirements for a 3DMark03 default score:



The Main Window

Upon loading 3DMark03, you will see this main window:

 

There are 4 sections to the window and 5 main buttons:

Sections:

Main buttons:

Note: All previous results are reset, when a new benchmark run is started.


Select Tests

Clicking the "Change…" button in the Tests section opens the Select Tests dialog:

By default, all the tests are selected (adjusted as for the appropriate hardware support) and each section is expanded to show the various benchmarks. By clicking on the checkboxes, you can choose which tests to run during the 3DMark03 benchmark. All supported game tests must be selected and run, in order to get a 3DMark overall score.

The test selection window also displays an estimated time as to how long it will take to complete all of the selected tests - it is an approximation due to the variable amount of time it may take to load the required data.

There are 4 buttons in this window:

See 'The Tests' section of this help file for more information regarding each specific benchmark test.

The Free version of 3DMark03 does not allow custom selections of tests. The Free version always runs all tests that the system is capable of running.


Benchmark settings

Clicking the "Change…" button in the Settings section opens the Benchmark Settings dialog:

There are various controls in this window, which have a direct impact on the benchmark tests in 3DMark03:

Resolution
This drop-down menu allows you to select the frame resolution that all the tests are rendered at. Choosing a setting higher than the default value of 1024 x 768 may result in test scores lower than the default values. The range of resolutions available depends on what the graphics adapter and monitor both supports. 3DMark always uses 32 bit colors, with a 8-8-8-8 bit RGBA color palette.

Anti-Aliasing
This drop-down menu selects which level of Full Scene Anti-Aliasing (FSAA) to use, provided the graphics adapter has support for FSAA via DirectX. Choosing higher levels of anti-aliasing will produce finer looking graphics, but the test results will be lower. You can choose between non-maskable anti-aliasing (super sampling), or anti-aliasing of certain sample counts, depending on what your hardware supports. Usually, a higher sample count produces a finer but slower rendering. If you choose non-maskable FSAA, 3DMark automatically uses the lowest quality level.

You can also switch off anti-aliasing, which is the default setting.

Texture Filtering Mode
This drop-menu allows you to select what form of texture filtering is applied through all of the tests. There are various options:

If you select anisotropic filtering, you can set the anisotropy amount below the Texture Filtering drop-menu. The default anisotropy level is 4.

The default setting for the texture filtering mode is Optimal.

Shaders
The Post-processing check box enables full frame filtering in game tests 2, 3 and 4. Depth of field and bloom filtering is done by rendering every frame to a texture and modifying the frame using 1.1 pixel shaders. The depth of field filter produces a cinematic effect where only objects at a certain distance from the camera are displayed sharply focused. Objects closer or further away are somewhat blurred. The bloom filter is another cinematic effect that make bright surfaces appear overbright. The bright surface not only gets brighter discarding any details too small to be visible in the intense reflection, the light partly spills over darker areas in the frame surrounding the bright surface.

The second checkbox controls whether the graphics adapter or the CPU processes the vertex shading routines in the tests. Note that if the graphics adapter does not support hardware processing, all vertex shaders will be processed by the CPU anyway. Selecting Force software vertex shaders, if hardware accelerated is available, may result in some tests producing significantly lower results. On the other hand, running the default benchmark with forced software vertex shaders is an excellent way to test CPU performance.

The third checkbox forces pixel shaders 1.1 for game tests 2 and 3. These game tests have two optional rendering paths, one more compatible path utilizing pixel shader 1.1, and a second performance optimized path for hardware with support for pixel shader 1.4 and above. The compatible path uses 3 rendering passes per light source (+ depth pass), but the optimized path does the same rendering in a single pass (+ depth pass). If you want to compare pixel shader 1.1 hardware to DX9 hardware and do the exact same rendering passes, this checkbox makes it possible.

The default benchmark settings are post-processing switched off, hardware vertex shaders if supported and use the optimized shader path, if supported. Note that the 3DMark03 demo always uses post-processing in demo parts 2, 3 and 4.

There are additional benchmark settings on the right side of the dialog:

The Free version of 3DMark03 can only run the benchmark using default settings


Default settings

The default settings in 3DMark03 are as follows:


Benchmark results

After every benchmark run, and when clicking the Options... button in the Result frame of the main window, the 3DMark Score dialog is displayed.

The 3DMark total score of the latest benchmark run is displayed in large letters. This dialog also has a number of buttons for result handling:

The 3DMark total score is calculated using the following formula:

3DMark score = (GT1fps * 7.3) + (GT2fps * 37) + (GT3fps * 47.1) + (GT4fps * 38.7)

GT1fps means the average frame rate of the first game test etc.

You can view the frame rates of the separate game tests and the results of the theoretical tests with the Online ResultBrowser. You can also view the result details by clicking the Show Details button, but the ORB contains many other services with valuable information about your system, which cannot be obtained off-line.


Image Quality

The Image Quality functionality helps taking screenshots of the tests in 3DMark03 for image quality comparison. Any frame from the benchmark tests can be selected, and that precisely same frame can be rendered on other systems for comparison, or using the DirectX reference rasterizer, for reference renderings. Take one or any number of screenshots and compare the rendering quality using your favourite Image Composer software.

Note: some objects and characters in the scenes might be placed a bit differently and look different. This is due to the usage of real-time physics. Also, particle systems get their look over time, this means that slower particle systems might look different than during test runtime.

The Image quality dialog has the following options:

Note: The DirectX 9 SDK must be installed, when using the reference rasterizer.

The image quality testing functionality is not enabled in the Free version of 3DMark03.


Texture Filtering test

The Texture Filtering test shows a 3D scene that highlights texture filtering artifacts and imperfections. You can move back and forth, and rotate the tunnel with the arrow keys. You can also change the texture filtering settings at run-time. The run-time keys are displayed, so go ahead and try different combinations of settings. Remember that your graphics hardware might not support all implemented filtering settings.

Note: The Texture Filtering test is also useful for anti-aliasing quality testing.

The Texture Filtering test has the following options:

The runtime keys and options are:

Words like magnification and minification might sound strange to 3DMark users unfamiliar with DirectX details. The following combinations of the settings are usually utilized in 3D games and applications:

Note: The Texture Filtering test goes on until you press the 'Esc' key.

The Texture Filtering test is not enabled in the Free version of 3DMark03.


Demo

The Demo button in the main window opens the Demo Settings. Select the resolution and the anti-aliasing level you want. You can disable the sounds and enable looping, if you want the demo to run continuously. The demo always uses optimal texture filtering and full frame post-processing effects, using 1.1 pixel shaders. Please read more about these settings in the benchmark settings chapter.

Note: The demo requires graphics hardware with 1.1 pixel shader support. Also, the post-processing effects require that the graphics card can render to textures of non-power-of-two dimensions. The fourth part of the demo is based on game test 4, and requires pixel shader 2.0 support in the hardware.

The Free version of 3DMark03 can only run the demo using default settings.


System Info

System Info shows you the hardware setup of your PC in detail. Click the Details... button in the System frame of the main window to view your system info. The detailed System Info display requires that Internet Explorer 6 is installed.

System Info shows you details regarding your PC's:

NOTE! 3DMark03 runs on a wide variety of PCs, equipped with very different hardware. We have done our very best to build a system info utility which detects any hardware we have been able to test it on. Still, it is possible that your PC contains for example a motherboard, which doesn't provide the CPU bus clock speed or amount of memory in the format our system info expects. Should you find yourself in this situation, please check our FAQs for more information.


Command Line Options

3DMark03 has a number of command line options. These can be used for benchmarking automation, automatic 3DMark startup after system startup, and some options are only available as command line options. Command line options are options that are entered when 3DMark is started. The traditional way is to open a command prompt, and type manually the name of the executable that is to be run and the command line options for it. The examples of using the '-help' options shows two other ways to use command line options.

All available command line options are presented in a window, when you start 3DMark03 with the switch -help. This can be done in at least two ways:

  1. Drag the 3DMark03.exe file or the 3DMark03 desktop shortcut to the Run dialog of your Start menu. Press the Start button on your desktop and then select Run to open the Run dialog. When the 3DMark03 file or shortcut is in the text field of the Run dialog, move the cursor to the end of the text, add a space, type -help and click OK.
  2. Edit the 3DMark03 shortcut on your desktop by clicking it with the right mouse button. Select properties and edit the Target: text field. Move the cursor to the end of the text, add a space, type -help and click OK. Next time you click the 3DMark03 desktop shortcut the command line switches will be displayed. Remember to change the shortcut back to the way it was, so that you can run 3DMark03 again.

Use all command line switches the same way you used the -help switch. The command line switches are the following:

-help - show all command line switches
-verbose - write a verbose file of benchmark transactions during loading and running of the tests.
-nosplash - start 3DMark03 without the splash screen.
-noscreens - do not display the test loading screens.
-res=<w>x<h> - set benchmark resolution <w>x<h>. Example: -res=1024x768
-filter=<x> - set texture filtering mode, where <x> can be 'optimal', 'bilinear', 'trilinear' or 'anisotropic'. Example: -filter=optimal which is the default.
-aniso=<x> - set maximum anisotropy to be used with anisotropic filtering.
-aa=<x> - set pixel processing mode, where <x> can be 'postprocess', 'nonmaskable' for non-maskable anti-aliasing or the sample count for anti-aliasing. Example: -aa=4 for four sample anti-aliasing. The default value is not to use any pixel processing.
-vshader=<x> - enable or disable forced software vertex shaders, where <x> can be 'hw' or 'sw. Example: -vshader=sw for software forced vertex shaders. The default value is 'hw'.
-repeat=<x> - set how many times each test should be repeated, where <x> is the amount of repeats. Example: -repeat=3 . The default is to run each test once.
-gt1 - run game test 1. By default all supported tests are run.
-gt2 - run Game Test 2.
-gt3 - run Game Test 3.
-gt4 - run Game Test 4.
-singlefill - run the single texture Fill Rate test.
-multifill - run the multiple layer Fill Rate test.
-vertex - run the Vertex Shader test.
-pixel - run the Pixel Shader 2.0 test.
-physics - run the Physics test.
-sound0 - run the reference sound test, with no sounds.
-sound24 - run the sound test with 24 3D sound sources.
-sound60 - run the sound test with 60 3D sound sources.

You can also put the name of a 3DMark03 result file after the switch. This will run the benchmark specified by the other options and save the results in the result file. Example: 3DMark03 myperf.3dr . This will run the default benchmark and save the results in the file myperf.3dr.

One good use of command line switches allows you to start the 3DMark03 benchmark automatically after Windows has booted up. Make a 3DMark03 shortcut with the desired command line switches. Copy this shortcut to the Windows Start-up folder. Then every time the PC is booted-up, 3DMark03 is automatically run.

The Free version of 3DMark03 cannot use command line options.


Batch Run

If you want to run a number of results on a system using different settings and test selections, this can easily be done using the batch run functionality. The batch run functionality works like standard Microsoft Windows batch files, which use the file ending '.bat'. Just write a text file with a number of 3DMark03 runs, using various settngs as command line options, save that textfile in the 3DMark03 installation directory, and give the file the ending '.bat'. The batch run can now be run by double clicking the .bat file. The obtained result files can be viewed with the Online ResultBrowser, with the 'Submit Saved Result' functionality of the right-click menu, or they can be viewed in Microsoft Excel, using the 'Open in Excel' functionality of the right click menu.

Example 1:
Save a textfile with the following contents with the file name batchrun1.bat:

3DMark03 results.3dr

This batch run will run the default benchmark and save the results in the file 'results.3dr'

Example 2:
Save a textfile with the following contents with the file name batchrun2.bat:

3DMark03 -res=640x480 -aa=4 -gt1 results1.3dr
3DMark03 -res=1024x768 -aa=postprocess -gt2 -gt3 results2.3dr

This batch run will run the game test 1 in 640 x 480 resolution and with 4 sample anti-aliasing, and save the result in results1.3dr. Then it will run game tests 2 and 3 in 1024 x 768 resolution and using full frame post-processing, and save the results in the file 'results2.3dr'

The Free version of 3DMark03 cannot do batch runs.


Right Click Menu

Place the mouse pointer over the main window and click the right mouse button to open the right click menu. This menu has some useful functionality:


Other functionality

When you start 3DMark03 for the first time, you will get a 'Tip of the Day'. These tips contain important information about 3DMark03. We recommend clicking the 'Next' button and reading through all the tips, or at least leaving the tips enabled. When the tips are enabled, you will get a new tip every time you start 3DMark03. If you have read through all tips, or find this kind of pop-ups annoying, you can disable the tips. The tips can later be re-activated from the right click menu.

You can interrupt an ongoing benchmark run, demo run or any other test by pressing the 'Esc' key on your keyboard. However, this functionality does not work during test loading.

You can take screenshots by pressing the F12 key during any of the tests and the demos. The screenshot will be saved in the 3DMark03\shots folder in .BMP format.

NOTE: Taking a screenshot affects your test result by a temporary performance drop. Therefore the result of the current test will be discarded, when you take a screenshot.