Developing Testing Strategies

The goal of your application testing is to verify that everything that works on your current platform also works on Windows 2000. If an application was written for a previous version of Windows, it will not necessarily use new Windows 2000 features to the best advantage, but its functionality should work as well on Windows 2000 as it does on your current platform.

Strategies for Commercial Applications

For commercial applications, the first step is to run Windows 2000 Professional Setup in Check Upgrade Only mode to check for potential incompatibilities. When you run Setup in this mode, it checks the installed software against a list of applications known to be incompatible and logs any that it finds. The command-line syntax for Check Upgrade Only mode is:

winnt32 /checkupgradeonly

Although this utility can alert you to potential compatibility problems, it addresses only a small percentage of your applications and only the applications installed on the computer you are checking. Even if an application is not on the list of incompatible applications, it does not mean that it is compatible. For more information about the Setup program, see the chapter "Automating Client Installation and Upgrade"in this book.

The next step is to check the directory of Windows 2000 applications to determine the compatibility of the applications you use.

Even if you find that some of your applications have already been tested by others, you should test them in your environment. In this case, focus your testing on the way your organization uses the applications. For example, test:

• Configurations your organization uses.
• Features most frequently used.
• Combinations of applications used together.

The section "Testing Tips" later in this chapter provides examples of ways to test application functionality. If the commercial applications you use have not been tested for compatibility by other organizations, you should test more extensively than otherwise.

Remember to test your antivirus software. Many of these applications need to be upgraded because of their use of file system filters. Many Windows NT 4.0 file system filters might not work on Windows 2000 due to changes in the NTFS file system.

Strategies for Custom Applications

If you use custom third-party products or develop applications internally, you need to develop a more extensive testing strategy than for pretested commercial applications.

Even if you are testing an application you did not develop, the Windows 2000 Application Specification can provide insight into testing. The MSDN Web site includes a downloadable version of the specification, as well as a test plan detailing all the Microsoft tests for Windows 2000 application certification. This test plan can provide you with ideas about functional areas and ways you should test. For more information about how to download the specification or test plan, see the Application Specification Download link on the Web Resources page at http://windows.microsoft.com/windows2000/reskit/webresources.

The MSDN Web site also contains other important information about testing, such as white papers about exploratory testing and the method that independent testing organizations use to test the functionality of applications that vendors submit for certification.

Testing Tips

The testing suggestions in this section are not comprehensive and do not apply to all situations. They are provided to help you start thinking about how to test.

Test Deployment Scenarios

You should test installing and running your applications using the scenarios you plan to use during deployment. For example, you might plan to deploy by installing on clean machines or by upgrading from Windows 3.x or a prior version of Windows NT. If you plan to upgrade, you might keep the applications on the computer during the upgrade, or you might uninstall them and reinstall them after the upgrade.

Consider repackaging applications for the Windows Installer or as .zap files so the application can be managed by IntelliMirror software installation and maintenance. For more information about packaging applications, see the chapter "Applying Change and Configuration Management"in this book.

Because of differences between Windows 3.x and Windows 2000, some application installations work differently depending on which operating system you use for the installation. For example, if you install an application on a computer running Windows 3.x and then upgrade the computer to Windows 2000, the application might not work the same way as it would have if it had been installed on Windows 2000. In this case, you might need to uninstall the application and reinstall it after the upgrade or obtain a migration dynamic link library (DLL).

A migration DLL allows an application that was originally installed on Windows 3.x to function correctly after the computer is upgraded to Windows 2000. Migration DLLs can resolve application problems by performing the following actions:

• Replacing or upgrading Windows 3.x–specific files with Windows 2000–compatible files.
• Moving application and user settings to the correct location for Windows 2000.
• Mapping Windows 3.x–specific registry keys to the appropriate Windows 2000 locations.

For applications developed internally, you might have to create migration DLLs or obtain them from your vendors. You can find more information about creating and testing migration DLLs by searching the MSDN Library using the keywords "migration DLL." For more information about the MSDN Library, see the MSDN link on the Web Resources page at http://windows.microsoft.com/windows2000/reskit/webresources. Some migration DLLs are also included with Windows 2000.

Because results can vary depending on how you upgrade, it is important to test using the same procedures and tools you plan to use during the rollout. If the procedures and tools are not ready when you are testing the applications, at least test the scenario you plan to use.

Upgrade Scenario

If you are planning to upgrade your computers, use the following steps:

• Install Windows 3.x or Windows NT 3.51 or later.
• Install the application.
• Upgrade to Windows 2000.
• Test the application.

If a Windows 3.x application does not function properly, contact the ISV for a migration DLL. If a Windows NT application does not function properly, contact the ISV for a patch or a new setup program.

Clean Installation Scenario

If you are planning to install Windows 2000 on reformatted computers, use these steps:

• Install Windows 2000.
• Install the application.
• Test the application.

If the application does not function properly, contact the ISV for a patch or a new setup program.

Test Installing and Uninstalling

Test the installation of the application in a variety of ways, such as the following:

• Terminate the installation before it is complete.
• Try all the installation options used in your environment.
• If your organization allows users to install applications, test the installation both as administrator and as power user; then test the application's functionality.
• Try to uninstall the application.
• Verify that an application can be installed by an administrator and uninstalled by a user. When a user is logged on as a user, the uninstall should be either complete or disallowed.

Test Basic Application Functionality

Test applications using the features, configurations, and application suites you use to accomplish business tasks. For example, you can try the following types of tests:

• Log on as a user and test the features most important to your end users. Test specific scenarios that are needed to accomplish business tasks.
• Log on as several users who are members of the Users group.
• Apply Group Policy to the system and to the applications.
• Test combinations of applications, such as standard desktop configurations.
• Run several applications on the desktop for several days or weeks without shutting them down.
• Test automated tasks that use Microsoft® Visual Basic® for Applications (VBA) in Microsoft® Office applications.
• Test to determine that long file names are consistently supported. Include embedded periods and verify that leading spaces are stripped.
• Manipulate large graphic files—for example, files over 1 MB.
• Perform extensive edits on word processing documents.
• Perform rapid development sequences of edit, compile, edit, compile.
• Test OLE custom controls (OCXs).
• Test with applicable hardware, such as scanners and Plug and Play devices.
• If you plan to deploy Terminal Services, test the applications on a Terminal Services server. Test with multiple users running the same and different applications and with user-specific settings.

To download a sample test plan, see the Application Specification Download link on the Web Resources page at http://windows.microsoft.com/windows2000/reskit/webresources.

Access Data

Try to access data in a variety of ways, such as the following:

• Access data on a server running the current version of Windows, as well as on a server running Windows 2000.
• Test concurrent use of a database, including simultaneous access and update of a record.
• Perform complex queries.

Test Printing

Print a variety of document types with a variety of printers, such as the following:

• Print documents with embedded files from several source applications.
• Print to printers with long file names.

Use Testing Tools

The Windows 2000 Software Development Kit (SDK), Driver Development Kit (DDK), and Microsoft® Windows® 2000 Server Resource Kit contain tools for testing and debugging applications.

• Dependency Walker, in the Windows 2000 Server Resource Kit, recursively scans dependent modules that are required by an application. This utility detects files that are missing, files that are not valid, import or export mismatches, circular dependency errors, and modules installed on mismatched computers. For more information about this utility, see the Help file that accompanies it.
• Apimon, in the Windows 2000 Server Resource Kit, monitors a running application for all application programming interface (API) calls, counting and timing them. Optionally, it can also monitor page faults. Apimon can report the following:
  • A count of all API calls and timings for each.
  • A trace of API calls in the sequence in which they occur.

© 1985-2000 Microsoft Corporation. All rights reserved.