PC-Test Pro

home *** CD-ROM | disk | FTP | other *** search

/ PC-Test Pro / PCTESTPRO.iso / disktool / qbench / entp / qbinstr.com (.txt) < prev next >

Wrap

Graham's TXT2COM | 1992-10-29 | 57.1 KB | 1,071 lines

Quantum Data Access Time Benchmark Operating Instructions and Technical Information Before You Begin As with any hard disk benchmark, QBENCH results may be affected by disk-caching utilities or TSR (terminate-and-stay-resident) programs. To obtain accurate results, remove any such programs before running the benchmark. QBENCH requires 535,000 bytes of free memory to run, primarily due to the large buffer necessary to obtain accurate results. If, during benchmark operation, the message "QBENCH has run out of memory" is displayed, it will be necessary for the user to remove any resident programs or otherwise free up memory before QBENCH can be run. In addition to disk caching utility software, caching disk controllers may also cause the data access times reported by QBENCH to be unusually low. NOTE: QBENCH is a DOS program which cannot be run under Windows or OS/2. Also, QBENCH cannot be used with drives which do not have a BIOS or INT13h interface. To stop the benchmark prematurely during test, press <Esc>. DO NOT stop the program by turning off the power or by restarting your computer. For instructions on running the QBENCH program, refer to Section 2, "Running QBENCH." How to Obtain Assistance for QBENCH If you need assistance, you can contact Quantum Product Support. When calling Product Support, please have the following information available: ∙ Type of computer ∙ Version of QBENCH ∙ DOS version ∙ Description of problem or error encountered We advise that, prior to contacting Quantum for assistance, you read this documentation file. This documentation file has explanations and examples that have been researched and developed to be self explanatory. If, after reading this documentation, you must contact Quantum for reasons that are not explained in the QBENCH help screens or this document, you should call the Quantum assistance line. The Quantum QBENCH assistance telephone number in the United States and Canada is: 1-800-624-5545 The time of operation for this assistance number is 8am to 8pm EST. International Residents: Outside the United States and Canada, contact the Quantum distributor in your country. Bulletin Board Quantum Product Support has a 24 hour, 7 day a week, bulletin board system. The bulletin board system allows you, if you have a modem, to download the latest version of the QBENCH files. In addition, the bulletin board provides answers to the most commonly asked questions. The bulletin board supports up to 9600 bps modems and uses 8 bits, no parity, and 1 stop bit (8, N, 1). The telephone number for the bulletin board is: 408-894-3214 Notice You are free to use, copy and distribute the QBENCH files (QBINSTR.COM, QBENCH.EXE, QBENCH.HLP, README.COM) for non-commercial use IF: - no fee is charged for use, copying or distributing. - the four files are distributed together. - the program and documentation files are not modified in any way. Quantum provides the software "as-is". Quantum shall not be liable for loss of data or other damages of any kind on any basis, even if Quantum or an agent of Quantum has been advised of the possibility of damages. The person using the software bears all risks as to the quality and performance of the software. The information in this manual is subject to change without notice. Quantum Corporation shall not be liable for technical or editorial errors or omissions contained herein; nor for incidental or consequential damages resulting from the furnishing, performance, or use of this material. Product names mentioned in this document and the QBENCH program are for identification purposes only and may be trademarks and/or registered trademarks of their respective companies. Copyright 1992 Quantum Corporation. Table of Contents Section 1 - Introduction Why Most Disk Drive Benchmarks Don't Measure Up Not The Same Old Game Current Seek Tests Are Based On Several Myths A Better Disk Benchmark QBENCH: How It Works Definition of terms See for yourself Section 2 - Running QBENCH Introduction Beginning The Test Test In Progress Screen Information Test Completed Screen Detailed Results Screen Compare Results Screen Appendix A - Command Line Options List Command Line Options - QBENCH /? Continue If Error Occurs - QBENCH /C Specify Drive Number - QBENCH /D:n Specify Test Iterations - QBENCH /I:n Specify Number of Single Sector Transfers - QBENCH /M1:n Specify Percentage of Sequential Accesses - QBENCH /PS:n Specify Percentage of Reads - QBENCH /PR:n Specify Quick Test - QBENCH /Q Change Maximum Transfer Block Size - QBENCH /127 Appendix B - Data Access Time Technical Description QBENCH Overview QBENCH Theory Definitions Appendix C - QBENCH Limitations and Erroneous Results Causes Data Access Time Limitations Erroneous Results Disk Errors Section 1 Introduction Why Most Disk Drive Benchmarks Don't Measure Up Hard disk drives are not created equal. In comparing drives, most users understand that support and system compatibility count. They also know that a 500 MB drive holds substantially more programs and data than a 50 MB drive. But they may not know that disk drives also vary in speed, much in the way race cars and paper copiers do. In disk-intensive applications and program environments (Microsoft Windows, for example), disk drive speed can greatly affect the computer's performance overall. A relatively slow disk drive can create a bottleneck, slowing down the workings of an otherwise fast system. Consequently, anyone who purchases a hard drive should be aware of speed. But how can you compare disk drive speeds? For years, average seek time has been used throughout the industry to measure drive performance. In addition, commercial benchmark tests have evolved from utility software vendors as well as laboratories that evaluate equipment on behalf of computer publications. However, these tests have not kept up with the times. Disk drive technology has changed radically since the first hard disk drives for personal computers were introduced in the early 1980s. But the criteria by which drives are assessed have not changed. As PC software grows more complex and as PC systems become more powerful, the importance of disk drive performance will grow. To provide a more meaningful assessment of how one disk drive compares with another, Quantum has developed a measurement of disk drive performance called data access time. This sophisticated measure takes into account the technologies incorporated into today's disk drives-- and considers the way people use their systems in their daily work. The objective is to help value-added resellers, system integrators and end-users alike make more informed decisions about hard drives. Not The Same Old Game Because reading and writing data is a physical process, a delay occurs between the computer's request for information from the disk and the time it actually receives that information. A comparable delay also arises when the computer wants to write information to the disk. Either way, the request must be transmitted, the head must be positioned correctly over the disk, and the data must be transferred. Currently throughout the industry, this sequence is called the average seek time. A typical benchmark test forces the disk drive head to move from one random location to another and times the overall results. The problem is that this process is artificial: data is never transferred to or from the disk. Imagine testing a soccer player by having him run to a set number of randomly selected locations on a playing field. That measure would tell you something about the player's ability to run fast. But it would not reflect his performance in an actual game, because speed is not the only skill needed--opposing players intervene and strategies change. In a similar way, testing for average seek times provides a good measure of a drive head's ability to move from place to place. But this measure is far weaker in showing how the drive would actually perform in the workplace. Current Seek Tests Are Based On Several Myths Myth: All disk drive requests are random access. Fact: Data is not randomly scattered all over the disk. Much of it is arranged sequentially on adjacent tracks. In a study conducted by Quantum and corroborated by a major personal computer manufacturer, MS-DOS and Windows machines were monitored to determine how data was read from and written to the disk in daily use. As shown in Table 1, sequential access predominates. The study also revealed that disk writes are much more important than previously thought-accounting for 40 percent of all disk operations. The reason: Disk caching-storing recently read data in memory for faster access-has reduced the number of times data must be read directly from the disk. Disk caching has become much more prevalent since the inclusion of the SMARTDrive disk caching utility in Windows 3.1 systems. Consequently, disk writes now account for a larger percentage of disk operations than before. Table 1. Typical Single-User DOS Workload Accesses Read Write Total Sequential 39% 26% 65% Random 21% 14% 35% Total 60% 40% 100% Myth: Only the seek portion of a random operation-the time it takes for the head to move from one region of the disk to another-is relevant. Fact: Other factors also influence a random request. For example, sometimes the head must wait for the data to rotate into place, causing a delay known as latency. But where disk drives once rotated at the same speed, some high-performance models now spin significantly faster, thereby reducing latency. And advancements in circuitry enable a drive to read data out of order, then reassemble it, thereby saving additional time. Myth: New disk drive technology that doesn't improve seek time can be ignored. Fact: Few technologies have benefited from as many innovations as the disk drive. And these design factors greatly influence both sequential and random operations. For example, write caching can greatly speed write procedures by avoiding excessive latency and assuring they are done as a single data stream in one disk revolution. Read-on-arrival speeds up read operations by reading information as soon as the head arrives, rather than delaying a few milliseconds for the head to settle into place. And multiple-zone recording stores more data on the larger tracks toward the outer edge of the disk, to increase disk capacity and disk performance. The interactions among these techniques also affect the drive's overall performance. A Better Disk Benchmark The data access time performance index is Quantum's contribution to the effort to measure hard disk performance in a more meaningful and rigorous way. QBENCH sets a new standard in these directions: ∙ The test replicates the way disk drives are used in the real world by combining the same proportions of random and sequential read-and-write operations that are found in the field. ∙ The test is almost completely independent of variations in system type and speed (e.g., 286/386/486). In years past, disks were invariably the weak link - the limiting factor on the data transfer rate. Today, disks are significantly faster, making for a substantial difference between the transfer rates of a slow machine and a fast one. ∙ The test pushes the disk drive to the limit, generating very high disk utilization that replicates the conditions of today's demanding software. ∙ The test provides a single composite speed index for drive comparisons and offers a methodology that produces this index accurately. QBENCH: How It Works Definition of terms Access time: The time from the issuance of a read or write command until the desired data is located but not transferred. Data access time: A workload weighted measure of drive performance showing the average access time for a typical workload mix. Workload mix - the proportion of reads vs. writes and sequential vs. random disk accesses for a typical DOS user. To compute data access time, a widespread study was conducted to determine how real users working with real applications use disk drives in the real world. Workload studies were conducted to determine the relative frequencies of sequential reads and writes (in which data is read from or written to adjacent areas of the disk) and random reads and writes (in which data is read from or read to scattered locations throughout the disk). The engineering team devised a test that performs all of these disk operations. The results are averaged, with extra weight given to those operations that occur most frequently in the field. First, QBENCH tests how long each operation takes to read or write data and send it to the central processor. Data is transferred in sectors, and QBENCH tests eight transfer sizes: 1, 2, 4, 8, 16, 32, 64, and 128 sectors. Next, QBENCH calculates the access time for each operation. Access time includes only those operations that are dependent on the disk drive, not the computer itself. In this way Access time is consistent whether the computer is a slow machine or a fast one. Finally, QBENCH calculates the weighted average of the four access times, depending on how often each occurs in the field. This final calculation is the data access time. See for yourself Data access time is a realistic, rigorous, system-independent index for measuring disk drive performance. Unlike typical benchmarks, data access time not only shows that one drive is faster than another, but also shows exactly how much faster it is. The QBENCH test takes into account the many technological advancements that have taken place in disk drives over the last decade, and also considers the way people actually use them in their daily work. We hope you will view QBENCH as a significant step in moving to a more meaningful assessment of drive performance. Its ultimate goal is to help you make more informed decisions on disk drive technology. We encourage you to share the diskette with your friends and colleagues. The QBENCH software may be upgraded in the future. For information on obtaining the latest version of QBENCH, refer to the "How To Obtain Assistance For QBENCH" section at the beginning of this document. Section 2 Running QBENCH Introduction This section provides a procedure for running the QBENCH program. Beginning The Test Step 1 Boot DOS on your computer system. Note: Before continuing and running the QBENCH program, you should read the section titled "Before You Begin" at the front of this manual. Step 2 Run the QBENCH program. a. Insert the diskette containing the QBENCH program into a floppy disk drive. b. Type in the following command line and press <Enter>: X>qbench [/option1 /option2 ...] where X = the drive letter of the floppy diskette containing the diskette with the QBENCH program. [/option1 /option2 ...] = command line options, if used. NOTE: The QBENCH program allows you to specify a variety of options from the command line if you wish to modify the QBENCH defaults. Refer to Appendix A, "Command Line Options," for additional information on the command line options. A welcome screen will be displayed while the QBENCH program is initializing the system. When the program has completed initializing the system, the Begin Test dialog box will be displayed. To make selections on the various screens displayed when you are running QBENCH, and to move around in the help screens, you may use a mouse, if installed on your system, or you may use the arrow keys, <Tab> key , <PgUp> and <PgDn> keys, and the <Enter> key. The arrow keys, and the <PgUp> and <PgDn> keys, may be used to scroll through the help screens. The <Tab> key may be used to move between selection buttons and the additional topic selections at the end of each help screens. Pressing <Enter> will always select the currently highlighted item. In addition, some selections contain a highlighted character which you may type in to select that option. Step 3 Select the hard drive you wish to test. The left side of the Begin Test dialog box contains a list of all the hard disk drives in your computer. Select the one you wish to test. After the test has completed you will have the opportunity to select another drive. Step 4 Select the Standard Test or the Quick Test. The right side of the Begin Test dialog box contains buttons that allow you to select the test you want to run. Select the test you wish to run by clicking on the button with your mouse, or by typing the highlighted character. The Standard test is the most accurate test. It takes longer to run than the Quick test because it performs more repetitions of the disk operations for a more accurate measurement. The Quick test performs the same operations as the Standard test, but does not use as many repetitions of the disk operations. It should complete in about one-fifth the time of the Standard Test. With most disk drives and most computers, the overall data access time calculated by the Quick test should closely match the data access time calculated by the Standard test. NOTE: To stop the benchmark prematurely during test, press <Esc>. DO NOT stop the program by turning off the power or by restarting your computer. The following sections describe the information presented in the screen that is displayed while the test is in progress, and the information screens that can be displayed after the test has completed. Test In Progress Screen Information When you have selected the test you wish to run, the program will begin the test and display a screen that is divided into four sections: Drive Map, Test Information, Drive information, and Map information. The main portion of this screen, the Drive Map, is a graphical representation of the disk drive being tested. Its purpose is to illustrate which portions of the drive are being accessed. The Test Information section of the screen describes both the current type of disk access that is being performed and the size of the read/write operations (in 512-byte sectors). The Drive Information section of the screen displays the number of the disk drive being tested, the drive size in megabytes, and the number of cylinders, number of heads and the number of sectors per track for the drive. The Map Information section of the screen displays information on the Drive Map. You can press the <Esc> key to halt the test at any time. NOTE: If you invoke the Help screens, by pressing the <F1> key, the test will resume when you exit from the help screens. The interruption to view the help screens will not affect the accuracy of the results. Test Completed Screen When the test you selected has finished, the program will display a large number and four buttons in the middle of the screen. The large number in the middle of the screen is the data access time which QBENCH has calculated for this drive. The four buttons allow you: - To see the data that was used to calculate this number by selecting the Detailed Results button. - To compare this drive's test results with those from a variety of other disk drives by selecting the Compare Results button. - To test a different drive, or run the test on the same drive again by selecting the Test Another button. - Exit the QBENCH program and return to DOS by selecting the Exit QBENCH button. The following sections provide information on the data fields presented in the Detailed Output screen and the Compare Results screen. Detailed Results Screen The Detailed Results screen shows the test data that was used to calculate the data access time for the hard drive which was tested. The following paragraphs describe each of the data fields on the screen. Hard Disk number is the number (1, 2, ...) of the hard disk which was tested. The number of Cylinders, Heads and Sectors per track for the hard disk tested is obtained by querying the test disk. The Size of the disk tested (in millions of bytes) is calculated from the number of Cylinders, Heads and Sectors per track. Test Date/Time is the date and time that the test was run. Test Type is either Standard Test (recommended in general for accurate results) or Quick Test (to obtain quicker but less accurate results) or Custom Test if you specified, through command line argument(s), either Iterations (/I:n) or a Sector 1 Multiplier (/M1:n) value. The Workload Assumptions are used to calculate the Weighted Average column and the Data Access Time. The relative importance of each type of disk access is based on its frequency of occurrence from workload studies. The default workload values are 65% sequential versus 35% random, and 60% reads versus 40% writes. Test Loops refers to the number of disk commands which are executed and timed for each block size and type of disk access. In the default case for the standard test, 100 loops or iterations are used. The access time calculations are based largely on the command service time for a transfer size of one sector. Thus additional iterations for this measurement are necessary to obtain an accurate Data Access Time value. The Sector 1 Multiplier indicates how many times more loops will be executed for single sector transfers than for other transfer sizes. In the default case the Sector 1 Multiplier is 4 indicating that 400 iterations, rather than 100, will be used for this case. The Test Results show the average time in milliseconds to perform transfers of eight different lengths. The Block Size (Sectors) shows the size of each transfer in 512-byte sectors. A complete test is run for each block size which is a power of two (i.e., 1, 2, 4, 8, 16, 32, 64, 128 sectors), since these sizes are of most interest. For each block size tested, the average Command Service Time for each of the four types of disk access (sequential read, sequential write, random read and random write) is calculated and displayed. The command service time is the average total time to complete a read or write command. The weighted average column is calculated from the other four columns using the percentages from the workload assumptions. The Access Time displayed at the bottom of each column is calculated by taking the command service time for a transfer of one sector and subtracting the transfer time for one sector. The weighted average of the four access times is the Data Access Time in milliseconds. This is the single performance index which describes the performance of this disk drive for the given workload mix. Data Access Time is a comprehensive index of drive performance which can be used for drive comparisons. For example, if drive A has a Data Access Time of 10 ms and drive B has a Data Access Time of 13 ms, then drive A can be said to be 30% faster than drive B for an average DOS workload mix. Finally, the Transfer Rate is calculated from the slope of the weighted average line when command service time is plotted against transfer size. The slope is the time in milliseconds to transfer one sector, and the transfer rate is obtained by converting this value to KBytes/second. Compare Results Screen The Compare Results screen presents the data access times, as measured by QBENCH, for a variety of disk drives. The drives in the list are sorted in capacity order. The list contains primarily Quantum drives, but drives from other manufacturers are included for comparison. The entire list of drives and their data access times can be either Printed or Saved to a file by selecting the appropriate choice. The drive just tested is displayed in the middle of the screen, with drives in the same general size category included on the screen for comparison. The Data Access Time entry for each drive in the list is based on an actual QBENCH run for that drive. For every drive in the table, including the drive just tested, complete detailed results are available by moving to the drive of interest and selecting Detailed Results. IMPORTANT: The data access time measure has been designed to be as consistent, repeatable and system-independent as possible. However, if the results for your drive differ from those in the comparative database for QBENCH, there are several possible sources of variation which can explain such a difference. These should be kept in mind when interpreting these comparative results. The first source of variation is in the drive itself. Drives are complex electromechanical devices, whose components are each subject to some variation. Many of the entries in the comparative table represent measurements for a single drive only, which may or may not be typical of drives of that particular model. In addition, changes in the firmware which controls drive operation tend to change the data access time. Since firmware is upgraded occasionally, the access time for a given drive model may change slightly over time due to this effect. In addition, the intense nature of the I/O generated by QBENCH may significantly change the operating temperature of the drive, particularly if it was brought up from as cold start right before running the test. Such a temperature change may in turn trigger what are called thermal recalibrations in some drives. Depending upon the point during the test at which these occur, recalibrations can also significantly affect the results. The second important source of variation in results is due to the host CPU type and speed, and the I/O bus implementation of the test machine. In particular, the command service time values, which make up the majority of the detailed results, include transfer time, which, along with the Data Transfer Rate, is highly system dependent. CPU speed can change transfer rates and times by a factor of two or more from a slow machine to a fast one. The data access time measure is designed to be as system-independent as possible. However, certain drives still show a variance from one system to another. Nearly all data in the comparative table was gathered on a Compaq DeskPro 386/33 MHz computer. A few drives which could not be installed using their full drive capacity on the Compaq DeskPro were run on a comparable IBM-compatible computer with a BIOS allowing programmable drive types. A final source of variation is due to the test itself. The magnitude of this effect for a given system and drive can be observed by running the test multiple times and observing the differences. Statistical variation should be expected in any measurement. For benchmarks like QBENCH, timer accuracy is a particular reason for statistical variation. To obtain the most valid performance comparison between two drives, run the test on the system for which they are being evaluated. Besides testing the drives on the same system, be sure that the test procedure is identical in both cases so that both drives are being evaluated under exactly the same conditions. The objective of including a comparative database in QBENCH is to give the user an idea of the relative performance of a sampling of different drives. This data is included strictly for information purposes for the interested user. No guarantees should be implied about the performance of any drive based on the data in this table. NOTE: Since all tests were run on an IBM-compatible PC/AT system, the results for any SCSI drives in this table necessarily include the additional overhead necessary to translate each AT command to a SCSI command. Due to this extra command overhead, the data access time for a SCSI drive can be expected to be approximately two milliseconds higher than that for the AT version of the same drive. SCSI results may show additional variation based on the SCSI host adapter used. All Quantum SCSI drives in the table were tested using a Future Domain FD-1660 host adapter. Other SCSI drives were tested using their own adapter if applicable. Appendix A Command Line Options If you wish to modify the QBENCH defaults, you can specify a variety of options from the command line when you start the program. These command line options include: - /? Displays the command line options - /C Continues on error - check help before using - /D Specifies the drive to be tested - /I Specifies the number of iterations or loops - /M1 Specifies the loops multiplier for 1 sector transfers - /PS Specifies the Sequential access percent - /PR Specifies the Read access percent - /Q Runs the Quick test, not the Standard test - /127 Makes maximum block size 127 sectors To use these command line options, exit QBENCH now and restart it with the desired options. The following paragraphs a detailed of each of the command line options. List Command Line Options - QBENCH /? The /? option will display the complete list of command line options, along with a short description of what each option is used for. Continue If Error Occurs - QBENCH /C The /C option specifies that QBENCH will continue with the test if an error occurs while trying to read or write a sector on the hard disk. Normally QBENCH will abort the test and notify you when it encounters an error. QBENCH uses low-level read and write commands and may attempt to read or write sectors which were marked as bad when the drive was formatted by DOS. Thus errors reported by the benchmark do NOT necessarily indicate a problem with the drive, since the sector or sectors in question may have been removed from use during normal operation. QBENCH operates by reading sectors of data, and then writing the same data back to the same locations. Thus if the /C option is used and a read error occurs, the effect of writing back the possibly erroneous data is unpredictable. To be safe, always backup the data on a hard disk before running QBENCH with the /C option. NOTE: The results of the test may not be accurate if you continue the test after encountering an error because the disk may spend several seconds retrying the attempt to read or write the bad sector. Specify Drive Number - QBENCH /D:n The /D:n options specifies that the test is to be performed on hard drive number n. For example, if n is 2 QBENCH will test your second hard drive. If the /D option is specified, the test starts immediately after QBENCH initializes without user intervention. Normally, the user selects the drive to be tested from the initial QBENCH screen. Specify Test Iterations - QBENCH /I:n The /I:n option specifies that the Sequential Read, Sequential Write, Random Read and Random Write tests will each be performed n times for each sector size. The default is 100 repetitions. NOTE: QBENCH has been designed to generate the most statistically random request patterns for the default number of iterations used in the Standard Test. If more accurate results are required, the number of iterations should be increased significantly (to at least several hundred). Slightly increasing the number of iterations may give less accurate results. Specify Number of Single Sector Transfers - QBENCH /M1:n The data access time calculation is predominantly based on the results for single sector transfers. Thus, to obtain an accurate estimate, it is necessary to perform additional iterations for the single sector measurement. The /M1:n option specifies the extra weighting factor that should be applied for one sector disk accesses. By default, QBENCH uses a value of 4 for n, which means that instead of doing 100 iterations for each of the one sector tests, it will do 400 iterations. Specify Percentage of Sequential Accesses - QBENCH /PS:n The /PS:n option specifies the percentage of Sequential accesses to be used when calculating the drive's overall performance. This value is used to calculate the Weighted Average column in the detailed output as well as the overall data access time. The importance of each type of disk access is weighted by its relative frequency of occurrence in an average DOS workload mix. The remainder of the accesses will be Random. The default percentage value for Sequential access is 65 and hence 35 percent of the accesses will be Random. The default values were obtained from studies of the access patterns of DOS (and Windows) users in their day-to-day work. Specify Percentage of Reads - QBENCH /PR:n The /PR:n option specifies the percentage of Read accesses to be used when calculating the drive's overall performance. This value is used to calculate the Weighted Average column in the detailed output as well as the data access time. The importance of each type of disk access is weighted by its relative frequency of occurrence in an average DOS workload mix. The remainder of the accesses will be Writes. The default percentage value for Read access is 60 and hence 40 percent of the accesses will be Writes. The default values were obtained from studies of the access patterns of DOS (and Windows) users in their day-to-day work. Specify Quick Test - QBENCH /Q The /Q option specifies that the Quick Test should be run, rather than the Standard Test. Change Maximum Transfer Block Size - QBENCH /127 QBENCH testing has shown that certain drives and/or adapters have a problem with benchmark transfers which are 128 sectors in length (the maximum size possible). The /127 option specifies that the largest block size that should be used by QBENCH is 127 sectors rather then the default 128 sectors. Use this option if your system cannot transfer 128 sector (64K) blocks. This will not change the overall test results. Appendix B Data Access Time Technical Description QBENCH Overview QBENCH performs a total of 32 different complete tests (each of the four different types of disk access for each of eight different transfer sizes). The access types are sequential read, sequential write, random read and random write. The transfer sizes are 1, 2, 4, 8, 16, 32, 64 and 128 sectors, where a sector is 512 bytes of data. For both sequential and random access, the read and write tests are interspersed. In either case, the same data which is read during a read test is written back to the location from which it was read during the corresponding write test. In this way data on the disk is not disturbed by the benchmark. Low-level disk requests (called BIOS or INT13h requests) are used to allow random and sequential measurements across the disk without regard to where data is stored on the disk. DOS file system overhead and issues such as file fragmentation are avoided with this approach. Multiple-zone recording, a feature common to modern disk drives, allows more sectors of data to be stored on the outer tracks of a disk (where there is more room) than on the inner tracks. This feature causes the off-the-disk transfer rates to vary depending upon where on the disk the data is located. QBENCH takes multiple-zone recording into account by dividing all sequential tests into four parts. Two of the tests are performed on the outermost and innermost tracks, while the other two are equally spaced in between. This allows the benchmark to get a representative sampling of track densities for drives which employ multiple-zone recording. Random tests are also done in four parts, for convenience. This also allows QBENCH to update the screen at least four times during a test, to indicate test progress, without affecting the timing of the disk requests being measured. All screen updates take place between test segments when no timing is taking place. While the test is in progress, a map of the drive layout is provided so that the user can see exactly where on the disk requests are being executed. Map information, including the size of each block in the map, is provided in the lower right box entitled Map Information. By observing the map, the user can first watch the four evenly paced sequential tests for a given transfer size, with the read portion of each directly followed by the write. The random tests for that transfer size follow in a similar manner. The same amount of data is transferred in each test, although the sequential tests appear to transfer less, since several sequential transfers occur in each display block in the map while random transfers are typically one per block. The lower left box entitled "Test Information" shows exactly which test is being executed, and what percentage of that test has been completed. The test type and block size are displayed, along with the percentage complete. The overall progress of the test is displayed at the bottom of the screen. A total of nearly 50 MBytes of data is transferred when the standard test is run. As the test progresses, the bar at the bottom of the screen shows the percentage of this total which has been completed. In terms of data transferred, the test runs more quickly at higher block sizes, so the bar provides a conservative estimate of the percentage of the total test time which is complete. To pause or abort the test while it is running, press the <Esc> key. The user can resume benchmark execution without affecting the accuracy of the results. QBENCH Theory This section contains information concerning the derivation of data access time. Definitions The terms command service time, access time and transfer time are used throughout the following text. The definitions below are provided to aid the readers understanding of the information presented in the following paragraphs. 1. Command Service Time is the total time that it takes to complete a read or write command. 2. Access Time is the time from the issuance of a read or write command until the desired data is located but not transferred. 3. Transfer Time is the time it takes to transfer data once it has been accessed. For all access types (sequential read, sequential write, random read and random write) the relationship between the transfer size (amount of data transferred per request) and the command service time (the time to complete a read or write command) is highly linear for all drives. Command service time is equal to access time plus transfer time. Access time is the fixed or constant portion of command service time, while transfer time is proportional to the amount of data transferred. Access time cannot be measured directly, since all read/write disk operations involve data transfer as well as access. The value closest to access time which can be measured is the command service time for a single sector (this service time includes the transfer time for a single sector in addition to the access time). Access time is calculated by subtracting the transfer time for one sector from the command service time for one sector. The transfer time for a single sector (transfer time per sector) is equal to the slope of the line when command service time is plotted against transfer size. To calculate access time: (1) measure the command service time values for several transfer sizes; (2) calculate the slope of the best fit straight line for command service time vs. transfer size - this slope is the transfer time per sector; (3) calculate the access time by subtracting the transfer time from the command service time for one sector. Definition: Data Access Time is the weighted average of the access times for each access class, where each access time value is weighted according to the frequency of occurrence of that class in a typical DOS user workload mix. Once each of the access times have been calculated, the data access time can then be calculated based on the workload mix percentages. The data access time calculation, as well as each entry in the Weighted Average column in the output, is calculated from the individual values for the access types as follows. The default workload values are 65% sequential (with 35% random), and 60% reads (with 40% writes). To find the exact weighting factor for a given access type, multiply the appropriate workload percentages. For example, for sequential read, use 0.65 for sequential, times 0.60 for read, which equals 0.39 or 39%. Applying this formula, other values are 26% for sequential write, 21% for random read, and 14% for random write. The slope of the line when command service time is plotted against transfer size is the average time to transfer a sector in milliseconds. The Data Transfer Rate displayed is calculated from the slope of the weighted average line. The calculation goes as follows. If the slope is .50 ms per 512 byte sector (for example), the complete transfer rate formula would be (512/1024)/(.50/1000) = 1000 KB/sec. The 1024 converts bytes to KBytes, and the 1000 converts milliseconds to seconds. Appendix C QBENCH Limitations and Erroneous Results Causes Data Access Time Limitations Data access time is designed as an improved measure of disk drive performance in DOS systems. A workload-weighted measure of drive access time, it provides a comprehensive index which describes the performance that a typical user will experience. However, there are limits to the applicability of data access time beyond the DOS systems for which it was designed. Other operating systems, even on PC-compatible computers, would certainly have different workload assumptions on which their data access time calculations would be based. Each workload mix would have to be determined individually for each operating system with a representative workload study. Measurement tools would have to be developed to take these measurements. Benchmark measurements for other hardware platforms could be significantly different than the results for PC-compatible computers. Bus implementation and I/O architecture would certainly have an effect on the raw measurements, and the data access time values calculated could be quite different as well. Data access time is designed to measure the access speed of the drive mechanism, therefore, it is important to disable system caching to accurately do this. File servers typically make heavy use of caching to achieve high levels of I/O performance. Thus data access time is not appropriate as a measure of file server performance, although with the right workload assumptions it could indicate which drives might function most effectively on a particular server. Although new disk drive technology which has an effect on access time is captured in the data access time measure, other more advanced features are not. For example, the effect of retaining Least-Recently- Used data in multi-segmented caches such as those implemented on all Quantum drives is not shown by this benchmark. The effect of prefetching data into a particular cache segment is captured, however. As another example, QBENCH will show no performance advantage for drives with the SCSI command queuing feature. This feature allows queuing and access time optimization for multi-user systems by allowing the drive to reordering requests to minimize seek time and latency. Drive array subsystems provide another example of systems for which the data access time measure is not likely to be appropriate. Drive arrays achieve improved performance through using several drives in parallel, while QBENCH measures data access time only for individual drives. In summary, data access time provides a single number which describes the performance of a drive on DOS and Windows single-user systems. This number should not be used as the indicator of drive performance for systems and situations beyond those for which it was designed. Erroneous Results For certain drives, particularly older technology drives with an interleave factor greater than one-to-one, the mathematical model which is the basis of the access time calculation is not correct. This does not mean that there is any problem with the drive itself or its operation. It only means that the data access time calculation procedure used by QBENCH does not apply. Asterisks are printed for any access times with this problem. Asterisks are also printed for data access time, since it is calculated from the other access time values. Only the access time calculation is affected. All other values which are printed for this drive are correct. Disk Errors A message may be displayed if a disk error has occurred during QBENCH testing. QBENCH uses low-level read and write commands and may attempt to read or write sectors which were marked as bad when the drive was formatted by DOS. Thus errors reported by the benchmark do NOT necessarily indicate a problem with the drive, since the sector or sectors in question may have been removed from use during normal operation. QBENCH has a /C command line option which instructs the benchmark to continue the test in the event of a disk error. Refer to Appendix A, "Command Line Options," for more information on the /C command line option. QBENCH operates by reading sectors of data, and then writing the same data back to the same locations. Thus if the /C option is used and a read error occurs, the effect of writing back the possibly erroneous data is unpredictable. To be safe, always backup the data on a hard disk before running QBENCH with the /C option. It the disk error occurs during the part of the test which transfers data 128 sectors at a time (i.e., the block size is 128 sectors when the test fails), it may be possible to run QBENCH successfully by invoking the /127 command line option when the benchmark is first invoked.