Checking Disk I/O

Checking Disk I/O

The system uses disks to store data and transfers data between the disk and memory. This input/output (I/O) process consumes a lot of system resources, so you want the operating system to be as efficient as possible when it performs I/O.

If you are going to run a large application or have a heavy system load, the system will benefit from disk I/O tuning. Run sar -A or timex -s and look at the %busy, %rcache, %wcache, and %wio fields.To see if your disk subsystem needs tuning, check your output of sar -A against the figures in the following table. (Note that in the tables that follow, the right column lists the sar option that will print only selected output, for example output for disk usage (sar -d) or CPU activity (sar -u).

Table 11-3 lists sar results that indicate an I/O-bound system.

Indications of an I/O-Bound System
Field Value sar Option
%busy (% time disk is busy) >85% sar -d
%rcache (reads in buffer cache) low, <85 sar -b
%wcache (writes in buffer cache) low, <60% sar -b
%wio (idle CPU waiting for disk I/O) dev. system >30
fileserver >80 sar -u

Indications of an I/O-Bound System
Field	Value	sar Option
%busy (% time disk is busy)	>85%	sar -d
%rcache (reads in buffer cache)	low, <85	sar -b
%wcache (writes in buffer cache)	low, <60%	sar -b
%wio (idle CPU waiting for disk I/O)	dev. system >30 fileserver >80	sar -u

Notice that for the %wio figures (indicates the percentage of time the CPU is idle while waiting for disk I/O), there are examples of two types of systems:

a development system that has users who are running programs such as make. In this case, if %wio > 30, check the breakdown of %wio (sar -u). By looking at the %wfs (waiting for file system) and %wswp (waiting for swap), you can pinpoint exactly what the system is waiting for.
an NFS system that is serving NFS clients and is running as a file server. In this case, if %wio > 80, %wfs > 90, the system is disk I/O bound.

There are many other factors to consider when you tune for maximum I/O performance. You may also be able to increase performance by:

using logical volumes
using partitions on different disks
adding hardware (a disk, controller, memory)

Using Logical Volumes to Improve Disk I/O

By using logical volumes, you can:

grow an existing file system to a larger size without having to disturb the existing file system contents.
stripe file systems across multiple disks--You may be able to obtain up to 50% improvement in your I/O throughput by creating striped volumes on different disks.
Striping works best on disks that are on different controllers. Logical volumes give you more space without remaking the first filesystem. Disk striping gives you more space with increased performance potential, but you do run the risk that if you lose one of the disks with striped data, you will lose all the data on the filesystem since the data is interspersed across all the disks.

Contiguous logical volumes fill up one disk, and then write to the next. Striped logical volumes write to both disks equally, spreading each file across all disks in the volume, so it is impossible to recover from a bad disk if the data is striped, but it is possible if the data is in a contiguous logical volume. For information on creating a striped disk volume, see the IRIX Admin: Disks and Filesystems guide.

Using Partitions and Additional Disks to Improve Disk I/O

There are some obvious things you can do to increase your system's throughput, such as limiting the number of programs that can run at peak times, shifting processes to non-peak hours (run batch jobs at night), and shifting processes to another machine. You can also set up partitions on separate disks to redistribute the disk load.

Before continuing with the discussion about partitions, let's look at how a program uses a disk as it executes. Table 11-4 shows various reasons why an application may need to access the disk.

An Application's Disk Access
Application Disk Access
execute object code text and data
uses swap space for data, stack /dev/swap
writes temporary files /tmp and /var/tmp
reads/writes data files data files

An Application's Disk Access
Application	Disk Access
execute object code	text and data
uses swap space for data, stack	/dev/swap
writes temporary files	/tmp and /var/tmp
reads/writes data files	data files

You can maximize I/O performance by using separate partitions on different disks for some of the aforementioned disk access areas. In effect, you are spreading out the application's disk access routines, which will speed up I/O.

By default, disks are partitioned to allow access in two ways, either:

three partitions: partitions 0, 1 and 6, or
one large partition, partition 7 (encompasses the three smaller partitions)

On the system disk, partition 0 is for root, 1 is for swap, and 6 is for /usr.

For each additional disk, you need to decide if you want a number of partitions or one large one and what file systems (or swap) you want on each disk and partition. It's best to distribute file systems in the disk partitions so that different disks are being accessed concurrently.

The configuration depends on how you use the system, so it helps to look at a few examples.

Consider a system that typically runs a single graphics application that often reads from a data file. The application is so large that its pages are often swapped out to the swap partition.
In this case, it might make sense to have the application's data file on a disk separate from the swap area.
If after configuring the system this way, you find that it doesn't have enough swap space, consider either obtaining more memory, or backing up everything on the second hard disk and creating partitions to contain both a swap area and a data area.
Changing the size of a partition containing an existing file system may make any data in that file system inaccessible. Always do a complete and current backup (with verification) and document partition information before making a change. If you change the wrong partition, you can change it back, providing you do not run mkfs on it or overwrite it. It is recommended that you print a copy of the prtvtoc command output after you have customized your disks, so that they may be more easily restored in the event of severe disk damage.

Also, if you have a very large application and have three disks, consider using partitions on the second and third disks for the application's executables (/bin and /usr/bin) and for data files, respectively. Next, consider a system that mostly runs as a "compile-engine."

In this case, it might be best to place the /tmp directory on a disk separate from the source code being compiled. Make sure that you check and mount the file system prior to creating any files on it. (If this is not feasible, you can instruct the compiler to use a directory on a different disk for temporary files. Just set the TMPDIR environment variable to the new directory for temporary files.) Now, look at a system that mainly runs many programs at the same time and does a lot of swapping.

In this case, it might be best to distribute the swap area in several partitions on different disks.

Adding Disk Hardware to Improve Disk I/O

If improved I/O performance still does not occur after you have tuned as described previously, you may want to consider adding more hardware: disks, controllers, or memory.

If you are going to add more hardware to your system, how do you know which disk/controller to add? You can compare hardware specifications for currently supported disks and controllers by turning to your hardware Owner's Guide and looking up the system specifications. By using this information, you can choose the right disk/controller to suit your particular needs.

By balancing the most active file systems across controllers/disks, you can speed up disk access.

Another way to reduce the number of reads and writes that go out to the disk is to add more memory. This will reduce swapping and paging.