Mapping a File for I/O

Mapping a File for I/O

You can use mmap() as a simple, low-overhead way of reading and writing a disk file. Open the file using open(), but instead of passing the file descriptor to read() or write(), use it to map the file. Access the file contents as a memory array. The memory accesses are translated into direct calls to the device driver, as follows:

An attempt to access a mapped page, when the page is not resident in memory, is translated into a call on the read entry point of the device driver to read that page of data.
When the kernel needs to reclaim a page of physical memory occupied by a page of a mapped file, and the page has been modified, the kernel calls the write entry point of the device driver to write the page. It also writes any modified pages when the file mapping is changed by munmap() or another mmap() call, when the program applies msync() to the segment, or when the program ends.

When mapping a file for input only (when the prot argument of mmap() does not contain PROT_WRITE), you can use either MAP_SHARED or MAP_PRIVATE. When writing is allowed, you must use MAP_SHARED, or changes will not be reflected in the file.

Memory mapping provides an excellent way to read a file containing precalculated, constant data used by an interactive program. Time-consuming calculation of the data elements can be done offline by another program; the other program also maps the file in order to fill it with data.

You can lock a mapped file into memory. This is discussed further under "Locking Pages in Memory".

Mapped File Sizes

Since the potential 32-bit address space is more than 2000 megabytes (and the 64-bit address space vastly greater), you can in theory map very large files into memory. To map an entire file:

Open the file to get a file descriptor.
Use lseek(fd,0,SEEK_END) to discover the size of the file (see the lseek(2) reference page).
Map the file with an off of 0 and len of the file size.

Apparent Process Size

When you map a large file into memory, the space is counted as part of the virtual size of the process. This can lead to very large apparent sizes. For example, under IRIX 5.3 and 6.2, the ObjectServer maps a large database into memory, with the result that a typical result of ps -l looks like this:

70 S 0 566 1 0 26 20 * 33481:225 80272230 ? 0:45 objectser

The total virtual size of 33481 certainly gets your attention! However, note the more modest real storage size of 225. Most of the mapped pages are not in physical memory. Also realize that the backing store for pages of a mapped file is the file itself--no swap space is used.

Mapping Portions of a File

You do not have to map the entire file; you can map any portion of it, from one page to the file size. Simply specify the desired length as len and the starting offset as off.

You can remap a file to a different segment by calling mmap() again. In this way you can use the off parameter of mmap() as the logical equivalent of lseek(). That is, to map a different segment of the file, specify

the same file descriptor
the new offset in off
the current segment base address as addr
MAP_FIXED in flags to force the use of addr as the base address (otherwise map the new portion of the file as a different, additional memory segment)

The old segment is replaced with a new segment at the same address, now containing data from a different offset in the file.

Each time you replace a segment with mmap(), the previous segment is discarded. The new segment is not locked in memory, even if the old segment was locked.

File Permissions

Access to a file for mapping is controlled by the same file permissions that control I/O to the file. The protection in prot must agree with the file permissions. For example, if the file is read-only to the process, mmap() does not allow prot to specify write or execute access.

Note: When a program runs with superuser privilege for other reasons, file permissions are not a protection against accidental updates.

NFS Considerations

The file that is mapped can be local to the machine, or can be mounted by NFS®. In either case, be aware that changes to the file are buffered and are not immediately reflected on disk. Use msync() to force modified pages of a segment to be written to disk (see "Synchronizing the Backing Store").

If IRIX needs to read a page of a mapped, NFS-mounted file, and an NFS error occurs (for example, because the file server has gone down), the error is reflected to your program as a SIGBUS exception.

Caution: When two or more processes in the same system map an NFS-mounted file, their image of the file will be consistent. But when two or more processes in different systems map the same NFS-mounted file, there is no way to coordinate their updates, and the file can be corrupted.

File Integrity

Any change to a file is immediately visible in the mapped segment. This is always true when flags contains MAP_SHARED, and initially true when flags contains MAP_PRIVATE. A change to the file can be made by another process that has mapped the same file.

A mapped file can also be changed by a process that opens the file for output and then applies either write() to update the file or ftruncate() to shorten it (see the write(2) and ftruncate(3) reference pages). In particular, if any process truncates a mapped file, an attempt to access a mapped memory page that corresponds to a now-deleted portion of the file causes a bus error signal (SIGBUS) to be sent.

When MAP_PRIVATE is specified, a private copy of a page of memory is created whenever the process stores into the page (copy-on-write). This prevents the change from being seen by any other process that uses or maps the same file, and it protects the process from detecting any change made to that page by another process. However, this applies only to pages that have been written into.

Frequently you cannot use MAP_PRIVATE because it is important to see data changes and to share them with other processes that map the same file. However, it is also important to prevent an unrelated process from truncating the file and so causing SIGBUS exceptions.

The one sure way to block changes to the file is to install a mandatory file lock. You place a file lock with the lockf() function (see Chapter 4, "File and Record Locking"). However, a file lock is normally "advisory"; that is, it is effective only when every process that uses the file also calls lockf() before changing it.

You create a mandatory file lock by changing the protection mode of the file, using the chmod() function to set the mandatory file lock protection bit (see the chmod(2) reference page). When this is done, a lock placed with lockf() is recognized and enforced by open().