DMA Operations

DMA Operations

As indicated in "Programmed I/O (PIO)," use DMA (direct memory access) when the device supports it. In its simplest form, DMA is easy to use: your driver gives the device the physical memory address, and the transaction begins. Your driver can then put itself to sleep while it waits for the transfer to complete, thus freeing the processor for other tasks. When the transfer is complete, the device interrupts the processor. On most systems, when large amounts of data are involved, DMA devices obtain higher overall throughput than devices that do only PIO.

DMA operations are categorized as DMA reads or DMA writes. DMA operations that transfer from memory to a device, and hence read memory, are DMA reads. DMA operations that transfer from a device to memory are DMA writes. Thus, you may want to think of DMA operations as being named the from the point of view of what happens to memory.

There are important cache considerations for drivers using DMA. The cache architecture of the system dictates the appropriate cache operations. Write back caches require that data be written back from cache to memory before a DMA write, whereas both write back and write through caches require the cache to be invalidated before data from a DMA read is used. See "Data Cache Write Back and Invalidation" in Appendix A and the dki_dcache_wbinval(D3X) man page for a discussion of these issues.

Another concern for driver writers is that DMA buffers may require cache-line alignment. If a driver allocates a buffer for DMA, it must use the kmem_alloc() function, using the KM_CACHEALIGN flag.

The interrupt service routine then calls your drvintr() routine. Your drvintr() routine can check that the transfer is complete (if necessary), set flags indicating the status of the transfer, and then awaken the sleeping process.

Unfortunately, the details of how you implement the simple scheme described above is complicated by the use of virtual memory, different VME addressing modes, and a variety of device and system implementations. To sort through these potentially confusing choices, ask the following questions in order. If the answer to any question is "yes," go on to the section indicated. Otherwise, proceed to the next question.

A32 Addressing Scatter/Gather Support

Modern computer architectures support virtual memory--memory in which the user's view of memory is logically contiguous, but the underlying physical pages are not. Because VME devices understand only physical page addresses, your driver would ordinarily be forced to do transfers one page at a time. At the start of each one-page transfer, your driver would have to awaken the sleeping process and compute the physical address for the next virtual page.

Because this is not efficient, many devices now provide a method to store the address mapping for the entire transfer up front. Your driver can usually do this merely by programming the device with a table of physical addresses for all of the upcoming transfer. This method of regrouping of noncontiguous physical memory is called scatter/gather.

If your VME device supports scatter/gather, uses A32 addressing, has less than 4 GB of physical memory, and you are not on a CHALLENGE/Onyx series system, proceed to "VME Devices with Scatter/Gather Capability."

DMA Mapping for High-end Systems and Older Systems

Older Silicon Graphics systems and current high-end systems provide for address mapping of physical addresses so that even devices that do not support scatter/gather in the controller can transfer to and from noncontiguous physical pages with ease. This facility, called DMA mapping, is available on 4D/100 through 4D/400, Crimson, CHALLENGE/Onyx and POWER CHALLENGE/POWER Onyx series systems. Indigo, Indigo2, and Indy workstations have no VME-bus support. DMA mapping works equally well for both VME A24 and A32 master addressing. See "Using DMA Maps" for a description of how to use DMA mapping.

Does the VME Device Perform A24 Master Addressing?

If the VME device uses A24 addressing, and your system does not support DMA mapping, the controller can access only the first 8 MB of physical memory. Because user programs may use physical pages beyond
8 MB, your device driver must do DMA into a kernel buffer and copy from that buffer to the user's pages. See "DMA on A24 Devices with No DMA Mapping."

A32 Addressing with No Scatter/Gather

If you are writing a driver for an A32 VME device that does not support scatter/gather on a workstation that does not support DMA mapping, see "DMA on A32 Devices with No Scatter/Gather Capability" for advice on how to implement this driver type.