At this time there are exactly two low-level I/O interfaces that clients commonly use to access parallel file systems. The first of these is the UNIX API as presented by the client operating system. The second is the MPI-IO interface.
In PVFS1 we provide access through the operating system by providing a loadable module that exports VFS operations out into user space, where a client-side UNIX process, the pvfsd, handles interactions with servers. A more efficient in-kernel version called kpvfsd was later provided as well.
PVFS2 uses a similar approach to the original PVFS1 approach for access through the OS. A loadable kernel module exports functions out to user-space where a UNIX process, the pvfs2-client, handles interactions with servers. We have returned to this model (from the in-kernel kpvfsd model) because it is not clear that we will have ready access to all networking APIs from within the kernel.
The second API is the MPI-IO interface. We leverage the ROMIO MPI-IO implementation for PVFS2 MPI-IO support, just as we did for PVFS1. ROMIO links directly to a low-level PVFS2 API for access, so it avoids moving data through the OS and does not communicate with pvfs2-client.