LAM 6.2 Beta Release Notes

Installation

Please see the installation guide for instructions on installing LAM 6.2 Beta.

New feature overview

RPI transport layers

LAM 6.2 provides three client-to-client transport layers which implement the request progression interface (RPI). As in LAM 6.1 the LAM daemon RPI transport is always available. It is no longer the default transport and must be explicitly invoked via the -lamd option to mpirun .

The three client-to-client transports are

tcp

The tcp transport uses TCP sockets for all interprocess communication.

usysv

The usysv transport is multi-protocol. Processes on the same node communicate via SYSV shared memory and processes on different nodes communicate via TCP sockets. It uses spin-locks for shared memory synchronization as well as a SYSV semaphore or pthread mutex for synchronizing access to a per node global shared memory pool.

The spin-locks require that the architecture has strongly ordered writes and this transport is only supported on such platforms. It should be relatively easy to modify this transport to work on systems with weakly ordered writes by adding memory barriers in appropriate places.

sysv

The sysv transport is the same as the usysv transport except that SYSV semaphores are used for message synchronization rather than spin-locks. On some uniprocessor systems (e.g. Linux) the blocking nature of semaphores can lead to better performance than when using spin-locks.

The usysv transport should give the best performance on SMPs. Please refer to the tuning notes for more on the performance of the various transports and on tuning them.

Mpirun

The default behavior of mpirun has changed. The default options now correspond to -w -c2c -nger. That is, wait for the application to terminate, use the fast client-to-client communication mode and disable GER. To get the old behavior use the options -lamd -ger -nw.

Mpirun now recognizes command lines of the form

 mpirun -np <nprocs> {LAM specific mpirun args} <program> {program args}
For example,
 mpirun -np 4 -lamd n0 n1 /bin/foobar 12 a b c
runs 4 copies of program /bin/foobar on nodes n0 and n1, passing the arguments, 12 a b c, to the program. The new syntax is equivalent to the following in the "-c" syntax which is still supported.
 mpirun -c <nprocs> {LAM specific mpirun args} <program> -- {program args}

Signal catching

LAM MPI now catches the signals SEGV, BUS, FPE and ILL. The signal handler terminates the application. This is useful in batch jobs to help ensure that mpirun returns if an application process dies. To disable the catching of signals use the -nsigs option to mpirun.

Internal signal

The signal used internally by LAM has been changed from SIGUSR1 to SIGUSR2 to reduce the chance of conflicts with the Linux pthreads library. The signal used is configurable.

New basic datatypes

Support has been added for the MPI_LONG_LONG_INT, MPI_UNSIGNED_LONG_LONG and MPI_WCHAR basic datatypes.

MPI-2 Support

LAM 6.2 adds support for many MPI-2 functions. It does not provide any MPI IO support but this can be obtained by installing the ROMIO package from Argonne National Labs. C++ bindings for MPI-1 functions can be obtained by installing the MPI-2 C++ package from Notre Dame University.

Inter-language interoperability is supported. It is now possible to initialize LAM MPI from either C or Fortran and mix MPI calls from both languages.

One-sided communication

Support is provided for get/put/accumulate data transfer operations and for the post/wait/start/complete and fence synchronization operations. No support is provided for window locking.

The datatypes used in the get/put/accumulate operations are restricted to being basic datatypes or single level contigs/vectors of basic datatypes.

The implementation of the one-sided operations is layered on top of the point-to-point functions and will thus perform no better than them. Nevertheless it is hoped that providing this support will aid developers in developing and debugging codes using one-sided communication.

The following functions related to one-sided communication have been implemented.

MPI_Win_create
MPI_Win_free 
MPI_Win_get_group 

MPI_Get
MPI_Put
MPI_Accumulate

MPI_Win_fence 
MPI_Win_post 
MPI_Win_wait 
MPI_Win_start 
MPI_Win_complete 

Dynamic processes

The dynamic process support provided in LAM 6.2 has been extended and the function names changed to conform to the final MPI 2.0 standard. The following functions related to dynamic process support are provided.
MPI_Comm_spawn
MPI_Comm_spawn_multiple
MPI_Comm_get_parent 
MPI_Comm_accept
MPI_Comm_connect
MPI_Comm_disconnect
MPI_Comm_join

MPI_Lookup_name 
MPI_Publish_name 
MPI_Unpublish_name 

MPI_Open_port 
MPI_Close_port 

Info

Full support for info objects is provided.
MPI_Info_create 
MPI_Info_free 
MPI_Info_delete 
MPI_Info_dup 
MPI_Info_get 
MPI_Info_get_nkeys 
MPI_Info_get_nthkey 
MPI_Info_get_valuelen 
MPI_Info_set 

Communicator and window error handling

The new communicator error handler functions are supported and window error handlers are also supported.
MPI_Comm_create_errhandler
MPI_Comm_get_errhandler
MPI_Comm_set_errhandler

MPI_Win_create_errhandler
MPI_Win_get_errhandler 
MPI_Win_set_errhandler 

Handle conversions

Handle conversions for inter-language interoperability are supported.
MPI_Comm_f2c 
MPI_Comm_c2f 

MPI_Group_f2c 
MPI_Group_c2f 

MPI_Type_f2c 
MPI_Type_c2f 

MPI_Request_f2c 
MPI_Request_c2f 

MPI_Info_f2c 
MPI_Info_c2f 

MPI_Win_f2c 
MPI_Win_c2f 

MPI_Status_f2c
MPI_Status_c2f

Attributes on communicators, datatypes and windows

Attributes may now be set on and retrieved from datatypes and windows. The new communicator attribute handling functions are also supported.
MPI_Comm_create_keyval
MPI_Comm_free_keyval 
MPI_Comm_delete_attr 
MPI_Comm_get_attr 
MPI_Comm_set_attr 

MPI_Type_create_keyval
MPI_Type_free_keyval 
MPI_Type_delete_attr 
MPI_Type_get_attr 
MPI_Type_set_attr 

MPI_Win_create_keyval
MPI_Win_free_keyval 
MPI_Win_delete_attr 
MPI_Win_get_attr 
MPI_Win_set_attr 

New derived type constructors and type enquiry functions

Support has been added for the following new derived type constructors
MPI_Type_create_struct
MPI_Type_create_hindexed 
MPI_Type_create_hvector
MPI_Type_dup 
MPI_Type_create_resized
MPI_Type_create_subarray
MPI_Type_create_darray
and for the type enquiry functions
MPI_Type_get_contents
MPI_Type_get_envelope
MPI_Type_get_extent 
MPI_Type_get_true_extent

Miscellaneous

Implementations of the following functions are provided. LAM 6.2 reports its MPI version as 1.2.
MPI_Get_version
MPI_Get_address