Club Amiga de Montreal - CAM/CAM_CD_1.iso/files/164.lha/IPC/ipc.doc

Text File | 1988-04-28 | 25KB | 487 lines

          An Inter-Process Communication Standard for the Amiga
          =====================================================

This is initial documentation for the IPC standard developed via usenet
over the past several months.  A large part of the standard is now fairly
stable (the IPCPort mechanism in particular), but there is a lot of work
still to be done on the details, on developing message IDs for specific
functions, and on publishing them.  Anyway, in its present state, here it
is.


Why IPC?
========

Up to now, even though the Amiga is the first really totally multitasking
personal computer, programs for it have been written much as for any of the
older one-job-at-a-time machines (though they often take good advantage of
the windowing environment): they run as single processes.  A few programs
spawn child tasks to perform some of their functions, but they are still
monolithic in their design.

There is an alternative.  A ... let's call it a "Job environment"... could
be serviced by a number of more specialized processes, communicating with
each other.  Depending on the needs of that environment, these processes
could be very specialized -- a note bender in a MIDI system, say -- or more
general, like a text editor that called up things like a spell checker or
output formatter when needed.  Such a system would give the user much more
choice in configuring it to his/her preferences. (Not forgetting though
that a default configuration would have to be as easy to use as any well
designed program!)  The modules in an environment wouldn't all have to come
from one company, as the interface to invoke a module's function would be
published with the module.  You could add special purpose functions to a
paint program say, just by starting up another process that would
recognize, or be recognized by, the paint program.

There are a lot of possible applications of this principle, with widely
varying requirements.  Here is a very incomplete list of a few that come to
mind:

    *   Communications -- connect terminal emulators, protocol handlers
                    [Kermit, XModem etc.], auto-answer modules, script
                    processors etc. to suit the user's environment.
                    [Note the wide range of parameters such as processing
                    speed demanded by different modules: protocol handlers
                    will have to operate character by character at baud
                    rates, while script servers and auto-answer can be much
                    more leisurely.]

    *   MIDI Modules -- a lot of possibilities here for a fast IPC
                    protocol; anything you might want to do to a MIDI
                    channel could be a "plug-in" module.

    *   Data Acquisition and Control -- there are untapped opportunities
                    for a multitasking machine in the laboratory; being
                    able to "plug" modules together like you can with lab
                    equipment would be a great attraction.

    *   Desktop Publishing -- put together a comprehensive and competent
                    system from the pieces YOU prefer.

    *   Desktop Video and Presentation -- lots of enticing possibilities
                    here; the real-time nature of presentation gives lots
                    of opportunities for fast IPC.  (One might even think
                    of coupling several machines together...!)

    *   Service Programs -- to do common jobs for other modules; small ones
                    -- print format servers, pattern matchers, directory
                    searchers etc. -- can take a lot of the load off other
                    modules that would otherwise have to include code for
                    those things; larger ones could handle jobs like spell-
                    checking for any program that wanted it.

    *   A Multitasking H*perC*rd -- a somewhat vaguely specified, but
                    highly enticing, goal; dream your own scenario...


This should give some flavor of the ways in which I think a general
"Cooperating Processes Paradigm" would be a winner.  You can see that a
wide range of schemes could fall under the term "IPC".  It would be nice if
they could all be handled by one underlying protocol.

There are a number of possible candidates.  Processes can pass information
between each other via files and pipes, for example.  This has the great
advantage that it is already built into the system, but its disadvantage is
the heavy overhead involved in file-type I/O.  Also, simple serial stream,
unstructured, communication isn't good enough for a system that is going to
pass around many different types of information.  The receiver of
information has to know exactly what it is getting, and the sender must
know exactly how to tell it.

This means some kind of standard structure for the data that can be
understood by all processes. IFF is such a standard, but again
unfortunately it is too complex and serial-oriented for very fast
communication.  Its principles make a good starting point though.

There is an obvious candidate built into the Amiga Exec: Messages and
Ports.  These are actually just about what we want, but there are a couple
of problems with them as they stand.

First, although ports work perfectly when child tasks spawned and managed
by a single master process use them to talk to each other, there is no
protection against accidents that are likely to occur when independent
processes communicate.  The problem is that, as ports are accessed by a
pointer, there is no way of being sure that the port pointed to still
exists when you want it, unless you lock out all other processes before you
get the pointer, and don't unlock them again until you've used it.  This
can get VERY cumbersome and inefficient if you want to send a lot of
messages to a port, as you have to do the Forbid/Find pointer/Use pointer/
Permit sequence for every message.

Second, there is the problem of a process knowing exactly what is in the
message it has received.  As we said, there must be a Standard Structure
for the data in the message so that it becomes easy for different authors
to write programs that can understand each other.

Hence the IPC Standard really breaks down into two parts: IPCPorts and
IPCMessages.  These are essentially independent of each other, in that the
Ports have no knowledge or concern about the message structure (aside from
the necessity for the standard Exec Message structure at its heart), and
conversely the Messages are not concerned with the Port procedures.

It should be pointed out that this is a "Foundation Level" protocol.  It
defines how ports should be managed, and the common structure of messages,
but says nothing more about how they are to be used.  There are many higher
level considerations, such as how the processes should appear to the user,
and how he is to control them, that need to be worked out.  There are a lot
of possibilities here: control panels, patch panels, "drop-in" menus; some
applications will need added capabilities such as command languages and
scripts; others would be slowed unacceptably by such overhead.  But with
luck this standard will be general enough to cover the range.

There are other schemes, actual and potential, that have to be considered
in relation to this one.  In general there is probably no need for them to
be competitors.  One such is AREXX, which has already been successful in
applications like integrating editors with text formatters; because it is
basically an interpreted script language, it doesn't seem so suitable for
some of the other potential applications mentioned above.  There seems no
reason though why an interface between AREXX and this standard could not be
written; only a restricted set of IPCMessages could be handled, but
probably enough to give great flexibility.

Another possibility is an "Object Oriented" communication scheme. This has
both similarities and differences to this protocol, but is still in rather
formative stages at this point.  We will have to wait to see if the two
schemes can coexist: I believe they can.


IPCPorts
========

An IPCPort is a rendezvous point between processes where messages can be
dropped off and picked up.  It is identical in concept to the standard
Amiga Exec MsgPort (see the Rom Kernel Manual), but is managed so that
messages cannot be sent to a non-existent port (if the processes obey the
rules!).  Any number of "Client" processes can be sending messages to a
port at one time, but there can be only one "Server" on a port.  If a port
doesn't have a Server -- either running or in the process of being loaded
--, it is "Closed", and won't accept messages.

Unlike an Exec MsgPort, an IPCPort doesn't "belong" to any particular
process, and a program must never create one directly or assign memory in
its own space for one: a port will be created by the first process that
needs it, and will remain available until all references to it have been
cleared; it may be finally deleted by the last user long after the first
has gone away. Either a Server or Client may make the first reference.

Each port is identified by a unique name (although unnamed anonymous ports
are also possible, as we'll see later); the name may be any appropriate
null-terminated byte string.  A port is located by an exact match to the
string: the case of the characters is respected.  The author of a server
will publish the names of the ports it services, so that others can write
clients to access it; if two programs offer alternatives for the same
service, they should use the same port name, so that the user may plug one
of them in without changing the client.

IPCPorts are managed by a procedure module linked in with each program that
uses IPC.  No separate manager process is needed, although for more
flexibility it is possible to run a "Port Broker" that manages the loading
of servers when their ports are requested (this will probably become the
usual mode).  At the moment, each program must have its own copy of the
code, but it will eventually be a resident library.


Clients and IPCPorts
====================

When a Client process first needs a port of a particular name, it must get
a pointer to that port.  The pointer will then remain valid until the
Client specifically drops its access request.  Each request made MUST be
paired eventually with a drop for correct port management.

There are three possible ways of getting a port pointer.  Each of these
procedures takes a names string as argument and returns the pointer if
possible.  They differ in the actions they take to ensure the status of the
port.

    GetIPCPort(name)    always returns a valid pointer (unless memory is
                        full, or some other disaster strikes).  The port is
                        created if it doesn't exist.  No indication is
                        given as to whether there is a current server.

    FindIPCPort(name)   only returns a pointer if the port already exists
                        and has a server; otherwise it returns NULL.

    LoadIPCPort(name)   (an addition to the basic procedures that is in its
                        own module) does a GetIPCPort, but if the port has
                        no server it calls on a "Port Broker" process
                        (also an addition to the basic system) to load one;
                        if the broker can't do this (or the broker doesn't
                        exist) the port will be dropped again, and NULL
                        will be returned.  If a valid pointer is returned,
                        you can assume that a server exists or is being
                        loaded.

                        LoadIPCPort does not wait for the server to be
                        loaded, but the "Loading" flag will be set in the
                        port so that it will accept messages for the server
                        to handle when it arrives.

Each successful call to one of these procedures increments the "Use Count"
of the port.  To reduce the use count again you must end the Client's
access to the port by:

    DropIPCPort(port)   where 'port' is the pointer returned by any of the
                        above calls.  The use count is reduced by one: if
                        it goes to zero, the port is deleted.

A Client must not exit without dropping all the ports it has acquired (and
of course must not drop them more times than it has acquired them!).


A Client sends messages to an IPCPort by a call exactly analogous to Exec's
PutMsg(), but the Exec call has no way of detecting the presence or absence
of a Server, so it cannot be used.  Instead use:

    PutIPCMsg(port,message) where 'port' is a valid pointer, and 'message'
                            is a pointer to an IPCMessage.  It returns TRUE
                            if the message was successfully queued on the
                            port; if the port has no server either present
                            or loading, it will not send the message and
                            will return FALSE.

Unless there are specific reasons otherwise, a message sent to a Server
will eventually be replied, and the Client must handle this also by
supplying a Reply Port.  The reply path does not need the IPCPort protocol,
because a Client MUST keep the reply port available until it has received
back ALL the replies it is expecting, so normally this can be a standard
Exec MsgPort.  There is no reason though why it should not be an IPCPort as
well if you prefer -- even one used for other communications -- as long as
you bear in mind that replies will be sent to it whether or not it has a
server assigned.


Servers and IPCPorts
====================

The Server uses a complementary set of procedures to a Client to manage a
port.  There is only one way it can acquire a port:

    ServeIPCPort(name)  makes this process the server for the named port
                        if possible and returns a valid pointer to the
                        port. The port is created if it doesn't exist.  If
                        a server already exists for the port, the call will
                        return NULL. (If successful, it also increments the
                        use count.)

Terminating service on the other hand is normally a two-step process.
First the server must shut the port against further incoming messages, then
handle and reply to any messages still queued on the port, and finally free
up the port for eventual reclamation or for another server to claim it.

    ShutIPCPort(port)   just marks the port as "Shut".  The server remains
                        attached to the port and can receive signals from
                        it (for example if the number of clients changes,
                        see below), but PutIPCMsg calls will be blocked.

    LeaveIPCPort(port)  removes this process from association with the
                        port.  Also does a DropIPCPort to decrement the use
                        count and delete if appropriate.

Between the ServeIPCPort and the ShutIPCPort, the server must be prepared
to accept messages on that port.  In most cases it will probably spend the
major portion of its time waiting on that port -- or possibly several ports
-- for a message to arrive.  All this area of the server's operation is
handled by standard Exec calls: WaitPort() or Wait(), GetMsg(), and
ReplyMsg() (see the Rom Kernel Manual).

If it only has one port to wait on, it can use:

    WaitPort(port)  which suspends the process until a message arrives at
                    that port.

This is liable to be inadequate though, because it will only be woken up by
messages arriving on that particular port.  It is NOT awakened by other
signals to the process, even if they use the same signal bit (see the RKM
for Task Signals).  Thus you are more likely to want to use:

    Wait(sigbits)   where 'sigbits' is a 32-bit mask of all the signals you
                    want to be awakened on.  It returns the signals that
                    were actually set, so the process can determine if it
                    needs to take unusual action.

This means, though, that you have to create the sigbits mask from the
signal bit number contained in the port itself (and other bits that you
want to respond to).  There is no specific call to get this (though there
should probably be at least a macro), but if the server is just using IPC.h
as a header (and not IPCPorts.h) IPCPorts are equated to MsgPorts, so you
can get the bit by:

    sigbit = 1L<<port->mp_SigBit;

Note that this bit is also used by the "Notify" feature (below) to awaken
the server if the number of clients changes.  No message is associated with
this signal, but all the server has to do is check the number of clients
(again, below) each time it is awakened.

To process a received message, the server simply uses GetMsg(), though it
is preferable to cast the returned value to the correct type:

    msg = (struct IPCMessage *)Getmsg(port);

When done with the message, it should use ReplyMsg() in the normal way
(using a cast if your compiler applies prototype checks):

    ReplyMsg((struct Message *)msg);


Checking IPCPort Status
=======================

Either Client or Server can get information about the current state of an
IPCPort with the CheckIPCPort() call.  The Server alone can also set
the state of certain flags in the port (only one is currently defined).

    CheckIPCPort(port,flags)    as long as the high bit of 'flags'
                                (a 16-bit value!) is not set, returns the
                                number of users (including the server)
                                currently aware of the port; if the high
                                bit is set (0x8000 -- this should be a
                                defined identifier... an omission in the
                                current IPC.h, sorry), the call instead
                                returns the current flags set in the port.
                                When called by the current Server ONLY, the
                                value in the low 8-bits of the 'flags'
                                argument will be set into the port's Flags
                                slot; system flags in the upper 8-bits of
                                the slot are not affected.

Although a Client may use this call, it will probably have no need to,
unless it wants to check for the presence of a Server without making a
PutIPCMsg call.  The Server, however, can use the call to keep tabs on how
many clients actually require its services, and optionally terminate if
there are none.  To do this it will want to be notified if the number of
clients changes; this can be done by setting the IPP_NOTIFY flag in the
port: while that is set, any call that acquires or drops a port will send a
signal to the Server process (using the signal bit defined for that message
port); no message is actually sent, but the Server should call CheckIPCPort
each time it is woken up.


Anonymous IPCPorts
==================

It is possible to create IPCPorts that are unnamed, but these obviously
cannot be acquired independently by other processes without a name to
access them by.  The associated pointers must be passed to other processes
via some standard message pathway (left up to the processes concerned).
They still should obey the IPC rules, though, so another procedure is
provided for acquiring them:

    UseIPCPort(port)    registers another user for that port (i.e.
                        increments the use count).

When the user process is done, it issues DropIPCPort in the usual way.

You create an anonymous IPCPort by:

    ServeIPCPort(NULL)  which will return a pointer to a NEW port for
                        each call, and set the caller as the Server.
                        (It will only fail if there is no memory or some
                        such fatal problem.)

For special cases -- e.g. ReplyPorts -- where you know PutIPCMsg will never
be used, you could instead use GetIPCPort(NULL), but there is NO way of
registering a server on an anonymous port except at creation time.

                            +++++++++++

IPCMessages
===========

The format of IPCMessages is intended to be very flexible, yet specific
enough that a Server can immediately determine the function and content of
a message it receives (or reject it if it does not recognize it).

The protocol has two levels where identification is specified.  There is a
fixed structure Message header (basically an Exec Message structure with
added information) which includes a Message ID, followed by an arbitrary
number of "Items", each again of fixed structure with its own Item ID.
Each item in turn usually points to a block of memory where the actual data
for the item is stored.  Instead of a pointer, a single 32-bit value can be
stored in the item itself.  In special cases, the data pointed to may be
more complex than a single data block (a list, say) but then the
restrictions on what can be done with the message increase markedly.
As long as all the items in a message are simple data blocks (or single
words), it can be passed between servers that do not have to understand the
contents of those blocks: deletion of the message, for example can be done
by any server.  Flags in the message and each item indicate whether the
data is private to the originating process or can be transferred to the
receiver, whether it is suitable to send out on a network, and so on.

An ID -- Message or Item -- is a 32-bit longword, normally a four-character
ASCII code.  All "Published" IDs should be of this form, but cooperating
processes could perhaps use non-ASCII numeric values internally.  If the
first (high) byte is zero, the ID is a private one.  Codes of less than
four characters may of course be used, but they preferably should be padded
on the right with blanks (rather than nulls).  Case is important naturally;
by convention (and analogy to IFF) upper case is preferred, but there is no
firm requiI5ent for this; lower case should probably be used to extend on
meanings that were orignially upper case.

For details on IPCMessage structure, and the various flags currently
defined, please refer to the IPC.h header file.  [A fuller discussion will
be included in a future revision of this document...]  For examples of IDs,
look at the demo example sources on this disk. [Again, at some point not to
far distant, I hope to publish a list of IDs suggested for various
functions.]  When assigning your own IDs, please do it with future
expansion in mind, and publish them on usenet or elsewhere as soon as
possible for comment.


Message Structure
=================

(briefly)

Each message has a header which is a standard message structure, followed
by an ID field (32-bits), a 32-bit Flags field, and an Item Count
(16-bits).  This is immediately followed by that number of items, each of
which has the following structure:  Item ID (32-bits), Flags (32-bits),
Size of data block (32-bits), and Pointer to data block.  This may in turn
be followed by a data area containing the data for some or all of the items
in a message; this may or may not be convenient, depending on the nature of
the data.  The mn_Length field in the standard message structure indicates
the complete length of the message (remember it is only 16-bits, so the
maximum size of a message containing in-line data is 64K).

As already remarked, the "Pointer" field of an item may not in fact point
to a single block of data, but may be a single value or a NON-STANDARD
pointer (a BPTR for example). In both these cases, the Size of the item
should be set to ZERO.  The Flags field will give other information about
the item, some system, some private (defined by the application).  For the
current system flags, please see IPC.h.


Managing IPCMessages
====================

Two system procedures are provided for convenient management of memory for
IPCMessages (much preferable to the program doing it all for itself).

    CreateIPCMsg(items, extra, replyport)   creates a new message block
                                            for that number of 'items'
                            (0..n), with 'extra' bytes of memory after the
                            items for data storage (you will have to get
                            its offset as the location of the "item"
                            following the last actual item); the
                            'replyport' pointer is inserted in the standard
                            message structure.  It returns a pointer to the
                            created message (or NULL if there is no space).

    DeleteIPCMsg(msg)       deletes the memory block created by
                            CreateIPCMsg.  It does NOT attempt to handle
                            memory occupied by data outside the message and
                            pointed to by its items.


[This document is still in progress -- apologies for omissions]


                            +++++++++++

                                            Pete Goodeve
                                            July 1988