Club Amiga de Montreal

home *** CD-ROM | disk | FTP | other *** search

/ Club Amiga de Montreal - CAM / CAM_CD_1.iso / files / 164.lha / IPC / ipc.doc < prev next >

Wrap

Text File | 1988-04-28 | 24.8 KB | 487 lines

An Inter-Process Communication Standard for the Amiga ===================================================== This is initial documentation for the IPC standard developed via usenet over the past several months. A large part of the standard is now fairly stable (the IPCPort mechanism in particular), but there is a lot of work still to be done on the details, on developing message IDs for specific functions, and on publishing them. Anyway, in its present state, here it is. Why IPC? ======== Up to now, even though the Amiga is the first really totally multitasking personal computer, programs for it have been written much as for any of the older one-job-at-a-time machines (though they often take good advantage of the windowing environment): they run as single processes. A few programs spawn child tasks to perform some of their functions, but they are still monolithic in their design. There is an alternative. A ... let's call it a "Job environment"... could be serviced by a number of more specialized processes, communicating with each other. Depending on the needs of that environment, these processes could be very specialized -- a note bender in a MIDI system, say -- or more general, like a text editor that called up things like a spell checker or output formatter when needed. Such a system would give the user much more choice in configuring it to his/her preferences. (Not forgetting though that a default configuration would have to be as easy to use as any well designed program!) The modules in an environment wouldn't all have to come from one company, as the interface to invoke a module's function would be published with the module. You could add special purpose functions to a paint program say, just by starting up another process that would recognize, or be recognized by, the paint program. There are a lot of possible applications of this principle, with widely varying requirements. Here is a very incomplete list of a few that come to mind: * Communications -- connect terminal emulators, protocol handlers [Kermit, XModem etc.], auto-answer modules, script processors etc. to suit the user's environment. [Note the wide range of parameters such as processing speed demanded by different modules: protocol handlers will have to operate character by character at baud rates, while script servers and auto-answer can be much more leisurely.] * MIDI Modules -- a lot of possibilities here for a fast IPC protocol; anything you might want to do to a MIDI channel could be a "plug-in" module. * Data Acquisition and Control -- there are untapped opportunities for a multitasking machine in the laboratory; being able to "plug" modules together like you can with lab equipment would be a great attraction. * Desktop Publishing -- put together a comprehensive and competent system from the pieces YOU prefer. * Desktop Video and Presentation -- lots of enticing possibilities here; the real-time nature of presentation gives lots of opportunities for fast IPC. (One might even think of coupling several machines together...!) * Service Programs -- to do common jobs for other modules; small ones -- print format servers, pattern matchers, directory searchers etc. -- can take a lot of the load off other modules that would otherwise have to include code for those things; larger ones could handle jobs like spell- checking for any program that wanted it. * A Multitasking H*perC*rd -- a somewhat vaguely specified, but highly enticing, goal; dream your own scenario... This should give some flavor of the ways in which I think a general "Cooperating Processes Paradigm" would be a winner. You can see that a wide range of schemes could fall under the term "IPC". It would be nice if they could all be handled by one underlying protocol. There are a number of possible candidates. Processes can pass information between each other via files and pipes, for example. This has the great advantage that it is already built into the system, but its disadvantage is the heavy overhead involved in file-type I/O. Also, simple serial stream, unstructured, communication isn't good enough for a system that is going to pass around many different types of information. The receiver of information has to know exactly what it is getting, and the sender must know exactly how to tell it. This means some kind of standard structure for the data that can be understood by all processes. IFF is such a standard, but again unfortunately it is too complex and serial-oriented for very fast communication. Its principles make a good starting point though. There is an obvious candidate built into the Amiga Exec: Messages and Ports. These are actually just about what we want, but there are a couple of problems with them as they stand. First, although ports work perfectly when child tasks spawned and managed by a single master process use them to talk to each other, there is no protection against accidents that are likely to occur when independent processes communicate. The problem is that, as ports are accessed by a pointer, there is no way of being sure that the port pointed to still exists when you want it, unless you lock out all other processes before you get the pointer, and don't unlock them again until you've used it. This can get VERY cumbersome and inefficient if you want to send a lot of messages to a port, as you have to do the Forbid/Find pointer/Use pointer/ Permit sequence for every message. Second, there is the problem of a process knowing exactly what is in the message it has received. As we said, there must be a Standard Structure for the data in the message so that it becomes easy for different authors to write programs that can understand each other. Hence the IPC Standard really breaks down into two parts: IPCPorts and IPCMessages. These are essentially independent of each other, in that the Ports have no knowledge or concern about the message structure (aside from the necessity for the standard Exec Message structure at its heart), and conversely the Messages are not concerned with the Port procedures. It should be pointed out that this is a "Foundation Level" protocol. It defines how ports should be managed, and the common structure of messages, but says nothing more about how they are to be used. There are many higher level considerations, such as how the processes should appear to the user, and how he is to control them, that need to be worked out. There are a lot of possibilities here: control panels, patch panels, "drop-in" menus; some applications will need added capabilities such as command languages and scripts; others would be slowed unacceptably by such overhead. But with luck this standard will be general enough to cover the range. There are other schemes, actual and potential, that have to be considered in relation to this one. In general there is probably no need for them to be competitors. One such is AREXX, which has already been successful in applications like integrating editors with text formatters; because it is basically an interpreted script language, it doesn't seem so suitable for some of the other potential applications mentioned above. There seems no reason though why an interface between AREXX and this standard could not be written; only a restricted set of IPCMessages could be handled, but probably enough to give great flexibility. Another possibility is an "Object Oriented" communication scheme. This has both similarities and differences to this protocol, but is still in rather formative stages at this point. We will have to wait to see if the two schemes can coexist: I believe they can. IPCPorts ======== An IPCPort is a rendezvous point between processes where messages can be dropped off and picked up. It is identical in concept to the standard Amiga Exec MsgPort (see the Rom Kernel Manual), but is managed so that messages cannot be sent to a non-existent port (if the processes obey the rules!). Any number of "Client" processes can be sending messages to a port at one time, but there can be only one "Server" on a port. If a port doesn't have a Server -- either running or in the process of being loaded --, it is "Closed", and won't accept messages. Unlike an Exec MsgPort, an IPCPort doesn't "belong" to any particular process, and a program must never create one directly or assign memory in its own space for one: a port will be created by the first process that needs it, and will remain available until all references to it have been cleared; it may be finally deleted by the last user long after the first has gone away. Either a Server or Client may make the first reference. Each port is identified by a unique name (although unnamed anonymous ports are also possible, as we'll see later); the name may be any appropriate null-terminated byte string. A port is located by an exact match to the string: the case of the characters is respected. The author of a server will publish the names of the ports it services, so that others can write clients to access it; if two programs offer alternatives for the same service, they should use the same port name, so that the user may plug one of them in without changing the client. IPCPorts are managed by a procedure module linked in with each program that uses IPC. No separate manager process is needed, although for more flexibility it is possible to run a "Port Broker" that manages the loading of servers when their ports are requested (this will probably become the usual mode). At the moment, each program must have its own copy of the code, but it will eventually be a resident library. Clients and IPCPorts ==================== When a Client process first needs a port of a particular name, it must get a pointer to that port. The pointer will then remain valid until the Client specifically drops its access request. Each request made MUST be paired eventually with a drop for correct port management. There are three possible ways of getting a port pointer. Each of these procedures takes a names string as argument and returns the pointer if possible. They differ in the actions they take to ensure the status of the port. GetIPCPort(name) always returns a valid pointer (unless memory is full, or some other disaster strikes). The port is created if it doesn't exist. No indication is given as to whether there is a current server. FindIPCPort(name) only returns a pointer if the port already exists and has a server; otherwise it returns NULL. LoadIPCPort(name) (an addition to the basic procedures that is in its own module) does a GetIPCPort, but if the port has no server it calls on a "Port Broker" process (also an addition to the basic system) to load one; if the broker can't do this (or the broker doesn't exist) the port will be dropped again, and NULL will be returned. If a valid pointer is returned, you can assume that a server exists or is being loaded. LoadIPCPort does not wait for the server to be loaded, but the "Loading" flag will be set in the port so that it will accept messages for the server to handle when it arrives. Each successful call to one of these procedures increments the "Use Count" of the port. To reduce the use count again you must end the Client's access to the port by: DropIPCPort(port) where 'port' is the pointer returned by any of the above calls. The use count is reduced by one: if it goes to zero, the port is deleted. A Client must not exit without dropping all the ports it has acquired (and of course must not drop them more times than it has acquired them!). A Client sends messages to an IPCPort by a call exactly analogous to Exec's PutMsg(), but the Exec call has no way of detecting the presence or absence of a Server, so it cannot be used. Instead use: PutIPCMsg(port,message) where 'port' is a valid pointer, and 'message' is a pointer to an IPCMessage. It returns TRUE if the message was successfully queued on the port; if the port has no server either present or loading, it will not send the message and will return FALSE. Unless there are specific reasons otherwise, a message sent to a Server will eventually be replied, and the Client must handle this also by supplying a Reply Port. The reply path does not need the IPCPort protocol, because a Client MUST keep the reply port available until it has received back ALL the replies it is expecting, so normally this can be a standard Exec MsgPort. There is no reason though why it should not be an IPCPort as well if you prefer -- even one used for other communications -- as long as you bear in mind that replies will be sent to it whether or not it has a server assigned. Servers and IPCPorts ==================== The Server uses a complementary set of procedures to a Client to manage a port. There is only one way it can acquire a port: ServeIPCPort(name) makes this process the server for the named port if possible and returns a valid pointer to the port. The port is created if it doesn't exist. If a server already exists for the port, the call will return NULL. (If successful, it also increments the use count.) Terminating service on the other hand is normally a two-step process. First the server must shut the port against further incoming messages, then handle and reply to any messages still queued on the port, and finally free up the port for eventual reclamation or for another server to claim it. ShutIPCPort(port) just marks the port as "Shut". The server remains attached to the port and can receive signals from it (for example if the number of clients changes, see below), but PutIPCMsg calls will be blocked. LeaveIPCPort(port) removes this process from association with the port. Also does a DropIPCPort to decrement the use count and delete if appropriate. Between the ServeIPCPort and the ShutIPCPort, the server must be prepared to accept messages on that port. In most cases it will probably spend the major portion of its time waiting on that port -- or possibly several ports -- for a message to arrive. All this area of the server's operation is handled by standard Exec calls: WaitPort() or Wait(), GetMsg(), and ReplyMsg() (see the Rom Kernel Manual). If it only has one port to wait on, it can use: WaitPort(port) which suspends the process until a message arrives at that port. This is liable to be inadequate though, because it will only be woken up by messages arriving on that particular port. It is NOT awakened by other signals to the process, even if they use the same signal bit (see the RKM for Task Signals). Thus you are more likely to want to use: Wait(sigbits) where 'sigbits' is a 32-bit mask of all the signals you want to be awakened on. It returns the signals that were actually set, so the process can determine if it needs to take unusual action. This means, though, that you have to create the sigbits mask from the signal bit number contained in the port itself (and other bits that you want to respond to). There is no specific call to get this (though there should probably be at least a macro), but if the server is just using IPC.h as a header (and not IPCPorts.h) IPCPorts are equated to MsgPorts, so you can get the bit by: sigbit = 1L<<port->mp_SigBit; Note that this bit is also used by the "Notify" feature (below) to awaken the server if the number of clients changes. No message is associated with this signal, but all the server has to do is check the number of clients (again, below) each time it is awakened. To process a received message, the server simply uses GetMsg(), though it is preferable to cast the returned value to the correct type: msg = (struct IPCMessage *)Getmsg(port); When done with the message, it should use ReplyMsg() in the normal way (using a cast if your compiler applies prototype checks): ReplyMsg((struct Message *)msg); Checking IPCPort Status ======================= Either Client or Server can get information about the current state of an IPCPort with the CheckIPCPort() call. The Server alone can also set the state of certain flags in the port (only one is currently defined). CheckIPCPort(port,flags) as long as the high bit of 'flags' (a 16-bit value!) is not set, returns the number of users (including the server) currently aware of the port; if the high bit is set (0x8000 -- this should be a defined identifier... an omission in the current IPC.h, sorry), the call instead returns the current flags set in the port. When called by the current Server ONLY, the value in the low 8-bits of the 'flags' argument will be set into the port's Flags slot; system flags in the upper 8-bits of the slot are not affected. Although a Client may use this call, it will probably have no need to, unless it wants to check for the presence of a Server without making a PutIPCMsg call. The Server, however, can use the call to keep tabs on how many clients actually require its services, and optionally terminate if there are none. To do this it will want to be notified if the number of clients changes; this can be done by setting the IPP_NOTIFY flag in the port: while that is set, any call that acquires or drops a port will send a signal to the Server process (using the signal bit defined for that message port); no message is actually sent, but the Server should call CheckIPCPort each time it is woken up. Anonymous IPCPorts ================== It is possible to create IPCPorts that are unnamed, but these obviously cannot be acquired independently by other processes without a name to access them by. The associated pointers must be passed to other processes via some standard message pathway (left up to the processes concerned). They still should obey the IPC rules, though, so another procedure is provided for acquiring them: UseIPCPort(port) registers another user for that port (i.e. increments the use count). When the user process is done, it issues DropIPCPort in the usual way. You create an anonymous IPCPort by: ServeIPCPort(NULL) which will return a pointer to a NEW port for each call, and set the caller as the Server. (It will only fail if there is no memory or some such fatal problem.) For special cases -- e.g. ReplyPorts -- where you know PutIPCMsg will never be used, you could instead use GetIPCPort(NULL), but there is NO way of registering a server on an anonymous port except at creation time. +++++++++++ IPCMessages =========== The format of IPCMessages is intended to be very flexible, yet specific enough that a Server can immediately determine the function and content of a message it receives (or reject it if it does not recognize it). The protocol has two levels where identification is specified. There is a fixed structure Message header (basically an Exec Message structure with added information) which includes a Message ID, followed by an arbitrary number of "Items", each again of fixed structure with its own Item ID. Each item in turn usually points to a block of memory where the actual data for the item is stored. Instead of a pointer, a single 32-bit value can be stored in the item itself. In special cases, the data pointed to may be more complex than a single data block (a list, say) but then the restrictions on what can be done with the message increase markedly. As long as all the items in a message are simple data blocks (or single words), it can be passed between servers that do not have to understand the contents of those blocks: deletion of the message, for example can be done by any server. Flags in the message and each item indicate whether the data is private to the originating process or can be transferred to the receiver, whether it is suitable to send out on a network, and so on. An ID -- Message or Item -- is a 32-bit longword, normally a four-character ASCII code. All "Published" IDs should be of this form, but cooperating processes could perhaps use non-ASCII numeric values internally. If the first (high) byte is zero, the ID is a private one. Codes of less than four characters may of course be used, but they preferably should be padded on the right with blanks (rather than nulls). Case is important naturally; by convention (and analogy to IFF) upper case is preferred, but there is no firm requiI5ent for this; lower case should probably be used to extend on meanings that were orignially upper case. For details on IPCMessage structure, and the various flags currently defined, please refer to the IPC.h header file. [A fuller discussion will be included in a future revision of this document...] For examples of IDs, look at the demo example sources on this disk. [Again, at some point not to far distant, I hope to publish a list of IDs suggested for various functions.] When assigning your own IDs, please do it with future expansion in mind, and publish them on usenet or elsewhere as soon as possible for comment. Message Structure ================= (briefly) Each message has a header which is a standard message structure, followed by an ID field (32-bits), a 32-bit Flags field, and an Item Count (16-bits). This is immediately followed by that number of items, each of which has the following structure: Item ID (32-bits), Flags (32-bits), Size of data block (32-bits), and Pointer to data block. This may in turn be followed by a data area containing the data for some or all of the items in a message; this may or may not be convenient, depending on the nature of the data. The mn_Length field in the standard message structure indicates the complete length of the message (remember it is only 16-bits, so the maximum size of a message containing in-line data is 64K). As already remarked, the "Pointer" field of an item may not in fact point to a single block of data, but may be a single value or a NON-STANDARD pointer (a BPTR for example). In both these cases, the Size of the item should be set to ZERO. The Flags field will give other information about the item, some system, some private (defined by the application). For the current system flags, please see IPC.h. Managing IPCMessages ==================== Two system procedures are provided for convenient management of memory for IPCMessages (much preferable to the program doing it all for itself). CreateIPCMsg(items, extra, replyport) creates a new message block for that number of 'items' (0..n), with 'extra' bytes of memory after the items for data storage (you will have to get its offset as the location of the "item" following the last actual item); the 'replyport' pointer is inserted in the standard message structure. It returns a pointer to the created message (or NULL if there is no space). DeleteIPCMsg(msg) deletes the memory block created by CreateIPCMsg. It does NOT attempt to handle memory occupied by data outside the message and pointed to by its items. [This document is still in progress -- apologies for omissions] +++++++++++ Pete Goodeve July 1988