Programming With OT AutoPush Support

This document describes the programming API to the "OT AutoPush Support" extension. You can use this API to force your STREAMS module to be autopushed on top of all TCP streams that are created on the system, for purposes of network debugging, firewall support, and so on.

Contents

Introduction

While working as the STREAMS expert at Apple Developer Technical Support, I was asked many times how to see all TCP data or connections under Open Transport. Typical uses for such technology include data filtering (ie looking for 'naughty' words in web downloads), firewall support (ie implementing SOCKS in a system-wide fashion), or debugging tools (ie logging all TCP traffic to disk while programming).

Seeing all TCP data was possible under MacTCP by patching the _Control trap and looking for all communications between applications and the ".ipp" driver. Although this code was ugly to write, it did work.

Under Open Transport all of these patches broke. Well, not exactly "broke", more like "didn't work properly". The patches continue to see all data and connections passing between MacTCP applications and Open Transport's compatibility ".ipp" driver, but they do not see data and connections initiated by OT-native applications.

I've seen some products try to emulate approach under Open Transport, ie they try to patch all the OT client APIs for sending and receiving data. This is the wrong approach, mainly because it involves a bunch of really ugly patches. In general, I strongly recommend against patching OT API routines. It is hard to do, and it introduces a significant compatibility liabity for your product. If you don't believe me, see the Caveats and Design Decisions section of this document.

STREAMS To The Rescue

The alternative to patching is to write a STREAMS module that does the filtering. To do this, you need a bit of background about STREAMS. Open Transport is based around a standard STREAMS architecture. This architecture divides networking code into client code (a normal program that wants to send or receive data over the network, eg an FTP client), and kernel code (a system plug-in that provides some network service, eg an Ethernet device driver). The key benefit of STREAMS is that kernel code components can be plugged together in a 'mix and match' fashion. This provides extensive flexibility in the networking kernel.

Note
If this sections doesn't make any sense to you, you need to learn something about STREAMS. I recommend you read a book about STREAMS, for example:

So, let's say you want to watch all the Ethernet traffic to and from the machine. You can do this by inserting your own kernel plug-in (a module) into every stream that is connected via the Ethernet driver. STREAMS provides this functionality through the autopush mechanism -- you can specify that your module is to be autopushed on top of every streams that's opened to the Ethernet driver.

This is really neat because it allows you to write a small plug-in to implement your functionality, instead of rewriting the entire Ethernet driver, and to get that plug-in called without patching.

Components in a STREAMS 'Patching' System

The component of a system-wide stream 'patching' system are:

As a real world example of this approach, imagine a hypothetical program that logs all the traffic on all the Ethernet streams to a file. It would contain two components. The first is the stream module itself, EnetWatcher, which is installed in the Extensions folder. The second is the assistant application, EnetWatcher Monitor. When you run the assistant application, it autopushes the EnetWatcher module on top of Ethernet driver. It then opens a special stream to the EnetWatcher module, and configures it through that stream instructs. The EnetWatcher module will now be inserted into all Ethernet streams that are created on the machine. Its purpose is to make a copy of all the Ethernet data that is sent or received on all the Ethernet streams, and forward a copy down the special control stream to the EnetWatcher Monitor. The monitor grabs this copy of the data and spools it on to the disk.

The primary advantages of this approach are:

  1. There is no patching to get the EnetWatcher module installed.
  2. The EnetWatcher module does not have to talk to the file system. This is good because talking to the file system is technically illegal in a STREAMS module. Instead, the file system operations are restricted to the EnetWatcher Monitor application.

The Way It Should Work

The only tricky part of implementing such a system is to get your stream module autopushed on top of all the streams you are interested in. STREAMS provides a good solution to this problem, in the form of the "sad" module. The "sad" module is a part of OT that affects how streams are constructed. Whenever a STREAMS driver is opened, the system consults the "sad" module to see if there are any modules that should be autopushed on top of the driver.

For example, you can ask "sad" to automatically push your "EnetWatcher" module on top of the "mace" driver (which controls the built-in Ethernet on the first generation PCI machines). Whenever a client opens the "mace" driver, OT consults "sad" and sees that "EnetWatcher" is supposed to be pushed on top of the driver. So the "EnetWatcher" module is pushed on top of the "mace" driver and can see all traffic up and down the stream.

The following listing shows how to autopush "EnetWatcher" on top of "mace":

static OSStatus EngageEnetWatcher(void)
{
    OSStatus err;
    StreamRef strm;
    struct strioctl stri;
    OTAutopushInfo myOTAutopushInfoStructure;
 
    strm = OTStreamOpen(kSADModuleName, 0, &err);
    if (err != noErr) {
        strm = nil;
    }
 
    if (err == noErr) {
        myOTAutopushInfoStructure.sap_cmd = kSAP_ALL;
        strcpy(myOTAutopushInfoStructure.sap_device_name, "mace");
        myOTAutopushInfoStructure.sap_minor = 0;
        myOTAutopushInfoStructure.sap_lastminor = 0;
        myOTAutopushInfoStructure.sap_npush = 1;
        strcpy(myOTAutopushInfoStructure.sap_list[0], "EnetWatcher");
 
        stri.ic_cmd = I_SAD_SAP;
        stri.ic_dp  = (char *) &myOTAutopushInfoStructure;
        stri.ic_len = sizeof(OTAutopushInfo);
        stri.ic_timout = -1;
        err = OTStreamIoctl(strm, I_STR, (void*)&stri);
    }
 
    if (strm != nil) {
        (void) OTStreamClose(strm);
    }
 
    return err;
}

Some things you should note about "sad":

  1. "sad" stands for System Administration Device.
  2. The actual interface to "sad" is defined in the Open Transport Client Note.
  3. It is not persistent across reboots. If you need your autopush to remain in effect when the machine reboots, you will have to write some code that runs at startup time to re-institute the autopush. Typically you would use an application, extension ('INIT'), or application extension ('appe') to do this.
  4. It may not be persistent when the OT kernel unloads. I have not had chance to examine this case. You may need to hold the OT kernel in memory (by calling InitOpenTransport in your extension) to prevent the "sad" setting being lost.
  5. Autopush only comes into play when drivers are opened. Autopush is not consulted when modules are pushed.

Sad About "sad"

The final note in the previous section is critically important if you want to autopush on top of TCP. Unfortunately, OT 1.x constructs TCP streams by pushing the "tcp" module on top of IP. Because autopush is not consulted when modules are pushed, you can not autopush on top of "tcp".

This problem was scheduled to be corrected with OT 1.5. As that project has be cancelled, it may never be fixed. So for 1.x there is nothing you can do to achieve this goal other than to patch Open Transport.

The purpose of "OT AutoPush Support" is to provide a single, well crafted, implementation of that patch. By doing this, I hope to increase the reliability of software that needs to autopush on top of TCP, and provide a single point of change when and if the problem is every addressed in OT itself.

Using OT AutoPush Support

As described above, a STREAM patching scheme requires a number of components:

  1. a STREAMS module that is suitable for pushing on top of driver.
  2. some mechanism for getting this module autopushed on top of the streams that you want to watch.
  3. an optional client-side assistant program.

The following sections discuss the how to implement the first two of these components. The remaining component, the client-side assistant, is very much dependent on the exact task you want to achieve, and is not covered in this document.

Writing Your STREAMS Module

The "OT AutoPush Support" software distribution contains a sample module, StreamNOP, that is a good basis for your stream module. In addition, the OT Module SDK comes with number of other sample stream modules. In addition, you should consult a STREAMS reference for general help on writing a STREAMS module.

Getting Your Module AutoPushed

There are two ways you can go about getting your module autopushed on top of all TCP streams. Firstly, you can actually ResEdit the standard "OT AutoPush Support" extension to autopush your module as the system starts up. This is a good technique to get started with because it's very easy to do. However, it is not a good technique for shipping code because it requires you to distribute a modified version of the extension, which is expressly forbidden by your licence.

To add your module to the list of modules that "OT AutoPush Support" autopushes on TCP at startup, use ResEdit to add your module's name to the resource 'STR#' ID=128. Note that module names are case sensitive.

A better solution than ResEdit is to use an auxilliary program to enter the autopush information. You can either wrap this up into your client side assistant program, or ship some separate extension or application to do it. Your registration program must include the "OTAutoPushSupport.[ph]" interface file, and link with the "OTAutoPushSupportLib" stub library. You can then call the library to register your module to be autopushed, as shown below:

    err = OTAPRegisterAutoPushOnTCP("MyModuleName");

When you are done, you can unregister your module, so it will no longer be autopushed on top of all TCP streams.

    err = OTAPUnregisterAutoPushOnTCP("MyModuleName");

Note that both registration and unregistration only affect newly created streams -- they do not affect existing streams. So your module may still be active on existing streams even after it has been unregistered.

Also note, your code should explicitly check for the error return kOTAPPatchingTechniqueBrokenErr. This indicates that the patching technique used by this extension is no longer viable (as determined by the Gestalt selector, see Gestalt Reference for details). You should report this to the user and recommend that they seek an upgrade to the "OT AutoPush Support" extension.

Checking for the Presence of OTAutoPushSupportLib

The API to the "OT AutoPush Support" extension comes in the form of a shared library. You should be sure to do two things before using calling this API:

  1. weak link to the "OTAutoPushSupportLib" stub library. [In CodeWarrior, use the popup next to the library in the project window.] This ensures that your application will continue to run even if the extension is not installed.
  2. programmatically check for the library's presence. If the library is not present, you should take appropriate action. [Typically, you either disable that part of your program, or complain to the user and ask them to install it.] You can check for the library this using the following snippet of code: 
	gOTAutoPushSupportLibAvailable =
                (long) OTAPRegisterAutoPushOnTCP !=
                kUnresolvedCFragSymbolAddress;

What About 68K?

The good news is that "OT AutoPush Support" works fine for both PPC and 68K clients on a PPC machine. The bad news is that the extension does not work on 68K machines. This is because the extension uses Code Fragment Manager to implement its patch, and the 68K version of OT is implemented in Apple Shared Library Manager (ASLM). It may be possible to build an ASLM version of this extension, but I have neither the expertise nor the desire to do it.

Reference

This section provides reference material for the "OT AutoPush Support" package.

Packing List

The package comes with the following files:

Gestalt Reference

The "OTAutoPushSupport.[ph]" file declares the following Gestalt constant:

enum {
    gestaltOTAutoPushAttr = 'PüSH'
};

The Gestalt selector gives you information about the current state of the library, and it is also be used by the system to control the behaviour of the extension. The following Gestalt bits are defined:

gestaltOTAutoPushLoaded
This bit is used by the extension to prevent itself from loading twice. The extension sets this bit when it successfully loads. If a second copy of the extension is placed in the Extensions folder, it will find this bit set and refuse to load.
gestaltOTAutoPushDisabled
This bit is provided for use by the system to control the behaviour of the extension. If this bit is set, the extension will not load, and it will report to the user that they should remove the extension from the Extensions folder. By setting this bit, a future version of the system could subsume the functionality of the extension. It is important that anyone setting this bit also provide a CFM library "OTAutoPushSupportLib" against which clients can link.
gestaltOTAutoPushHackSubsumed
This bit is provided for use by the system to control the behaviour of the extension. If this bit is set, the extension will refrain from patching Open Transport, and call the "sad" module to perform its functionality. A future version of Open Transport that supports autopush on TCP using "sad" could set this bit so the extension implements its functionality using "sad" rather than hacking. Clients that rely on the extension will benefit from this behaviour without modification.
gestaltOTAutoPushHackBroken
This bit is provided for use by the system to control the behaviour of the extension. If this bit is set, the extension will not patch Open Transport at all, even if that results in it being unable to perform its functionality. Instead, it will return the kOTAPPatchingTechniqueBrokenErr error code to its clients. A future version of OT that breaks this hack should set this bit. Of course, one would hope that such a version would also set the gestaltOTAutoPushDisabled bit or the gestaltOTAutoPushHackSubsumed bit so that clients of this library continue to function, but that may not be possible.

Error Code Reference

The "OTAutoPushSupport.[ph]" file declares the following error constants:

kOTAPPatchingTechniqueBrokenErr
The patch code is explicitly marked as broken by the system using the Gestalt gestaltOTAutoPushHackBroken bit. You should report this error to the user as a need to upgrade the "OT AutoPush Support" extension
kOTAPVectorNotFoundErr
The patch code failed to find the transition vector for OTAsyncCreateStream. Chances are this means that the patching technique is broken.
kOTAPVectorChangedErr
The patch code could not undo the patch because someone has overpatched on top of our patch to the transition vector. You should never get this error code because the extension never attempts to unload once it has loaded.

Routine Reference

The "OTAutoPushSupport.[ph]" file declares the following routines:

extern pascal OSStatus OTAPRegisterAutoPushOnTCP(const char *moduleName);
Registers moduleName to be autopushed on top of all TCP streams created from now on. moduleName must point to a C string that contains the name of a valid, installed, Open Transport module.

Errors include:

kEEXISTErr
the module has already been registered for autopush
kENOENTErr
the module does not exist
kVectorNotFoundErr
could not find the right vector to patch
kOTAPPatchingTechniqueBrokenErr
the patching technique used no longer works
other Open Transport error codes
these generally have their common Open Transport meaning
extern pascal OSStatus OTAPUnregisterAutoPushOnTCP(const char *moduleName);
Unregisters moduleName, so that the module will no longer be autopushed on top of all newly created TCP streams. moduleName must point to a C string that contains the name of a valid, installed, Open Transport module that has been registered with OTAPRegisterAutoPushOnTCP.

Errors include:

kENOENTErr
the module has not been registered with OTAPRegisterAutoPushOnTCP
other Open Transport error codes
these generally have their common Open Transport meaning

How It Works

On PPC, all OT routines are exported by Code Fragment Manager. It is possible to patch CFM entry points, at the cost of a significant level of compatibility risk. The "OT AutoPush Support" extension does this patching, so that you don't have to.

Theory of Operation

The general theory of operation of the "OT AutoPush Support" extension is that it patches a specific OT routine, OTAsyncCreateStream to modify the stream created to contain the modules that have been registered for autopush.

The critical parameter in OTAsyncCreateStream is the OTConfiguration parameter. An OTConfiguration is an abstract representation how how a stream should be assembled. If you've done any work with the OT API, you'll know that you usually create a TCP endpoint with the code: 

   ep = OTOpenEndpoint(OTCreateConfiguration("tcp"), ...);

The OTCreateConfiguration takes a string and returns an OTConfiguration data structure. That data structure is then passed to OTOpenEndpoint to create the actual connection to the transport provider. Internally, OTOpenEndpoint (and OTAsyncOpenEndpoint) both call through to another OT routine, OTAsyncCreateStream. It turns out that this routine is exported for the benefit of people doing low-level STREAMS code on OT, and so the prototype is available in the public header file "OpenTptClient.h". [This header file is not in the Universal Interfaces. If you want to see it, you have to get it from the OT Client SDK.]

The purpose of the "OT AutoPush Support" extension is to modify the OTConfiguration passed in to OTAsyncCreateStream to prepend all the modules that have been registered for autopush with the extension. For example, if the input configuration is tcp, we would modify it to be StreamWatcher->tcp and then call through to the real OTAsyncCreateStream routine to do the real work. Fortunately, "OpenTptClient.h" contains a number of API routines for modifying OTConfigurations, so this job can be done using documented APIs.

Implementation Specifics

The "OT AutoPush Support" extension patches OTAsyncCreateStream using CFM. The basic idea is to use GetSharedLibrary to create a connection to the library (OTNativeClientLib) containing the routine, and then use FindSymbol to look up the address of the symbol. This address is actually the address of a transition vector. The code grabs the old value of the transition vector and stores it in a global variable, and then modifies the transition vector to point to its own internal patch routine.

So, when OTAsyncCreateStream is called, it actually calls the patch routine. The patch routine does the necessary munging of the OTConfiguration parameter, and then calls through to the old transition vector as a simple C function call.

The technique works for OT 1.x only because OT's client libraries are globally shared. So you can patch the entry point and expect all clients to be affected. The plan for OT 1.5 was to use per-context CFM libraries, which would break this sort of patching. The plan was to fix the "autopush" on TCP at the same time; if OT for traditional Mac OS is ever significantly enhanced, chances are that the "OT AutoPush Support" extension will have to be revised to accomodate the changes.

Note
If your eyes glaze over every time someone says "transition vector", consult the "Mac OS Runtime Architectures" book on ETO. 
    E.T.O. #22
      Documentation
        Mac OS Runtime Architectures
          Mac OS Runtime Architectures

Caveats and Design Decisions

This section describes a number of important design decisions and caveats in the design of the extension. Many of these are notes on the internal structure of the code, and only make sense if you have the source. [Which I don't generally distribute.] But they might make interesting reading for some.

Credits

"OT AutoPush Support" was written by Quinn "The Eskimo!". Please see the document "Read Me for OT AutoPush Support" that comes with this distribution for more details on its distribution conditions.

Quinn "The Eskimo!"
Late Night Silly Software

5 Aug 1997

Special Disclaimer

While I wrote this product while working as an employee of Apple Computer, Inc, I developed it on personal time using personal tools. Thus, I claim full copyright for it. In a similar vein, Apple Computer, Inc was not involved in the development of this product, and has is in no way endorsed it.