IBM Shop IBM Support Download
HomeNewsProductsServicesSolutionsAbout IBM
Search 

ICAT for OS/2 kernel- and source-level debugger FAQ

Table of Contents

  1. What is ICAT for OS/2?
  2. Hardware and software requirements
  3. Setting up the victim machine
  4. Setting up the host machine
  5. What is the Passthru window?
  6. A sample debug session
  7. Threads vs. slots
  8. Performance tips
  9. Ring-3 debugging
  10. Paged-out code
  11. Special situations

What is ICAT for OS/2?

The IBM Interactive Code Analysis Tool (ICAT) Debugger is used for debugging OS/2 device drivers, kernel code, IFS's, and applications that are running on an OS/2 remote machine. The debugger helps you detect and diagnose errors in code written in C, C++, and MASM languages at the source level.

The ICAT for OS/2 debugger is a source-level debugger which uses OS/2 Warp 3.0 or higher to assist in detecting and diagnosing errors in an OS/2 Warp 3.0 or higher system. It provides a graphical user interface and debugs PM and non-PM (single- or multi-threaded) applications as well as device drivers and other system-level binaries. It has been designed to debug remotely an OS/2 system using the Kernel Debugger (KDB).

KDB runs on a victim OS/2 machine and replies to debug service requests that are sent from the debugger which is running on a host OS/2 Warp 3.0 or higher machine. The supported communication mode between machines is serial.

Supported debug file formats include HLL, which is IBM VisualAge C++ (VACPP), and CodeView (CV).

Hardware and software requirements

This section lists the hardware and software requirements, considerations you need to be aware of, compiling and linking options, environment variables, limitations, and helpful tips.

Hardware Requirements

The debugger must be run on an Intel-based system capable of running OS/2 Warp 3.0 or higher. Slightly less than 11 megabytes of disk space is required to install the debugger. We recommend that you run with at least a 486SX processor for performance reasons.

Software Requirements

Since you are remotely debugging, there are software requirements for both the victim and the host machines. Note that the victim machine can be at Warp 3.0 or higher, but you will need the latest fixpack levels for all versions of OS/2 3.x. See Setting up the victim machine.

Setting up the victim machine

You will have to obtain or build an OS2KRNLD and OS2LDRD that have the Lexington packet communication code. This code is already part of the regular 4.0 or greater KDB builds. For OS/2 3.*, you will need fixpack 23 or greater for the binaries to contain this code. Place these binaries on the target system and rename to OS2KRNL and OS2LDR, respectively.

If you want to debug the demo device driver, then copy SAMPLEDD.SYS over to your victim system. Edit config.sys w/ DEVICE=C:\SAMPLEDD.SYS. This allows the sample device driver to instantiate on the next reboot of the victim system. Copy SAMPLE.EXE over to your victim system, too.

Setting up the host machine

icatgam.exe runs from an OS/2 host. This can be OS/2 Warp 3.0 or OS/2 Warp Connect. By unzipping icatmerl.zip, you have already set ICAT to run on the host machine.

You will need to edit the seticat.cmd file. Please set the environment variables carefully. If you don't, communications won't work or source will not be found. These are the variables and what they mean:

  REM  This variable sets the com port of your host system and the baud rate at
  REM  which ICAT communicates w/ KDB.  Note that at speeds > 19200, you will
  REM  need buffered UARTs on both the host and victim machines.

  SET  CAT_MACHINE=COM2:57600

  REM  This variable is used by the iprobe library to start initial
  REM  communication w/ KDB.  If it's not specified, the iprobe library defaults
  REM  to 9600.  In most cases, you do NOT need to set this variable.  The
  REM  iprobe library will try to communicate initially w/ KDB at this rate, and
  REM  if that fails, it will try to communicate at the CAT_MACHINE baud rate
  REM  (the rate at which you want to have ICAT and KDB communicating).  We
  REM  invented this variable in case you were previously communicating w/ KDB
  REM  (say using zoc or t) at a rate different from 9600 or the CAT_MACHINE
  REM  rate.
  REM
  REM  This variable is important, however, when ICAT is closed down.  If ICAT
  REM  has this variable set, it will reset KDB to this rate on shut down.
  REM  This facilitates those who want ICAT to communicate at the higher baud
  REM  rates yet want to use terminal emulators before and after that can not
  REM  handle the higher baud rates.

  SET  CAT_SETUP_RATE=

  REM  This variable tells ICAT where to find your debug binaries (the .sys
  REM  and .exe files w/ debug information) on your host system.  You should
  REM  put doscalls.cvk (renamed to OS2KRNL) in this host machine subdir IFF
  REM  you are debugging OS/2 kernel code.  Ensure that the version of
  REM  doscalls.cvk lines up w/ the OS2KRNLD that you boot on the victim
  REM  machine, please!

  SET  CAT_HOST_BIN_PATH=i:\sde\samdetw

  REM  This variable is for async communications.  W/ the uKernel, we could
  REM  use token ring and ethernet, too.  Maybe someday w/ Merlin we can.  For
  REM  now, use ASYNC_SIGBRK.

  SET  CAT_COMMUNICATION_TYPE=ASYNC_SIGBRK

  REM  This variable tells ICAT where to find your source (SAMPLEDD.ASM and
  REM  SAMPLE.C) in the demo.  If you are debugging the kernel, ensure that
  REM  your source paths are included here for the kernel components of
  REM  interest.

  SET  CAT_HOST_SOURCE_PATH=i:\sde\samdetw

  REM  This variable causes a recursive search of the subdirectories below the
  REM  subdirectories listed in CAT_HOST_BIN_PATH and CAT_HOST_SOURCE_PATH.
  REM  For example, with the CAT_HOST_SOURCE_PATH variable above, ICAT will
  REM  search the samdetw subdirectory and all subdirectories below samdetw as
  REM  well as their subdirectories, ad nauseam.  This variable defaults to NULL
  REM  so that ICAT will NOT do the recursive search.  Setting this variable to
  REM  any non-null value causes the recursive search to be performed.

  SET  CAT_PATH_RECURSE=

  REM  This variable allows ICAT to disregard all modules other than OS2KRNL and
  REM  the ones specified in the string.  If this variable is null, the iprobe
  REM  library will return all modules to ICAT.  ICAT takes time processing each
  REM  module, so it is often a good idea for performance to supply this
  REM  variable.
  REM
  REM  You can separate the various modules w/ a ';' or a ' '.  (Actually, use
  REM  any delimiter you want as the iprobe library uses strstr on the list to
  REM  see if it should tell ICAT about a module or not.)  The module name must
  REM  be the full name; e.g., mmpmcrts.dll instead of just mmpmcrts.

  SET  CAT_MODULE_LIST=sampledd.sys sample.exe

  REM  This variable allows ICAT to resume the system on its initialization.
  REM  You will rarely (if ever) want to do this.  When you invoke icatgam, it
  REM  initializes the COMx port and breaks into KDB.  Then it changes the baud
  REM  rate (when necessary), and it leaves control of KDB in icatgam's hands.
  REM
  REM  This is really important if you are attaching to a system that has
  REM  already reached a failure or has an embedded int3 (device drivers often
  REM  do).  As such, the default is for CAT_RESUME to be NULL.
  REM
  REM  There could be situations where you want to initialize icatgam, but
  REM  somehow time your attach to the OS2KRNL via the [Attach] option.  In
  REM  this rare case, you may want to set CAT_RESUME=ON, and then icatgam will
  REM  resume the victim system waiting for your attach command to stop it
  REM  again.

  SET  CAT_RESUME=

  REM  This variable allows ICAT to configure KDB with some of your favorite
  REM  commands.  An example would be CAT_KDB_INIT=vsf *     .  This would have
  REM  KDB trap back to ICAT when a fatal trap/fault occurs in a ring-3 app.
  REM  You can use multiple KDB commands separated by semicolons in the string.

  SET  CAT_KDB_INIT=

  REM  If either of the next two environment variables is defined, icatgam sets
  REM  up the COMx port provided by the CAT_MACHINE environment variable so
  REM  that icatgam can talk to a modem and issues the modem attention string
  REM  (+++).  If CAT_MODEM_INIT is defined, icatgam then sends the modem the
  REM  string contained in the CAT_MODEM_INIT environment variable.  If
  REM  CAT_DIAL is defined, icatgam then sends the modem the string contained
  REM  in the CAT_DIAL environment variable and waits 500 seconds for a
  REM  connection to be established.
  REM
  REM  Our intention is that you will figure out the magic AT commands for
  REM  CAT_MODEM_INIT based on whatever you currently use in your terminal
  REM  emulator (which presumably doesn't change very often).  You can then
  REM  leave CAT_MODEM_INIT alone and set CAT_DIAL to the command to actually
  REM  place the call (which presumably changes frequently).

  SET  CAT_DIAL=ATDT4840

  SET  CAT_MODEM_INIT=ATZ

What is the Passthru window?

The Passthru window allows you to send commands to KDB and view its responses. The window is divided into two areas. The command entry field area is where you enter the KDB commands, and the response area is used to display the KDB output.

The KDB response area is scrollable, which allows you to review previous KDB responses from prior commands. This output area also appends a status icon for each line to indicate your input, ICAT's acceptance of that input, and text from KDB.

When you send KDB commands using the Passthru window, ICAT does NOT update its state. Thus, you should use this window for obtaining information instead of altering registers or data. The Resync push button allows you to force ICAT to update its state if you've made changes via the Passthru window that involve the victim machine's state.

To send a command to KDB, do the following:

  1. Type the command you want to send in the command entry field.
  2. Select the Send push button. You can also press the enter key if the Enter Key Performs Send Function check box is enabled in the Display Style window.

A sample debug session

Reboot your victim machine to get SAMPLEDD.SYS instantiated and the packetized OS2KRNL and OS2LDR files instantiated, too. You can perform a quick check that the victim machine is OK by running your normal terminal emulator (t, zoc, etc.) to see that KDB is alive and well and communicating at 9600 baud. Please shut down this terminal emulator before you run icatgam as they will compete over the COMx port.

After you have edited seticat.cmd, run it to set these environment variables on your host system. Type icatgam, and you will be prompted w/ an initialization dialog.

When you see this dialog, select [Attach]. Running at 9600 baud, this took 35s on my setup. At 57600, this took about 10s. So if you can run at 57600, it will speed things up quite a bit. We are working on ICAT bandwidth problems to enhance performance.

Look over in the component window and ensure that you see SAMPLEDD.SYS. If so, click on the '+', and notice that one part is displayed. Click on the '+', and notice eight entries are displayed. Click on STRATEGY. The strategy routine of the sampledd device driver should come up in the source window. Note: this is masm source, so you will see assembler directives, BUT it is source. We support both CodeView (cl and cl386) and HLL (IBM C Set) debug formats for C.

Set a breakpoint on line 209 inside the Strategy routine. Now have ICAT "go" by clicking on the green light icon.

On the victim machine, type sample. SAMPLE.EXE will emit a DevIOCtl to open the sampledd device driver. You should hit your bp if you are patient.

When icatgam notifies you of the bp, click on the register window. Click on the storage window, and edit its value to that of the PC reg in the Status Flags section of the register window. You will see memory that corresponds to the code space.

Now click on the Source window's Monitors pulldown. Then click on Passthru. In the Passthru window, type dg cs and click on Send OR just hit the enter key. This command will be "passed thru" to KDB, and you will see its reply in the response area. Notice that it gives information about sampledd's CS descriptor. You can emit all the KDB commands from the Passthru window, but if you change state from the Passthru window, you must push the Resync button to allow icatgam to collect the changes.

Set another bp on line 275 inside the Open subroutine. Go again, and hit the bp. Now do a step into on the Ccall macro to SubrWFrame. You are in the SubrWFrame subroutine. Do a call stack unwind. Click on the Call Stack window's Options pulldown. Click on Display style, and enable Return Address before clicking on the OK button. Do a step over and notice the change in the call stack window.

Look over in the component window and find SAMPLE.EXE. Click on the "+" and notice that one part is displayed. Click on that "+", and you will see main. Click on main and set a bp at line 44. Go. You will hit the bp in the Strategy routine again. One more go, and you will hit the bp in SAMPLE.EXE in main.

Now go crazy and debug away. We will be fixing bugs and supplying new debug spins. When done, close icatgam and use your terminal emulator to get KDB back in shape.

Threads vs. slots

The Debug Session Control (DSC) window is the control window of the debugger and displays during the entire debugging session. This window is divided into two panes. One pane shows the process name, the process id (PID) and thread id (TID), and the processor number (for SMP) for each of the ICAT virtual threads (slots), while the other pane shows the components for the modules you are debugging. The slots pane contains the ICAT virtual threads and their state, but each ICAT thread maps to a KDB slot.

In KDB parlance, a slot is a unique system thread identified by its PID and its ordinal (or TID). The ICAT slot number doesn't (necessarily) match the KDB slot number, which you could examine via the Passthru window.

To display the state of a thread, select the plus icon to the left of the thread. To enable or disable the thread listed in the slots pane, double-click on the word Enabled or Disabled depending on the state of the thread. You can also toggle the Thread enabled choice from the Run menu. When a check mark symbol displays beside the Thread enabled choice, threads are enabled, and the debugger allows the highlighted thread to execute. When the check mark symbol does not display, threads are disabled and do not execute.

Note that ICAT only shows the KDB slot information in the DSC slots pane. In all other places you will see the ICAT thread, which is just a convenience mapping to the KDB slot.

Performance tips

ICAT WILL be slower than IPMD as a native debugger. There is no getting around this fact! ICAT has to obtain debuggee information over a serial wire from KDB. In addition, there is an ACK/NACK protocol that pads the information coming and going to ensure that no data is dropped. Having said that, there are some things you can do to speed up ICAT. The next sections address these items.

CAT_MODULE_LIST

This environment variable allows ICAT to disregard all modules other than os2krnl and the ones specified in the string. If this variable is null, the iprobe library will return all modules to ICAT. ICAT takes time processing each module, so it is a good way to enhance performance. Usually, you should just set it to the names of the 2 or 3 (say) modules that you will be debugging.

Baud rate

ICAT (and KDB) can run up to 115200 baud. You must have buffered UARTs on both host and victim machines to support this rate (or any rate > 19200 baud). If you are running on a Warp Connect host machine, you will need to obtain the com.sys that comes with Warp 4.0 if you want to run at 115200 baud, too.

No VDMs!

If you have a VDM running in the background on your host machine, it interferes with the device driver cycles needed for ICAT's communication. You can disable VDMs executing in the background though a DOS Window's DOS Settings. Choose All DOS settings and select Off for DOS_BACKGROUND_EXECUTION.

RAM

Large, complex programs all benefit from more RAM. ICAT is a memory pig, so this works wonders for it. Yes, we need to curb its appetite for memory, but you'd be surprised what an additional 4-8M of RAM will do for ICAT's performance.

Ring-3 debugging

You can debug applications (.exe and .dll files) at ring 3 with ICAT (see A sample debug session where ICAT is used to debug SAMPLE.EXE as part of the demo). The application's modules will show up in the DSC window, and you can click on these modules to show their constituent parts.

Ring-3 context

Note that you need to be in the context of your application when you bring up its source under ICAT. By context, we mean the active thread must be one of your application's threads (slots). If not, then you are mapping source for (say) app1.exe on top of (say) app2.exe's address space. You don't run into this problem with device drivers or IFS binaries because they are mapped into the global address space.

One way to establish this correct context is to double-click on one of the slots (threads) (see Threads vs. slots for a discussion on ICAT threads and KDB slots) under the desired process name in the slots pane of the DSC. Another way to establish the correct context is to double-click on a module part of interest in the modules pane of the DSC. ICAT will force a slot change in KDB (if necessary) to one of the n threads that the module has coursing through it.

The simplest way to establish the correct context when you have control over the launch of your application (read, when you are NOT attaching to a system where your bug has already occurred) is to put a spin loop in your application right after main(). Essentially, you use while( i != 0 ); in your code, and then you launch the application on the victim machine. Attach with ICAT, and then you should be at that spin loop in the correct context. Use the jump to location command to move over the spin loop or change i to 0, etc.

Ring-3 single stepping

When you single step a thread that is in ring-3 context, KDB (which is used by ICAT) does NOT freeze the other n-1 threads (slots) in the victim system. So other events may occur while a single step of a ring-3 application is taking place.

This is even more insidious when ICAT does an implicit single step after you emit the run command. For example, when you ask ICAT to run and you are sitting on a defined break point, ICAT first single steps the current thread, puts in all of the break points, and runs. It does this so that you don't stay stuck on any given break point. However, this means that when you run, some threads may be given time slices BEFORE ICAT can put in its break points! What happens is that some threads seem to _miss_ breakpoints that you expected them to hit.

Ring-3 traps and faults

Again, ICAT is notified of events via KDB. At ring 3, the default for KDB is NOT to notify the end user of fatal traps and faults. Use the Passthru window to emit the vsf * command to KDB if you want ICAT to capture the exceptions at their point of failure in your ring-3 application. You can also set your CAT_KDB_INIT environment variable to vsf * and let ICAT remember to do this action for you each time you attach.

Paged-out code

ICAT does not know if a page of code text is swapped out. If it is, you will NOT be able to set breakpoints via the source listing. ICAT will emit a message that indicates that the line number is invalid. ICAT will NOT be able to show you a mixed or disassembly view of the source when the page of code text is swapped out as well.

When a page is swapped out, you can use the Passthru window and the KDB .i command to try to page the code in. If that fails, you can use an instruction fetch watchpoint for the routine in question, and when the code is executed, ICAT will get a watchpoint notification. Then you can set breakpoints via the source view, and you can get a mixed or disassembly view, too.

If you have placed a breakpoint in a page that has been paged out, KDB now notifies ICAT when that page is paged in. This allows ICAT to restore the breakpoint before the code is executed.

Special situations

There are a number of situations in which the behavior of the debugger and the debugged system is counterintuitive (or wrong), either due to limitations in KDB, limitations or bugs in ICAT, or bugs in the kernel:

Nonsteppable areas

Instructions in the following routines cannot be single stepped: _Debug_CtrlC32, _DebugLoadSymMTE, and _PGSwitchContext. An attempt to single step an instruction in one of these routines sets a breakpoint at the end of the routine and resumes the system. ICAT does not support the equivalent of the KDB tx command.

Thread cleanup

If you use ICAT to debug a multi-threaded program that relies on the system to terminate secondary threads (i.e., one that does not use _endthread or DosExit to explicitly end a thread), the system may not do so properly when the program ends. For example, we have a small multi-threaded testcase in which each thread writes a character to a random position in a window. On normal exit, the program clears the screen, and then the OS/2 prompt reappears. On exit after some debugging sessions, the program does not end cleanly, and the OS/2 window is hung (it cannot be closed and Ctrl-Break attempts fail). This behavior can be duplicated using KDB, so it is either working as designed or a bug in the kernel. We have opened a defect against the kernel.

Discarded device driver segments

Most device drivers jettison their initialization code and data after the initialization phase. If you use ICAT to view these labelled areas (by this, we mean the source laid on top of memory) after initialization, you'll see some odd mappings particularly in the code segment. ICAT doesn't take into consideration that the code or data segment has shrunk, so it doesn't warn you in some instances when you try to look at initialization code or data that has been discarded.

Profile breakpoints

When you exit an ICAT session, information about the various modules under debug is saved in a profile. This includes breakpoint information for these modules. Ring-3 applications present a special problem regarding breakpoint information in the profile.

Realize that you have to set the right context for ICAT before you read or write memory, etc. for a specific application. See Ring-3 debugging for the dos and don'ts concerning process-specific information.

This also applies to ring-3 breakpoints from the profile. If when you attach you are not in the context of the ring-3 application for which the breakpoints apply, you will not hit these breakpoints in your application. Worse, these breakpoints will be arbitrarily written into some other process space perhaps in the middle of opcodes!

So be careful with profile breakpoints for ring-3 applications when you fire up ICAT on subsequent runs. We will try to figure out a way to handle this better down the road. Profile breakpoints for ring-0 applications (say device drivers) don't have this problem because their code is in the global space.

Deferred breakpoints

Deferred breakpoints do not work for any modules currently. The problem has to do with the way KDB notifies ICAT of a module load. When the module is loaded, KDB signals ICAT. ICAT then tries to put in the deferred breakpoints (if any) for that module. But this can't work as none of that module's pages are paged in at this point.

We will try to modify code in the ICAT debug probe library to address this if we solve the problem of paging code in from the probe. Until then, please use a load on occurrence breakpoint, set the proper context (if necessary), and then use an instruction fetch watchpoint on the function of interest.

PrivacyLegalContact---
⌐ 1999 IBM Corporation