The Need for Instrumentation and Tracing

A major challenge for programmers is to get a handle on the behavior of complex programs. Often this challenge is part of finding and eliminating a bug. In other instances, the challenge is part of an effort to characterize program operation so routines can be targeted for optimization. Sometimes the challenge is part of testing to verify that obscure execution paths are being taken and indeed perform correctly.

The phrase get a handle on is a sloppy way of saying that the programmer needs a way of following the execution of a program at the necessary level of detail to determine why a program is performing as it does.

There are five common methods of achieving this end:

Each of these methods have their own advantages and disadvantages. For example, careful code reading can be very comprehensive, but it often fails to expose subtle, dynamic workings of the code. Also it takes a large amount of time. Interactive stepping through code with a debugger gives the programmer a great deal of control and detail but is often tedious because it requires the active involvement of the programmer. Profiling is very valuable for determining real performance timings but many profiling methods suffer from presenting too much poorly targeted data; from the sampled nature of the data; and from the need for specific termination points.

Where code reading, interactive debugging, and profiling are less effective, instrumentation and tracing can fill in the gaps. To the advantage of instrumentation and tracing, they require less interactive involvement during use. To their disadvantage, they require the program to be augmented by statements to perform the instrumentation and tracing.

When used, methods for instrumentation and tracing tend to be very ad hoc. Tracing usually consists of scattering print statements through the program source code. Instrumentation usually means introducing variables used to count events and the associated code to do the counting. When the tracing and instrumentation support needs to be stripped out, this can often be error-prone leading to unused variables, print statements left in, or possibly even deletion of code not for instrumentation or tracing.

A system for performing program instrumentation and tracing designed to be easily disabled, portable, and highly functional would be extremely desirable for X server developers and testers. It would be even more desirable if it was accessible through the same client-server mechanism other X services are made available. XYZ is an implementation of such a system.