Simtel MSDOS 1992 September/Simtel20_Sept92.cdr/msdos/sysutl/fdk.arc/FDK.DOC

Text File | 1988-05-24 | 13KB | 270 lines

                          Filter Development Kit
                        (C) 1988 -- Chris E. Greer


NOTICE: This software may be used and freely copied for personal use only.
        Any commercial or business use requires permission of the author.
        The author makes no guarantees about the operation of the software,
        and cannot be held liable for any problems or loss resulting from
        the use or misuse of this software.




                                Overview

   The Filter Development Kit is a project that grew out of my desire to
try out my new copy of Turbo C, and to learn the C language in general.
What it consists of, essentially, is two separate programs that can be used
to create many different kinds of DOS filters.  By a filter, I refer to a
program that takes a stream of characters as input, changes them in some
way, and outputs the changed stream.  An example of a filter is a program
that strips off the high bit of all the characters in a stream.  A filter
may also change lowercase characters to uppercase, or strip all the form
feeds from a stream.  All these types of filters can be created with FDK.

   Although filters can be pretty complex, such as the SORT, FIND, and MORE
filters supplied with DOS, the filters created by the FDK are primarily
substitution filters.  A swap table is maintained by the filter, and any
character can be mapped to any other character, or simply removed from the
stream altogether.  In other words, you can set up the filter so that every
time an 'A' is encountered on the input stream, an 'a' is substituted for
it.  This would have the effect of changing all uppercase A's to lowercase.

    In addition to character substitution, the FDK also gives you the
option of completely removing NULLs, or characters with a value of 0, from
the stream.  Thus, by mapping any unwanted character or characters to NULL,
and then stripping NULLs from the stream, you can delete unwanted
characters from the stream.

    The final option available is the expansion of tabs into spaces.  Tab
'stops' can be set at any interval you want, and any tabs encountered in
the input stream would be turned into spaces up to the next tab stop.



                            Creating a Filter

   The two files needed to create a filter are FDK.EXE and FILTER.EXE.
FDK.EXE is the program that allows you to modify the swap table and set the
NULL and TAB options for your new filter.  FILTER.EXE is the template
filter; actually, it is a 'non-filter' - nothing has been remapped in the
swap table, and NULLs and TABs are passed through unaffected, but it still
is a working filter.  Although you can use any filter created by FDK as a
template filter, it's still best to keep this one around when you need to
start from scratch.

   Using FDK is fairly simple.  To create a filter, make sure FDK.EXE and
the template filter are in the same directory, or at least that the
template filter is in the current directory and FDK.EXE is available
through the current PATH setting.  Enter the following at the DOS prompt:

    FDK <template> <newfilter>

where <template> is the name of an existing filter created by FDK, and
<newfilter> is the name of the new filter you are creating.  Both must be
valid DOS file names, both must have the extension .EXE, and they cannot be
the same name (i.e., the command: FDK filter.exe filter.exe will generate
an error.  For example, to use FILTER.EXE as the template, and to create a
filter named UPPER.EXE, enter the following:

    FDK filter.exe upper.exe

The template filter is read, and the current settings for the low-order
bytes of the swap table will be displayed.  Each column has two numbers;
the left one is the input value, which can't be changed.  The right number
is the substitution character and can be changed to any one-byte value you
choose (00-FF).  You can move through the table using the following editing
keys, and remap any character as desired.

      Editing Keys:

        Arrow keys        move in the direction of the arrows

        Tab, Shift-Tab        move right or left one column

        PgUp, PgDn        move to top or bottom of current
                    column

        Home, End        move to upper left and lower right
                    characters of the displayed table

        F1            toggles between low-order charac-
                    ters (00 - 7F) and high-order
                    characters (80 - FF)

        F10            toggles between fast and slow
                    screen updates.  Fast is the
                    default, use this toggle only if
                                        you get snow on your monitor

        ESC            exits the swap table


All values in the swap table must be entered as a hex value ranging from 00
to FF.  The original character and its mapped character are shown at the
bottom of the screen.  This line also displays the two or three letter
abbreviation for the ASCII control codes (these are the first 31 characters
of the ASCII character set, from 00 - 1F hex).

   After setting up the swap table for your new filter, you will be shown
the current status of the NULL handler.  Use the space bar to toggle
between keeping and stripping NULLs, and hit Enter to accept the new
setting.  You will then be shown the current status of the TAB handler, and
can use the space bar to toggle between expanding them or passing them
through.  If you want to expand the tabs, you will be prompted for a number
to use to create the tab stops.  Enter a new value or press Enter to accept
the current setting.  That's all there is to it; the new filter will be
written to disk and you will be returned to DOS.




                            Using your Filter

   These filters work by taking advantage of redirection and pipes in DOS.
All the references to character streams so far may be a little confusing,
so here's a brief explanation.  DOS treats all files as streams of bytes.
DOS also treats keyboard input, or stdin,  and console output, or stdout,
as files.  Redirection takes advantage of this fact, and remaps stdin or
stdout to point to a regular file instead of the keyboard or monitor.  To
redirect input, add a left arrow (Shift ,) to the name of the file you want
to use as an input file.  To redirect stdout, add a right arrow (Shift .)
to a valid DOS filename.  That file will be created and output intended for
stdout will go there rather than to the monitor.

   The filters created with FDK take advantage of DOS redirection.  They
actually get their input from stdin, and output the changed stream to
stdout.  You can verify this by running the filter called CAPS.EXE supplied
with the FDK.  If you run CAPS with no argument, the program will accept
anything you type at the keyboard as input, and will change any lowercase
letters to uppercase every time you press Enter (try it if you want; stop
the program with a Ctrl-C).  By using redirection, you can use any DOS file
as an input, and place the output in a new file.  If you have not used
redirection before, try using it to change sample text file provided with
the FDK to all caps.  Enter the following at the DOS prompt:

      CAPS <lower.txt >upper.txt

LOWER.TXT is a text file containing all lowercase characters.  After
running LOWER through the filter CAPS, all the letters will be changed to
uppercase and placed in the file UPPER.TXT.  LOWER.TXT itself is unchanged.
Any filter you create with FDK can be used in the same way.

    There is also one other way to use these filters - through the use of
pipes.  Pipes take their input from the output of another program.  Pipes
are specified by using the vertical bar (|) before the name of the program
that is to receive the output.  You can try out an FDK filter using a pipe
by entering the following at the DOS prompt:

      TYPE lower.txt |caps

The result is the same as in the previous example, except that the
transformed file is displayed on the monitor rather than saved to a file.
Using your filter with pipes is useful for viewing files that would
normally be unreadable.  You can also chain pipes, by placing them one
after the other on the command line.  Try entering the following:

    TYPE fdk.doc |caps |more

This will display this documentation file on the screen in all caps,
pausing after each screenful to wait for a keypress (more.com and caps.exe
need to be in the current directory or on the current PATH).

    For an example of the usefulness of a filter, suppose you have a
document created by your favorite word processor, and you want to take a
quick look at it.  Most word processors insert formatting characters into
the file, creating lots of garbage characters if you try to view the
document using the DOS type command.  Using FDK, you can create a filter
that either strips the high bit off of the control bytes, or removes them
altogether.  This way you can view any file created by your word processor
without loading the whole program.  The ASCII.EXE filter included with FDK
is an extreme form of this kind of filter.  It removes every non-ASCII
character, and also all non-printing ASCII control codes except for CR, LF,
FF, and SUB (Ctrl-Z, the DOS end-of-file character), from the input stream.
If you want to see how this filter was set up, start FDK using ASCII.EXE as
the template filter.




                                  Etc.


   The last filter supplied with FDK is one that remaps seldom used
characters in the ASCII set to single-line box drawing characters from the
IBM high-order character set.  This is useful for someone whose word
processor doesn't allow the high-order set to be used in a document.  Draw
the box using the characters below, and print the file to disk.  If your
program won't allow you to print to disk, use one of the utilities found in
the software libraries of most BBS and commercial computer services such as
GEnie, that automatically capture printer output and save it to disk.  Use
BOX.EXE to remap the characters to box characters, and use the DOS print
utility to print the new file.

      Box drawing characters:

        {        upper left corner

        }        upper right corner

        [        lower left corner

        ]        lower right corner

        |        vertical line

        \        horizontal line

        ~        left intersection

        `        right intersection

To verify this, enter:

    TYPE fdk.doc |box |more

Be sure that more.com and box.exe are either in the current directory or in
the current PATH.  Press any key until this table comes on the screen.
Now, instead of the ASCII characters, you'll see the box characters next to
their descriptions.  Of course, you can use FDK to change the character
assignments to any character you want, or to change them to double line
characters.  Also, you may have to experiment as to which characters can
be used to create boxes since printer control sequences will be imbedded in
the print image file.  If one of the above characters is part of a printer
control sequence, remapping it to a graphics character can cause the
printer to choke.  There is one potential conflict in the above characters
for an Epson printer - the \ character used for horizontal lines is part of
the control string used to set relative printhead position (ESC \).  This
will probably not cause a conflict, but if it does, try using the caret
(Shift 6) instead.  You'll need to check your printer documentation if you
encounter a problem.



   There are a lot of uses for filters, and it can be a pain to code an
entire program to create a special filter for one or two uses.  Since the
vast majority of filters are just simple substitution-type filters, FDK can
really save a lot of time and aggravation.  The examples shown are just a
scratch on the surface of the problems that can be solved using these kinds
of filters.

   If you have any questions or comments, or find any bugs, I can be
reached via my mailbox on GEnie (send mail to C.Greer).  I'm not very
consistent at logging onto GEnie, so it might take a week or so for me to
respond to a letter.  This never has been intended to be commercial
software or shareware or anything like that, just a way to learn the C
language, so feel free to use it or share it with friends (just don't
charge anyone for it).  The only thing I ask is that you keep the DOC file
and sample filters together with FDK.EXE, since FDK.EXE is less than
useless without a template filter.

    Whew!  Writing documentation is harder than I thought it would be.  I
hope someone finds this program useful like I did.  One last note:  I want
to give credit where credit is due.  The filter itself is based on a sample
filter in the book "Advanced MSDOS" by Ray Duncan.  I took his code and
modified it to use a swap table, strip or keep NULLs, and keep or expand
TABs (his expanded tabs all the time).  This is an excellent reference book
for experienced or inquisitive computerphiles (I'm the latter), and is
highly recommended.