% $Id: txtcut.tex 1.4 1998/02/07 19:35:00 brian Exp $

\chapter{txtcut---Text Preprocessor}

Text files that contain blank lines, comments preceded by \verb+#+
(pound signs), a varying amount of whitespace between tokens, and
other features to make them well-documented and readable (such as
INI-style ASCII text files) are not easily processed by a Korn shell
or REXX script and can even require a bit of extra work for Perl
scripts. Additional logic can be used to skip over blank lines and
remove comments, but this logic adds undesirable complexity and slow
performance to the script.

One solution is \textbf{txtcut}. This program prepares a text file,
stripping out comments and blank lines and handling simple strings. It
was originally developed as a fast preprocessor for the AIX
\textbf{cut} command. However, the output of \textbf{txtcut} can be
piped into \textbf{rxqueue}, giving OS/2's REXX the ability to easily
and quickly handle text-style INI-like files. Even \textbf{awk} and
Perl (of which excellent Win32 and OS/2 ports exist) can benefit from
\textbf{txtcut}.

Information can be stored in an INI-style text file in an easily-
maintained and readable fashion using comments, strings, and blank
space. The \textbf{txtcut} program preprocesses this text file,
removing comments and blank lines, processing simple strings, and
delimiting the tokens in a consistent manner. The tokens produced by
\textbf{txtcut} can be easily extracted using the AIX \textbf{cut}
command, a REXX script, or even an \textbf{awk} or Perl script.

The REXX, awk, Perl, and the Unix shells cover a lot of ground, but I
have wished for a long time for the capabilities of \textbf{txtcut} to
make the basic AIX, DOS, Win32, OS/2 toolsets more complete. I even
maintain an AIX port of \textbf{txtcut} which I use daily and on which
the DOS, Win32, and OS/2 ports are based. Text INI-style files that
are highly readable by people \textit{and} easily processed by command
scripts: \textbf{txtcut} gives you the best of both worlds!

\section{Usage}

\verb+txtcut [ -dchar ] [ -n ] [ -l ] [ -c ] [ textfile ]+
\medskip

This program prepares a text file for the AIX \textbf{cut} command. If
the name of the text file is missing, then \textbf{txtcut} reads from
standard input.

For each line that contains one or more tokens, \textbf{txtcut} writes
one line to stdout that contains the tokens. A delimiter character is
placed between each pair of tokens.

The default delimiter character is a tab character. You may use the
\texttt{-d} flag to change it to another character.

\section{Flags}

\begin{itemize}

    \item[\texttt{-d\#}]

        Specify a character for \verb+#+ to be used as the token
        delimiter. This character is specified just as it is for the
        \textbf{cut} command. The default is a tab character.

    \item[\texttt{-n}]

        List the filename as the first token in each line. If the text
        file is being read from standard input, then the filename is
        listed as ``\verb+(stdin)+''.

    \item[\texttt{-l}]

        List the line number within the file as a token.

    \item[\texttt{-c}]

        List the number of tokens on the line as a token. This count
        does not include the additional tokens that may be added by the
        \texttt{-n}, \texttt{-l} or \texttt{-c} flags.

\end{itemize}

\section{Input Text File Format}

Blank lines are ignored. Lines that begin with a \verb+#+ or \verb+;+
are assumed to be comments and are ignored.

Each nonblank, noncomment line consists of one or more tokens. A token
can be:

\begin{itemize}

        \item An \verb+=+ (equal) sign.

        \item A string of characters enclosed by either double
        quotes, single quotes, or square brackets.

        \item A series of contiguous nonblank characters.

\end{itemize}

If a non-string token that begins with a \verb+#+ or a \verb+;+ is
encountered on a line, it and the remaining tokens are assumed to be
comments and are ignored.

\section{Examples}

\begin{enumerate}

%====================================================================

\item Assume that the \textbf{sample.txt} file contains the following:

\begin{verbatim}
# This is a comment line

        f1 = a b c    # First line of tokens
        x y z         # Second line of tokens
        aaa bbbb  "cccc   ddddd"  # Last line of tokens
\end{verbatim}

Run the following command:

\begin{verbatim}
        txtcut -d: sample.txt
\end{verbatim}

The \textbf{txtcut} command writes the following to stdout:

\begin{verbatim}
        f1:=:a:b:c
        x:y:z
        aaa:bbbb:cccc   ddddd
\end{verbatim}

%====================================================================

\item Using the same \textbf{sample.txt} file from the previous
example, we'll cut the third field from each line of the file by
running the following command:

\begin{verbatim}
        txtcut sample.txt | cut -f3
\end{verbatim}

The following is written to stdout:

\begin{verbatim}
        a
        z
        cccc   ddddd
\end{verbatim}


%====================================================================

\item Again, using the same \textbf{sample.txt} file from the previous
examples, we'll cut the third field from each line of the file. But
this time we'll pipe the file into \textbf{txtcut}'s standard input
and we'll use a colon for the delimiter:

\begin{verbatim}
        cat sample.txt | txtcut -d: | cut -f3 -d:
\end{verbatim}

Again, the following is written to stdout:

\begin{verbatim}
        a
        z
        cccc   ddddd
\end{verbatim}

%====================================================================

\item Using the same sample.txt file in example 1, we will add the
filename, the line number within the file where each line of tokens
was found, and the number of tokens in the line. The following
shows a set of commands and the output from each of them:

\begin{enumerate}

\item \verb+txtcut -d: sample.txt+

\begin{verbatim}
f1:=:a:b:c
x:y:z
aaa:bbbb:cccc   ddddd
\end{verbatim}

\item \verb+txtcut -d: -n sample.txt+

\begin{verbatim}
sample.txt:f1:=:a:b:c
sample.txt:x:y:z
sample.txt:aaa:bbbb:cccc   ddddd
\end{verbatim}

\item \verb+txtcut -d: -l sample.txt+

\begin{verbatim}
3:f1:=:a:b:c
4:x:y:z
5:aaa:bbbb:cccc   ddddd
\end{verbatim}

\item \verb+txtcut -d: -n -l sample.txt+

\begin{verbatim}
sample.txt:3:f1:=:a:b:c
sample.txt:4:x:y:z
sample.txt:5:aaa:bbbb:cccc   ddddd
\end{verbatim}

\item \verb+txtcut -d: -n -l -c sample.txt+

\begin{verbatim}
sample.txt:3:5:f1:=:a:b:c
sample.txt:4:3:x:y:z
sample.txt:5:3:aaa:bbbb:cccc   ddddd
\end{verbatim}

\end{enumerate}

\end{enumerate}
