Power Programming

home *** CD-ROM | disk | FTP | other *** search

/ Power Programming / powerprogramming1994.iso / progtool / filutl / brik2src.arc / BRIK.DOC < prev next >

Wrap

Text File | 1989-08-04 | 29KB | 705 lines

BRIK(1) USER MANUAL BRIK(1) NAME brik - calculate 32-bit CRC SYNOPSIS brik -h brik -gcGCbafvsqWHT file ... DESCRIPTION Brik generates and verifies 32-bit CRC values (checksums). It is designed to generate CRCs for text files that are the same on all computer systems that use the ASCII character set, provided each text file is in the usual text file format for each system. Brik will also optionally use binary CRCs calculated using every byte in a file. Such binary CRCs are portable across systems for binary files that are moved from system to system without any newline conversion. Brik can be asked to decide by examining each file whether to calculate a text mode or binary mode CRC for it. Changes from version 1.0 are summarized at the end of this document. The general usage format is: brik -gcGCbafvsqWHT [ file ] ... The brackets mean that file, which is the name of a file, is optional. The three dots indicate that more than one filename may be typed (separated by blanks). Exactly one of the options -c, -C, -g, -G, or -h, is required. The -h option gives a help screen. In addition to -h, the Brik options available (as they appear on the help screen) are: -g look for Checksum: header, generate CRC for rest of file -c get CRC from header, verify CRC of rest of file -G generate CRC for entire file (add -b for binary files) -C verify all file CRCs from output of -G (-f is not needed) -b use binary mode -- read file byte by byte, not line by line -a automatically decide whether each file is text or binary -f read filenames (wildcards ok) from specified files -v be verbose, report all results (else only errors are reported) -s be silent, say nothing, just return status code -q be quiet, don't print header for -G -W after generating CRC with -g, write it to original header -H after generating CRC with -g, print header to stdout -T include trailing empty lines, normally ignored (text mode only) VERIFYING CRC HEADERS The primary intended use of Brik is to verify Checksum: headers in Usenet postings and in C and Pascal source files. A Checksum: header looks like this: Checksum: 9485762645b (verify with "brik") The word Checksum: must occur at the beginning of a line. It is followed by a colon, an optional blank, a ten-digit decimal 32-bit CRC, and any arbitrary optional string such as the comment shown above. The CRC value may be followed by a one-character suffix identifying the type of the CRC. These suffixes are described later. When Brik is invoked with the syntax brik -c file ... it will search for the Checksum: header in each specified file, read the CRC value from that header, calculate the CRC-32 for all lines in the file (except trailing empty lines) that occur *after* the header line, and report an error if the two values do not match. CALCULATING CRC HEADERS The command brik -g file ... tells Brik to look for the Checksum: header in each specified file, calculate the CRC of the lines in the file following the header, and print the CRC and filename without changing the file in any way. You can also ask Brik to update the Checksum: header with the following command: brik -gW file ... This causes Brik to update the Checksum: header in the file with the newly-calculated CRC. If there is not enough space for the CRC in the existing header, Brik reports an error and does not change the existing header. To initially add a Checksum: header to a file, insert a line of the following form into the file with a text editor: Checksum: XXXXXXXXXX -- this is an optional comment The word Checksum must be at the beginning of a line. The ten `X' characters shown can be any ten characters. They will be later replaced by the calculated CRC value. They do not need to be `X'. The comment following them is optional and can be any arbitrary string of characters after the CRC field, separated from it by blanks. As an example, in a C source file called main.c, you might have: /* Checksum: XXXXXXXXXX (verify or update with brik) */ Once a header like this exists in the file, invoke Brik as follows: brik -gW main.c This will cause the ten `X' characters to be replaced with the calculated checksum, giving something like this: /* Checksum: 1922837484 (verify or update with brik) */ Later you can use the command brik -c main.c to verify that the contents of the file following the header have the same CRC as the stored value. If Brik is invoked with the -c or -g options and it cannot find a Checksum: header in a file, it prints a row of question marks. If it is invoked with the -gW option and it does not find enough character positions after the Checksum: string to hold the CRC, it does not fill in the CRC and prints an error message. Brik can be asked to generate a complete Checksum: header but print it to standard output instead of writing it to the file. Do this by adding the -H option. If both -W and -H are given, the Checksum: header will be written both to the file and to standard output. WHOLE-FILE CRCS A "whole-file" CRC calculation is done for the entire contents of a file, without looking for a Checksum: header. The command brik -G file ... asks Brik to do this calculation for all specified files. The output from this command is a list of files and their whole-file CRCs. It is sent to the standard output stream, which in most cases is your screen. The output should be saved in a file by redirecting it. For example, the command brik -G *.h *.c > crc.lst on an MS-DOS system might yield the following in the output file crc.lst: # CRC-32 filename # ------ -------- 2566277976 getopt.c 100594287 brik.c 1151475469 vms.c 3929503738 turboc.c 2424271922 addbfcrc.c 1943472856 assert.h 2601923477 brik.h The output of the -G option is in free format. The output file may be edited by hand. Empty lines and lines beginning with '#' will be ignored by Brik when it is later asked to read this file. This list of filenames and whole-file CRCs may be verified by a command like this: brik -C crc.lst This makes Brik read the file crc.lst, get the CRCs and filenames from it, do the CRC calculation again for each file, and report an error if it does not match the CRC recorded inside crc.lst. **IX and MS-DOS users (and others who can pipe the output of one command into another) could use a command like the following to see if Brik's -G and -C options are working: brik -G file ... | brik -C This invokes "brik -G" on some files, sending the output to a pipe where it becomes the input of "brik -C". If no filename is specified, Brik reads from standard input, so "brik -C" will read from the pipe and concurrently verify all the CRCs that are being generated by "brik -G". Whole-file CRCs are normally generated in text mode, in which a file is treated as lines of text. You can also generate whole-file CRCs in binary mode. See below. BINARY VERSUS TEXT MODE Brik can work in two different modes. The most common mode, used unless you specify otherwise, is text mode. In this mode Brik reads all files line by line, and forces each line of text to be terminated by a newline character of constant value before doing a CRC calculation. Thus, no matter what newline character is used by your system, the CRC calculation will be unaffected. This means that whether your operating system uses linefeeds to terminate lines (e.g. **IX), or a carriage return and a linefeed (e.g. MS- DOS) or a carriage return only (e.g. Macintosh) or nothing but a record length field (e.g. VAX/VMS), the CRC calculation in text mode gives the same results. If a file is not intended to be a text file, the text mode CRC calculated by Brik may not be reliable and may be different on different systems. If Brik is calculating a text mode CRC on a file that appears to contain binary data, it still calculates the text-mode CRC but adds a "*" character after the CRC to indicate to warn the user. Brik can be asked to operate in binary mode by adding a -b option. Binary mode is applicable only to the -G command, which acts on whole files. Thus brik -G file ... finds whole-file CRCs of all specified files in text mode, while brik -Gb file ... does the same in binary mode. In binary mode Brik simply reads and calculates the CRC for all bytes in a file. If a file is moved from one system to another without any newline conversion or any other changes, Brik should calculate the same binary mode CRC for it on both systems. The output from "brik -Gb" includes a trailing b suffix in each CRC, so "brik -C" will correctly use binary mode for such CRCs and it is never necessary to type "brik -Cb" instead. In its auto-check mode, Brik will itself examine each file and determine whether it is a text or binary file and calculate a CRC accordingly. To do this, use the -a option. Although Brik can determine the type of each file with a high degree of reliability, it is still possible for it to come to a wrong conclusion about some files. The output from "brik -Ga" will include a trailing b character in the CRC for those files that appeared to be binary to Brik. You may find both text and binary CRCs in the output. TRAILING EMPTY LINES The normal behavior of Brik in text mode is to ignore any trailing empty lines in a file. An empty line is a line that contains nothing, not even blanks or tabs. (Just hitting a carriage return when you are editing a text file usually produces an empty line.) If manipulating a text file causes trailing empty lines to be added or deleted, the CRC calculated by Brik will be unaffected. You can override this if necessary with the -T option, which makes Brik include trailing empty lines in the CRC calculation. For example, brik -gWT will update the Checksum: header with a CRC that includes all lines in a text file. Similarly brik -GT will find whole-file CRCs that include all lines in a text file. When Brik is given the -T option, it adds a T suffix to each generated CRC. Then, when the CRC is verified with -c or -C, Brik will correctly include trailing empty lines when needed without having to be explicitly told to do so. In binary mode Brik always reads all bytes in a file, so it does not ignore trailing empty lines, and the -T switch is not needed. The effects of the -T and -b options are mutually exclusive. If both are given, whichever is used later overrides the first. So -bT is equivalent to -T and -Tb is equivalent to -T. FILENAME CONVENTIONS Under MS-DOS and VAX/VMS, wildcards are allowed on the command line and will be expanded by Brik. Under **IX, wildcards are expected to be expanded by the shell and are not expanded by Brik. If no filename is given, Brik reads its standard input. By default this is keyboard input, but it is expected that input will usually be redirected to come from a file or a pipe. Also, if a filename is explicitly specified as a dash ("-"), it stands for standard input. The following five commands (valid under the **IX operating system) brik -c myfile # "myfile" brik -c < myfile # stdin = "myfile" cat myfile | brik -c # stdin = a pipe brik -c - < myfile # "-" = stdin = "myfile" cat myfile | brik -c - # "-" = stdin = a pipe all have the effect of verifying the Checksum: header in the file myfile. Standard input can also be read when using the -C option. For example, suppose we have already given the command brik -G *.c *.h > crc.lst to create a file called "crc.lst" that contains a list of whole-file CRCs. We can now verify the integrity of these files with any of the following commands: brik -C crc.lst # "crc.lst" brik -C < crc.lst # stdin = "crc.lst" brik -C - < crc.lst # "-" = stdin = "crc.lst" cat crc.lst | brik -C # stdin = a pipe cat crc.lst | brik -C - # "-" = stdin = a pipe INDIRECT FILE NAMING The -f option allows you to have filenames in a list rather than on the command line. For example, suppose you want to generate whole-file CRCs of files whose names are selected by some other program. You could use find (under **IX) or Stuff (under MS-DOS) to generate a list of filenames. When the -f option is given, Brik reads filenames, one per line, from the file(s) specified. Thus you could do the following: find . -mtime +3 -print > flist brik -Gf flist > crc.lst The first command asks find to generate a list of names of all files that were modified more than 3 days ago, and sends the output to the file flist. The contents of flist might now look like this, as an example: ./sez.doc ./brik.doc ./stuff.doc ./looz.doc The second command above asks Brik to read the file called flist, get a list of filenames from it, one per line, and generate the whole-file CRC of each of these files. We additionally redirect the output of "brik -Gf" to the file called crc.lst. Given the above contents of flist, the following two commands are exactly equivalent: brik -Gf flist >crc.lst brik -G ./sez.doc ./brik.doc ./stuff.doc ./looz.doc >crc.lst The advantage of the -f option is that once you have filenames in a file, you need not type them again at the command line. The -f option also allows you to feed Brik through a pipe filenames generated by another program. Consider this command: find . -mtime +3 -print | brik -Gf - > crc.lst Under **IX this concurrently invokes both find and Brik. As find generates a filename and sends it to its standard output (a pipe), Brik reads the filename from its standard input and generates its whole-file CRC. The following pipeline find . -mtime +3 -print | brik -Gf - | brik -C - invokes find to generate filenames, Brik to print the whole-file CRCs of those files, and Brik again to immediately verify them. Under MS-DOS and VAX/VMS (and any other operating system under which Brik has been installed to expand wildcards itself), a file specified by the -f option can itself contain wildcards in the filenames. For example, suppose the file "wildfile" contains the following lines: /bin/*.exe /bin/*.com /doc/*.doc Now if we invoke Brik with the command brik -Gf wildfile it will read filespecs from "wildfile," expand wildcards in each filespec, and generate whole-file CRCs for all matching files. If you are checking whole-file CRCs with the -C option, you do not normally need to use the -f option. When used with -C, the -f option introduces an additional level of file naming indirection. For example, the command brik -C crc.lst takes a list of CRCs and filenames from "crc.lst" and verifies them. However, the command brik -Cf master.lst does not do the same thing. Instead, it takes a list of files from "master.lst," looks inside each of those files for a list of CRCs and filenames, and verifies them. As an example of the use of -Cf, consider this sequence: brik -Gb /bin/*.exe > exelist brik -Gb /bin/*.com > comlist brik -GT /doc/*.doc > doclist brik -G /doc/*.man > manlist Now we have four files called "exelist," "comlist," "doclist," and "manlist" containing whole-file CRCs of various types. We can test the integrity of files listed in these in four separate commands like this: brik -C exelist brik -C comlist brik -C doclist brik -C manlist But we could also do this in a single step by first creating a file called "biglist" that contains the names of these four files: exelist comlist doclist manlist and then use -Cf thus: brik -Cf biglist This causes Brik to read filenames from "biglist," look inside each of those files ("exelist," "comlist," "doclist," and "manlist") and check the CRCs found there. A **IX example to do the same thing in a more compact way might be: cat exelist comlist doclist manlist | brik -Cf - The above examples are somewhat contrived, however. We could also use the following command: brik -C exelist comlist doclist manlist SILENT VERSUS VERBOSE VERSUS QUIET Brik accepts options -s, -q, and -v that control the degree of verbosity. Normally Brik prints a message only if it detects an error. For example, if you give the command brik -c *.c *.h and Brik finds that all Checksum: headers contain CRCs that match the calculated value, it prints nothing. The -v switch makes Brik print a message for each file tested that contains the word "ok" if stored and calculated CRCs matched and "BAD" if they did not. In all messages reporting file CRCs, Brik prints the actual CRC it calculated, or a row of question marks if it could not calculate one. Brik fails to calculate a CRC if it is trying to calculate a header-based CRC (commands -c and -g) but does not find a Checksum: header in a file. The -s switch tells Brik to be silent. This will cause nothing to be printed if Brik does not find a Checksum header or if CRCs don't match. However, the status code on exit is still set. The -s option is useful for invoking Brik from within a shell script (**IX) or a batch file (MS- DOS) and later testing the error code it returned when it exited. The -s switch does not suppress the error message printed if Brik is given a filename and the file cannot be found. For example, if the command brik -cs myfile is given but "myfile" does not exist, Brik will print an error message even though the -s option was given. The -q switch makes Brik slightly quieter. Its only effect is to suppress the introductory comments that are otherwise generated by the -C option. VAX/VMS BUGS o Under VAX/VMS, file manipulation is unpredictable and the VAX/VMS C runtime library in particular is very unreliable. One result is that under VAX/VMS, the -gW option will work only for stream-LF files, which are the only type of file that VAX/VMS seems to handle correctly. Fortunately, the -c option will correctly work for VAX/VMS standard text files. o The VAX/VMS implementation of Brik simply ignores filenames that correspond to nonexistent files instead of giving an error message. VMS users are invited to fix this bug (probably somewhere in the nextfile() function in the file vms.c) and send me the fix. o To avoid annoying error messages, Brik under VAX/VMS always exits with a status code of 1. o Due to a problem in the VAX/VMS command interpreter, if any uppercase option characters are typed (e.g. -C, -G, -T), Brik will not recognize them as uppercase unless they are enclosed in double quotes. An example of a correct command under VAX/VMS: brik "-GT" *.* An example of a command that won't work: brik -GT *.* USAGE HINTS This section discusses some ways of using Brik. - Brik is currently used to create and verify checksums of articles posted on the Usenet newsgroup comp.binaries.ibm.pc. While reading Usenet news with rn, for example, you can verify the Checksum: header of such an article with the command: | brik -cv This feeds the current article to Brik as standard input and asks it to verbosely verify the checksum. - C and Pascal source files that are being distributed can be given their own Checksum: headers. The advantage of this over listing CRCs separately is that each file contains its checksum no matter where it goes. The recipient can easily verify that each source file was received intact. This is especially valuable when source files are being sent by electronic mail through an IBM mainframe-based network, such as BITNET, that can cause corruption of text files. Create the Checksum header with a text editor and initialize its value with the command: brik -gW file ... The recipient can verify the checksum with the command: brik -cv file ... - To keep track of any unexpected filesystem corruption or unauthorized changes to system files, Brik can be used to create a list of checksums. This should be done using the -b option (all binary checksums) or the -a option (use text or binary as appropriate). Under **IX this can be done with a command like this one: find /bin /usr/bin /lib /usr/lib /usr/new \ -type f -print | \ grep -v '^/usr/bin/beta' | \ grep -v '^/usr/new/lib/news' | \ brik -Gbf - > crc.list This example uses find to print the pathnames of certain files, grep to filter out certain directory subtrees, and then Brik to print checksums of all selected files. Periodically a background job can be run to compare the file list against current files and report any discrepancies: brik -C crc.lst | mail -s 'brik output' postmaster Under MS-DOS, create a list of checksums with the help of Stuff, a find-like file finder program that is distributed with zoo 2.01: stuff /bin /msdos /turboc | brik -Gaf - > crc.list This is a simple mechanism that can be used to detect unexpected file changes caused by hardware problems or malicious programs such as viruses and worms. Be warned, however, that since Brik uses a published CRC algorithm, very clever deviant programs are possible that could change a file without affecting its CRC. To avoid too many false alarms, only files that do not change much (such as system binaries) should be selected. CHANGES Changes from version 1.0 are as follows. - The CRC calculation has been recoded in 8086 assembly code suitable for Turbo C/MS-DOS. As a result Brik 2.0 under MS-DOS is much faster than version 1.0. - The new -a flag makes Brik automatically determine for each file whether it is text or binary and calculate the CRC accordingly - When Brik is asked to calculate a text mode CRC but the file appears to be binary, it now does not print a warning message but instead adds a "*" suffix to the calculated CRC to indicate that the CRC may not be reliable. - The new -H flag makes Brik print the formatted Checksum: header to standard output. - Detection of binary files is now more reliable under MS-DOS. If a control Z character occurs near the beginning of a binary file, Brik will not be fooled by the apparent end-of-file but will in most cases reliably detect that the file is binary. - The new -q flag suppresses the introductory comments from the output of the -G command. BUGS Brik is designed to work with computer systems that use the 7-bit ASCII character set and 8-bit bytes with the eighth (parity) bit set to zero. Brik has not been tested with computer systems that use ASCII characters with the eighth bit set or those that use EBCDIC characters. Although it will calculate a CRC on such machines, the probability of this CRC being the same as the one calculated on machines that use 7-bit ASCII is approximately 0.00000000023. DIAGNOSTICS Error messages are intended to be self-explanatory. The exit status returned is 1 if Brik was invoked with incorrect arguments, else it is the number of files found with missing or invalid CRCs, but not greater than an operating-system-dependent maximum value. COPYRIGHT Both this documentation and Brik are Copyright 1989 Rahul Dhesi, all rights reserved. Permission is granted to copy, use, and distribute for any commercial or noncommercial purpose in accordance with the requirements of version 1.0 of the GNU General Public license. Note: This software has not been endorsed by the Free Software Foundation, the creator of the GNU license, and I am not affiliated with that organization. AUTHOR Rahul Dhesi UUCP: {iuvax,pur-ee}!bsu-cs!dhesi Internet: dhesi@bsu-cs.bsu.edu BRIK(1) USER MANUAL (1989/07/11)