Programmer 7500

home *** CD-ROM | disk | FTP | other *** search

/ Programmer 7500 / MAX_PROGRAMMERS.iso / PROGRAMS / UTILS / COMPRESS / NARC25.ZIP / NARC.DOC next >

Wrap

Text File | 1988-12-31 | 77.7 KB | 1,977 lines

NARC - A STAND-ALONE DE-ARCHIVE UTILITY (no other files required) Documentation for NARC.EXE Written by Gary Conway Infinity Design Concepts, Inc. Louisville, Kentucky Copyright (c) 1987-89, All rights reserved Version 2.5 NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. NARC.EXE is a shareware program. NARC.EXE is and will remain the property of Gary Conway. This program may not be used in any connection with commercial ventures, nor as a sales aid, without the expressed written consent of the author. All rights are reserved. Infinity Design Concepts, Inc. 1052 Parkway Drive Louisville, Kentucky 40217 (502)-636-1234 Member IEEE PCCL KKUG NSPE All new releases of NARC.EXE and all other IDC software can be located -FIRST- on ; The SoftStone FOG #24 (supporting CP/M and MSDOS) (502)241-4109 60 MEGs 300/1200/2400 baud 24 hrs. Louisville, Kentucky Curt Edwards - SYSOP Sponsored by: Kentucky Kaypro Users Group Accounting Computer Systems First Osborne Group NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. REGISTRATION If you find yourself using NARC, please take the time to do the right thing and that is register your copy. You have been provided the opportunity to freely test the program before even thinking about registering. This is only fair, so, in fairness, you should reciprocate and register your copy, if you continue using the program. Why register ? 1) You get the NARCCFG.EXE program for customizing NARC. 2) You get notification of updates to all IDC software. 3) You get patch table information. 4) You get FREE net-mail services for contacting IDC. Disk only with current version .................. $20.00 (includes manual on disk) Printed manual .................................. $15.00 (bound and printed manual only, NO REGISTRATION) Printed manual and reg. disk .................... $35.00 Site License .................................... $50.00 (required for business use) (applies for up to 20 machines) (please call for quantity discounts) Registered users can obtain update disks for $10.00. Customers residing outside of North America, should add $5.00 U.S for airmail postage. Add $3.50 for shipping and handling. Customers residing outside of the United States of America should send a check or money order drawn on a U.S. bank or in U.S. funds or use a credit card. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. You will find the registration form in the ARChive with this document under the name REGISTER.FRM. Please use this form for registration. Infinity Design Concepts, Inc. 1052 Parkway Drive Louisville, Kentucky 40217 (502)-636-1234 The user is granted the right to make unlimited copies of the SHAREWARE versions of the program (except NARCCFG.EXE and NARCCFG.DOC) and to distribute these copies as desired without profit, EXCEPT that Infinity Design Concepts, Inc. reserves the SOLE right to distribute the program(s) for profit. NOTE: NARCCFG.EXE is NOT a shareware program and may not be released to the general public in any form under any circumstances whatsoever. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. ............................................................................. ........................ ............................ ........................ TABLE OF CONTENTS ............................ ............................................................................. Page FEATURES UNIQUE TO NARC..............................1 WHAT IS IT ANYWAY.................................. 2 ACKNOWLEDGEMENTS................................... 2 COMPATIBILITY...................................... 3 Author's Ramblings............................. 3 ARChive Storage Methods Supported By NARC...... 3 Packing Squeezing Crunching Squashing OVERVIEW........................................... 4 Video Modes ................................... 4 Mouse Support ................................. 4 Sound ......................................... 5 Environment Variables.......................... 5 HELP............................................... 7 COMMANDS........................................... 7 Extract Command................................ 7 View Command................................... 8 Print Command.................................. 8 ARC-wind Command............................... 9 DRV-wind Command............................... 9 DirTree Command..........* NEW *............ 9 Quit Command................................... 10 ALTERNATE COMMANDS................................. 10 Function keys.................................. 10 Find Command................................... 11 Kill File Command.............................. 11 Page UP, DOWN, HOME,END........................ 11 Using extensions besides .ARC and .ARK......... 11 Calling IDCshell.EXE........................... 11 Command Line Processor......................... 12 Trivia status screen........................... 12 OPERATING HINTS AND SHORTCUTS...................... 12 ERROR MESSAGES..................................... 13 ARCHIVE FILE FORMATS AND GENERAL INFORMATION....... 14 Packing........................................ 15 Huffman coding (SQUEEZING)..................... 16 Crunching (LZW Compression).................... 20 DETAILS OF STORAGE VERSIONS........................ 22 ARChive file Header Structure.................. 24 HASHING............................................ 25 CRC - calculations............................. 25 ARC RELEASE DATES AND VERSIONS..................... 26 ........................................................................ Narc (c) 1987-89 Infinity Design Concepts, Inc. all rights reserved NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. ........................................................................ NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. ═══════════════════════ FEATURES UNIQUE TO NARC ═══════════════════════ 1) allows random access to archives. ALL of the other archive utilities allow ONLY serial access to archives. 2) allows viewing DOC files before extraction. This feature allows you to determine which, if any of the subfiles that you want to extract. This can save considerable time and disk clutter when you extract only the necessary files. 3) has a view feature that allows page up, page down, home end and arrow key movement through a file. Binary files can also be viewed in HEX mode. Also includes a text search function. 4) allows printing files from an archive without extracting the file first. Supports many printing options. 5) allows users with floppy systems to load NARC once and examine many archives from all floppy drives. Other archive utilities must reload (and reread the arc) for each command. 6) directory tree allows one to easily move from directory to directory or drive to drive. 7) large archives are much easier to deal with in NARC since you may page through the subfiles. 8) mouse support for all functions. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 1 ═══════════════════ WHAT IS IT ANYWAY ? ═══════════════════ NARC is a menu driven de-ARChive facility, written entirely in assembler. NARC allows you to easily move from ARC file to ARC file, with the option of viewing, printing, extracting or deleting the subfiles from the ARChive. The program may be operated from the mouse or the keyboard. Menus are of the musical popup variety to add a little "TechNoFlash" to the proceedings. NARC is the culmination of about 18 months of frustrating effort and 13000 + lines of 8088 source code. NARC was developed using the Norton Editor and the Microsoft Macro assembler on a Kaypro PC and an ALR 386/20. Why.... Because I use a lot of ARC files and ARC.EXE and the clones are reminiscent of the early Ward Christensen CP/M days in user interface etiquette, I wanted something a little more flexible and friendly to use. I would like to pause here for a second and give a little credit to Mr. Christensen ( the Don Garlits of CP/M ) for the fine FREE utilities he has given to ALL of us over the years. The next time you do a modem transfer, you can thank him and Keith Petersen for the original XMODEM from which all others have transpired. Why NARC... It seemed like a good idea. Short for uN-ARC. The idea was originally Bob Freed's. Acknowledgments.. I would like to thank my parents for not killing me at any early age and my wife for not killing me at a later age... I would like to thank Bob Freed for his allowing me to examine his Z80 code before writing NARC. Bob wrote UNARC for the CP/M world and is ( as of this writing 4/28/87) working on NOAH the ARCing program for CP/M. I would also like to thank System Enhancement Associates for releasing their source code in "C". Without both of the above, NARC would have been a much larger chore than it was. Note also that the crunching algorithm used in ARC.EXE was taken from COMPRESS, used in UNIX. A special thanks to the following kind folks. Curt Edwards Jerry Taylor Chuck Crumpton Frank Roemer Paul Bowling Dr. Matthew Witten Paul Clarke Kevin Able Carlisle Crutcher H.W. Straley John R. Gernert Mr. Dudley Hoffman Dr. Melvin Maron NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 2 ═════════════ COMPATIBILITY ═════════════ NARC is compatible with all known "skrunching" algorithms, that is up to and including Squashing. NARC is compatible with ARC.EXE version 5.21 and PKxARC. NARC supports Squashing which is nothing more than a variation of the crunching algorithm, yet it is the easiest (and most logical) of the crunching methods to code. I have heard a lot of criticism of squashing, but those folks need to get up with the times, squashing is (and should be) here to stay. NARC also recognizes the .ARK extension soon to be prevalent in the CP/M world via Bob Freed's CP/M archive facility, NOAH. Author's Ramblings System Enhancement Associates, I am told is dropping the ball as far as ARC.EXE goes. I think that Thom Henderson deserves a great round of applause for his contribution and help with a formidable problem, namely, storage space (or lack thereof). The oldest version of ARC.EXE that I can find is version 3.10, released 5-1-85. This version supports storage methods up to and including squeezing (no crunching). If anyone has an older version I would be interested in seeing it. Here is a list of the versions that I do have and would be interested in getting any other versions floating around. 3.10 4.10 4.50 4.52 5.00 5.10 5.12 5.20 ═════════════════════════════════════════ ARCHIVE STORAGE METHODS SUPPORTED BY NARC ═════════════════════════════════════════ Packing - all versions Squeezing - Huffman Coding Crunching - all versions (LZW encoding) Squashing - one version Note: LZW stands for Lempel-Ziv-Welch NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 3 OverView... When NARC is first invoked, it saves the current drive/path for use again on exit, so you always end up where you started. Some folks like that and some don't. I DO. NARC first searches the default path for ARC/ARK files and if any are found they are displayed in a window on the left side of the screen. The arrow keys (or the mouse), PgUP,PgDN, Home and End may be used to move the cursor bar up and down the window, there are two ways to select the highlighted ARC file. (1) Hit the ENTER key (2) Press the left mouse button After selecting the sub-file of interest, NARC displays all of the ARC sub-files and their statistics on the screen. You are also given a menu bar at the bottom of the screen. You may use the arrow keys or the mouse to move the cursor bar to the desired selection and then select with the ENTER key or the left mouse button. As the cursor bar is moved, you are also given a brief description of the highlighted command. A file may be located in the list by the now common ALT-letter key combination. This applies to all menus in NARC. The commands will now be discussed individually. Note: You may also select any option from the command bar by entering the first letter of the command. The ESCape key will abort any operation or window. Video Modes NARC will determine whether a monochrome or color video adapter is being used and act accordingly. If a CGA or EGA card is detected, then NARC will select 80 x 25 color mode, unless color has been turned off with NARCCFG. The users video mode and cursor shape is saved when NARC is invoked and restored upon exit or shell. Mouse Support NARC will determine if a mouse driver is present. If so, then NARC's mouse support routines are enabled and working at all times within the program. The left mouse button ALWAYS emulates the ENTER key (for selecting highlighted items) and the right mouse button ALWAYS emulated the ESC key (for aborting operations). Highlight bar movement can always be accomplished with mouse motion. The sensitivity of mouse motion can be changed with NARCCFG. The mouse routines were written with the Microsoft mouse in mind and any mouse that is compatible with the Microsoft mouse should work. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 4 Sound NARC makes use of the 8253 programmable interval timer chip and speaker port to create some gurgles,bleeps,whistles and burps in general, that some folks like and some don't. The chip is programmed for square wave output in mode 3. The sound may be permanently disabled with NARCCFG. NARC determines microprocessor speed and adjusts the sound routines accordingly so that the sounds are pretty much the same on all machines. ENVIRONMENT VARIABLES If you don't know what an environment variable is, then continue reading. If you DO know, then skip to "The Goods" section below. What are they ? An environment variable is a convenient way for programs to communicate with DOS or with each other through a commonly shared spot(s) in memory. If you type "SET" <ENTER> at the DOS prompt, DOS will respond by showing you the current spot of shared memory which we call the environment. These text strings can be created,read, modified and deleted at any time by any program. Each variable has a certain format that must be adhered to. The format is as follows, variable_name = string An example would be IDCED=C:\UTILITY\WS Spaces are allowed before and after the equal sign. This variable will mean absolutely nothing to DOS, but, to NARC, it will show the way to your editor. The variable name here is IDCED and the string is C:\UTILITY\WS which is the pathname for finding WordStar on the disk. You can change this string to anything you want by simply typing --- SET IDCED= (and type your pathname for your editor). NARC uses several of these environment variables to communicate with you and you should set them up in your AUTOEXEC.BAT file. What do I do if DOS says, "Out of Environment Space" ? DOS reserves only so much space for the environment strings, and if you have a long search path or a lot of strings, then you may run out of space. I can offer two suggestions, one for DOS 3.1 and one for DOS 3.2. There is a "switch" that you can add to your CONFIG.SYS file (If you don't have one, then you should !). One of the lines in your config.sys file should read Shell=C:\COMMAND.COM or something similar. You can add the following to the end of this line to increase your environment space. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 5 DOS 3.1 - /E:nn where nn = 10 up to 62. This number times 16 is the length of your environment. example: Shell=c:\command.com c:\ /P/E:10 This would reserve 160 bytes. DOS 3.2 - add /E:nnnn where nnnn is the actual number of bytes to reserve. example: Shell=c:\command.com c:\ /P /E:160 This would reserve 160 bytes. Environment variables can be set up in your Autoexec.bat file as follows. Set IDCARCPATH=c:\utility\idcshell.exe Set IDCED=c:\wordproc\editor.ext The Goods .. IDCED=c:\editors\ws.exe Use this variable to tell NARC where your editor is located, so that when the F9 key is pressed to invoke the editor, NARC will know where to locate it. The drive and path and editors name must be given (in upper or lower case). The file extension (.COM or .EXE) MUST be given. IDCARCPATH=c:\utility\idcshell.exe Use this variable to tell IDCshell where to find IDCshell when the "I" key is pressed. As with IDCED, the drive and path and filename and extension must be given. NARCHELP=c:\utility\narchelp.hlp Use this variable to tell NARC where to find the NARC help file when the F1 key is pressed. IDCTEMP=e:\ramdisk Use this variable to tell NARC where it can place temporary work files. If you have a ramdisk set up, then this would be a good use for it since all file operations will be MUCH faster from the ramdisk. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 6 THE HELP SCREEN From the files screen, it is always possible to get a context sensitive help screen. Hitting the F1 or F10 key will cause a full screen of help to pop up that has information specific to the command that the cursor bar was pointing to. Once the help window is opened up, it is then possible to get help on any of the commands. The Home key will take you to the help index (as will the "I" key). The window may be put at any time with the ESC key. NOTE: There are several commands that are listed under miscellaneous commands in the INDEX section of the help screens, so be sure to check them out. ════════ COMMANDS ════════ ═══════════════ Extract Command ═══════════════ Selecting this option will cause another prompt to be displayed, asking whether you wish to extract the highlighted file or tagged files. (Files are tagged with the space bar). "Point and shoot" here again as before. The ESC key or the right mouse button will abort the operation. If the disk becomes full, you will be informed and have the option of aborting or continuing. Highlighted File When EXTRACT is selected, you will be asked for a drive/path to extract the file to. If you simply hit ENTER or the left mouse button, the file will be extracted to the default drive/path. You may also enter any valid DOS drive/path. The ESC key or the right mouse button will abort the operation. NOTE: Control-R can be used to recall the last path that files were extracted to. Tagged Files The Space bar (or F3 key) is used to TAG the current file. When a file is tagged, a diamond will be displayed on the line with the current file in column 80. The space bar will also unTAG a file, thus it is a "toggle". When the space bar is pressed, a diamond will appear as described above and the cursor bar will move to the next file. When the "TAGGED" option is selected from the command line, all files that have been tagged, will be extracted to the SAME drive/path. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 7 After the file is extracted, it's date and time are set to those contained in the ARC file. The file is also checked for size and CRC, if both of these do not match exactly what was contained in the ARC file header, then an error has occurred and the user is notified. The files will also remain tagged after the extraction. ════════════ View Command ════════════ This option will display the currently highlighted file on the screen. The PgUP, PGDN, Home and End keys, as well as the cursor keys allow movement through the file. The file to be viewed is first extracted to the default drive (or to whereever the IDCTEMP variable points) to a file called NARC.TMP. This file is deleted when the view is ended. The extraction is performed due to the sequential nature of ARC files, which makes it nearly impossible to perform the page up,down operations on the compressed file itself. If NARC finds that there is not enough disk space or directory space to create NARC.TMP, you will be asked for another drive where the temporary file can be created. The newest addition to the view command in NARC 2.1 is the text search command. Simply hit "S" and you will be prompted for a string to find. When located, the text will be highlighted on the screen, otherwise a message is issued stating that the text was not found. The search is case insensitive, that is you may enter upper or lower case letters and NARC will find a match. The mouse is also supported now in the view function. ═════════════ Print Command ═════════════ The print option will print the currently highlighted file. After selecting the print option, you will be asked which character pitch you want to print in. Enter the number that you wish (or 0 for the default pitch) and the printer will be set to that pitch. NOTE: The printer strings that come installed with NARC are compatible with EPSON printer strings. If you wish to install NARC with your own strings, see NARCCFG.DOC for complete instructions. After selecting selected the printer pitch, you will be shown three more options for formatting the printout. Use the arrow keys to move left and right. The space bar is used to toggle the options ON or OFF. When you have finished and are ready to print, hit ENTER (or left mouse button). The ESC key or the right mouse button will abort at any time. The options are; NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 8 Format - YES - This option causes NARC to format the output with page breaks and page numbers. NO - NARC does not format the file. The following two options work independently of the Format option. Strip High - YES - NARC will strip the high bit off each character before it is sent to the printer. Some word processors set this high bit on some characters as a means of text formatting. These characters will print as garbage usually. NO - NARC will not strip the high bit. Strip Control - YES - NARC will strip all control characters from the file before it is printed. This is useful on files that have embedded formatting characters, and you wish to have NARC provide the formatting. NO - NARC will not strip the control chars. NOTE: On all of the following windows, the PG UP, PG DN, HOME and END keys in addition to the cursor keys allow movement through the window. ════════════════ ARC-wind Command ════════════════ This option will display the window containing all of the ARC/ARK files in the current sub-directory. Move the cursor bar up and down with the mouse or arrow keys and select with the ENTER key or left mouse button. ════════════════ DRV-wind Command ════════════════ This option will pop up a window that contains all of the logical drives that DOS reports to NARC. Select as before and the ARC-window will be popped up so that you can then choose an ARC file to examine. ═══════════════ DirTree Command ═══════════════ This command is totally new with release 2.1. The old window approach won a lot of criticism and rightfully so. I had a tree routine that I wrote some time ago, that I just wasn't happy with speed-wise and didn't want to stick that thing in NARC. I did have some nice ideas for a new routine that would be much faster and slicker all around and finally found the time to write it and NARC release 2.1 has it, you may judge for yourself. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 9 A visual tree is presented that displays all subdirectories on the disk. Once the tree has been read, NARC will not read it again, unless you have changed drives. From the tree, you can create and delete subdirectories, select a new drive or subdirectory or return with no action at all. The ESC key will abort the tree operations. ════════════ Quit Command ════════════ The exit options have been changed so that now you will be prompted for where you wish to exit the program. You have the choice of exiting to the current directory or the original directory where NARC was invoked from. This can be made totally automatic with the ARCCFG program in the registered version. ══════════════════ ALTERNATE COMMANDS ══════════════════ The extra commands can be located on the help screens, which are invoked by the F1,F10 or ? keys. F1 - key Invokes the context sensitive help system. F2 - key This keys function varies with the window that is on the screen at any one time. When an ARC file is opened and the subfile list is onscreen, this key will pop up the ARC file window again. Any other use of this key is given at the appropriate time on the screen. F3 - key This key will tag a subfile and move the cursor bar on to the next subfile. This key also has other functions, and they are also shown on the screen when necessary. F4 - key The F4 key will print an image of the screen less the advertisement and command lines. F5 - key Invokes the NARC-DOS command processor. You may then enter any valid DOS command. When finished, simply hit ENTER by itself and you will be returned to NARC. You may also enter "COMMAND" which will invoke a second copy of COMMAND.COM, if the file COMMAND.COM is in your search path. To return to NARC, you would then type "EXIT". NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 10 F6 - key This key will tag all of the subfiles in the ARChive for subsequent extraction. F7 - key This key will invert all of the tags on the subfiles, that is all files that were tagged will become untagged and vice- versa. F9 - key This key will call your editor. Note that you MUST have first set up the environment variable IDCED before this function can be used. See the section on environment variables. F10 - key This key will call up the context sensitive help system. ALT-F10 - key This sequence displays the trivia screen, where your serial number and other information is located. (F)ind command. Will prompt for a wildcard filename to find in the sub-file list. Any number of characters may be used, for example, you may enter a single character and NARC will find the first file whose name begins with that character. Alternatively, you may enter a complete wildcard specification and NARC will attempt to find a match. (K)ill file command. Will remove the currently highlighted sub-file from the ARChive. No additional disk space is required for temporary files. PgUP,PgDN,Home and End These keys do what you might expect. These functions work in all windows. (S)et search spec command. When NARC searches the current directory for archive files, it normally searches for .ARC and .ARK files. The .ARK may be changed to another extension (or wildcard extension) with this command. All subsequent searches will use both the .ARC extension and the newly defined mask. This command may be used from the ARC file window or from the subfile screen. The extension may also be passed from the command line. Enter NARC /h for complete information. (I)dc-arc command. This command will run IDCshell.EXE. Note that the environment variable IDCARCPATH must be set up before this command will work, unless IDCshell is in the current directory. IDCshell is the companion program to NARC and will be released in April - 88. This command provides a simple link between the arcing functions of IDCshell (and its NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 11 copy/rename/delete/print/view etc...) functions. Note that NARC can also be called from IDCshell via its "N" command along with a filename. THE NARC COMMAND LINE PROCESSOR A command line processor has been added that is compatible with both ARC and PKXARC. There are some variations on the implementation of these commands that are not wholly compatible with ARC and PK. For example, the ONLY command that operates on wildcard subfiles is the extract command. The "T" (test ARC integrity command) will simply open the ARC file and display the main NARC screen. This is done, since NARC does the integrity check when reading the ARC file. The "D" (delete subfile) command will only delete one file at a time. This is done since it is very seldom that files are deleted from an ARC anyway and the wildcard can be very dangerous. Command line syntax: NARC [command] [ARC file] [submask] where; [command] may be any of the following and may be preceded by "-" or "/" D,-D = delete submask file from ARCmask X,-X,E,-E = extract submask file(s) from ARCmask P,-P = print submask file from ARCmask V,-V,L,-L,T,-T,Z,-Z = open ARCmask file and display files C,-C = view submask file in ARCmask H = shows command line options [ARC file] may be any ARC file with or without the extension. An extension of .ARC is assumed, but may be explicitly included as .ARC or .ARK. The drive/path may also be included. [submask] may be any valid wildcard filename. TRIVIA STATUS SCREEN This screen shows some useful information about the current NARC environment. It is invoked with the ALT-F10 command sequence from the subfiles screen. Operating Hints and Philosophy and Shortcuts When NARC is first invoked, it pops up the window showing all of the ARC files in the current directory. The first thing that you must always do is to select an ARC file. The reason for this is that when I want to look inside an ARC file, I NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 12 will move to that drive/directory and then call NARC. This saves me from having to select where I want to look and what drive and all that mess. This way, when the program comes up, I can go right to work. NOTE: When the ARC file window is onscreen, you may also choose another file extension beside .ARK with the "S" command. See the section entitled "Using extensions besides .ARC and .ARK. When there are no windows popped up, the "A" key (or F2 key) will pop up the ARC file window, regardless of where the command line cursor bar is. This is handy when you have a lot of ARC's that you want to thumb through, with just the 2 mouse buttons or ENTER and F2 keys, you can look through a whole directory of ARC files in nothing flat ! Also along these lines, when the ARC window is on the screen, the right mouse button or the ESCape key will exit the program. NARC buffers 64k of the input file at one time, thus speeding up file operations. The output buffer is 32k and should be larger in the next version. NARC requires about 194K of RAM to operate. (This prime advertising space for rent) Note that the arrows keys may be used to move the error message box up and down the screen. This feature is provided so that if the message covers up something on the screen that you would like to see, you may move the box. ══════════════════════ Error Messages. ══════════════════════ Memory Allocation Error. - NARC allocates memory when it is invoked, this says that DOS told NARC that there was not enough memory left to run the program ERROR: Extraction Failed due to CRC error, Hit ENTER - After a file is extracted, the CRC contained in the ARC header is compared to the CRC that NARC calculates, this message says that the two were different. This is the CRC-16 polynomial. ERROR: Extraction Failed due to FileSize error - Same as above, except with filesize ERROR: Disk File Inconsistency. Hit ENTER - This will usually mean that the user has swapped disks just after telling NARC to View,Print or Extract and NARC does not recognize the file. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 13 ERROR: Incompatible Crunch Format - Says that either the stowage code for this file is not supported by NARC -OR- there is an error in the ARC header ERROR: Extraction Failed due to Lack of Disk Space - (A)bort (C)ontinue - Abort will stop tagged extraction, continue will try to fit next file. Squeezed File Has a Diseased Decode Tree. - When unsqueezing a file, NARC has found a bad entry in the decode table. ERROR: No directory space on destination! - Self explanatory Bad Path Name, Hit ENTER - The destination path that the user entered for extraction is not a valid DOS pathname, re-enter. Requires DOS version 2.0 or above. - NARC requires DOS 2.0 or above to operate. Invalid archive file format - NARC could not find any ARC headers, this is probably not an ARC file Warning: Bad archive file header, bytes skipped = xxxxx - If an entry has a bad header, NARC will examine the next 64k bytes looking for a good header. This is to maintain compatibility with ARC v.5.20 which allows self-unpacking ARC files. Unexpected end of ARChive file - Says that NARC couldn't find the last ARC header No matching file(s) in ARChive - ARC file is empty Cannot create work file, enter drive for temporary file - there was not enough directory space or disk space to create NARC.TMP for viewing. Enter a drive letter where NARC can create the necessary work file. The file will be deleted when the view has ended. ════════════════════════════════════════════ ARCHIVE FILE FORMATS AND GENERAL INFORMATION ════════════════════════════════════════════ For Those With a Little More Curiosity... The following are the currently supported stowage methods. 1 unpacked (obsolete) 2 unpacked 3 packed 4 squeezed (after packing) NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 14 5 crunched (obsolete) 6 crunched (after packing) (obsolete) 7 crunched (after packing, using faster hash algorithm) 8 crunched (after packing, using dynamic LZW variations) 9 Squashed c/o Phil Katz (no packing) (var. on crunching) NOTE: LZW is Lempel-Ziv-Welch crunching algorithm A little about the stowage methods. Packing - This is the simplest of the storage methods. Suppose that you have a line of text and at the end of the line, you have 40 spaces. These 40 spaces are compressed into 3 bytes in the ARC file by the "packer". The first byte is the actual character to be expanded (in our case a space). The second byte is a special "flag" byte that indicates that we need to expand these bytes when we extract the file. The third byte is the count byte (in our case it would be 40). So you can see that any time the ARC'er finds repeated bytes like this, it can compress them into 3 bytes. The anomalous case to watch out for here is when the count byte is the same character as the "flag" byte, this proved to be a difficult roach to kill ! NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 15 ══════════════════════════ HUFFMAN CODING (SQUEEZING) ══════════════════════════ It does, at first, seem that making a file smaller would be an impossible task. I will make an attempt here to shed a little light on this subject since that is a question that I hear pretty frequently and it is not a two minute discussion question. It does require some thought. To compress a file with the Huffman algorithm, commonly called squeezing, the first thing that must be done is to read the file completely and count the occurrences of each character. That is you count the "A" 's and the "B" 's and so forth. There are 256 characters in the extended ASCII character set, of which approximately 90 are "printable", that is you can see them on the screen. The IBM set has more "printables", but that is of no consequence, since the squeezer deals only with the numbers and doesn't care whether or not the file is an ASCII text file or an EXE file. Once the squeezer has counted the occurrences of each character, thus the frequency of occurrence, it scans the table for the characters that appear the least number of times and forms an imaginary link between them, called a node. Somewhere else in the tree, we will later develop a pointer that points to this node. When you start putting all of these things together, you will form a binary tree in memory. Confused enough ? Let us try an example. We have a file that is 100 bytes long and has 6 different characters in it. We have counted the occurrence of each of the characters and found the following. quantity character 5 - D 10 - A 10 - F 20 - B 25 - E 30 - C The spelling in the file wasn't very good, but we don't care. Now we take these numbers and will call them the frequency of each character. We then arrange the table as below. This is an arbitrary arrangement, but it is useful here so as to make our tree readable on the screen. The arrangement makes no difference. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 16 Frequency 20 10 5 10 30 25 Character B A D F C E We then examine the table to find the two characters with the smallest frequency of occurrence. In our case, it is obvious that one of them is 5,but which 10 do we choose. As it turns out, it doesn't matter which one you choose, we will arbitrarily choose the F. We draw lines from the D and the F to form our node (the box below). Frequency 30 10 5 10 20 25 Character C A D F B E \ / \ / ╔══╗ ║15║ = 5 + 10 ╚══╝ The number in the box is the sum of the frequencies of the D and F characters. Now we again look for the lowest two frequencies, except, this time we do not consider the D and F characters individually, we instead consider the node. The lowest two now are the A and the node, that is 10 and 15. We again do some artwork. Frequency 30 10 5 10 20 25 Character C A D F B E \ \ / \ \ / \ ╔══╗ \ ║15║ = 5 + 10 \ ╚══╝ \ / \ / ╔══╗ ║25║ = 10 + 15 ╚══╝ We look at the table again for the next two lowest frequencies and now find B and E. We continue in this fashion until the entire "tree" is built, that is until it all condenses to one node. The leaves are the actual characters at the top of the tree and the nodes represent branch joints with the root at the bottom. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 17 Frequency 30 10 5 10 20 25 Character C A D F B E \ \ \ / \ / \ \ \ / \ / \ \ ╔══╗ ╔══╗ \ \ ║15║ ║45║ \ \ ╚══╝ ╚══╝ \ \ / / \ \ / / \ ╔══╗ / \ ║25║ / \ ╚══╝ / \ / / \ / / ╔══╗ / ║55║ / ╚══╝ / \ / \ / ╔════╗ ║ROOT║ ╚════╝ Now that our tree is made up, we can encode the file. We start at the root (always). To encode the first character (leaf) of the tree (the letter C), we trace up the tree until we hit the letter C at the top. Along our journey, if we make a left turn, we record a 0 bit, and a 1 for a right turn. So for the C, we would go left to 55 (and record a 0) and then left again to the letter C (and record another 0),so the Huffman code for our letter C is 00. For A we go left to 55, right to 25 and left to A and it becomes 010. By doing all of the letters this way, we find the following. C = 00 ( 2 bits ) A = 010 ( 3 bits ) D = 0110 ( 4 bits ) F = 0111 ( 4 bits ) B = 10 ( 2 bits ) E = 11 ( 2 bits ) Mind that the zeroes and ones above are bits and not bytes. Each character was represented in the original file by 8 bits (one byte) and since we have reduced the number of bits needed to represent each character, we therefore reduce the size of the file. The savings add up as follows, NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 18 character frequency original bits squeezed bits savings C 30 30 x 8 = 240 30 x 2 = 60 180 A 10 10 x 8 = 80 10 x 3 = 30 50 D 5 5 x 8 = 40 5 x 4 = 20 20 F 10 10 x 8 = 80 10 x 4 = 40 40 B 20 20 x 8 = 160 20 x 2 = 40 120 E 25 25 x 8 = 200 25 x 2 = 50 150 ══════════ ══════ ═════ ═════ Totals 100 800 240 560 │ │ original file size ──────┘ │ squeezed file size ───────────────────────┘ 240 is 30% of 800, so we have compressed this file by 70%. Golly Wally, that seems pretty good. The rub lies in the fact that in order to reconstruct the original file, we must have access to the decode tree and since each tree will be different for each file, we must therefore save the tree with the file. It turns out that the tree can have only 256 nodes in a bytewise compression technique and each node will hold 4 bytes as pointers,a full table will be about 1k long. The table in our example has 5 nodes plus the 6 leaf nodes (where our characters are), totaling 11. 4 times 11 is 44 and if we add a few bytes for storing the node count and some other statistics, our table is about 50 bytes long. If we look at the 240 in the above table this gives the total number of bits that it will take to encode the file, divide 240 by 8 to get the number of bytes (30) and add it to our 50, we get a compressed file size of 80 bytes. Since our original file was 100 bytes, we have achieved a 20% reduction in file size. Not bad. What we have really accomplished is a translation of character sets, with our new set requiring less space than the original ASCII set. How far can we go ? If we look at the maximums that we can obtain for the different bit combinations in a optimally skewed tree, that is a tree that is not exactly symmetrical, we find that we can have only 4 - 2 bit codes, 8 - 3 bit codes, 16 - 4 bit codes, 32 - 5 bit codes, 64 - 6 bit codes, 128 - 7 bit codes, the remaining 4 will be 8 bit codes. 2 - 1 bit codes 4 - 2 bit codes 8 - 3 bit codes 16 - 4 bit codes 32 - 5 bit codes 64 - 6 bit codes 128 - 7 bit codes -------- 254 NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 19 And since we have a total of 256 different bytes to encode, the remaining 2 characters must have 8 bit codes. If we add the number of bits that this represents,we find a total of 1554 bits or 195 bytes. So at maximum, we have compressed the 256 bytes to 195 or 33%, thus the idealistic maximum that can be achieved with the Huffman algorithm is 33% when using a byte level implementation. One final note; The Huffman scheme requires the input file to be read twice, once to count characters and frequencies and then again to do the actual encoding. The major differences in Huffman coding and crunching lie in the fact that crunching is a one pass operation and does not require the table to be stored with the file. Both, however, are extremely vulnerable to errors, for example, imagine what would happen if you skipped one bit when squeezing the file, all of the remaining characters in the file would become the proverbial garbage, since we are looking at the file on a bit level. NARC uses the method described in K. & R. pp. 130 for setting up the binary tree with several modifications. The simple binary tree is acceptable for this, since the tree never grows and therefore will never become unbalanced. If you followed that, now go back and read the second paragraph again, maybe it will make sense this time. ══════════════════════ CRUNCHING ══════════════════════ Crunching began with an article by J. Ziv and A. Lempel in IEEE Trans. Information Theory, May 1977, where the method was originally described. Terry A. Welch wrote a definitive application article in IEEE Computer, June 1984 which described in detail how to apply the algorithm and some common problems encountered. Thus the name LZW compression. Crunching takes the Huffman coding method a step further as it does not include a table with the crunched file. The crunching algorithm also "learns" as it proceeds through the file. If it finds repeated strings in the file, they will be encoded into a table. This table is set up (in NARC's implementation) as a 4096 by 3 table. Each entry is formatted as <PREF>,<SUFFIX>, where PREF is a 2 byte pointer to another entry. SUFFIX is the byte for this entry. Representing the PREF's as pointers rather than values speeds up most operations in NARC. This idea came from Bob Freed and is very trick. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 20 One obvious benefit of crunched files is the fact that there is no need to include the encoding table in the compressed file as was the case with squeezing. Another great benefit is the fact that crunching is a one pass operation as opposed to the two pass situation in squeezed files. Crunching begins by creating an "atomic" table, that is a table in RAM that contains 256 entries, one for each character in the extended ASCII set. The values range sequentially from 0 to 255. The table entries are arranged as follows. Prefix Pointer (2 bytes) and Suffix byte (1 byte) In this initial table setup, the Suffix bytes are the 0 through 255 mentioned above. These are the "atomic" characters, in that they must be in the table before crunching or uncrunching can begin, since all files contain some portion of this character set. We do not know which characters will be included in any given file and which ones will be excluded,so we must include them all in our initial table. Once this table is set up, we can begin crunching. The Prefix pointer will contain a value that is a pointer to another string. When the table is initially set up, there are no other strings, so this Prefix pointer is set to a special "Null" string, that is it points nowhere. We must be careful when crunching the file, to look for this special pointer and act accordingly when we encounter it. This Prefix and Suffix business is used to "build" long strings. If we read the input file and found that the first character was the letter "I", we would look this letter up in the string table by hashing (computing an address). If we found the letter in the table (which we certainly will on the first character), then we output it's "hashed" address as a code to the output file (the crunched file). Suppose then, that the next character input from the file was the letter "D", the cruncher would then look at the I and the D together to see if they exist as a string in the table. Well of course, since this is the second character of the file, we know that it doesn't, so the cruncher forms a new entry in the string table. This entry has as its' Prefix pointer, a value that "points" to the letter "I" entry in the table, that we made a minute ago. The suffix byte in this case would be the letter "D". Now another code is output to the crunched file, representing the letter "D". Well this is great, we have now turned 2 bytes from the input file (16 bits) into 3 bytes in the output file (24 bits). You are correct, crunching begins by "not crunching" , but it is a crazy world ! The real value becomes apparent when we run into this same sequence "ID" in the input file again. Now we will find an entry for it in the string table and we can output a single 12 bit code that stands for "ID", thus saving 4 bits. The cruncher continues "learning" strings like this until the file is crunched. It should be noted that the string table is dynamically changing as the input file is processed. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 21 The early versions of crunching implemented, stopped "learning" once the string table was full. This gave a very poor compression ratio in some files. Versions 8 and 9 have an additional feature called adaptive reset, where the string table is cleared and crunching begins all over again ! This capability really helps the larger files more than smaller files. Details of Storage Versions NARC supports all of the current "skrunching" algorithms. A brief description of each follows. Version 1 - "STORED" File is simply stored (obsolete now, 25 byte header) NOTE: Files stored with this version are rare. Version 2 - "STORED" Current version of simple storage. This version was implemented with the implementation of file compression. The main difference in version 1 and 2 is the ARC header (see header section below), version 1 has a header length 4 bytes smaller than any of the rest of the storage methods since in version 1 there was no need to store the original file length separately from the stored file length since they were the same. Version 3 - "PACKED" Repeated bytes are packed into a three byte string (see Packing above) Version 4 - "SQUEEZED" after packing. The file is first packed as described above and then squeezed Version 5 - "CRUNCHED" This is the first implementation of LZW released. Uses fixed length codes and a complex hashing function. (obsolete now) (See hashing below) NOTE: Files compressed with this version are hard to find. Version was released only one month when next version came out. Version 6 NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 22 - "CRUNCHED" after packing. The file is first packed and then crunched. Uses fixed length codes and the same hashing function as version 5. Version 7 - "CRUNCHED" after packing. Same as version 6 except a faster hashing function is used. NOTE: Thom Henderson (author of ARC) has this to say about version 7. "This approach was abandoned because dynamic Lempel-Ziv works as well, and packs smaller also. However inadvertent release of a developmental copy forces us to leave this in." Version 8 - "CRUNCHED" after packing. Uses variable length codes in the crunched file (9 to 12 bits) and has a faster hash function (actually not hashing at all, but for the sake of consistency, we will call it that). This version also resets the string table when it becomes full which benefits the compression ratio of larger files. This resetting is commonly called an adaptive reset. Version 9 - "SQUASHED" (variation on crunching scheme). This version uses the same hashing function as version 8 but varies the crunching codes from 9 to 13 bits. There is also no packing, which affords the string table the opportunity to "learn" longer codes and thus improve the compression ratio (benefits "real large" files). NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 23 ARC file header structure is same for both DOS and CP/M Byte number Value(s) Meaning ══════════════════════════════════════════════════════════════════════════ 1 1Ah Header Flag 2 0-9 Compression Version 3-15 --- ASCIIZ compressed filename 16-19 --- Compressed file size in bytes Low Word, High Word with each word in LoHi format 20-21 bits DOS date format 15-9 Year 8-5 Month 4-0 Day (all zeroes means no date) 22-23 bits DOS time format 15-11 Hours (military) 10-5 Minutes 4-0 Seconds 24-25 --- CRC-16 in LoHi format of uncompressed file. ------- This is important. 26-29 --- Original uncompressed file size NOTE: Version 1 files are not compressed so the length can be found with bytes 16-19, also the header len for version 1 files is 25 bytes. NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 24 Hashing..... Hashing is simply an arithmetic way of coming up with an address in a table. You have a set of input numbers (codes) that will map one-to-one with the output codes in an ideal situation. That is, each time you input a certain number, you can rest assured that the output will always return the same output number. This is not quite the case in the current versions of .ARC files. The reason is that the algorithm would require a MEG or so of RAM for implementation. The utilization of a smaller string table in all of the ARC programs introduces the possibility of producing the same output number for 2 or more input numbers. This is called a hash collision. This is handled separately in .ARC files with what is called a "collision table", which helps to locate the correct table entry. There are three versions of "hashing" used in ARC files. Hash1 - Key = upper 12 bits of lower 18 bits of unsigned square of (prefix code + suffix byte) OR 800h Used in stowage versions 5 and 6 Hash2 - Key = lower 12 bits of unsigned product of (prefix code + suffix byte) * 15073 Used in stowage version 7 Hash3 - Key = next available address in table. Used in stowage versions 8 and 9 CRC calculations - NARC does not use the traditional table lookup method for calculating the CRC of the file. The table approach requires the table to be in RAM and takes up more space. NARC calculates the CRC on the fly, which requires no table and saves space. The algorithm is taken from the definitive article by Aram Perez in IEEE Micro, June '83. The polynomial is X^16 + X^15 + X^2 + X^1 which is not compatible with the Xmodem CRC. Gary Conway NARC.DOC Copyright (c) 1987-89 Infinity Design Concepts, Inc. Page 25 ══════════════════════════════ ARC RELEASE DATES AND VERSIONS ══════════════════════════════ These are the various versions of ARC.EXE that I have and what versions of storage they supported. PKxARC supports all of these methods as well since they were all originally created by ARC.EXE. Date Stowage Methods Released Version Supported 05-01-85 3.10 Storing,Packing,Squeezing (1-4) ( version 5 in here somewhere) 06-26-85 4.10 Up to version 6 of crunching 11-18-85 4.50 Up to version 6 of crunching 12-04-85 4.52 Up to version 6 of crunching ( version 7 in here somewhere) 01-21-86 5.00 Up to version 8 of crunching 01-31-86 5.10 Up to version 8 of crunching 02-05-86 5.12 Up to version 8 of crunching 10-24-86 5.20 Up to version 8 of crunching This list is compiled in an attempt to start some kind of historical record as to what transpired in the ARC world. I would be interested in hearing of additions. End of NARC.DOC Copyright (c) 1987,88 Infinity Design Concepts, Inc. Page 26