Software Recommendations

home *** CD-ROM | disk | FTP | other *** search

/ Software Recommendations - 1998 Season 1 / DNBCD4.iso / share / compress / carcom11 / CAR.DOC < prev next >

Wrap

Text File | 1994-11-11 | 42.3 KB | 870 lines

WARRANTY ------- This is a free evaluation copy of the programs subsequently described. As such no warranty is offered against the programs and/or associates contained within this package. You are free to copy and share/distribute copies of this package provided you do not modify any of the files. No liability will be accepted for any damage or losses arising out of the direct, or indirect usage or inability to use all or any parts of this package. The programs included should be considered as beta evaluate/test only. Being intended purely as introductory and widely distributed, this package will not be maintained/updated against any bugs/defects etc. that may exists. For the latest copies of the programs and documentation you should contact the package author (see Register.doc). If you intend to use any of the programs within this package beyond a reasonable evaluation trial period, you MUST register your intent with the package author. Such registering will ensure you receive the full source code in order that you might ensure the program(s) are fit for your own particular purpose or can be suitably modified for such, thereby in effect creating your own warranty/maintenance agreement. Registering will additionally ensure that you are kept aware of any upgrades, bugs or improvements that may occur. A reasonable period for evaluating the programs within this package is considered to be around one calendar month. All trademarks mentioned within this package are recognised to their respective holders. CONTACT ------- Compuserve (CIS) 100014,3141 Fido 2:254/151.3 or 2:254/152.3 Postal CarComp c/o 111 Deer Park Gardens, Mitcham Surrey. CR4 4DX. England INTRODUCTION ------------ Cross system integration within computing is increasingly becoming an important part within the home, small and corporate business operational environments. This package contains evaluation copies of the executables for the CAR, GZIP, ATOB, BTOA, UNIX2DOS, DOS2UNIX, SPLT and JN tools for use under DOS and SUN OS 5.0 systems that may be of some benefit for such cross systems working. In order to limit the size of the evaluation package to an acceptable level such that transmission times/costs via modem link distributions is kept at a minimum, no source code has been included within this package. Having tried out the executable versions however, if you believe that the utilities may be of benefit to you, then you can register your interest via our service desk which will result in your receiving the full source code (portable C) so that you might tailor the tools to your own particular requirements. Using the tools within this package, it is possible to compress files to levels that are often better than that of other commonly available archivers/compressors whilst being able to share such files independently of system architecture. In brief, there are several tools included within this package: CAR - Compressed file archiver (enables storing of multiple compressed, standard or mixed files into a single hold file). GZIP - Compressor/Decompressor distributed under the GNU General Public licensing terms and conditions. Provided by Jean-loup Gailly and Mark Adler. Caters for compressed files in Compress, pack or gzip formats and is currently the defacto for the Linux operating system. BTOA - Converts binary files to ASCII format files so that binaries might be transmitted via textual services. ATOB - Restores BTOA files (ASCII) back to the original binary format. UNIX2DOS - Converts UNIX text file formats to DOS DOS2UNIX - Converts DOS text file formats to UNIX. SPLT - Splits a data stream into one or more files of a specified size. JN - Recreates a stream from SPLIT files. Combined, these tools enable you to perform compressed file exchange across multiple system architecture. For example a file or group of files may be compressed and archived into a single hold file (archive) on one system (e.g. UNIX) and then transported (e.g. via modem link, floppy copies etc.) to another system (such as a DOS based PC) where the original file(s) can be restored. Although only DOS and Sun example executables have been included within this package, this does not mean that the tools are restricted purely to these platforms; In practice, the tools may be used on practically any system that has access to a suitable C compiler. All of the source code for the tools (portable C code) is available via the package author. Throughout this document it is assumed that you are already familiar with one or more of the compression/archive programs that are currently around (typical programs being Compress, Pack, PKZIP, LHA, ARJ etc.). CAR/GZIP -------- The gzip compressor provided within this package can automatically handle the decompression of several formats of previously compressed files, namely :- compress, pack, gzip So as an example, you might collect a number of files into a common archive (hold) file on a Unix platform using the car tool, and compress this archive using either the Unix compress, pack or the supplied gzip tool in readiness for sending the compressed file to another system such as a DOS based system. On the DOS system, it is then possible to simply uncompress the archive hold file using gzip and extract the individual files using the car x function. gzip is capable of automatically detecting the compression method used (compress, pack or gzip) in order to correctly decompress the file. Naturally though, binary files under Unix cannot be expected to work under DOS and visa-versa. For compression, a derivative of the LZ77 and Huffmans coding methods are utilised, thereby being free of current patent restrictions (of which other programs such as the standard UNIX compress may currently infringe). gzip will only compress to its own gzip format, so if you intend to say receive and send files to and from a UNIX/DOS environment then ideally you should have a copy of gzip on both systems. If you are however only sending files from say Unix to DOS, then you'd only need to have a copy of gzip on the DOS system which could be used in order to uncompress files that were compressed on the Unix machine using 'compress'. It should be noted that the gzip program is NOT the package authors own work but is provided within the CarComp package purely to relieve the need of having to acquire the executable from alternative sources. Further details with regard to gzip can be obtained from the package author if required. The Compressed file Archiver tool (CAR) caters for storage and retrieval of files into a single hold file in a system independent manner. This is aided by the packages ability to convert such files as text from UNIX format to DOS and visa-versa. CAR is not reliant purely on using gzip as its associated compressor. In practice, almost any compressor of your choosing should also work with CAR. gzip was selected purely as a result of the availability of its source code, together with its abilities to handle multiple compression techniques that, in my opinion, will make it a very strong contender as a future 'most popular' compressor choice. Unlike many other archive programs where the archiver and compressor are contained within a single unit, this package provides two separate files for such functions. CAR (compressed archiver) deals purely with storing of file(s) into a single hold file, whilst gzip deals purely with compression. Several advantages in using such separate files are apparent; Firstly, tighter overall compression can be achieved when a number of files are to be compressed into a single archive. Secondly, by keeping the compressor separate, such functionality as manipulating files in a compressed format becomes much easier. For example, assuming we have a file containing a list of names and telephone numbers that have been compressed to a gzip file, then such commands as gzip -dc <telnos.gz | sort | gzip >t_sorted.gz can be used. (Effectively sorts the compressed telnos.gz file into a new file called t_sorted.gz). Such capabilities are even more apparent under UNIX systems where highly complex manipulation tasks can be achieved using gzip in combination with the likes of Bourne/C Shell, awk etc. statements. In total, the package enables a much higher degree of general flexibility of storing, manipulating and sharing compressed files. This package is by no means intended to be a replacement to your current most favoured archiver program, but more as a complement. It is ideally suited for use with the likes of BBS sysops who may prefer the flexibility and higher compression levels capable during such data exchanges as Echo/NetMail etc. where, in some cases of a BBS with a large nightly data exchange, hours can be knocked off the transfer time/user lock out over the course of a year, whilst also opening up the board to cater for other platforms to connect and exchange mail automatically using the common compression and archive/storage technique. Typically you could expect to see around a 5-10% smaller file than that of using say PKZIP in collecting and batching mail packets using the standard file archive add option. For larger BBS's with many echo areas, the potential savings even over a week can therefore be quite substantial. This package is ideal for developers who wish to include data compression functions into their package(s) or anyone who may wish to generally share files in a compressed format with others. Or for anyone who may be interested in compression and archive program coding in general. Compression Levels ------------------ Generally, usage of the files within this package will show at least comparable compression levels to that of many of the other currently available compressors/archivers around at the current date. A suggested test is to select all of the files in any directory of your choosing and try archiving all of those files within that directory into a single archive hold file using your currently preferred compressor. Then repeat this using the tools provided in this package. As a guideline, we performed such a test on a randomly selected directory. We logged into a local BBS and acquired copies of the following programs and applied a compress/archive time/size trial. Our randomly selected directory turned out to be that of the UC compression program. The following is the results of that test:- PROGRAM FLAGS START TIME END TIME SIZE ------- ----- ---------- -------- ---- CAR/GZIP -9 11:39:27.91 11:40:49.96 269,810 HAP3 11:41:49.67 11:47:57.28 292,498 UC2 11:48:57.04 11:51:34.13 262,698 PKZIP (2.04g) -ex 11:52:35.43 11:53:10.30 295,620 SQZ 11:54:20.72 11:55:18.45 292,530 ZOO 11:56:12.11 11:57:55.92 331,543 ARJ 12:02:20.33 12:03:17.45 294,341 LHA 12:07:15.50 12:08:09.98 294,255 HPACK 12:03:56:17 12:06:40.24 297,298 (Original directory contained 52 mixed files totalling 439,247 bytes.) Applying weights to the results yields: OVERALL PROGRAM TIME SIZE WEIGHT ------- ---- ---- ------ CAR/GZIP 5th 2nd 7 HAP3 9th 3rd 12 UC 7th 1st 8 PKZIP 1st 7th 8 SQZ 4th 4th 8 ZOO 6th 9th 15 ARJ 3rd 6th 9 LHA 2nd 5th 7 HPACK 8th 8th 16 Combining the weights, it can be seen that overall CAR/GZIP and LHA share joint 1st for compression rates/times UC2, ZIP and SQZ share joint 2nd ARJ 3rd etc. The test was performed on an old 12MHz 286 with each run being performed under similar disk space/machine resources. You might notice that the UC program was the only one to better the gzip/car combination on compression. However in contrast UC's compression rate turned out to be around 5 times slower than that of the likes of PKZIP. Additionally we're comparing an unoptimized version of CAR/GZIP here with the likes of highly optimised versions of PKZIP etc. In general then it seems reasonable to assume that an optimised version of car/gzip would therefore come out as the overall winner for general compression. Naturally these figures are by no means intended to be definitive, and are provided purely as a general example of car/gzip's performance. Every compressor/archiver package around has its own unique benefits/costs. However for general use, we hope we have indicated how car/gzip may be of benefit to you. Especially in its ability to compress down by around 10% more than that of some of the other more commonly used compressors. Combined with gzips ability to handle piping and redirection etc in order to manipulate data streams using compression, overall we believe that the tools included within this package are valuable additions to most set-ups. BTOA/ATOB --------- BTOA and ATOB are provided in order that binary format files, such as a compressed file, may be converted to a format that is suitable for transmission via textual services (such as via mail), and restored again at the receiving end. Warning! Both of these tools will OVERWRITE the output file without pre-warning! UNIX2DOS/DOS2UNIX ----------------- These utilities enable you to convert text files from one format to another. Simply, they strip the Carriage returns off DOS text files that are bound for a Unix system or insert such in the opposite direction. As per the above, both unix2dos and dos2unix will overwrite redirected outputs without any pre-warnings. SPLT/JN ------- Split and Join are useful for where you may have a largish distribution that may require say two or more floppies/CDROMS etc. Splt requires at least two command line parameters, a size in (multilples of 100KB's) and a filename prefix (no extension). It then reads from standard input or from the named file (3rd command line variable) and writes to a file named <prefix>.1 until the size parameter bytes is met. It then starts to write subsequent data to the file <prefix.2> etc. Thus a large continuous file of say 6MB that is in either binary or text form, can be easily split over several files of a size that is useful for say floppy disk transfer. Join (JN) takes either a single or dual command line parameter(s), that of the prefix used for the split and/or a second paramenter for an output filename (standard out is used when not specified) and rejoins the data as prior to the 'split'. For an example of using split and join, see the USING section below. PACKAGE FILE CONTENT -------------------- file_id.diz File Description register.doc Registration form car.doc Main documentation gnu.doc GNU General Public License covering gzip/gzip.exe gzip.exe DOS executable compressor/decompressor car.exe DOS executable archiver btoa.exe DOS executable 'convert binary to ascii' atob.exe DOS executable 'restore ascc to binary' unix2dos.exe DOS executable for text file conversion (Unix to Dos) dos2unix.exe DOS executable for text file conversion (Dos to Unix) splt.exe DOS executable for split jn.exe DOS executable for join gzip SUN OS 5.0 executable compressor/decompressor car SUN OS 5.0 executable archiver. btoa SUN OS 5.0 executable binary to ascii atob SUN OS 5.0 executable ascii to binary unix2dos SUN OS 5.0 executable unix2dos dos2unix SUN OS 5.0 executable dos2unix splt SUN OS 5.0 executable for split jn SUN OS 5.0 executable for join INSTALLING ---------- It is recommended that you install the files provided with the package into a common directory that is pointed to within your PATH statement. *** WARNING *** Under Unix systems, you should be careful not to overwrite any other programs of the same name (e.g. atob, btoa, unix2dos, dos2unix, jn etc.) If necessary, rename the functions as desired. USING ----- *** CAR (Compressed file archiver), works along similar lines to the Unix Tape Archiver (tar). Use the command 'car' alone to see the available options. Note that CAR can be set to store files either with or without CRC checking (much faster without). When such CRC checking is enabled, both the header and data content will have a CRC maintained. In general, subsequent usage of the gzip program (which in itself maintains a CRC check) should mean that the use of CAR's CRC checking should only be required when very tight data integrity/checking is required. I would therefore recommend usage of the 'car a ....' option more than that of the 'car s.....' option. CAR enables you to store multiple files into a single hold file. It is anticipated that you will CAR the relevant files into a single hold file prior to using gzip against that car file. This is the method that ensures greater compression over that of other archive programs. It is commonly known that LZSS compressors start off compression poorly due to the lack of pre-loaded information to compare against. By storing all of the uncompressed files within a single hold file (such as a .car file), then effectively the number of occurrences of 'poor' starts is reduced to only one compared to that of other archive programs that compress/add each individual file to the archive (e.g. 100 of such archived files would have 100 poor starts). The downside of using the CAR files, compress CAR file method, is that individual files within the archive cannot be easily maintained as is possible with some other archivers. However for such cases as where an group of archived files is intended to be manipulated as a whole, then the method can pay dividends. Additionally, there is nothing to stop you from employing a gzip file, car, gzip, car approach in order to enable files to be maintained more individually. Alternatively given the source code, you might code a new version along the lines of that as outlined in the 'Improvements' section of this document (see later). If you do maintain individually compressed files within a car file, then remember you could use a command such as car p myarc compfile | gzip -d in order to view the compressed file compfile contained within mycar.car. Or even car p mycar compfile | gzip -d >restored.fil One other additional limitation in the car program is that it only stores filenames, not pathnames. The benefit from such however is that files car'd on say a Unix system can equally be extracted on say a DOS system without concern of pathname limitations (provided you store such files using a maximum filename of the 8.3 format as used under DOS). Again, given the source code, you might prefer to adapt the car code in order to cater for pathnames etc. if so required by your own particular circumstances. The car all files, gzip single car file approach will produce tighter compression levels when files of a similar type are stored sequentially together. Thus if you intend to archive a complete directory, it is more preferable to say car a arcfile *.doc, car a arcfile *.exe etc rather than adopting a car a arcfile *.* approach where the sequence of storage may result in something like File 1. a.doc, 2. b.exe, 3. c.exe, 4. d.doc,.. etc (e.g. 1. a.doc, 2. d.doc, 3. b.exe, 4. c.exe is more preferable). One area you should be aware of is that under UNIX, car is set to only store files of Regular, block or character type files. Directory, FIFO/Pipe, Link etc. type files are skipped. An interesting side use of CAR is as a disk defragmentor tool. Simply by archiving all files within a directory to a CAR file using the move option (car m c:\workdir) followed by a car x to restore those files will result in the possible reduction of fragmented files within that directory. CAR currently caters for the following options R - Replaces attempts to replace all of the files in the archive with a version within the current directory. If a given file in not available, then that files replacement is skipped. M - Move moves a files from disk into the archive. A data content CRC and header check CRC are attached. S - Similar to the move option but leaves the original file intact on disk. A - Add is similar to the S option with the exception that no data CRC is stored. Not having to calculate CRC data significantly improves overall storage run times. D - Deletes a file from the archive. L - List archives contents V - As per the list option (compatibility) T - Tests the files in the archive where a CRC is included (files stored with the Move or Store methods). X - Extracts file(s) from the archive P - Print the content of a file within the archive. Note: Wildcards are permissible with the options e.g. car a myarc \dos\mydir\*.* will stores all of the files in the \dos\mydir directory to the archive files myarc.car. Individual files may be named for selected options e.g. car x myarc myfile will extract the single file (if exists) called myfile from the archive file myarc.car When creating an archive, a default extension of .CAR is used unless another value is specified. CAR performs no filename validation, so if you intend to transfer files from say a Unix system that can cater for long filenames, then you should endevour to rename the files to a naming format that is suitable for the lowest common denominator filename maximum size for which system(s) you wish to share such files with. Typically this will be the DOS 8.3 format e.g. If you have a file called this.is.my.text.file on a Unix system and you want to use CAR to transfer this file to a DOS system, then you should rename the file to something like myfile.txt prior before adding the file to the CAR archive. If disk space is at a low level, you may also find that you may have to 'move' a few files at a time into an archive as during the archive process, the system may have to be handling up to three copies of the actual files being added to the archive (original, temporary and archive versions). ********************* *** GZIP the compressor/decompressor, caters for INDIVIDUAL file compression only. As previously mentioned it can automatically decompress files that have been compressed using compress, pack and (of course) gzip. It is a commonly used version within the Linux world and is free of any current patent restrictions. To gain an insight into gzip's functions use the command gzip -h. gzip can cater for stream redirection or pipe filtering. For tightest compression (slowest), use gzip -9 <filename> For quickest compression (fastest) use gzip -1 <filename> To decompress files use gzip -d <compressed_file> where compressed files may be any of a previously compressed, packed or gzip'd file. As previously mentioned, when a group of files are to be compressed to a single archive file, the benefits of first storing all of the files into a single hold file and then compressing this hold file can often produce much tighter compression levels than that of compressing each file individually to a common hold file. In practice you'll probably find that the compression levels when using this approach nears or exceeds that of the best of the commonly available compressors currently around. Often in much quicker compression times too! ********************* *** BTOA/ATOB - Convert Binary to ASCII, Convert ASCII to Binary are similar to the atob and btoa programs that you may be familiar with. They simply enable you to convert a binary file such as a compressed file into a pure ASCII file (but larger in size) using the BTOA tool so that the resulting ASCII file may be effectively transmitted via a text only service and suitable restored at the recipient end using the ATOB tool. Both work on standard input and output redirection. A typical usage might be if you wish to send an executable (binary) file to a colleague via Netmail. Say the exe file was called myexec.exe, then a typical preparation would be gzip myexec.exe | btoa >myexec.bta OR gzip myexec.exe btoa <myexec.exz >myexec.bta (myexec.exz being the output filename from the gzip). The pure text file 'myexec.bta' could then be Netmailed as required and all the recipient would have to do is atob <myexec.bta >myexec.exz gzip -d myexec.exz OR atob <myexec.bta | gzip -d >myexec.exe ********************* *** DOS2UNIX/UNIX2DOS Both of these utilities take the form of a redirected or piped input and output e.g. unix2dos <unixfile >dosfile. Both expect the input file to be of a textual format. The only conversion performed in to strip or insert carriage returns as required. Thus if you create a file containing textual information such as a Comma Separated Values file of say names and addresses called adds.txt under a Unix system, and you know that this file is bound for a DOS based system then a command something like unix2dos <adds.txt >addsdos.txt (under DOS) will ensure that the file is in the correct format for reading by the recipient. Note under Unix (SUN), the command syntax is unix2dos infile outfile OR unix2dos infile >outfile Similarily dos2unix syntax is DOS dos2unix <infile >outfile SUN/UNIX dos2unix infile outfile or dos2unix infile >outfile ********************* *** SPLT/JN Lets say we have a directory that contains 100 files, each of 100 KB (10 MB in total). Lets also say that these represent a distribution that you wish to share with a colleague via a floppy disk posting. We can car the files using a command like car a forbob c:\thedir\*.* which will archive the contents of c:\thedir into a newly created forbob.car file. We can then compress this file using a command such as gzip forbob.car, thereby creating a file called forbob.caz. Lets assume the gzip compressed the files down to around 50% of its original size, thus we have a single file of around 5MB that is almost ready for transfer. In order to put this file on floppies we can use a command such as splt 13 bob forbob.caz OR splt 13 bob <forbob.caz OR type forbob.caz | splt 13 bob What this does is to split the data stream (a redirection from 'forbob.caz') into several sequential files named bob.1, bob.2, bob.3 etc, where the split occurs at 1300 (KB) (being a suitable size in this case for copying onto 1.4 MB floppies). In our example we will now have files bob.1 to bob.5 that can be copied onto 5 different floppy disks and posted off to Bob. When Bob receives these floppies, all he has to do is to copy the floppies onto his hard disk and use a join command such as jn bob >forbob.caz OR jn bob forbob.caz Note no filename suffix is include as jn automatically assumes suffices of .1, .2 etc. This will take each of the bob.1 to .5 files and recreate a single file of around 5MB. He can then gzip -d the file in order to extract the car file, and then unarchive the 100 individual files to the required directory using car x forbob. Note how the usage of splt and jn maximised usage of the floppy disks to their full capacity. There was no need to concern ourselves with what files should be copied onto which floppy in order to be able to send the files on the fewest number of floppy disks as possible. This potential makes split and join ideal utilities for developers or anyone who may wish to share data/files with others via floppy, CDROM, tape etc. medium. As an example, I archived a directory containing over 34MB of comma seperated file format data on a Sun system into a single car file. This was then compressed using the standard Unix compress down to a level of around 2.6MB. I then the used splt tool in order to split this compressed archive file down onto two floppy disks using something like splt 1380 myfiles <arcfile.caZ. The whole process took literally only a few minutes to perform. Not bad considering the original 34 MB could then subsequently be transferred to another platform such as a DOS based system and equally restored using the gzip/car combination (also possibly having to use unix2dos). One thing to bear in mind when using splt and jn with floppy disks is that ideally you should try not to specify exactly 1.4 MB split file sizes as some floppies may have bad sectors which in practice would reduce the amount of available space on that floppy. If your split files were exactly 1.4MB in size, then obviously you wouldn't be able to copy such split files onto a floppy with minor disk defects. Hence the usage of 1.3MB as in the above example. BUILDING PACKAGES ----------------- With the additional flexibility offered in using the CarComp programs, you also have the addition of extra responsibilities to whom ever you might share such compressed archived files with. Typically if you are building a package of several programs and documents into a single distribution file, then you might like to consider using batch/script files in order to prepare such a release. Such an approach ultimately simplifies matters should you ever wish to add or modify any of the files and then rebuild the distribution copy. Similarly, you might like to consider including a batch or script file within your distribution that aids in extracting the files to the relevant intended destination areas. As an example, lets say we have an executable program that runs on both Unix and DOS systems (one executable for each system) and a single common manual document. To this you might add a file description and a short introductory readme file. One method to create a suitable archive would be to initially add both of the executables and the document file. You might then create a script file for the Unix executable installation and a .bat file for the DOS installation. These batch/script files would include commands in order to extract and uncompress the relevant files from the overall hold file as well as any other general installation tasks as required by your program. Finally you could then car all of the files into a single car archive file and compress this in readiness for distribution. Any recipients would then simply identify the car/archive by its common .caz extension (as per .taz, .zip etc) and could then decompress this and apply the car l option, probably followed by a 'car p readme' command, in order to view the conents of the pacakge. The readme file would provide the recipient with an overview of the package and advise of how to install such for their system (e.g. via the .bat of script file). Quite a similar approach to how existing packages are created really! But with the additional bonus of using up to 10% less disk space or modem transmission times. For larger package distributions, the additional benefits of using the split function against such a distribution comes into its own. You can simply create a highly compressed copy of all of the files within a single archive, and split this into sizes that would be best accomodated on say floppy disks. Each part can then be labelled as such for general distribution. All without needing to concern youself with which files to place into which archive in order to mimimise the number of floppy disks required in order to distribute the package. An alternative to the above however, is to design your own custom archived distribution using the source code provided with CarComp. Making the potential to provide highly professional in appearance installation programs without the concern of someone elses copyright/advertisment poping up during the installation process. REGISTERING ----------- If you like any or all of the programs within this package then if you register the package, you will be sent the full source code of the latest available copies of the programs. Details of registration are provided in register.doc. Additionally you will be advised of any bugs, improvements, additional functions etc. as and when they become available. SOURCE CODE ----------- All source has been written in portable C and can be compiled under many platforms/compilers. The code is in almost all cases in pure C only to alleviate the necessity of having a low level, machine specific, assembler compiler for compilation purposes. As such, the performance of the example programs can be significantly enhanced for more specific local requirements under specific platforms, if so required, by adapting the source code to your own particular needs such as replacing selected functions with assembler functions. Some assembler routines are included within the source listing for the more critical functions in order that you might link these in if desired. IMPROVEMENTS ------------ This section is aimed at those who may already be familiar with compression algorithm coding and those of you who might be considering modifying the source code of car/gzip in order to adapt/improve such code for your own particular requirements. It principally outlines how the source might be used/modified in order to provide a functionally comparable program to that of the more generally used compressors/archivers, whilst maintaining the 'bit extra' compression levels. As previously mentioned, the car all uncompressed files to a single file and gzip'ing the resulting file generally leads to tighter compression levels than that of other techniques. LZSS compression basically maintains a window of previously seen values (bytes) and a look ahead window. The algorithm effectively compares byte sequences within the look ahead window with that of the previously seen window and, when a match is found, encodes those bytes in the look ahead with a pointer to the occurrence within the previously seen window together with a 'length' count. e.g. if a 16 bit pointer and 8 bit length values are assigned, then the previously seen window would be 16384 bytes and the maximum length range would be 256 bytes. Thus if say a sequence of 20 bytes in the look ahead match that of the bytes within the previously seen window at location 4567, then that byte sequence would be stored as something like 4567 020 (3 bytes in effect coding 20 bytes). The break even for these match position/length group (24 bits) is 3 bytes. If therefore no matches of 3 bytes or more are found for a particular instance, then the bytes are output as is. (I've omitted the description of overhead bits and the usual adoption of Huffmans post process encoding, block compression levels checks etc.). The problem with this method is that at first, the previously seen window is effectively initialised as all nulls. Thus the method tends to start off encoding poorly. If however, the previously seen window was to be initially pre-loaded in a set manner with some fixed data, then obviously this would improve upon the prospects of being able to encode byte groups at an early stage. The problem here though is what data values to use for such fixed pre-loading. The simple method as suggested with the car/gzip approach is to effectively use the data from the tail end of the previous file. And this works relatively well as you will see if you make a few tests of the compression provided by car/gzip (especially if you order the storing of the files into the car file by file type e.g. docs, exe's etc.). However the downside is that this does restrict the extraction of individual files. A suggested alternative would be to change the code to something like the following approach:- gzip and add to the car file the first file. gzip the next file using the x number of decompressed bytes from the tail end of the first file as the pre-load 'previously seen window' data for this second file (where x corresponds to the size assigned to the previously seen window (65536 in my example). Add the second file to the car file. Repeat for subsequent files. Such an approach would result in a car file which could now have individual files extracted using a reverse approach. The compression levels should be comparable to that of the existing suggested car all/gzip car file approach, whilst catering for the individual file extraction function. Obviously though, the overheads and management would need to be carefully maintained as, such as if one intermediate file was requested to be deleted, then the deletion would require that the two files either side of that file would also partially need to be processed as well. In general however, the additional processing required to maintain these file start/end adjacent areas, might justify the additional compression that should be obtained overall. Alternatively, you might prefer to simple predefine a set of window pre-load values, where the choice of pre-load used corresponds to particular file types. Providing both the decompressor and compressor used the same pre-loads, then such an approach could be maintained with a very small (if any) overhead being added to the compressed file. Other areas of possible improvements include the archivers header/data file structures. Significant improvements towards faster updating could be made if required. I do intend to make the above and additional improvements to the current CarComp code for subsequent releases of the package. So if your interested you should register in order to ensure you are kept up to date with the latest versions. GENERAL ------- You'll find that the programs in this package are open to a number of other possible improvements. For example currently they perform limited validations and version checking (only gzip has any magic number checks etc. at the moment). You should therefore be careful in using the programs as if you do something like atob <infile >outfile, then that's what will occur, EVEN if 'outfile' already exists! Similarly CAR will add or extract any files that are requested, again even if such files already exist within the archive/disk directory. Additionally the user interface is purely provided for usabillity, not for appearance. Future versions are expected to improve upon this. Once you have the source code, you can modify the code to suit your own particular requirements/expectations. Alternatively we can provide assistance if required for specific tailoring. For those who might be considering creating their own packages based around the source code provided from this package, there are very few restrictive rights that have to be adhered to. Simply put, gzip can be modified and shared at will (as under the GNU product licence) on the principle that you pass on the rights you received to anyone else whom you might share that source code or executables with. For the other units, I only impose a restriction that the source code is shareable only with other registered users unless you have specific written permission from the package author to the contrary. If you do therefore opt to register for the source code, be aware that you will be under the obligation NOT to post the likes of car.c etc. to public areas without suitable consent. This restriction arises out of our funding of the package improvement/enhancement/distribution via moneys received as a result of source code registrations. Finally if you have any suggestions or recommendations for additions to this package we are always delighted to hear from you. We would also welcome any submissions of compiled versions of the CarComp files under other platforms/operating systems. OTHER ----- If you are a Shareware programmer and would like assistance with the distribution of your program(s), then CarComp may be able to help. For a small fee, we can often help you to globally distribute software packages such that you might best benefit from any potential registrations from a very wide audience. Methods include international backbone file distribution, general commonly avaiable shareware CD-ROM file inclusion, major Internet and BBS file areas etc. Further details are available upon request.