World of A1200/World_Of_A1200.iso/programs/compress/misc/compressdisk/compressdisk.doc.eng

Text File | 1995-02-27 | 9KB | 209 lines

      Compressdisk.device 37.14 (release 1.0)

           by Antonio Schifano


 1 - Introduction.

Data compression is not a novelty in the Amiga scene: since many years
there are many commercial and public domain utilities that give to all
Amiga users the possibility to save disk space. We can roughly divide data
compressors in two main classes: the class of archivers (like ZOO, LHARC,
etc.), and the class of "file oriented" compressors (like COMPRESS,
POWERPACKER, IMPLODER, etc.). Archivers' main feature is their ability to
preserve the files hierarchy; the price to pay is the impossibility to use
archived files in the compressed form. On the contrary, file oriented
compressors sacrifice this feature to achieve a better usability of
compressed files: programs compressed by PowerPacker, for instance, remain
executable; moreover, the package includes some useful tools (like PPMORE)
that compensate for the versatility lost because of the compression (for
instance, you can still read compressed text files, but you cannot edit
them). Compressdisk.device tries to resolve the defects of both types of
packers with a different approach to the problem.



 2 - What is compressdisk.device?

As its name probably suggests, it is an Exec device, similar in the way of
operating to Matt Dillon's fmsdisk.device. Like that, in fact, it allows
you to create virtual disks, whose tracks are saved on a file. Unlike that,
however, it uses a different file for each track, and compresses the
tracks. This allows you to create virtual partitions on your hard-disk,
usable like normal partitions, but having the great advantage that every
file you write is automatically compressed (and obviously, it is decompressed
whenever you read it back).




 3 -Installation.

The device can be installed only on Amigas with kickstart V37+. The
installation is (almost) totally performed by the programs "CDInstall" and
"CDPartition", who take care of copying all necessary files in the right
directories:

 - The file "compressdisk.device" is copied in the DEVS: directory.
 - For each virtual partition desired, an opportune entry is created in the
   DEVS:mountlist file. If you are using Workbench V38+, all entries will
   be created in the DEVS:DOSDrivers directory.
 - The preference editor "CDPrefs" is copied in the SYS:Prefs directory.

For the device to work properly, you should add the following line to your
s:user-startup file, just after the command SetPatch:

    assign CDISK: <directory>

where <directory> is the complete path of the directory in which you want
the device to store the compressed tracks.
Otherwise, the device will store the tracks in the root directory of the
boot partition.
Once mounted, each virtual partition must be initialized; this can be
easily accomplished with the system command "format". Since the device
stores only the tracks written at least once, it is advisable to use the
QUICK option: this will let you save disk space and time (formatting a 40M
partition can take more than thirty minutes on a standard Amiga 500).
To speed up read/write operations, the device features a built-in cache in
which it stores the most recently used tracks; the cache size can range
from one to thirtytwo tracks. The default configuration is eight tracks per
unit; from tests the device proved to be usable with four or, if you want
to save your memory, even with two tracks, whereas with only one track the
performance reduced significantly, expecially in writing.



 4 - How to use compressdisk.device.

You can use virtual partitions created with compressdisk.device like
usual ones; nevertheless, it is better to follow some guidelines. First of
all, it is recommendable not to write any files into any virtual partition
if the device work disk (CDISK:) is full: if the device cannot store some
tracks, you lose some data. Moreover, you should not use virtual partitions
for storing temporary files; remember that deleting a file (on the standard
file system), does not clear the blocks that it occupied (in fact, you can
recover it using proper utilities), until the file system does not
overwrite them. This means that when you delete a file you will NOT free
any space on the device work disk. Virtual partitions created with
CDPartition use the Fast File System; however you can use any file system
you want. Since Standard File System and the correspondent International
version reserve part of the data blocks for internal informations, they can
reduce the tracks compression; for this reason their use is not
recommended.



 5 - The compression algorithm.

The more decisive choice in the device realization was the selection of the
compression algorithm: it is clear that the device performance depends
tightly on the data compressor and decompressor ones. The algorithm had to
be as fast as possible, to use little memory, and reasonably good at
compressing. The used algorithm was ideated by Ross Williams, who named it
LZRW3-A; it requires only 16K of work memory, and has an average
compression of 54%. The version I wrote and optimized in assembler has a
compression speed of 16K/sec and a decompression speed of 64K/sec (on a
68000 at 7.16MHz). The following table gives a rough idea of the
performances, obtained by compressing two text files, an executable, a link
library and a soundtracker module:

 -------------------+------------+-----------+--------+----------+----------
  file            | dim. ante  | dim. post | % comp | K/sec c. | K/sec d.
 -------------------+------------+-----------+--------+----------+----------
  intuition.doc     |    237916     |     98950 |    58.4  |  19.07     |  84.80
  dkbffp        |    144648     |     81606 |    44.6  |  13.17     |  60.36
  dkb212.doc        |    157458     |     74523 |    52.7  |  17.43     |  76.12
  amiga.lib        |    157880     |     43866 |    72.2  |  26.40     | 118.60
  mod.trial        |    108946     |     86847 |    20.3  |   8.35     |  39.99


For further informations about LZRW3-A refer to the original documentation
given by Ross Williams (directory "pub/compression" on the FTP site
sirius.itd.adelaide.edu.au [IP=129.127.40.3]).



 6 - Implementation.

From a technical viewpoint, compressdisk.device may look a bit strange,
since it seems to break the hierarchy of the Amiga OS. In fact it is
unusual for a device to use the DOS, since this one is at an higher level
in the hierarchy. The hierarchy is purely functional: devices are used
for management of physical resources (eventually virtual ones) and
handlers for O.S. logical resources. For this reason a device is allowed
to implement its primitives using a logical resource, provided that the
tasks which use the device are not aware of the implementation of the
primitives.
The device core is a process which satisfies all I/O requests: whenever it
receives a CMD_READ command, it loads and decompresses all the needed
tracks; in a similar way, whenever it receives a CMD_WRITE or CMD_FORMAT
command, it compresses and stores the tracks. If possible, the process uses
the cache both in reading and writing: in reading, if a track is already in
the cache, there is no need to reload and decompress it; in writing, tracks
are compressed and stored only if there is no space left in the cache.
Tracks are updated upon receiving a CMD_UPDATE command; data consistency is
thus ensured. The track buffers can range from 1 to 32. Every track has 64
sectors of 512 bytes each, for a total of 32K. The device needs about 60K
of working memory, plus about 32K for each track buffer.

The highest cylinder number supported by the device is 65000. The largest
size of a virtual partition is thus 2031.25M.



 7 - Acknowledgements.

I wish to thank the following people, in no particular order:

 - Raffaele Dell'Aversana, for the academic discussions prior to the
   realization of the device, as well as for testing my brand new 2MB chip
   ram expansion on his A500+ (still frightened of seeing your caps lock
   key flashing? );

 - Ross N. Williams, for the data compression algorithm;

 - Jan van den Baard, for the (useful!) utility "GadToolsBox";

 - Giuseppe Sacco, for the utility "ZeroUno" (see the DISCLAIMER!! ;-);

 - The organizers of I.P.I.S.A. '92;

 - my sister Letizia, for no particular reason.



 8 - Copyright/distribution.

Compressdisk.device,  Copyright © 1992 Antonio Schifano.
All rights reserved. This program is SHAREWARE; if you use it, send
Lit. 15000 if you live in Italy, 20DM if you live in the European
Community, or 20USD if you live somewhere else.
The program can be distributed in its original archive format; can be
included in collection disks of public domain software. It can also be
distributed in electronic form via BBS, FTP, and similar. You may not
charge more than Fred Fish does for a single library disk. Also no profit
is to be made by selling this program. The commercial use and distribution
of this program is NOT allowed without written permission from the author.
The author cannot be held liable for any damage or data loss directly or
indirectly caused by the use or misuse of this program; by deciding to use
it the user assumes all the responsibilities.
Suggestions, bug reports and comments are welcome.

Send the registration form to:

           Antonio Schifano
           via Don Bosco, 3/28
           56100 Pisa, Italy
(until 31 January 1993);

           Antonio Schifano
           via dell'Oca, 11
           91100 Trapani, Italy
(always);

The FidoNet address is: 2:332/607.666