Hide full article
and Adam C. Engst -- TidBITS Editor
Transparent compression utilities have become quite popular since they promise to make more space available for storing files while at the same time not slowing down your Mac. Some people, most notably David Ramsey in his column in MacWEEK, have suggested that it's easier just to buy a larger hard disk. I'm sure that's true for some people, but it can be easier to justify $50 for a program than $200 for a new hard disk. The popularity of Salient's AutoDoubler continues to rise, Alysis's More Disk Space (MDS) has recently entered the market to mixed reviews in CompuServe discussions, and Aladdin will release StuffIt SpaceSaver as some point in the near future.
As David Ramsey points out, however, these utilities are not truly transparent, and one of the more confusing interactions they have is with backup programs, most notably Dantz Development's powerful Retrospect. Lots of people have had some trouble specifically with how AutoDoubler and Retrospect work together, and there's a simple fix for the problem that we'll get to in a bit. First, please bear with us for some discussion of how the transparent compression utilities work, because you'll need to understand that before you can figure out the interaction with Retrospect (and possibly other backup programs).
Literals -- A fair amount of discussion on the Internet has pertained to AutoDoubler and how it handles files, especially across the network and with Retrospect and other backup programs. Here are some answers that may also apply to More Disk Space and Aladdin's forthcoming SpaceSaver. Keep in mind that this is all conjecture based on lots of experience and a bit of investigation; we don't work for any of the compression companies and most of our experience is with AutoDoubler.
AutoDoubler contains within it a list of file types and creators to avoid compressing. It also maintains a list of what we'll call "literals"; these are applications which are handed files "literally", or without change regardless of whether they are compressed or not. MDS also maintains two exception lists, one of which tracks files not to compress, the other of which tracks applications which should see files literally. I presume SpaceSaver will have something similar.
Imagine that you have a file named "Bob" which has been compressed with AutoDoubler. While inside Nisus, you instruct Nisus to open "Bob". AutoDoubler will intercept the call from Nisus to open the file and will decompress it first (into memory if the application only wants the file read-only, or onto disk if the application wants write permission to the file as well); then it will allow Nisus to continue opening the decompressed "Bob".
If, on the other hand, you were running ZTerm, which AutoDoubler has in its list of literals, and asked ZTerm to send "Bob", AutoDoubler would NOT decompress the file prior to ZTerm sending it. The advantage to literals is that it gives programs capable of their own form of compression the ability to see files pre-compressed. In the case of ZTerm, it would be silly for me to decompress "Bob" prior to sending it over a telecommunications line using a protocol like MNP, which is going to turn around and recompress the file and probably not get it as small as AutoDoubler did originally.
The problem with using literals is that if I'm uploading a text file to America Online, whose software is included in the AutoDoubler literals list, I don't want that file to be compressed on the other end since I don't know if a downloader will have AutoDoubler or DiskDoubler. There are two workarounds currently, and as we said above, Salient is considering allowing the user to edit the literals list in the next version of AutoDoubler. The first workaround is to simply expand the file with DiskDoubler, if you have it. The second is to open the file and make a change like adding a space and then deleting it. When you save the document it will be in expanded form again because of that change. More Disk Space puts this literal exception list out front, which is good because it's better to allow the user to decide when a program should and should not view compressed files literally.
There is another advantage to literals is in the case of backup programs, most notably Retrospect, which AutoDoubler considers a literal. When you backup your Mac with Retrospect, you might think that you are going to have a problem (or at least slowdown) because Retrospect is asking for files, which AutoDoubler is then decompressing, which Retrospect is then compressing (if you have compression on). In reality, since Retrospect's compression can be disabled and AutoDoubler considers Retrospect a literal, you can get fast backups of pre-compressed files. You can tell that Retrospect sees compressed files in compressed form because they will show up in the Retrospect browser as ADExpandUtil (the application that can expand AutoDoubler files if the extension is not loaded) or DiskDoubler files. This can be something of a pain, actually, if you normally select files by type and creator, since most everything will have AutoDoubler's type and creator.
However, this assumes that you have compressed all the files on your hard disk. In all likelihood, this will not be true, since AutoDoubler won't compress your System Folder and both More Disk Space and AutoDoubler can optionally exclude files from compression, as we said above. In addition, files you've just modified are unlikely to be compressed. So how do you save that disk space on your backup? You don't want to just compress all the files, since compressed files that are compressed again generally grow larger and it doesn't make sense to take the time to compress files twice.
Retrospect Selectors -- Here's where one of Retrospect's little-known features comes in. If you've looked into Retrospect's depths, you'll know that it features something called Selectors. A Selector is merely a collection of criteria that can select a set of files that match those criteria. It's easy to set up Selectors to include all files and folders with the word "temp" in the name, or files that haven't been modified in eight months, or even all files created only by a certain application. It turns out that Retrospect's compression is not merely a checkbox in the Options screen of the Archiving process (you won't see it if you only click Backup in Retrospect's initial window). If you enter the Extended Options (via a radio button at the top of the window), you'll see that there is a pop-up menu of Selectors at the bottom of the window next to the Compression label. The Selector that's selected in the pop-up menu (one called Compress is the default) will govern which files Retrospect compresses in both the Archive and Backup processes.
The trick then, is to modify the default Compress selector in Retrospect (from the Selectors... item in the Config menu) by first duplicating it (select it and click Duplicate), and then modifying it by adding a new "File Kind" item from the Item pop-up menu on the right hand side of the window. Then check the "Direct Entry" view button, and the "Creator only" match button. You'll get a little text entry box to enter a four letter creator code, and there are two checkboxes labeled Not and Enable that you also want to check (so the criteria is a negative one - we don't want these files included - and so that it is enabled). The code for AutoDoubler and DiskDoubler is DDAP. If you look at the other "Not File Kind" items in the Compress Selector, you'll see that Dantz has already added ones for Compact Pro and StuffIt and the like, and I'm sure Dantz will be modifying that Compress selector in the future so it includes AutoDoubler/DiskDoubler by default as well.
Let's explain what just happened and what it will do. You modified the Compress selector so that it knows NOT to compress files created by AutoDoubler or DiskDoubler (or any of the other common compression programs). Now there's no problem keeping compression on in Retrospect because Retrospect will look at each file to backup and see if it was created by a compression program. If that's true (i.e. the file matches the Compress selector), then Retrospect will simply backup the file and not compress it. If the file is not compressed by one of the compression programs, then Retrospect will use its own compression, saving space on the backup disks. This sort of stuff may not be all that easy, but at least it's possible. Hats off to Dantz for providing a program with this level of power and flexibility so it can handle strange situations like this.
Retrospect Remote -- One confusing issue which has arisen on the Internet is what happens with something like Retrospect Remote? This is an extension (INIT) which resides on networked Macs and acts as a file handler for Retrospect running on a "backup server" on the network. When Retrospect wishes to backup John's Mac, it sends a message to the Retrospect Remote extension on John's Mac, which scans John's disk and hands back a list of files. Retrospect (on the backup server) decides what needs to be backed up and then asks the Retrospect Remote on John's Mac to start sending it files over the network.
These files will come across the network expanded. This is because Retrospect Remote uses standard toolbox calls to copy the files; AutoDoubler intercepts the calls (as usual) and expands the files. This is the opposite of how it works on your local Mac, which receives the files literally. Why the difference? Because there's no easy way for an extension (like Retrospect Remote) to indicate to another extension (like AutoDoubler) that the extension is asking for a file; on the other hand, it's easy for AutoDoubler to tell that, for example, Retrospect is asking for a file on a local Mac. On a networked Mac running Retrospect Remote, all AutoDoubler knows is that something is asking for a file and, if that file is compressed, it needs to be expanded. It appears that there is no workaround - you'll just have to leave Retrospect compression on or else all your files will be in your backup in expanded form and will take up twice as much space. Lars Holm of Dantz Tech Support said that they were testing all the compression programs to ensure compatibility and that Dantz is looking for ways to let other companies detect the presence of the Retrospect Remote extension so that files do not have to be expanded all the time.
Dantz Development -- 510/849-0372
Alysis -- 415/566-2263
Salient -- 415/321-5375
Aladdin -- 408/761-6200
Information from:
Lars Holm, Dantz Development-- DANTZ.TECH@applelink.apple.com
Lloyd Chambers, Salient -- salient@aol.com
