Amiga Developer CD 2.1

home *** CD-ROM | disk | FTP | other *** search

/ Amiga Developer CD 2.1 / Amiga Developer CD v2.1.iso / Contributions / Cloanto / Tools / DirDiff / DirDiff.man < prev next >

Wrap

Text File | 1997-10-06 | 24.7 KB | 581 lines

Cloanto DirDiff v. 5.1 1. Introduction DirDiff is a powerful tool to scan, compare, synchronize and replicate files. It was originally designed for internal use at Cloanto to simplify the maintenance of software master disks and for backup management. Over the years, new features were implemented, the file comparison engine was enhanced and a Workbench interface was added. 1.1 What can DirDiff do? DirDiff allows you to compare (and optionally update) two disks. Several options can be set to finely tune this process. DirDiff allows you to copy some files from hard disk to floppy, then use the floppy on another computer (for example at home), and from time to time automatically move the files which have changed to whichever medium contains the older version. DirDiff allows you to compare bootable disks against new versions of the Amiga Workbench disks. It can selectively compare only the files used in both disks, skipping icon files if desired. So you get only messages regarding the files that count. If you wish, you can upgrade the system files on your work disks in the same process. When copying tens, or hundreds of Mbytes from one device to another, errors are more likely to occur and pass undetected than with normal, interactive use. Using DirDiff, at Cloanto we discovered faulty I/O configurations which caused only one or two bytes out of 100 million to be corrupt. We now always use DirDiff after each step in the preparation of master disks, both for floppy disks and for CD-ROMs, to make absolutely sure that no errors have occurred. When it finds certain errors, DirDiff does an additional check to make sure that the storage medium gives consistent results. It even double-checks the RAM locations it uses. DirDiff is an invaluable tool to back-up a hard disk to a slower device, such as a magneto-optical disk, because it can copy only the files which have changed. Results can be achieved faster and in a less resource-consuming way. To make the process bullet-proof, the source directory can be locked, and differing files can be forced from hard disk to the optical disk even if they have an older date. Files from older backups which become unused can be deleted in the process. The combined use of these functions can deliver identical copies at a fraction of the time a standard "Copy" procedure would normally require. DirDiff can detect even subtle differences between two items, such as a directory renamed from "WBstartup" to "Wbstartup". Before DirDiff reports that two directories are identical, it checks not only the contents, but also the attribute bits, file comments, dates and exact names of all items. DirDiff can scan a directory or disk and give an electronic signature (for example "DD5-A89CE287"), derived from the contents of all files. This short code can then be used to refer to a particular disk or product release without detailing the entire contents. For example, two distant offices which want to verify if they have the same version of a disk can execute DirDiff separately, and then check the signature in a phone call taking only a few seconds. Or the signature can be used as a reference in a licensing agreement, as a confirmation of what is being licensed. DirDiff includes additional checksum capabilities, and features such as clearing of unused blocks on a device (useful when you do not want others to "undelete" data on a medium which you considered safe to give away). 2. The user interface DirDiff can work in two modes: single directory mode and two-directory mode. The first mode is mainly a scanning mode. It can be used to check the integrity of files (all data is read from beginning to end), and to calculate file checksums and a final signature based on the total data. When working with two directories, DirDiff can perform complex comparison and update tasks, while remaining very easy to control. For certain tasks (as in the "Lock", "Maximum Priority" and "Delete" modes), the first directory is considered the primary, or source directory, and the second directory is the destination. The first directory can be locked, so that data only flows in one direction. The first directory can also be assigned "maximum priority", so that data is transferred from the first directory to the second even when files in the first directory appear to be older than those in the second. Existing files in the second directory can optionally be deleted, if they do not exist in the first directory. DirDiff can be executed from Workbench or from the Shell. In Workbench mode, all options can be set from the program window. Menus allow to do additional things such as saving the current settings. The window has a status line indicating the current file or action, plus an area showing either the settings or the results. In Shell mode, a help list containing the keywords associated to the program settings is displayed whenever the program is executed without parameters (that is, just entering "DirDiff" in the Shell). The execution results are printed in the Shell, and upon termination DirDiff exits with an AmigaDOS return code ("OK", if no errors or unresolved differences were found, or "WARN" otherwise), which can be used for conditional branches when DirDiff is executed from scripts. 2.1 Window Gadgets The window displayed by DirDiff incorporates standard Amiga gadgets. The Window Close gadget on the upper left corner is equivalent to the Quit gadget. The Zoom gadget, on the right, allows the user to quickly expand the window size to the maximum possible size. The Depth gadget is located on the window title bar next to the Zoom gadget. It is used to bring the window to the front or back with respect to other windows. DirDiff automatically pops the window to the front of other windows when it finishes a job. The bottom of the window contains five gadgets: Run, Pause, Abort, Iconify and Quit. Run instructs DirDiff to begin a new processing cycle. Pause temporarily halts processing, which can be resumed by pressing the same gadget again. Abort concludes processing on the current file and suspends further processing. Iconify closes the window (but does not interrupt processing). A "DirDiff" icon appears on the Workbench to indicate the iconified state. A double-click on the icon reopens the window. The Quit gadget closes the window and terminates the program (aborting any processing in progress, if necessary). 2.2 The Mode Gadget This gadget toggles between single and dual directory mode. By default DirDiff works with two directories. If the same directory is indicated twice, DirDiff switches to single-directory mode. 2.3 The Path Gadgets These two text gadgets specify the working directories. Icons can be dragged from the Workbench screen and dropped over the gadgets. The small gadget to the right of the string gadgets activates a file requester which can be used to select the path. Note: If one of the two devices is much slower than the other, or if it has considerably fewer files, and if the processing involves a simple comparison and no updates, then DirDiff is faster if the directory with fewer files, or a slower device, is indicated as the first directory. This is because in the main phase the second directory has to be searched for all files found in the first one, and this occurs in an order arranged after the first directory. 2.3 The Page Gadget The central area of the program window always contains one of three pages: the Settings Page, the Output Page or the Status Page. The Page gadget allows the user to cycle through these pages. 2.4 The Settings Page This page contains all options which can be set by the user. A particular configuration of options can be saved from the program menus. 2.4.1 Dates By default, whenever DirDiff finds the same file or directory in both the first and in the second directory, it also compares the dates. On the standard Amiga filing system this date indicates the date of the last write to the file or directory (although directory dates are not always up-to-date). If DirDiff if used to compare two directories, and when it is known in advance that the files in a directory had their dates modified during an incomplete copying process, this option could be disabled in order to leave the list of differences more readable, without excessive messages. The date associated by AmigaDOS to each directory entry is accurate to 1/50 of a second (although fractions of a second are not indicated by the Amiga Workbench, nor by the Shell's List command). Some filing systems (e.g. MS-DOS, or ISO 9660 as used for CD-ROMs) and archiving software (e.g. LhA) store dates with a slightly different precision than the Amiga filing system. So it may happen that when a file is copied to a device employing a different filing system, the date of the copy shows a small difference (usually less than a second). The "Dates ± 2 sec" option instructs DirDiff to ignore small differences in the file dates, within a range of ± 2 seconds. In Update mode, it also informs DirDiff that file dates are not reliable within the ± 2 second range. When this option is disabled, dates are neither compared nor updated, unless a file or directory is being updated for a different reason. Note: It is recommended to always preserve the original file date when copying files. This is a piece of information which can give precious hints about the history and the current status of the file. In case of updates, DirDiff may use these dates to determine which item is older and should be overwritten. Unfortunately, by default the Amiga Copy command of the Shell changes the date of the destination file. To avoid this, Copy should be followed by the CLONE option. At Cloanto, we use a "cp" command, which is defined as follows in the S:Shell-Startup file: Alias cp Copy [] CLONE 2.4.2 Attributes Just as it compares the dates of items found in both directories, DirDiff also compares attributes such as the comment associated to that item and its status bits (see the Filenote and Protect AmigaDOS commands). Under AmigaDOS, matching files could even have slightly different file names, because capital and lower case letters are treated as if they were equal. Some or all of these additional attributes could be altered or even lost when files are archived or moved to other filing systems. For example, most Amiga implementations of the ISO-9660 CD-ROM filing system do not represent the original Amiga file comments and attribute bits. In a conversion to the MS-DOS filing system, it is possible that all file names are automatically converted to capitals. When this option is disabled, attributes (comments, attribute bits and exact names) are neither compared nor updated, unless a file or directory is being treated as different for another reason. 2.4.3 Icons DirDiff normally also processes the ".info" files which contain information about the Amiga Workbench icons associated to files and directories. This option allows to leave out icon files from this process. This could be useful, for example if two disks are known to have similar contents, but differing icon positions. 2.4.4 Quick On a reliable medium, it is normally safe to consider that two files having the same name, length and date are identical. The Quick option allows DirDiff to make this assumption, so that the contents of files having the same name, size and date is not analyzed. This is much faster than scanning two files from beginning to end. When the dates differ, DirDiff scans the files automatically, even if they have the same name and length. It is recommended to disable the Quick option whenever the reliability of a medium or of a copying process is questionable. This causes all files to be byte-scanned and compared from beginning to end. 2.4.5 Couples Only This option restricts the set of items being processed to those which exist in both the first and in the second directory. This is useful whenever a medium contains a base set of files which has to be checked against an original, plus some additional items which do not need to be checked or updated. Activating this option could be useful, for example, to compare a new Workbench disk with the Workbench installed on the computer's hard disk. In this case, both the hard disk and the Workbench disk could contain items which do not exist in the other directory, and which are not relevant for a quick difference check. 2.4.6 Update This option controls the part of DirDiff that not only reads, but also writes to the directories being processed. When Update is disabled, no writes occur. When Update is active, DirDiff will take actions in order to resolve the differences between the two directories. These could be: copying the entire file and/or setting the date and other attributes. Disabling the Update option is also useful to execute a test run of DirDiff to see which files would be affected if Update were enabled. If the results are acceptable, the Update option can be enabled to run DirDiff a second time. 2.4.7 Lock Directory 1 This option acts as a "valve", limiting the direction in which the update process is allowed to write data. It is normally a reassuring option to use when using DirDiff to make backups. In this case, the source directory (containing the data to be copied) should be indicated as the first directory. Regardless of the state of this option, older files are never copied over newer ones. This is only possible by activating the "Maximum Priority" option. 2.4.8 Maximum Priority This is a potentially "dangerous" option. It forces data to be copied from the first directory to the second one whenever two items are different, even if this means copying an older item over one having a more recent date. The Maximum Priority option only causes file to be copied from the first directory to the second, and not vice versa. From this point of view, it is similar to "Lock Directory 1", only that it is considerably more "active". Maximum Priority is very useful to restore a directory regardless of the file dates, and to make sure that all items in the second directory are identical to those in the first one. 2.4.9 Delete in Directory 2 This is another potentially "dangerous" option. It extends the update process by deleting those files or directories in the second directory which do not exist in the first directory. This option is normally used together with Maximum Priority. It could be employed, for example, to maintain a backup copy of a hard disk up-to-date, removing those files which have been removed (or moved, or renamed) in the source since the previous backup. The result is that the two directories are forced to be identical. 2.4.10 Fill Directory This function can be activated in single-directory mode. It causes empty blocks on the storage device to be cleared. Normally, when files are deleted it is still possible to recover some information with low-level disk tools. When delivering material to third parties, this potential security risk can be prevented either by formatting a new disk and copying the files to it, or by using a function like DirDiff's "Fill Directory", which fills the directory with a file as large as allowed by the existing space, and then deletes that file. 2.5 The Output Page This page displays the scrollable output list of DirDiff, which contains entries for the directories and files processed, plus errors and other messages in chronological order, followed by the final results as displayed in the Status Page. The list can be saved to file, printed or cleared (useful in case of memory shortage) by selecting the corresponding gadgets. The output, just as processing, occurs in two phases (in two-directory mode). The first phase does a preliminary check to see if the second directory contains any items which are not present in the first directory (in Delete mode, these are immediately deleted). The second phase scans all items in the first directory, comparing them against the second directory. In single-directory mode, DirDiff lists the size and checksum of each file. The checksum is a simple sum of all bytes in the file. At the end of the process an electronic signature is also printed. This has the format "XXX-YYYYYYYY", where XXX stands for the program version, and the Y digits are the actual signature. This is not a simple checksum. A change in a single attribute bit of one of hundreds of files can completely change the signature. Note: The signature method employed by DirDiff does not employ encryption technology, thus it does not pretend to offer any protection against the possibility that a set of source files can be artificially prepared to produce a given signature. By being only 8 characters long (32 bit) the signature leaves open the theoretical possibility that two different set of files produce the same signature. The comparison of two directories produces an output showing the differences found and the action being taken, if any. The possible differences include: non-existence of an item or differences in contents, name, attribute bits, comments and date. When DirDiff finds matching items with different names (upper/lower case differences) it shows both names. When it finds files with a different content but the same length, it shows the offset in which the first difference was found. To resolve the differences (in Update mode), DirDiff can take one of the following actions, on a case by case basis: copy the entire file, update (copy newer over older) or force (copy older over newer) the parts which are different, delete the item (directory or file), create a directory. At the end of a directory comparison process (two directories), DirDiff prints final result, indicating whether it found any errors or differences that could not be resolved. If the settings allowed for all attributes of all files to be checked, and no differences were found, then DirDiff will state that it has found the directories to be identical. 2.6 The Status Page This page contains information on the number of files and directories which have been processed, updated, created or deleted, and on the total number of errors or unresolved differences. This information is updated in real time as processing occurs. It should be noted that the counters indicating the total items being processed are increased once for each item and for each separate access. They are not indicators of the numbers of files and directories present on the disk. If an item exists in both directories, it is accessed at least once in each directory, thus counting as two. Plus, the counters are also increased in the first phase. 2.7 Menus The items in the Project menu are standard Amiga application items, equivalent to the corresponding gadgets in the window of DirDiff. The settings menu allows the user to save and reuse particular configurations of settings, in a way similar to the standard menus found in the Amiga system Preferences. The Save as Defaults command makes the current settings the default startup settings. Last Saved reapplies the settings which were last saved, whereas Restore resets all values to the initial program settings. If the Create Icons option is enabled, DirDiff will associate a Workbench icon to all settings files written. A double-click on the icon loads DirDiff with the settings stored in the selected file. 2.8 Examples The following examples show typical applications of DirDiff, indicating which options have to be set in a position different than the default. To show missing or different files and directories: - Default configuration To verify a disk after a copy or optimization: - Quick option disabled To verify a CD-ROM ISO image before mastering: - Dates set to "± 2 sec", Quick and Attributes options disabled To compare Workbench files on an application disk: - Couples Only option active To synchronize hard disk and floppy files: - Update and Couples Only options active To make or refresh a backup copy (keeping history of "orphan" files): - Update and Lock Directory 1 options active To make or refresh a backup copy (deleting "orphan" files): - Update, Lock Directory 1 and Delete options active To make or refresh an exact copy: - Update, Maximum Priority and Delete options active To check file integrity, show checksums and produce final signature: - Single directory mode To clear unused blocks on a disk: - Single directory mode, Fill Directory option active 2.9 Program Messages Most error messages displayed by DirDiff are self-explanatory. For example, simple AmigaDOS errors may occur when accessing (locking, opening, examining, deleting, creating, renaming, copying or writing) a file or directory. This may occur if an update exceeds the storage capacity of the disk, for example. For most messages, an AmigaDOS error code is also indicated. "Not enough memory" messages indicate that there is insufficient free RAM. The following is a list of other messages, in alphabetical order. Data length different from file length ... This message indicates that the number of bytes which DirDiff was able to read from a file was different than the data associated to the file in its directory entry. This could be caused by a problem in the filing system structure (or also - but not necessarily - by a virus program inserting some hidden link code into executable files). At Cloanto we noticed this discrepancy to be quite common when processing other computers' filing systems on the Amiga. Data OK after retry on medium (possible random match) After discovering a difference in the contents of two files having the same size, DirDiff read the location showing the difference a second time, and found no difference between the files. The fact that two reads of the same data produced two different results is likely to point to a medium or I/O error, while the fact that the second read matched the contents of the other file could be caused either by the error being (temporarily) recovered, or by another error which caused the results to match by pure chance. Dates within ± 2 sec range - No update possible DirDiff is operating in Update mode, but encountered corresponding entries in each directory which required an update, but had dates too close to indicate with precision which item is newer. This only happens if Dates is set to "± 2 sec". Directory entries are of different types DirDiff encountered two entries (one in each directory) having the same name, but different types, presumably one being a directory and one a file. The two entries are skipped, as they can neither be compared, nor updated. Identical dates - No update possible DirDiff is operating in Update mode, but encountered corresponding entries in each directory which required an update, but had identical dates, making it impossible to determine which file is newer. Indirect date change... As a side effect of DirDiff updating the files in a directory, the filing system "touched" the date of the directory. DirDiff detects this and restores the original date. Internal error (...) These are internal errors which "should not occur". Reporting the error condition to Cloanto would be helpful to correct the problem. Medium Error (unconfirmed read on ...) Data consistency checks are performed routinely by DirDiff after certain errors occur, or when unlikely file differences are detected (as in files having the same name and length, but different contents). This message indicates that DirDiff obtained results which were different from the previous read. This could be caused by defects in the medium or in the I/O system. RAM error RAM inconsistency DirDiff checks the RAM locations it uses to store its file comparison buffers. These messages indicate that an error occurred during one these tests. An "inconsistency" is an error which could not be confirmed by a second test. Read beyond end of file This message indicates that DirDiff was able to read more data (before reaching the end of file) than the file length information in the directory entry indicated. This message is similar to "Data length != File length", only that it occurs in a different context. For additional information on this distribution of DirDiff please refer to the "DirDiff.readme" text, also in the archive containing this file.