Simtel MSDOS 1992 September

home *** CD-ROM | disk | FTP | other *** search

/ Simtel MSDOS 1992 September / Simtel20_Sept92.cdr / bbs / zdcs / zdcs_ref.txt < prev next >

Wrap

Text File | 1991-12-31 | 84.0 KB | 2,112 lines

Zipfile Duplicate Checking System (ZDCS) Ver. 1.65 Copyright (C) 1991, Michael W. Cocke --------------------------------------------------------------------- Technical Reference Manual TABLE OF CONTENTS ----------------- ABOUT THE DOCS..................................................2 PURPOSE OF ZDCS.................................................2 GENERAL TECHNICAL INFORMATION...................................3 ZDCS OPTIONS....................................................4 Deletion of Duplicate Files................................4 Allowed Dupicates..........................................4 BBS Ads....................................................4 Pre-Testing................................................5 INSTALLATION OVERVIEW...........................................5 THE ZDCS CONFIGURATION FILE.....................................5 Function...................................................5 Line 1.....................................................6 Line 2.....................................................6 Line 3.....................................................6 Line 4.....................................................6 Line 5.....................................................7 Line 6.....................................................7 Line 7.....................................................8 Line 8.....................................................8 THE ZDCS DATABASE BUILD.........................................8 Purpose....................................................8 Creation of the Initial Database...........................9 The Screen Display During the Database Build...............9 Additions to an Existing ZDCS Database....................10 ZDCSDB Without a Separate File Integrity Checker..........11 The Database Build Log File ZDCS-DBB.LOG..................11 THE ZDCS DUPLICATE REPORT GENERATOR............................12 THE ZDCS DATABASE PURGE........................................12 BBS ADS........................................................13 Function..................................................13 Creation of the BBS Ads Database..........................14 Updating the BBS Ads Database.............................14 Selecting Deletion or Flagging of BBS Ads.................15 ALLOWED DUPLICATES.............................................16 THE UPLOAD FILE CHECKER........................................17 Purpose...................................................17 Function..................................................17 Maximum and Actual Percent Dupes in an Upload.............18 Updating ZDCS Database(s) After an Upload.................18 Calling ZDCS from EXZTEST.................................19 Calling ZDCSFC from the PCBTEST.BAT file..................19 DOS Error Levels..........................................21 - 1 - PROCESSING LOCAL UPLOADS.......................................21 PRE-TESTING....................................................22 ZIPs WITHIN ZIPs...............................................24 CD-ROMs........................................................24 AV STAMP INTEGRITY.............................................25 ACCURACY OF THE CRC32 METHOD...................................26 MEMORY LIMITS..................................................26 TROUBLE-SHOOTING GUIDE.........................................26 ZDCSDB (the database build) is crashing...................27 ZDCS is reporting a device I/O error (in any module)......27 ZDCS reports memory corrupt errors (in any module)........27 ZDCS reports path/file access errors......................27 QEMM exception 13 errors occur when running ZDCS..........28 REGISTRATION...................................................29 SUPPORT........................................................30 FUTURE ENHANCEMENTS............................................31 ACKNOWLEDGEMENTS...............................................32 COPYRIGHTS AND LEGAL STUFF.....................................32 ABOUT THE DOCS -------------- Welcome to the technical reference manual ZDCS-REF.TXT for ZDCS version 1.65. This manual has been prepared to give you an easy reference guide for the various parts, functions, and options in ZDCS. It is deliberately arranged to be modular in organization so that you can look up whatever interests you without having to read the entire manual. There is a second major piece of documentation in this package, a walk- through called ZDCSWALK.TXT. This is a friendly guide that is meant to hold your hand and whisper sweet explanations in your ear as you install and first explore ZDCS. It's arranged to take you through all the steps you need to know from beginning to end. PURPOSE OF ZDCS --------------- ZDCS is a shareware set of utilities intended to help a PCBoard sysop deal with the problem of duplicate files, whether those files are already on the bbs or are being uploaded by a caller. It provides specific support for looking inside ZIP files (including PKZIP version 1.93) and self-extracting files made with PKZIP (SFX). Since both ZIPs and SFXs are treated the same by ZDCS, we'll refer to both as zipfiles. ZDCS also provides support for accepting unzipped GIFs. ZDCS helps to manage the problem of duplicate files in two ways. 1. It provides a method for weeding out duplicate files from an existing collection of files, like a bbs file system. 2. It provides a duplicate checking method for any prospective addition to the existing file base, such as uploads. - 2 - ZDCS was written by a sysop and programmer who knew that sysops never have enough time to do all the things they would like to do on their boards. Making ZDCS friendly to use for both sysop and caller has been important since version 1.0 first debuted. Although ZDCS is intended for use on PCBoard bbs's, it can also be used to look for duplicates on other systems as well, such as a shareware CD-ROM or even multiple directories on your hard drive system. However, it's on a bbs that ZDCS really shows its stuff. ZDCS can be told to decline an upload, to automatically remove duplicate files, to delete those pesky little bbs ads from uploads, and to recognize "allowed duplicates" - or any combination of the above. There's even a pre-test capability that lets callers find out ahead of time whether or not their intended upload duplicates files already on your board. All of these are discussed in this reference manual and described in the walk-through guide ZDCSWALK.TXT as well. GENERAL TECHNICAL INFORMATION ----------------------------- ZDCS handles file archives created with the .ZIP extension by PKZIP (including version PKZIP 1.93) and with the .SFX extension (self- extracting .EXE files) also created by PKZIP. ZDCS is able to see inside each of these archives to deal with the individual files inside them. Since ZIP files are by far more common that SFX files, and since they are both treated the same way by ZDCS, we'll use the term "zipfile" to refer to both of them. ZDCS also handles GIF files. These are individual graphic files that already have their own compression. GIFs may be uploaded to some bbs's as individual files. They are not archives of multiple freestanding files, so ZDCS does not need to look "inside" them. In fact, ZDCS thinks of them rather like poor zipfiles with only a single file in them. Whenever ZDCS encounters a file that is not a ZIP or an SFX, it treats that file like a GIF. Although we will continue to use the term GIF to refer to these files, ZDCS could actually handle any "other" type of file by treating it as a single file, the same as it does with GIFs. ZDCS does not provide support for other archiving methods. This means that if ZDCS encounters an ARJ or LZH archive, for example, it cannot see inside the archive to look at the individual files. (Support for other archive formats is under development.) Instead, the entire archive would be treated as a single GIF-type file. ZDCS makes use of the 32-bit CRC, often called the CRC32, used internally by PKZIP for ZIPs and SFXs. When ZDCS encounters any other file, it considers that file to be a GIF and calculates the CRC32 for it. ZDCS checks files for duplicates by keeping a database of the CRC32 values. When new files are added to the system (uploads), ZDCS - 3 - compares the CRC32s of the newcomers to those in the database to determine if there are any matches, indicating duplicate files. This method uses the identity of the files and is independent of the names of those files. The ZDCS database of CRC32s uses a B-tree index, so there are no sort utilities or regular file maintenance requirements of any kind. ZDCS is compatible with any Netbios compatible lan, such as Lantastic or Novell Netware. ZDCS OPTIONS ------------ ZDCS has four sets of options that can be configured independently of each other and changed at any time. You can try out the different options and change your mind about which ones you want to use without re-installing ZDCS. For a good guide to choosing the options to fit your system, take a look at the walk-through ZDCSWALK.TXT. As part of the installation guide it explains what each option can mean to your bbs and what some of the possible consequences are. Each of these options is also covered in greater technical detail in other sections of this reference manual. Deletion of Duplicate Files --------------------------- ZDCS can be set to either flag or delete duplicate files from uploads to your bbs. Either way, ZDCS will still recognize the files that are duplicates of ones already in the bbs file system and leaves you messages in a log file. This deletion feature does not operate on a GIF file. Allowed Dupicates ----------------- You can designate certain files as allowed duplicates. Although these files may already be present on your bbs, you can tell ZDCS not to treat them like duplicates. This prevents ZDCS from deleting them if you have selected the option to delete duplicate files. It also prevents the allowed duplicates from being counted in the calculation of the percentage of duplicates in an upload. This feature does work for GIF files. BBS Ads ------- You can specify certain files as truly obnoxious. These are collectively referred to as bbs ads after one of our favorites. ZDCS can be told to either flag or delete these files from all uploads. - 4 - This option is completely independent of the option to delete duplicate files. A major distinction is that bbs ads are files that are known and despised ahead of time. This option provides a safe way to delete them without risking the removal of authors' unchanged files from newer shareware versions. The bbs ads option does not operate on a GIF file. Pre-Testing ----------- ZDCS offers a simple method for callers to your bbs to pre-test an upload before actually sending the full file to the board. The pre- test is quick and gives the same information that callers would see after sending the full file. There are no upload credits granted for using just the pre-test feature without sending up the full file. INSTALLATION OVERVIEW --------------------- There are five basic steps to installing ZDCS to work with the bbs: 1. Setting up the configuration file. 2. Creating the initial database. 3. Creating the bbs ads database (optional). 4. Creating the list of allowed duplicates (optional). 5. Setting up the check for uploaded duplicates. There is a walk-through called ZDCSWALK.TXT included in this package to help you install ZDCS. It's a friendly guide to understanding the functions and installing the programs on your bbs. It is especially good for taking you through the steps slowly and in order so that installation goes smoothly on your system. This technical reference manual has information on all the parts, functions and options of ZDCS, but it's arranged with a different purpose in mind: to allow you to find and read only those specific sections you want to know more about. If you want to install ZDCS, we highly recommend using the walk-through as your primary guide and all- around hand-holder. THE ZDCS CONFIGURATION FILE --------------------------- Function -------- The ZDCS configuration file ZDCS.CFG is central to the entire process. All the modules of ZDCS look for and use at least parts of this file. It's also the place where you set many of the options and all of the parameters of ZDCS. For such a powerful component, the ZDCS.CFG is surprisingly simple: just eight short lines of ASCII that you can create with any text - 5 - editor. A sample configuration file is included with this package. The contents of each line are discussed in excruciating detail further on in this section. Under most circumstances, ZDCS.CFG should be in the same directory as the rest of the ZDCS files. This is always true if you are running DOS version 3.x or higher. The one time that the configuration should be located elsewhere is for those poor souls still running DOS 2.x. In this unusual case, ZDCS.CFG belongs in whichever directory will be the current directory when the ZDCS program modules are run. Line 1 ------ This line is the complete drive, path and filename of an ASCII text file. This is a file that you create listing all the pathnames, one on each line, that contain the zipfiles / GIFs to be included in the database. To process a new collection of files into the ZDCS database, like those on a CD-ROM, just change either this line or the contents of the file it points to. There is no upper limit on the number of pathnames that can be processed. But make sure that you've included the trailing backslash for each pathname. If you are not using the index file feature in PCBoard 14.5a, then you can use the DLPATH.LST file from PCBoard to point to the complete bbs file system if you want to. Line 2 ------ This line is the drive and pathname where you want the finished ZDCS database to be located. It makes no difference if you include the trailing backslash here or not. You can put the ZDCS database in the same directory as the rest of the ZDCS files and programs, or you can decide to put it on a different drive or even across the network. Line 3 ------ This line is either the letter "Y" or the letter "N". It controls whether you want ZDCS to delete all duplicate files from an upload (Y) or just flag them and leave them intact (N). Line 4 ------ This line is an integer - that's a whole number, no decimals - between 0 and 100. It sets the maximum percentage of dupes that your bbs will accept in an upload. - 6 - ZDCS will calculate the actual percentage of duplicates in the upload and compare it to your maximum percentage. If the actual percentage is lower, the upload is accepted. If the actual percentage is equal to or higher than the maximum you specified, the upload is declined and moved to your private upload directory for review. Setting the percentage to 100 effectively bypasses this filter, since it permits a duplicated GIF or a zipfile with nothing but duplicates to pass. At the other extreme, setting the percentage to 0 effectively requires that the uploaded GIFs and zipfiles have no duplicates at all. If you make a mistake and enter a decimal on line 4, ZDCS will not crash. It will simply truncate your number (lop off everything after the decimal point) and use the resulting integer as the maximum percentage of dupes. This works out quite well in actual practice. Uploads that are a fraction of a percent under the maximum percentage of duplicates are the only files where this makes any difference, and the use of truncating instead of rounding means that they will always be passed. IF ZDCS rounded the decimal points instead, there would be some uploads that would round up, making their actual percentage of dupes the same as the maximum value. Such files are declined by ZDCS. For more discussion of maximum and actual percentage of dupes, including an example of the above situation, please see the section on the upload file checker ZDCSFC in this manual. Line 5 ------ This line is the complete drive, path and filename you want ZDCS to use for the log file created by the upload file checker ZDCSFC. This log is an ASCII text file that contains information from the upload file checker ZDCSFC for each upload it has processed. Each message about a specific upload includes the following: name of the uploaded zipfile or GIF list of all component files inside a zipfile which files are flagged as bbs ads which files are marked as duplicates which files are marked as allowed duplicates actual percentage of dupes in the upload whether the upload was accepted or declined If PCBOARD.SYS is in the current directory when the upload file checker is run, then the name of the currently logged caller is also included in the log file. Line 6 ------ This line is either the letter "Y" or the letter "N". It controls the switch to tell ZDCS whether to delete bbs ads (Y) in an uploaded - 7 - zipfile or to just flag them (N). If you've decided not to enable any checking for bbs ads at all, just set this to N. Line 7 ------ This line is reserved for a single line of text by the sysop. The contents of this line are appended to the PCBFAIL.TXT file whenever an upload is declined. The caller who has just uploaded the declined file sees this line of text as a message on the screen. This line is where you can express from 1 to 72 characters' worth of creativity. Some callers have become quite fixated on the idea that "declined" is the same as "thrown out" - which is of course not true. You can use this line to tell the caller what has happened or will happpen with the upload. One possible line to use is (without the quotes) "Too many duplicate files - upload must be reviewed by sysop." If you don't want to display any message to the caller, just place something innocuous like a period or even a blank space on this line. Just don't leave the line completely blank! You also shouldn't use any quotation marks in this line. You can make use of the PCBoard @codes; they are fully supported here. Line 8 ------ This line consists of the single letter "Y" or "N". It controls whether ZDCS displays the one line "registered to" message after the board receives an upload (Y) or turns off the display of this message (N). Either way, the caller still sees the file by file breakdown of the upload and the status (duplicate, bbs ads, etc.) of each file. This line is only recognized by the registered version of ZDCS. It has no effect on the three line message displayed by the unregistered version. It is also the only line of the configuration file that you can forget to include without causing major problems. If the line is missing, ZDCS defaults to (Y) and displays the message. This would probably be a fine time to wax poetical about the advantages of registration. Instead, we'll just direct you to the section in this manual on registration for more information about that, and to the section on support for some of the reasons why. (Actions speak louder than words.) THE ZDCS DATABASE BUILD ----------------------- Purpose ------- In order for ZDCS to process new uploads, it must first have created a database for the existing files to serve as a standard of comparison. - 8 - This is referred to as the duplicate file database, the ZDCS database, or the CRC32 database. Information stored in this database includes the full pathname and file name of every zipfile and GIF examined by ZDCS and the names and CRC32 values of every individual file. There is another purpose to building the ZDCS database besides enabling ZDCS to check uploads. It also serves as a tool to help you weed out duplicate files from the existing collection. Creation of the Initial Database -------------------------------- THE ZDCS database of duplicate files is created with the database build program ZDCSDB. The finished database consists of three files: ZDCS.NDX (the index), ZDCS.DAT (part 1 of the data) and ZDCS.PTH (part 2 of the data). A fourth file is also created during this process: ZDCS-DBB.LOG, which is an ASCII text file logging messages from the database creation process. To create the database, you simply run ZDCSDB. This program looks for the ZDCS configuration file to tell it which files you want to process. These can be in multiple directories, on different drives, or across a network. Once you start the build process, there is nothing more to do until it is complete. Everything will run automatically with no need for additional input from you. While the program is running, there is a screen display that gives you the current status of operations and one very important piece of information: use the F10 key if you wish to abort the process. Additional explanation of the screen display is covered in another section of this manual. The Screen Display During the Database Build -------------------------------------------- The ZDCSDB status summary contains both useful and esoteric information. The most important of these is the notice that the F10 key is the one to use in order to abort the database build. Aborting the process any other way will almost certainly lead to lost and/or cross-linked clusters on your hard disk, not to mention a very unhappy sysop. In general, ZDCS treats an SFX file like a ZIP file, and this documentation has reflected that fact by using the term "zipfile" to refer to both types. For the purposes of this section, we will use the individual terms SFX and ZIP, since the screen display does distinguish between them. One line in the middle of the screen will change frequently as ZDCSDB is processing files. Besides letting you know that the system is working, this line tells you something about the processing steps as they take place, just in case you were curious. - 9 - The possibilities are: Directory C1.. ZDCS has just begun processing a new directory Directory C2.. ZDCS has just retrieved the next ZIP / SFX / GIF file within the same directory ZIP Directry.. ZDCS is retrieving information from within an individual zipfile SFX........... ZDCS is retrieving information from within an individual SFX file ZIZ........... ZDCS is processing a ZIP within a ZIP CRC........... ZDCS is calculating the CRC-32 for a GIF file Indexing...... ZDCS is writing the index and data to the Btree files Underneath this changing line is a line that tells you which ZIP, SFX or GIF file is currently being processed, with the full pathname included. Below that is the phrase "Member files" followed by the number of individual files within the ZIP, SFX or GIF being processed. (Remember, a GIF shows up as having only one member file.) The next line down starts with either "Share" or "NoShare" to indicate the type of "file opens" being used. The presence of the DOS share utility in memory is detected by all ZDCS programs to permit automatic use of the appropriate type of file access. The second item on the same line is the word "Files" followed by the number of ZIPs, SFXs and GIFs that have been processed so far. The third item on the same line is "Members" followed by the total of all the individual files that have been added to the CRC32 database so far. Since a GIF is thought of as a ZIP with only one file in it, each GIF that is processed is counted as just one more individual file here. The fourth item is "PDupes" followed by a number. This is arcane internal status information that will not be explained here. (Consider yourself lucky.) The only time you are likely to need this is if there is a probelm with your system during the database build and we ask you to read the screen display to us. The very last line is the time of day when the ZDCSDB program was started. The end time will appear right after the start time on the same line. Additions to an Existing ZDCS Database -------------------------------------- Adding new files to the existing ZDCS database is very simple. If you run the database build ZDCSDB when there already is an existing ZDCS database, the records for the new files will be added to those already in the database. (If you temporarily rename or move the database files mentioned in the database creation section, then ZDCS will not recognize them until you restore them.) This provides a very simple method for adding large or small file collections to an existing database. All you have to do is edit the first line of the configuration file to point to the new files, and - 10 - you're ready to go. Issue the command ZDCSDB and from here on the process will proceed just the same way as it does for creating the initial database. There is a second way to add files to the ZDCS database. Available free on The Hacker Central BBS is a merge utility that lets you combine two ZDCS databases into one. This allows you to create smaller databases at a time and then merge them together, which can be an advantage if you have a very large collection to process. The main advantage of the database merge option is that it allows sysops to use pre-built ZDCS databases for common file collections, like the popular CD-ROMs of shareware programs. Pre-built databases are also available on The Hacker Central and are mentioned in more detail in the section in this manual on support. ZDCSDB Without a Separate File Integrity Checker ------------------------------------------------ ZDCS is not a file integrity checker and does not intend to replace the fine checkers that are already available. Normal operation of ZDCS assumes that you have already processed your files with a file integrity checker. Human nature and circumstances being what they are, a special switch has been included with the database build module to permit some simple file integrity checking to be done. This method has ZDCS call in PKZIP to use the -T switch to test zipfiles. There is no checking performed on GIFs at all. To use this feature, you issue the database build command as ZDCSDB T instead of ZDCSDB. Because ZDCS now calls PKZIP for the file integrity checking in addition to the usual functions of the database build, the processing time is greatly increased. If you are only processing a small collection of files, such as a batch of local uploads, the difference in time is probably unimportant in practice. But if you are processing a large set of files, you'll be better off making your first pass with a file integrity checker and then using the regular ZDCSDB command to build the database. If you run ZDCSDB on a collection of files that includes a corrupt zipfile, the database build will very likely crash. While you can find out after the fact about the damaged zipfile by reading the log, it's still far preferable to avoid the problem entirely by using file integrity checking first. The Database Build Log File ZDCS-DBB.LOG ---------------------------------------- There is a log file called ZDCS-DBB.LOG created by the database build operation. This is an ASCII text file that logs messages from the database creation process. If ZDCSDB encounters damaged zipfiles while trying to build the database, there would be messages alerting you to that fact in the log - 11 - file. Another case that would generate messages here is when ZDCSDB hits a zipfile with more than 650 component files. The first 650 files would be processed, and then ZDCS would leave you a message in the log file letting you know about the situation. If you have any problems while running ZDCSDB, look in this log file for help in understanding what happened. THE ZDCS DUPLICATE REPORT GENERATOR ----------------------------------- The purpose of the duplicate report generator ZDCSDR is to use the information in your ZDCS database to give you a report of all duplicate files in your file collection. The usual time to do this is after you have created the initial database. It is entirely likely that when you first create the initial database you will already have some duplicate files in your collection of zipfiles and GIFs. To find out about them, run ZDCSDR to generate a flat ASCII text file called ZDCS-DUP.LST. This is a list of all duplicate files in the database, including the name and CRC32 of the duplicated file and the identity with full drive and pathname of the zipfile or GIF containing the dupe. The format of the ZDCS-DUP.LST file is the standard comma-separated variable to make it easy to import this file into a database or parse it into a .BAT file. When you run ZDCSDR, it asks you whether you want the results sorted by the CRC32, the individual file name, or the name of the zipfile or GIF containing the dupe. This sort is called the Wichita sort after some roundabout and historical reasons. Once you have the list of duplicates ZDCS-DUP.LST in hand, you are ready to clean out your file system. Use the information to remove any duplicate files from the file collection. On some of the larger bbs file systems, this step alone has freed up megabytes of hard drive space. After you are sure you have finished cleaning up the file system, it is possible to purge duplicate entries from the CRC32 database in order to reduce its size. This is covered in this manual in the section on purging the database. THE ZDCS DATABASE PURGE ----------------------- Every duplicate file in your initial collection of files on the bbs will be reflected by a duplicate entry in the CRC32 database, which ZDCS uses as its standard of comparison for checking new uploads. Only one entry for each unique file is needed in order to recognize future uploads of the same file. You can reduce the size of this ZDCS database by purging duplicate entries from it. This leaves just one single reference to each individual CRC32 in the database. - 12 - Before you make any changes to the ZDCS database at all, please read the section in this manual on the duplicate report generator ZDCSDR. The reporting function of this module will supply you with an accurate list of all your duplicate files, including the necessary path information - but only if you run it *before* you purge the database. Without this list, you'll be hard pressed to find and remove the duplicate files from your system. We strongly recommend that you not only generate this report but also finish cleaning out the file system before you purge anything. When you feel you are ready to purge the database, start by making a backup copy of it first. This is very important. *Please* make a copy of the database and put it away somewhere safe before you start purging info from it. Once you purge this information, the only way to restore it is to rebuild the database. While it is always possible to rebuild the database, this takes a fair bit of time, especially for larger systems. And what sysop has enough time? By now you are probably really ready to get on with it. To purge the database of duplicates, you use the duplicate report generator module ZDCSDR with the P (for Purge) switch. All you have to do is type ZDCSDR P and hit the enter key. The "P" will redirect the program to purging the duplicates instead of creating a report on them. There is nothing else to do until the purge is complete. This is not a fast process, but it will pay you back with a smaller database. A second way of modifying the ZDCS database is to use the database editor, a free utility available on The Hacker Central BBS. This is actually a debugging tool, but it also works to modify the database and to delete specific records. Unlike the database purge with ZDCSDR P, you can't let this one run on its own with no input from you. On the other hand, the database editor give you greater flexibility over which duplicate entries you delete. ZDCS doesn't quite have the intelligence to recognize a bbs ad unless you tell it which files to look for. To do this, you have to create the bbs ads database. BBS ADS ------- Function -------- ZDCS offers a way to recognize specific annoying files that can be targeted for deletion or simply flagged for manual removal, independently of whether or not you have configured ZDCS to delete duplicate files. The distinction here is that the files, known collectively as bbs ads, are *known* files that you have given ZDCS specific orders to deal with. In order to recognize a file as a bbs ad, you must first tell ZDCS about it. This is done by setting up the bbs ads database. After that, every time ZDCS seems the same file again it knows that it's a bbs ad. - 13 - This option is completely independent of the option to delete duplicate files, so you don't have to take a chance on removing authors' unchanged files from newer shareware versions. The bbs ads option does not operate on a GIF file. Creation of the BBS Ads Database -------------------------------- All you need to create the bbs ads database is the program ZDCSBA and a collection of bbs ads, pyramid schemes, chain letters or other obnoxious files. Once created, the bbs ads database will consist of a single file, ZDCS- BBA.NDX, which will be located in the same directory as the rest of the ZDCS files. You can delete this file at any time if you want to start over again with a fresh collection of files. The easiest way to do create the bbs ads database involves collecting all those nasty ads together and zipping them up into one zipfile. You can use whatever name you like for the ZIP. Then you run the program ZDCSBA from the directory containing all the ZDCS files on that zipfile. This is easy to see with an example. If your collection of files to be added is in a zip called BBS-ADS.ZIP, then the command you issue to create the bbs ads database with these files is: ZDCSBA BBS-ADS.ZIP You don't have to do anything else. The program handles everything for creation of the bbs ads database. If you want to create a new bbs ads database in the future, just delete the old database file (ZDCS-BBA.NDX) and follow the steps for creating a new bbs ads database. If you don't delete the old database, then the new ads will be added to the old ones in the database, which is an easy way to add new bbs ads. Updating the BBS Ads Database ----------------------------- There are three easy ways to add new bbs ads as they are foisted off on the bbs community. 1. You can run ZDCSBA on a new zipped collection of ads when you already have an existing bbs ads database. This is good even if the zipped collection only has a single file in it. ZDCSBA works just the same way here as it does for creating a new database, with one exception: the files being processed are added to the existing database. That's all there is to it. Example. Let's say that you've collected some bbs ads from your board, your own downloads, and even from friends. You zip - 14 - the entire set together and call it JUNK.ZIP. To include these files in the bbs ads database, just issue the command: ZDCSBA JUNK.ZIP 2. You can run ZDCSBA on an individual bbs ad without bothering to zip it up first. This is a quick way to catch a new ad as soon as you are lucky enough to get it. Example. You've just come across an obnoxious get-rich-quick scheme called MOOLAH.TXT. To put this file in the bbs ads database, just issue the command: ZDCSBA MOOLAH.TXT 3. You can even run ZDCSBA to include a file in the bbs ads database when you don't have the original file, as long as you have the CRC32 value for it from PKZIP. Just run ZDCSBA on the CRC32 value immediately preceded by a dollar sign $ instead of on the file itself. Example. You don't have the original ad anymore, but you have a record that the CRC32 of a new bbs ad is E1E10999. To put this file in the bbs ads database, just issue the command: ZDCSBA $E1E10999 You can use any or all of these methods to add new records to the bbs ads database. If you have an assortment of zipped collections, a couple of unzipped ads, and a few ads known only by CRC32, the data can all be entered into the same database without having to use the same technique for each individual file. Selecting Deletion or Flagging of BBS Ads ----------------------------------------- If you have built the bbs ads database, ZDCS will automatically detect its presence and will compare uploads against the records in the bbs ads database. What ZDCS does with any matches it finds depends on how you set up the configuration file. In line 6 of the ZDCS configuration file ZDCS.CFG, you typed one of two letters: Y or N. The letter Y tells ZDCS to delete the offending bbs ad. The letter N tells ZDCS not to delete it, but to just flag it as a bbs ad. When a bbs ad in an upload is merely flagged, that information is still displayed to the caller and recorded in the log for the upload file checker. If you wish, you can go back and manually delete such files later on. You can change your mind about using deletion or flagging to deal with bbs ads at any time. All you have to do is edit that one line in the configuration file. No other changes need to be made to run with the new option. - 15 - ALLOWED DUPLICATES ------------------ Allowed duplicates are those files that may already be present in your file collection, but which you do not want ZDCS to treat as duplicates. You can establish a list of files that are allowed duplicates. When ZDCS evaluates an upload, these allowed duplicates will not be flagged as dupes. They are protected from deletion, even if you have configured ZDCS to delete duplicate files. Furthermore, when ZDCS evaluates the actual percentage of duplicates in an upload, these files are not counted. The reason behind the concept of allowed duplicates is that there are some files that reappear frequently as critical components of more than one major shareware package or upgrade version. These might be order forms, registration guides, general text files, small utilities or other files. Two common examples are the OMBUDSMN.ASP file found in all ASP-ware and the VALIDATE.COM program included in every revision of McAfee's SCAN package. Both files turn up unchanged in many zipfiles. It would be misleading to treat these files as ordinary duplicates. There are two functional advantages to using allowed duplicates. First, by not counting allowed duplicates in the actual percentage of dupes calculation for new uploads, your system does not "penalize" a zipfile for having one or more of these files when it comes to deciding whether to accept or decline the file. This is especially significant if you have set a pretty high standard for the "uniqueness" of uploads that your bbs will accept. The second advantage is for those systems that configure ZDCS to delete duplicate files. The use of allowed duplicates can protect popular shareware packages from being stripped of important files. Not only does the deletion of such files alter the content of the author's shareware package, but it also destroys any AV stamp the author placed on the work. Whether or not this option is enabled is controlled by the presence or absence of an ASCII text file called ZDCS.ADN in the ZDCS directory on your system. This file is one that you create with any text editor to list all the files, one per line, that are allowed duplicates. You can designate an allowed duplicate by either its file name or its CRC32. To specify an allowed duplicate by name, just type the dollar sign $ followed immediately by the name of the file with its extension. This preserves a file with a distinctive name (like OMBUDSMN.ASP) even if it undergoes some revisions. To specify an allowed file by its CRC32 value, type the pound sign # followed by the CRC32 for the file. If you don't know the CRC32 offhand, you can get this information from PKZIP. Issuing the PKZIP - V command on any zipfile gives you the CRC32 values for each individual file inside the zipfile. The ZDCS.ADN file may be up to 256 lines long, but must contain no blank lines and no blank spaces. You can edit, delete or recreate - 16 - this file at any time without having to set any other switches, alter the configuration file, or make any other changes to the setup of ZDCS. While the expected application of the allowed duplicate feature is with zipfiles, it does work for GIF files too. If a GIF is included among the allowed duplicates, then a repeat upload of the same GIF will not be flagged as a duplicate, and the upload will be accepted. THE UPLOAD FILE CHECKER ----------------------- Purpose ------- Once you have created the initial ZDCS duplicate file database, you can get the bbs to check all uploaded zipfiles and GIFs against it from then on. This is done by processing the uploaded zipfiles and GIFs with the upload file checker ZDCSFC as the files are received. The primary purpose is to catch uploaded files that duplicate ones already on the bbs. ZDCSFC can also flag bbs ads, as described in that section of this manual. Depending on whether you have enabled any deletion options in the configuration file, ZDCSFC can also perform deletions of duplicate files and/or bbs ads. With the use of a list of allowed duplicates, also described in a separate section in this manual, the file checking performed by ZDCSFC can be made even more sophisticated. Function -------- ZDCSFC works with the CRC32 value for each file in an upload. If the uploaded file is a GIF, it first calculates the CRC32 for that file. If the uploaded file is a zipfile, ZDCSFC reads the value of the CRC32 for each individual file in the zipfile. Then ZDCSFC compares the CRC32 for each file in the upload against the ZDCS database. If you have created the bbs ads database and left it in the ZDCS directory on your system, ZDCSFC also compares the files in the upload against the bbs ads database. The results are displayed file by file for the caller after the upload has been received and processed, providing real time feedback. The same information is also available to the sysop as part of the ZDCSFC log file. The complete path and file name for this log were specified as line 5 of the configuration file. ZDCSFC also calculates the actual percentage of duplicates in any upload and compares this against the maximum percentage you have set for the board. On the basis of this comparison, an upload may be accepted or declined. Declined uploads are not deleted from the system. PCBoard moves a declined upload to your private directory for sysop review. - 17 - The calculation of percent duplicates is covered in more detail in a separate section of this manual dealing with maximum and actual percent duplicates in an upload. Maximum and Actual Percent Dupes in an Upload --------------------------------------------- ZDCSFC calculates the actual percentage of duplicate files in the upload. Since a GIF is a single file, it will either be 0% (not a dupe) or 100% (a dupe). For zipfiles, this actual percentage can vary anywhere between 0 and 100. ZDCSFC compares this actual percentage against the maximum percentage you set in line 4 of the configuration file. If the actual percentage is lower than the maximum, the upload is accepted. If the actual percentage is equal to or higher than the maximum you specified, the upload is declined. PCBoard moves these declined files into your private upload directory, where you can review them. Some callers have become frantic at the idea that a "declined" file is thrown out. This does *not* happen with ZDCS. If you want to bypass this filter, set the percentage to 100. This permits a duplicated GIF or a zipfile with nothing but duplicates to pass the filter and never be declined. At the other extreme, you can set the percentage to 0. This effectively requires that the uploaded GIFs and zipfiles have no duplicates at all, or they will be declined. The section in this manual that describes the configuration file specifies that the maximum percentage of dupes in line 4 must be a whole number, not a decimal. This is because ZDCS deliberately truncates any decimal from this value. ZDCS does not round up or down but performs a straightforward truncation on both the actual and the maximum percentage of dupes. This was done for reasons of memory management. An example may make it clearer how this approach works in actual practice. Suppose that the maximum percentage of duplicates is set to 20. An upload with an actual percentage of 19.9 would be truncated down to 19. ZDCS would see that 19 is less than 20 and would accept this upload. If the actual percentage were rounded instead of truncated, it would be 20 instead of 19. ZDCS would then decline this upload because the actual percentage would not be less than the maximum percentage of dupes. Updating ZDCS Database(s) After an Upload ----------------------------------------- ZDCSFC automatically updates the duplicate files database with the CRC32s of all uploads. You do not have to do anything special to include this information in the database for comparison with future uploads. ZDCSFC does not modify the bbs ads database at all. It's still not smart enough to recognize a bbs ad until you've pointed it out first - but it does remember them next time it sees them. Please see the section in this manual on the bbs ads database for more information. - 18 - Calling ZDCS from EXZTEST ------------------------- ZDCS should be used in conjunction with a file integrity checker. If you have decided to use EXZTEST, then how you call ZDCSFC to process new uploads depends on which version of EXZTEST you are running. The newer versions of EXZTEST provide a seamless integration with ZDCS that takes care of calling the ZDCSFC module, feeding it the information it needs, and completing any file deletions that you have enabled. This is available in EXZTEST version 2.0 and higher. A sample PCBTEST.BAT file that calls the newer (2.x) version of EXZTEST is included with this package under the file name PCBTEST.ALT. For more information on the ZDCS integration implemented in EXZTEST 2.x, please see the documentation for EXZTEST. The older versions of EXZTEST before version 2.0 can still be used in conjunction with ZDCSFC to check new uploads. Instead of relying on EXZTEST to call ZDCS, you must call the upload file checker from the PCBTEST.BAT file directly. A sample PCBTEST.BAT file that calls the older (1.x) version of EXZTEST and then calls ZDCSFC is included with this package under the file name PCBTEST.BAT. You'll have to modify it a bit to reflect the pathnames and other details specific to your system, but it's a good general guideline. More detailed information can be found in this manual in the section on calling ZDCSFC from the PCBTEST.BAT file. This section is more general and addresses the use of any file integrity checker. Calling ZDCSFC from the PCBTEST.BAT File ---------------------------------------- ZDCS should always be used in conjunction with a file integrity checker. When processing new uploads, the file integrity checker should be called from the PCBTEST.BAT file before ZDCSFC. If you are using the PCBDescribe feature in PCBoard 14.5a, that command should be used first of all. To use ZDCS with the file integrity checker of your choice (even if it's just PKZIP -T), there are six basic pieces you need to include in the PCBTEST.BAT file. 1. Include the following three lines to clean out old copies of these files left over from processing other uploads: @IF EXIST PCBFAIL.TXT DEL PCBFAIL.TXT @IF EXIST PCBPASS.TXT DEL PCBPASS.TXT @IF EXIST ZDCS-DEL.LST DEL ZDCS-DEL.LST PCBFAIL.TXT contains information displayed to the caller when an upload is declined. PCBPASS.TXT contains the information displayed when the upload is accepted. - 19 - ZDCS-DEL.LST is a list of files targeted by ZDCS for deletion from an upload. These might be duplicate files, bbs ads, or both. If you haven't enabled any deletion options in the configuration file, or if there were no files meeting the criteria for deletion, ZDCS would not generate the ZDCS-DEL.LST file for that upload. By deleting the files here, you make sure that no old deletion instructions are hanging around when a new file is processed. 2. Call your file integrity checker to process the upload. The exact command to do this will depend on which integrity checker you are using. The most basic and simple one of all is to make use of this ability in PKZIP by using the -T switch, but there are other programs to do file integrity checking and more. 3. Call ZDCSFC to check the upload. The appropriate command is: ZDCSFC %1 %2 %3 If you are running a version of PCBoard older than the May 22, 1991 version 14.5a, PCBoard will provide only two parameters. Since the third one is absolutely necessary for pre-testing, that means you must disable the pre-testing feature. There is a way to get around this limitation if you absolutely must. It's not exactly elegant, but it does work. If you need to know more about it, contact us in one of the conferences mentioned in the support section of this manual. If you are using a newer version of PCBoard but you don't want to use the pre-testing right now, it is still strongly recommended that you leave the third parameter in place. It does no harm and could save you some grief if you change your mind in the future. At the end of processing by ZDCSFC, you might have a new file called ZDCS-DEL.LST. This depends on whether you have enabled any deletions and on whether there were any files in the upload that met your criteria for deletion. If no files are to be deleted, this control file won't be created. 4. Skip to the end if there are no files to be deleted: IF NOT EXIST ZDCS-DEL.LST GOTO END This line tests for the presence of the ZDCS-DEL.LST file. If the file is not found, it means that there are no deletions to be performed on the current upload. 5. Perform the deletion of files specified by ZDCSFC: PKZIP -D %1 @ZDCS-DEL.LST Note that the actual deletion is done by PKZIP. ZDCSFC creates the control file ZDCS-DEL.LST to specify the deletions and PCBTEST.BAT passes this file to PKZIP at this point. This is also the only time an existing AV stamp on an upload is affected by ZDCS. More information on AV stamp integrity is available in a separate section of this manual. - 20 - 6. Mark the end: :END When there are no files to be deleted, this statement marks the location where control in the PCBTEST.BAT file is passed. DOS Error Levels ---------------- In addition to creating the required PCBPASS.TXT and PCBFAIL.TXT files, ZDCSFC also sets the DOS error level when it exits. These levels are: 0 No duplicate files were found within the upload. 1 Some duplicates were found, but the upload passed the percentage test. 2 Too many duplicates were found, and the upload failed the percentage test. 3 Every file within the upload was a duplicate. 4 There is no number 4. (Reserved for future expansion.) 5 Upload checking by ZDCSFC was aborted. Please see log for error message. PROCESSING LOCAL UPLOADS ------------------------ There probably isn't a sysop alive who hasn't scavenged new files for the board and uploaded them locally. ZDCS offers two ways of processing these local uploads. The first method is to use the database builder ZDCSDB; the second method is to use the upload file checker ZDCSFC that processes all the regular uploads from callers. In the first case, ZDCSDB can be used to add all the new files to the database. Just like the initial creation of the ZDCS database or subsequent database merges, ZDCSDB adds the CRC32's of the new files to the database. The duplicate report generator ZDCSDR can still be used to tell you about duplicates in the database. A disadvantage to this method is that it does not give the sysop the same kind of immediate feedback for a local upload as the sysop and caller get for a regular upload. It also does not delete bbs ads or duplicate files (if you have those options enabled). To use ZDCSDB to process local uploads, you only have to change the first line of the ZDCS configuration file. This line points to the paths of the files to be processed when running the database builder. (The line contains the full path / name of the text file that lists all the directories whose files are to be processed.) Change this to - 21 - point to the uploads directory, a holding directory, or whatever directory will hold the local uploads to be processed. Now just run ZDCSDB and the files will be added to the existing database. This method does not interfere with the realtime upload checking performed by ZDCSFC on regular uploads, because the path information in line 1 of the configuration file is not used by ZDCSFC. If you've decided on a permanent holding directory, you won't even have to touch the configuration file again to process any future uploads. Just remember to run ZDCSDB before posting new files on the board. This is a good time to repeat one note of caution: ZDCS does not do any integrity testing. ZDCS is intended to work with your choice of integrity tester, not to replace it. If you have not used an integrity checker on the local upload files (why not?), then you need to use the T (for Test) switch. Instead of using the command ZDCSDB to run the database build, use the ZDCSDB T command. This calls PKZIP to do an integrity check on the zipfiles. Any file that fails this test is not processed by the database builder. There will be a message in the database building log file ZDCS-DBB.LOG for each such damaged file. Don't use the T switch without reason - it adds another step and that slows the processing down tremendously. But if you have a collection of files that haven't passed through an integrity checker, like prospective local uploads, ZDCSDB T is the way to deal with them. If ZDCSDB has to skip a file because it's flagged as damaged, that information will show up in ZDCS-DBB.LOG. The second method mimics regular uploading to get the local uploads processed by the upload file checker ZDCSFC. This is handled via a small utility and batch file combination available free on The Hacker Central BBS as LOCALUP.ZIP. You'll have to modify the batch file for your own system, of course. The result is that the local uploads will see the same processing that a regular caller's upload does on your board - including your usual file integrity checker, bbs ad deletion (if enabled), duplicate file deletion (if enabled), and all the rest. This has the additional advantage of giving you the ZDCSFC log file entries for the local uploads, which gives the status of the files inside the zipfiles: duplicates, bbs ads, allowed duplicates, etc. Local upload processing is an area targeted for future enhancement in ZDCS. PRE-TESTING ----------- Callers can pre-test an upload to find out how it compares to files already on the bbs in terms of duplicates and bbs ads. The idea is to see how the prospective upload compares to existing files in terms of duplicates, bbs ads, etc. before actually uploading the full file to the bbs. It can save both caller and sysop a certain amount of frustration. - 22 - The procedure is wonderfully simple from a caller's point of view and requires nothing that might tax a relatively novice uploader's skills. There are no special files to download, no complicated operations to get right, no arcane rituals to perform. This makes it more likely that callers will take advantage of the pre-test feature. Included in this release of ZDCS is a sample bulletin that can be posted by the sysop on the bbs to explain to callers the ZDCS pre- testing feature for zipfiles. The bulletin holds the caller's hands, figuratively speaking, through the whole process. If your bbs permits the uploading of SFX or GIF files, you might want to add those initials where you see ZIP in the bulletin. A ZIP-only bbs can use the canned bulletin right from the package. To pre-test an upload called GOODSTUF.ZIP, the caller simply issues the DOS command: PKZIP -V GOODSTUF.ZIP > ZDCSTEST.CHK This command is not case sensitive, so it's pretty hard to make a mistake with it. The caller then uploads the file ZDCSTEST.CHK directly to the bbs. ZDCS will read the information in it and will give the caller a breakdown of the zipfile's contents, noting which individual files are duplicates or bbs ads. This output looks just like the response a caller gets after actually uploading the zipfile, with one exception: instead of telling the caller whether the file is accepted or declined, ZDCS tells the caller what percentage of files in the upload are duplicates, and also gives the maximum percent duplicates which the sysop has allowed for the board. Now it's up to the caller to decide whether to upload or not. Enabling the pre-test capability is not tough on the sysop, either. The key to whether or not pre-testing is permitted on your bbs lies in the UPSEC file. If you want to prevent callers from using pre- testing, just disallow uploads that have the .CHK extension. On the other hand, if you want callers to have the option to pre-test, make sure that .CHK files may be uploaded to the system. There's one more thing to check before letting your callers use the pre-testing feature. If you are not using EXZTEST version 2.0 or higher, remember to make sure that the PCBTEST.BAT file contains the following line: ZDCSFC %1 %2 %3 This is already discussed elsewhere in this manual as part of the ZDCS installation. That third parameter is necessary to let the pre- testing work for more than just the first caller. If you leave it out, the first ZDCSTEST.CHK file uploaded to your system will work just fine, but all subsequent attempts to pre-test will be told that the ZDCSTEST.CHK file is a duplicate! And that means they won't be recognized as attempts at pre-testing. If you are using EXZTEST version 2.0 and up as your file integrity checker, you won't have a ZDCS line in the PCBTEST.BAT file. That's - 23 - because EXZTEST has a seamless interface to ZDCS and calls it directly. Since this third parameter is critical to the proper functioning of the pre-test option, you must have a recent enough version of PCBoard that supports the third parameter. The earliest version of the board code that satisfies this requirement is PCBoard 14.5a from May 22, 1991. This version and anything after it can handle ZDCS pre-testing. If you track your callers' uploads and downloads, you might like to know that the ZDCSTEST.CHK will not count as an upload. According to ZDCS and PCBoard, the ZDCSTEST.CHK file always "fails" validation and is not credited to the caller as an upload. The bulletin does mention that pre-testing doesn't count as an upload, just in case some of your callers are too enthusiastic about getting upload credits. ZIPs WITHIN ZIPs ---------------- Processing of zipfiles contained within zipfiles is accomplished with some caveats. Zipfiles within a zipfile are only checked one level deep. The simplest explanation is to look at an example. ZIP A contains assorted files and ZIP B. In turn, ZIP B contains more files and another ZIP, C. ZIP C contains still more files. How does the whole melange get processed? All the files in ZIP A and in ZIP B have their CRC32 signatures entered into the duplicate files database. If you have configured ZDCS to delete any files (dupes or bbs ads), then those deletions are done automatically only for the individual (non-zipped) files inside ZIP A. No files are deleted from inside ZIP B. Of course, all the duplicates in ZIP B are still listed in the log, so you do know about them and you can decide whether to remove them manually from your file system. What about ZIP C? That's easy: any zipfile embedded more than one level deep in the uploaded zipfile (and C is two levels deep) is not processed as a zipfile at all. No file deletions, no CRCs, nothing. In fact, *any* file that has the .ZIP extension will not be treated as an individual file in the database or the log file. The reason is simple: it's the individual files inside the zipfiles and not the zipfile itself that are important when looking at duplicates. (Besides, it plays havoc with the percentages.) CD-ROMs ------- CD-ROMs have been growing in popularity among bbs's because they enable a large, static collection of files (such as shareware programs) to be available for downloading. ZDCS works just fine with CD-ROM drives, but the sheer size of these disks means that it can take quite a while to build the ZDCS database for a CD-ROM. - 24 - To make life easier, a new ZDCS database merge utility is now available on The Hacker Central. By merging databases, it is possible to take advantage of the existence of prebuilt ZDCS databases for the more popular CD-ROMS. This lets you merge a prebuilt database into your own bbs's database in a fraction of the time it would take to add all the CD-ROM information from scratch. Prebuilt ZDCS databases for some CD-ROMs are available free on The Hacker Central BBS. If you have a CD-ROM that doesn't already have a prebuilt ZDCS database, please consider creating a ZDCS database for the individual CD-ROM and sharing it with other ZDCS users by uploading it to The Hacker Central. During 1992, we are offering a free ZDCS registration as a thank you to anyone who uploads such a database. AV STAMP INTEGRITY ------------------ Registered versions of PKZIP can "brand" a zipfile with the unique AV (authenticity verification) number of the registered owner. The AV stamp is used by some shareware authors for their official distribution packages. It is also abused by some people, including a few bbs sysops, sad to say. Whether ZDCS will always leave that AV stamp intact is determined by the configuration of ZDCS. The first possibility is that ZDCS is configured never to delete any files, whether duplicate files or bbs ads. In this case, the original AV stamp is always retained. The second possibility is that ZDCS is configured to delete files, but no files need to be deleted from a particular zipfile. In that case, any AV on the original zipfile is left intact. The third possibility is that ZDCS deletes one or more bbs ads. Whether the AV stamp is left intact depends on whether the bbs ad was included in the zipfile inside or outside the AV branding. If the offending bbs just added their advertising to the outside of an AV stamped zipfile and left the original author's AV stamp in place, ZDCS will do the same. If that bbs added their overhead file and then rezipped the entire package with their own AV stamp (don't laugh, there's at least one large bbs that's done this), then they have destroyed any original AV stamp. ZDCS does not preserve the AV stamp in this case. The fourth possibility is that ZDCS deletes one or more duplicate files. If any deleted file was included in the AV branding, then the AV stamp is not preserved. If all deleted files were outside the original AV stamp, then the AV stamp is intact. (This could happen with some advertising file that you've already been hit with but which doesn't appear in your bbs ads database.) This last possibility deserves a little thought from the shareware author's point of view. It is not uncommon for a new version to contain some files that are unchanged from the previous version. If - 25 - these files haven't been flagged as allowed duplicates, it is possible to choose to configure ZDCS so that these files are removed. If that happens, the AV stamp is also removed. This is in keeping with the ideal that the stamp should represent an authenticity verification from the original author. ZDCS is quite capable of handling and retaining all AV stamps. This is a matter of choice on the part of the sysop and is the direct result of the simple configuration options. ACCURACY OF THE CRC32 METHOD ---------------------------- In order to detect duplicates, some unique signature is needed for each file. While the only truly unique signature is as long and cumbersome as the file itself, there are more manageable alternatives. Any algorithm is a trade-off between these two factors: manageability and accuracy. The more unique signatures a method can generate, the smaller the possibility that two different files might generate the same signature. ZDCS makes use of the CRC32 used by PKZIP. This signature has a total of eight "places", each filled by independent assortment with one of sixteen different values. That makes for a total of 16*16*16*16*16*16*16*16 (16 to the 8th power) unique signatures. With over four and a quarter billion possible signatures, this method has good accuracy. It is possible to use other methods to permit an even greater number of unique signatures. The longer signatures and different calculation methods carry a corresponding cost in complexity, disk space and processing speed. MEMORY LIMITS ------------- The ZDCS upload file checker ZDCSFC.EXE requires a minimum of 384K of free memory (RAM). There is an internal limit of 650 files within a zipfile for processing by both the upload file checker ZDCSFC.EXE and the database creator ZDCSDB.EXE. If there are more files inside the zipfile, ZDCS will handle the first 650 it encounters and will leave a message in the log. There is nothing to reset and no effect on processing the next file ZDCS encounters. These limits are imposed by the DOS 640K Limit. TROUBLE-SHOOTING GUIDE ---------------------- There is no better help for solving problems than experience, especially someone else's experience with the same problems. - 26 - ZDCSDB (the database build) is crashing. --------------------------------------- There is a log file ZDCS-BBA.LOG that collects messages when ZDCSDB is used to create a new duplicate file database or add to an existing one. Read the messages in the log file to find out why the database build program is crashing. Most of the time, the culprit is a damaged zipfile. ZDCS normally depends on the fact that most sysops use a separate integrity file checker on their boards. If a corrupted zipfile gets into the collection of files being processed into the database, it can bring things to a sudden halt. The recommended solution is to process all files with a file integrity checker first. However, there is a second alternative using just ZDCS. You can use the T (for Test) switch to tell ZDCSDB to call PKZIP and perform a file check first. Only those files passed by this check are processed by the database builder. The syntax is simple: ZDCSDB T instead of ZDCSDB. The penalty in speed for doing it this way is very significant, which is why it is not the usually preferred solution. ZDCS is reporting a device I/O error (in any module). ---------------------------------------------------- So far, we have seen this happen twice. Each time, ZDCS was running on a LAN. In both cases, the problem was traced back to bad spots on the hard drives. The best medicine for this error message is to run Norton Disk Doctor, Spinrite, or something similar on your hard drive system. ZDCS reports memory corrupt errors (in any module). -------------------------------------------------- This sounds like an earlier version of ZDCS, probably version 1.62 or earlier. Those versions required more free RAM and sometimes ran out of memory, which resulted in the misleading "memory corrupt" messages. Time to stop reading the docs and get this newer release (version 1.65) installed! The improved memory management in ZDCS 1.65 should solve your problems. ZDCS reports path/file access errors. ------------------------------------ One possibility is that you are using an earlier version of ZDCS on a LAN *and* you are not loading SHARE before running ZDCS. This is most likely to happen on a Novell Netware LAN. You can solve this one by loading SHARE before you run any of the ZDCS modules on a network. Better still, you can upgrade to this version 1.65 of ZDCS. If you have loaded SHARE high under DOS 5.0, you might still be experiencing the same symptoms. There seems to be a problem with - 27 - loading SHARE high; it's not anything specifically to do with ZDCS. Try loading SHARE low instead. Another possibility is that you haven't specified enough files in your CONFIG.SYS file. Because ZDCS makes such heavy use of disk I/O, you should make sure that at least 30 files are specified in CONFIG.SYS (FILES=30). Of course, if you are trying to use the upload file checker ZDCSFC before you've built the duplicate file database with ZDCSDB, you shouldn't be surprised if ZDCS acts a little weird. Try building the database first and then comparing new files against it. Things work much better that way. QEMM exception 13 errors occur when running ZDCS. ------------------------------------------------ There are five possible reasons for this error message. 1. You're running an earlier version of ZDCS, probably version 1.62 or even earlier. Those versions required more free RAM and sometimes ran out of memory, which could result in the QEMM error messages. Upgrade to the current version 1.65 of ZDCS, which has improved memory management. 2. Your QEMM command line is incorrect. The problem isn't with ZDCS itself, but ZDCS is likely to be the first program to notice this QEMM error because of the amount of data ZDCS needs to juggle. The result is that ZDCS is able to write its data someplace it shouldn't. To solve this problem, check that all areas of memory that should be excluded from QEMM are being properly excluded. Be especially careful if your system has super VGA, SCSI devices or a LAN: all of these devices usually require specific areas of memory to be excluded from remapping by QEMM. 3. You have one or more "mis-behaved" device drivers. You can check out this possibility by removing all drivers from your CONFIG.SYS file, rebooting the system, and trying ZDCS again. One example of this type of problem was with an older version of the Hyperdisk cache. 4. You believed the QEMM optimize program when it told you to get rid of DOS stack space. Put it back! QEMM is egocentric enough to believe it can handle all types of stack requirement for any application properly. Let's just say that putting back the DOS stack space solves this problem nicely. 5. You are trying to use the "stealth technology" feature of QEMM 6.x. There have been persistent reports of problems traced to interactions between stealth and various applications from other vendors. It wouldn't be at all surprising if ZDCS experienced the same trouble with stealth. This is not a bug in ZDCS. Contact Quarterdeck tech support. In the meantime, disable stealth by removing the ST:F or ST:M from the DEVICE=QEMM.SYS line in the CONFIG.SYS file. - 28 - REGISTRATION ------------ ZDCS is a fully functional shareware package. There are no critical limits, crippled features or "drop dead" dates. The only difference in the unregistered version is a brief message to sysop and caller that the version is unregistered. A customized ZDCS.KEY file is available to registered users to change the "unregistered" message to show registration in the name of <BBS Name> or to eliminate the display of any message at all. Registration entitles the user to a license for use of the Zipfile Duplicate Checking System on one bbs, no matter how many nodes it has. This includes all future versions of ZDCS. No additional fees will be charged for registration of future versions of this product. If you try ZDCS out on your system and decide that you want to continue using it, please register your copy by sending a check for $25.00 (US). Include the name of your bbs (up to 25 characters) as you want it to appear to your callers. An order form (ORDER.FRM) is included in this release for your convenience. We will prepare a key file for your copy of ZDCS with this information in it to replace the "unregistered version" line in the display. This keyfile will be available for download from The Hacker Central BBS. If your bbs is located in the continental United States or Canada, you may take advantage of the keyfile delivery service any time during 1992. For an additional $5.00 (US), we will prepare your keyfile and upload it to your bbs. Just set up an account in the name of ZDCS SUPPORT with the password KEYFILE and with sufficient security to upload a private file for the sysop. When you send in your registration of $25.00 plus $5.00, remember to include the name and number of the bbs. As of 1991, ZDCS can now be registered online at The Hacker Central BBS with a valid Visa or Mastercard. Look for the ZDCS script questionnaire in order to register. The keyfile delivery service is also available as part of the online registration. Processing usually takes from one to three days. If you prefer to register by check or money order, please make your registration check payable to Michael W. Cocke and mail it along with the necessary information (like a completed order form) to: Michael W. Cocke 11 Cedar Road Montville NJ 07045-9582 Please be assured that ZDCS will continue to receive support through future revisions. For over a year now it has been in use on its home board, The Hacker Central BBS. When the programmer is also a sysop and has to live with the results of that programming work every day, you can be sure that the support will be there! - 29 - SUPPORT ------- ZDCS product support will be handled on the ZDCS home board and via echoed conferences. Questions, comments and discussion of ZDCS are welcome in the following conferences: SHAREWARE (ILink) ZDCS (Intelec) ZDCS (Hacker Central, also echoed by many ILink bbs's) Additional product support in the form of downloadable files (including registration keys) is available on the ZDCS home board, The Hacker Central BBS. First time access to The Hacker Central BBS must be done via the public node. After completing the new user and visiting sysop scripts, all sysops and ZDCS users (whether or not they have registered ZDCS yet) are granted access to the two private nodes, including the high speed line. The Hacker Central phone numbers are: Node 1 201-334-2555 Public 2400 Node 2 201-316-8840 Private 2400 MNP Node 3 201-335-9343 Private USR HST/DS Please leave a comment to sysop on your first call indicating your interest in ZDCS in order to get access to the ZDCS support conference. Requests are usually handled in a day or two. Another aspect of ZDCS product support is the collection of utilities and pre-built CD-ROM databases available for download from The Hacker Central BBS for all ZDCS users. This collection is constantly growing in response to suggestions, requests and cries for help. A few of the special utilities are described below. The database merge module enables you to combine more than one ZDCS duplicate file database into a single database. This has the tremendous advantage of permitting pre-built ZDCS databases to be created for the more popular CD-ROMS. Once accomplished, a single CD- ROM database can be merged into any number of bbs databases in a fraction of the time it would take to create the CD-ROM database from scratch. Pre-built CD-ROM databases are available on The Hacker Central. We encourage anyone who has a CD-ROM without a prebuilt database to create the ZDCS database for the CD-ROM alone and to upload it to The Hacker Central to share with other sysops. To encourage this, we are offering a free ZDCS registration to anyone who uploads a new CD-ROM ZDCS database to The Hacker Central BBS during 1992. The FWK to ZDCS converter will take the data from an FWKCS(tm) database and convert it to a ZDCS database without requiring the original source files to be available. If you have tried FWKCS(tm) and want to evaluate or switch over to ZDCS, this makes the process easier and faster. If you have used an earlier version of ZDCS (1.5x) and haven't upgraded to the smaller and faster version 1.6, there is a free - 30 - upgrade kit available on The Hacker Central to help you do that without starting over again from the beginning. The database editor permits you to modify or delete records in a ZDCS database. It is intended primarily as a debugging tool. It is strongly recommended that a backup copy of the database be made before embarking on any editing. A small local uploads package has been put together to make procesing local uploads easier for the sysop who also scavenges files for the board. Additional work is also in progess for improving the handling of local uploads. Development on other utilities continues. One module in progress is geared toward ferreting out instances of bbs ads that might have crept into the database. With some bbs's changing their ads frequently, this can happen before the sysop realizes it and flags the offending ads. Suggestions for other new utilities are always welcome. All ZDCS utilities to date are free but are only available on The Hacker Central BBS. Please do not upload these modules to other bbs's or distribute them as part of the shareware ZDCS package. A final note about ZDCS support concerns documentation. We feel that clear and helpful documentation is an important part of a sysop utility package like ZDCS. In addition to the plain text files included in this package, we also have the documentation available in WinWord 2.0 with full use of styles, headers, outline processing and other features. If you are interested in obtaining a more presentation-quality copy of the documentation or in discussing conversion to other formats such as WordPerfect, please contact us on The Hacker Central BBS. If you have any questions, comments or suggestions concerning the documentation, we would like to hear from you, too. FUTURE ENHANCEMENTS ------------------- ZDCS is a dynamic product. In response to suggestions, comments and the changing bbs scene, ZDCS continues to be upgraded. Future enhancements under consideration or in progress include support for other archive formats besides PKZIP, improved local upload handling, and database path updating. Path updating would keep the file location information in the ZDCS database current even after files are moved around in the file system. Right now, the essential identification information (the CRC32 signature) remains fine, but the path pointing to the file becomes out of date when the file is moved. This is usually the case with uploads that are moved from a recent uploads directory to another destination. Path updating would allow the sysop to locate these and any other moved files more quickly by keeping the path information current as well. - 31 - ACKNOWLEDGEMENTS ---------------- Thanks to all the beta testers who have helped to make ZDCS a better and smoother program. Their patience and input are definitely reflected in the final product. Thanks to Tom Hughes for his prowess as a bug hunter extraordinaire, and for wanting a database merge utility. That module was a great idea, Tom! Thanks to David Terry for several clever ideas on better ways to do things. That kind of help is always appreciated. Thanks to Bob Jacobson for his thoroughness in finding a particularly persistent and recalcitrant bug during the recent long beta cycle. It (the bug, not Bob) has been sighted and squashed. Special thanks to Andy Keeves for pointing out a better hammer in the toolbox. It was by making use of his suggestion for a different sorting routine that the duplicate report generator was made significantly faster. Finally, thanks to everyone who had the patience to wait while version 1.65 went through six months of intense beta bashing. COPYRIGHTS AND LEGAL STUFF -------------------------- ZDCS (Zipfile Duplicate Checking System) is copyright (C) 1991 by Michael W. Cocke. ZDCS 1.65 is fully functional shareware. It may be freely copied and distributed, provided that no files in this package are removed or altered in any way. The individual documentation files ZDCS-REF.TXT, ZDCS.BLT and ZDCSWALK.TXT are copyright (C) 1991 by Evelyne Stalzer. Permission is granted to distribute these files as part of the complete ZDCS shareware package. Permission is also granted to bbs sysops to modify ZDCS.BLT for use as needed, including as a bulletin, message or downloadable help file. ZDCS.KEY is the individual registration key file and may not be copied, distributed or otherwise shared with individuals beyond the registered user. Neither Michael W. Cocke, Evelyne Stalzer nor MWC Enterprises will accept responsibility for the function, failure to function, or side effects of any function of the Zipfile Duplicate Checking System (ZDCS). ZDCS is provided in good faith, but its use is solely at the risk of the operator. EXZTEST and EXZIP are copyright Andy Keeves. FWKCS is trademark and copyright Fred Kantor. Lantastic is trademark of ArtiSoft Inc. MS-DOS is copyright and trademark of Microsoft Corp. Netware is trademark of Novell Inc. PCBoard is copyright and trademark of Clark Development Company. PKZIP and PKUNZIP are copyright and trademark of PKWARE, Inc. ZDCS is copyright (C) 1991, Michael W. Cocke.