Monster Media 1994 #1/monster.zip/monster/PCBOARD/ZDCS203.ZIP/ZDCSWALK.TXT

Text File | 1994-03-28 | 44KB | 982 lines

          ZIPFILE DUPLICATE CHECKING SYSTEM (ZDCS)  Ver. 2.0
                Copyright (C) 1991-1994 by Stacy Smith
 --------------------------------------------------------------------

                     The Bloom Beacon-Picayune BBS
       [phone]  (USR Courier HST/V.32terbo Dual Standard)
                 [phone]  (USR Courier V.32bis)
                [phone]  (Hayes Optima 288 V.FC)



  A Walk-Through Guide for Installing and Using ZDCS on a PCBoard BBS


          Table of Contents
          -----------------

          HELLO!
          OVERVIEW OF ZDCS
          ZDCS OPTIONS
               Deleting Duplicate Files
               Allowed Duplicates
               BBS Ads
               Privileged Users
               Pre-Testing
          INSTALLATION
          STEP 1 - Copy the ZDCS files
          STEP 2 - Set up the config file
               Line 1
               Line 2
               Line 3
               Line 4
               Line 5
               Line 6
               Line 7
               Line 8
               Line 9
               Line 10
               Line 11
               Line 12
               Line 13
               Line 14
               Line 15
          STEP 3 - Create the initial database
          STEP 4 - Create the bbs ads database
          STEP 5 - Create the list of allowed duplicates
          STEP 6 - Set up the upload file check
          PRE-TESTING
          THAT'S ALL, FOLKS!



HELLO!
------

ZDCS is a shareware set of utilities intended to help a PCBoard sysop
deal with the problem of duplicate files, whether those files are
already on the bbs or are being uploaded by a caller.  It provides
specific support for looking inside archives made with PKZIP,
including PKZIP version 1.93 (ZIP files), archives made with ARJ (ARJ
files), and self-extracting files made with either PKZIP or ARJ (SFX
files).  ZDCS processes all of these archives so that the differences
between them are transparent to the sysop and caller.  To make this
walk-through easier to read, we'll refer to all of these file types as
zipfiles.

ZDCS also provides support for accepting unzipped GIFs.  ZDCS thinks
of such a GIF file as a poor zipfile with only one file in it.  You'll
find more detailed information on GIF files and others in the
reference manual, ZDCS-REF.TXT.

This walk-through will guide you in getting ZDCS up and running on
your bbs.  The reference manual, ZDCS-REF.TXT, has more detail on all
sorts of things (probably more than you will want or need), but it's
arranged to be the sort of file where you look things up when you want
more information, not something you read all the way through.  On the
other hand, this walk-through is a friendly guide that will take you
by the hand and show you how easy it is to install ZDCS.

Don't be put off by the length of this file.  ZDCS is quite simple to
install.  We've added lots of extra words to explain the process and
your choices for how you want to run ZDCS.



OVERVIEW OF ZDCS
----------------

There are two basic functional parts to ZDCS:  checking an existing
collection of files for duplicates, and checking newcomer files
against the existing collection.  It's set up this way to permit use
on a dynamic collection like a bbs file system, but it can also be
used to look for duplicates on other systems as well, such as a
shareware CD-ROM or even multiple directories on your hard drive
system.  However, the focus is on bbs systems and that's how the
documentation is written.

To check an existing collection of files, you have to create a ZDCS
database for that collection.  Then you can use the duplicate report
generator to tell you which files are duplicates and where they are
found.  A little housecleaning based on this report can weed out
duplicates and recover hard drive storage space, something no one ever
has enough of.

When new files are added to the collection, you need to be able to do
two things with them:  add the files to the database;  and find out if
there are any duplicates among them.  The most common addition of new
files is via uploads to the bbs.  ZDCS will enable you to process an
upload as soon as it is received, and tells both you and your caller
about duplicate files in that upload.  It can be set to accept or
decline an upload based on the percentage of dupes it finds.  It also
adds any new files to the ZDCS database to keep that up to date.

There's a bit more, of course.  ZDCS can be told to decline an upload,
to automatically remove duplicate files, to delete those pesky little
bbs ads from uploads, and to recognize "allowed duplicates" - or any
combination of the above.  You can tell ZDCS to accept anything that a
privileged user uploads, and change your mind on who's "privileged"
and who's not.  There's even a pre-test capability that lets callers
find out ahead of time whether or not their intended upload duplicates
files already on your board.

Installing ZDCS to work on your bbs is relatively simple.  The hardest
part may be deciding which options you want to enable.  Let's look at
those first.



ZDCS OPTIONS
------------

Running ZDCS without any of the extra options enabled gives you a
basic duplicate checking system that will not delete any files
automatically.  It will still tell you about all duplicate files, but
it won't be able to distinguish between files you always want to
delete (like bbs ads), files that duplicate exactly ones already on
your board, and files that are duplicates but still important to keep
(like VALIDATE.COM from Macafee's SCAN program).

You can try out the different options and change your mind about which
ones you want to use without re-installing ZDCS.


Deleting Duplicate Files
------------------------

When a new zipfile is uploaded to the board, ZDCS checks it against
the existing database to see if there is any duplication of files.
You can permit ZDCS to not only flag the duplicates but also to delete
them from inside the zipfile, leaving the rest of the upload intact.
This deletion feature does not operate on a GIF file.

This is a good time to point out one consequence of deleting all
dupes.  Some shareware authors issue updates that consist of both new
files (executables, perhaps) and unchanged files (registration
information, maybe).  If you enable the deletion of all duplicate
files in an uploaded zipfile, you can lose some of the files that
belong in the author's package.  This brings us to the concept of
allowed duplicates.


Allowed Duplicates
------------------

You can choose to tell ZDCS that certain files are allowed duplicates.
When you create a list of allowed duplicates, ZDCS knows not to treat
these files as dupes no matter how often they appear.  You can add new
files to the list or delete old ones with any text editor.

Why would you want to have allowed duplicates?  There are some files
that reappear frequently as part of shareware or freeware packages,
such as OMBUDSMN.ASP (found in ASP-ware), or VALIDATE.DOC and
VALIDATE.COM (from Macafee's SCAN program).  Especially in a case like
Macafee's where new versions of the program come out frequently, each
with certain standard files included, it would be useful to
"recognize" these duplicate files as being acceptable.

There are two places where enabling the allowed duplicates option
makes a difference in how ZDCS runs.  If you have selected the option
to delete duplicate files, it prevents these allowed files from being
deleted.  It also does not figure these files into the calculation for
what percentage of an uploaded zipfile is duplicates.

This feature does work for GIF files.  If a GIF is included among the
allowed duplicates, then a repeat upload of the same GIF will not be
flagged as a duplicate, and the upload will be accepted.


BBS Ads
-------

There are some specific files that you may want to delete every time
you run across them.  The prime example of this would be those rude
and annoying bbs ads from you-know-who that a few boards feel it
necessary to add to every zipfile they carry.  You might also have
other problem files that you want to keep off your board, like some of
the pyramid scheme get-rich-quick files that circulate from time to
time.

You can opt to tell ZDCS to recognize these nuisances by creating a
bbs ads database.  (This is always referred to as the "bbs ads
database" to distinguish it from the main ZDCS database, which is
bigger and not the least bit optional.)  When you run the upload file
checker, ZDCS will flag any files it finds that match the ones listed
in the bbs ads database.  If you want to do more that just flag these
pests, you can also tell ZDCS to delete all bbs ads automatically.  Of
course, you can include new bbs ads as they are perpetrated.

This option is completely independent of the option to delete
duplicate files, so you don't take a chance on removing authors'
unchanged files from newer shareware versions.  The bbs ads option
does not operate on a GIF file.


Privileged Users
----------------

You can designate one or more user names as privileged users.  Any
file that is uploaded by a privileged user is never declined by ZDCS,
no matter how many duplicate files there may be in it.  Even if you
have enabled deletion of duplicate files and / or bbs ads, ZDCS still
will not delete them if the original zipfile was uploaded by a
privileged user.  The privileged user status takes precedence.  The
list of privileged usernames is a simple ASCII text file that can be
changed without needing to reinstall ZDCS.


Pre-Testing
-----------

You can decide whenever you'd like to allow or disallow pre-testing.
Since it's very easy to enable and won't affect the installation of
ZDCS at all, we've covered it in a separate section at the end of this
walk-through.  After you've gotten ZDCS installed on your bbs, you can
take your time about making the decision to pre-test or not to pre-
test.



INSTALLATION
------------

There are six basic steps to installing ZDCS to work with the bbs:

1.   Copying the ZDCS files to a new ZDCS directory.   (required)
2.   Setting up the configuration file.           (required)
3.   Creating the initial database.               (required)
4.   Creating the bbs ads database.               (optional)
5.   Creating the list of allowed duplicates.          (optional)
6.   Setting up the check for uploaded duplicates.     (required)

None of the steps is at all difficult.  We'll cover them one by one.



STEP 1 - Copy the ZDCS files
----------------------------

Create a new directory on your system to contain your ZDCS files.  A
good choice is C:\ZDCS - it's clear, concise, and short enough to fit
in your PATH statement.  Please add the directory to your PATH
statement right away.  We'll wait.

Now copy the ZDCS files into your new directory.  You can put all the
files from this package into that directory if you want to keep them
all together, but at a minimum you =must= put all the executable
(*.EXE) files from this package there.  There will be more files
coming later, as you create the configuration file and enable some of
the options.

From now on, this new directory will be referred to as your ZDCS
directory.



STEP 2 - Set up the config file
-------------------------------

The first step in the installation is to create the ZDCS.CFG file for
your configuration.  This is a flat ASCII text file containing
thirteen short lines.  A sample configuration file is included in this
package, so you might want to look at that.  It takes more time to
describe the file than it does to write one from scratch.

Put the finished configuration file into the same directory that
contains the executable ZDCS files, unless you're running DOS 2.x.  If
that's the case, either upgrade to at least DOS 3.x or go to the ZDCS
reference manual ZDCS-REF.TXT and read the answer there.


Line 1
------

This line is the complete drive, path and filename of an ASCII text
file.  The text file identified by this line is one which you create
listing all the pathnames, one on each line, that contain the zipfiles
and GIFs to be included in the database.

PCBoard refers to this as the download path list and defaults to
DLPATH.LST as the file name in PCB setup.  You can use that file to
point to the complete bbs file directory if you are =not= using the
index file feature in PCBoard 14.5a.  You can also use the freeware
utility DIRTREE (by Mike Cocke, available on The Central Core BBS)
to create this file.

There is no upper limit on the number of pathnames that can be
processed.  It does not matter whether or not you've included the
trailing backslash for each pathname.

To process a new collection of files into the ZDCS database, like
those on a CD-ROM, just change either this line or the contents of the
file to which it points.


Line 2
------

This line is the complete drive, path and filename giving the location
of the ZDCS database (ZDCS.NDX and ZDCS.DAT).  It makes no difference
if you include the trailing backslash here or not.

You can put the ZDCS database in the same directory as the rest of the
ZDCS files and programs, or you can decide to put it on a different
drive or even across the network.  This was made possible because the
database is the largest pair of files, and some sysops needed the
flexibility of locating them on a drive with more spare room.  All the
rest of the ZDCS files (such as the bbs ads, the allowed duplicates,
and whatnot) are grouped together in the same directory with the
executables.


Line 3
------

This line is the complete drive, path and filename giving the location of
the privileged user list file.  If you don't want to have any privileged
users on your system, simply leave this line blank.

What's a privileged user?  Someone who can upload no wrong.  <grin>
Whenever a file is uploaded by a user named on the privileged user list,
the file is explicitly passed by ZDCS, no matter how many duplicates
there might be in it.

Why would anyone want a privileged user?  This feature was added for a
couple of sysops who wanted to pass specific files (beta code) to each
other as part of unattended mail runs, and wanted those files
automatically posted for public access instead of possibly held for later
sysop review.

The format of the file containing the privileged user list is
straightforward:  one name per line, ending each line with a CR/LF.  The
list is not case sensitive.  There is no maximum number of names you may
put in the privileged user list, but remember that if you make this a
long list, ZDCS will take longer to check new uploads because it will
have to check every name in this list.


Line 4
------

This line is either the letter Y or the letter N.  It controls whether
you want ZDCS to add the disposition line to the end of the upload
description.  The disposition line shows the total number of files in an
upload and the number of those that were duplicate files.  It is used
when ZDCS declines an upload.  Here is a sample disposition line:

               ZDCS:  12  Duplicates /  12  files

The actual numbers will change depending on the file.

ZDCS has to be running in standalone mode for this, not with any of the
gateways such as with Extest.  When ZDCS is running with a gateway, then
the actual contents of this line are ignored by the program.  You still
need to have the line present so the total line count doesn't fall off,
but the line can contain anything as a placeholder because the
information will be handled directly via the gateway.

This new feature was added by request.  Please note that you must be
running PCBoard 14.5a in order to make use of it, and that the third
command line parameter must be specified (ZDCSFC %1 %2 %3) in your
PCBTEST.BAT file.  Otherwise, leave this fourth line of the config set
to the letter N.


Line 5
------

This line is either the letter Y or the letter N.  It controls whether
you want ZDCS to truncate nulls from the end of "other" type files
before performing any operations on them.  The truncation is actually
done on a copy of the file and the original is left intact.

Nulls (ASCII 0) can be added to the end of a file by some transfer
protocols (such as Ymodem) in order to make the entire file come out
on a block segment.  Other protocols (such as Zmodem) do not add the
null characters.  That would make the identical file uploaded by the
two different protocols slightly different files by the time they
arrived on your system.  If the nulls were included in the file
contents when any calculation or comparison was done, it would look
like two different files instead of the same file transferred by two
different protocols.  Setting this line to the letter Y enables ZDCS
to ignore those extra nulls.

The advantage of using this feature is increased accuracy when the
same file is uploaded by different transfer protocols.  The
disadvantage is that it adds a bit of time to the upload checking and
database build operations.  The time is needed for ZDCS to create a
temporary copy of the file in order to remove the exrta nulls, and
again to delete that temporary file, leaving the original upload
intact.


Line 6
------

This line is an integer - that's a whole number, no decimals - between
0 and 100.  It sets the maximum percentage of dupes that your bbs will
accept in an upload.

ZDCS will calculate the actual percentage of duplicates in the upload
and compare it to your maximum percentage.  If the actual percentage
is lower, the upload is accepted.  If the actual percentage is equal
to or higher than the maximum you specified, the upload is declined
and kept in your private upload directory for your review.

Setting the percentage to 100 effectively bypasses this filter, since
it permits a duplicated GIF or a zipfile with nothing but duplicates
to pass.  At the other extreme, setting the percentage to 0
effectively requires that the uploaded GIFs and zipfiles have no
duplicates at all.

If you make a mistake and enter a decimal on line 6, ZDCS will not
crash.  It will simply truncate your number (lop off everything after
the decimal point) and use the resulting integer as the maximum
percentage of dupes.

This works out quite well in actual practice.  Uploads that are a
fraction of a percent under the maximum percentage of duplicates are
the only files where this makes any difference, and the use of
truncating instead of rounding means that they will always be passed.
IF ZDCS rounded the decimal points instead, there would be some
uploads that would round up, making their actual percentage of dupes
the same as the maximum value.  Such files are declined by ZDCS.

For more discussion of maximum and actual percentage of dupes,
including an example of the above situation, please see the section on
the upload file checker ZDCSFC in the manual.


Line 7
------

This line is the complete drive, path and filename you want ZDCS to
use for the log file created by the upload file checker ZDCSFC.  This
log is an ASCII text file that contains information from the upload
file checker ZDCSFC for each upload it has processed.  Each message
about a specific upload includes the following:

     name of the uploaded zipfile or GIF
     list of all component files inside a zipfile
     which files are flagged as bbs ads
     which files are marked as duplicates
     reference to the file already in the database which is duped by
the UL
     which files are marked as allowed duplicates
     actual percentage of dupes in the upload
     whether the upload was accepted or declined

The fifth line in the list above is new in ZDCS 2.0.  It means that
when a duplicate file is found in an upload, ZDCS will provide the
reference to the zipfile or GIF containing the copy of that file which
was already on the bbs.  It happened that callers who had uploaded
files with lots of dupes were not always able to find where those
files already existed on the system, often because files had been
renamed.  Those callers sometimes pestered the sysop for this
information, which wasn't always easy for the sysops to locate,
either.  Now this reference is written directly to the upload checker
log file The information includes the name of the zipfile or GIF and
its pathnumber in the system.

If PCBOARD.SYS is in the current directory when the upload file
checker is run, then the name of the currently logged caller is also
included in the log file.


Line 8
------

This line is either the letter Y or the letter N.  It controls the
switch to tell ZDCS whether to delete bbs ads (Y) in an uploaded
zipfile or to just flag them (N).  If you've decided not to enable any
checking for bbs ads at all, just set this to N.


Line 9
------

This line is either the letter Y or the letter N.  It controls whether
you want ZDCS to delete all duplicate files from an upload (Y) or just
flag them and leave them intact (N).


Line 10
-------
This line is reserved for a single line of text by the sysop.  The
contents of this line are appended to the PCBFAIL.TXT file whenever an
upload is declined.  The caller who has just uploaded the declined
file sees this line of text as a message on the screen.

This line is where you can express from 1 to 72 characters' worth of
creativity.  Some callers have become quite fixated on the idea that
"declined" is the same as "thrown out" - which is of course not true.
You can use this line to tell the caller what has happened or will
happpen with the upload.  One possible line to use is:

     Too many duplicate files - upload must be reviewed by sysop.

If you don't want to display any message to the caller, just place
something innocuous like a period or even a blank space on this line.
Just don't leave the line completely blank!  You also shouldn't use
any quotation marks in this line.  You can make use of PCBoard
@variables and &filespec to your heart's content;  both are fully
supported here.


Line 11
-------
This line contains the filename ZDCSFC.OUT and nothing else.  For all
practical purposes, this line is planning for the future.

Technically, this line points to the name of an ASCII text file that
will be created every time an upload is processed.  It's a spare copy
of the ZDCS summary information that most upload checkers overwrite.
ZPEND and some utilities-in-progress use it.

Leave this line set to ZDCSFC.OUT until further notice or features.


Line 12
-------

This line is the complete drive and pathname of a RAM drive that is
available to ZDCS for certain types of processing work.

ZDCS can make use of a RAM drive to speed up the processing of
embedded zipfiles.  It can also use the RAM drive to process "other"
files if you set config line 5 to Y in order to truncate nulls.

The existance of the RAM drive is verified by ZDCS, but the amount of
space available on it is not checked.  If you run out of space on the
RAM drive while ZDCS is processing an upload, the upload will be
declined.

If you do not want to use a RAM drive, leave this line blank.


Line 13
-------

The complete drive, path, and file specification for PKUNZIP.EXE -
INCLUDING the extension (EXE)!  If you omit this or enter it
incorrectly, you'll see a warning message in your ZDCSFC.LOG file
explaining that embedded archives will not be processed.


Line 14
-------

The complete drive, path, and file specification for ARJ.EXE -
INCLUDING the extension (EXE)!  If you omit this or enter it
incorrectly, you'll see a warning message in your ZDCSFC.LOG file
explaining that embedded archives will not be processed.


Line 15
-------
This line consists of the single letter Y or N.  It controls whether
ZDCS displays the one line "Registered to..." message after the board
receives an upload (Y) or turns off the display of this message (N).
Either way, the caller still sees the file by file breakdown of the
upload and the status (duplicate, bbs ads, etc.) of each file.

This line is only recognized by the registered version of ZDCS.  It
has no effect on the three line message displayed by the unregistered
version.  It is also the only line of the configuration file that you
can forget to include without causing major problems.  If the line is
missing, ZDCS defaults to (Y) and displays the message.

This would probably be a fine time to wax poetical about the
advantages of registration.  Instead, we'll just direct you to the
section in this manual on registration for more information about
that, and to the section on support for some of the reasons why.
(Actions speak louder than words.)



STEP 3 - Create the initial database
------------------------------------

The next step after creating the configuration file is to create the
initial database of CRC value values.  To do this, you simply run the
database build program ZDCSDB.EXE.   Assuming that you have created
the configuration file properly, there is nothing more to be done
until this program finishes processing the files.

While ZDCSDB.EXE is running, the display points out that you may press
the escape key at any time to abort.  This is the only safe way to
abort the database build!  Ignore this warning and the chances are
good that you will be rewarded with lost clusters on your hard disk.
(This has been an official warning.)

There will also be other useful and esoteric bits of information on
the screen, but you can read more about those in the reference manual
ZDCS-REF.TXT.  The one that will probably interest you the most is on
the very last line, right after the start time.  After the database
build is complete, the end time appears here.

Usually you will be using ZDCSDB.EXE on a collection of files that
have already passed a file integrity checker.  If for some reason this
is not the case, you can still use ZDCSDB on those files by making use
of the T (for Test) switch.  This slows the processing down
tremendously.  It's really a better solution to use a file integrity
checker to handle this duty, but ZDCSDB T is a workable alternative if
you need it.

What happens when you use the T?  ZDCS determines whether the zipfile
was made with PKZIP or with ARJ, and then sends out to the appropriate
one to come in and test each zipfile.  Any one that is damaged is
marked and not processed by the database builder.  GIF files are
passed over, so they receive no file integrity checking at all.  The
GIFs are still processed by ZDCS to add them to the database.

There is a log file called ZDCS-DBB.LOG created by the database build
operation.  This is where you would find messages about corrupt
zipfiles ZDCS encounters while trying to build the database, for
example.  If you have any problems while running ZDCSDB.EXE, look in
this log file for help in understanding what happened.

It is entirely likely that when you first create the initial database
you will already have some duplicate files in your collection of
zipfiles and GIFs.  To find out about them, use the database report
generator ZDCSDR.EXE to generate a flat ASCII text file called ZDCS-
DUP.LST.  This will create a three part report that is ready for
viewing or printing.

Part one is a list of zipfiles or GIFs that are 100% duplicates of
other files in the database.  Part two is a list of zipfiles that have
some level of duplication, but also contain at least one non-
duplicated file.  Part three is the complete list of all duplicate
files in the database, including the name and CRC value of the
duplicated file and the identity with full drive and pathname of the
zipfile or GIF containing the dupe.

Note that no duplicate files are deleted by ZDCSDR.EXE when you create
the initial database.  The list of duplicates, ZDCS-DUP.LST, can be
used by the sysop to remove any duplicate files in the system.

After the sysop has cleaned up the file system and removed any
duplicate files, it is possible to purge duplicate entries from the
CRC database in order to reduce its size.  This is covered in the
reference manual ZDCS-REF.TXT.



STEP 4 - Create the bbs ads database
------------------------------------

This step is optional.  You need it only if you want to flag and/or
delete bbs ads from uploaded zipfiles.  If you want to complete the
basic installation and do this step at a later date, that's fine too.

ZDCS doesn't quite have the intelligence to recognize a bbs ad unless
you tell it which files to look for.  To do this, you have to create
the bbs ads database.  This consists of a single file, ZDCS-BBA.NDX,
which will be located in the same directory as the rest of the ZDCS
files.

The easiest way to do this is to first collect all those nasty ads
together and collect them up into one zipfile.  You can do this using
either PKZIP or ARJ.  Use whatever name you like for the file;  this
example is going to call it BBS-ADS.  Then run the program ZDCSBA.EXE
from the directory containing all the ZDCS files.  The syntax (now 6%
in NJ) is:

     ZDCSBA BBS-ADS.ZIP       or        ZDCSBA BBS-ADS.ARJ

(If you have used a different name for your collection of bbs ads,
just use that name in place of BBS-ADS.)  The program will create the
database files and you will be ready to flag or delete all that free
advertising.  Whether or not you delete the bbs ads is controlled by
line 8 of the configuration file.  You did read the section on setting
up the configuration file, right?

If you want to create a new bbs ads database in the future, just
delete the old bbs ads database file (ZDCS-BBA.NDX) and follow these
same steps to create the new one.  If you don't delete the old
database, then the new ads will be added to the old ones in the
database, which is an easy way to add new bbs ads.

To make it even easier to include single ads as they come on the
market, you can also run this program on an individual bbs ad, whether
it is archiveded (with PKZIP or ARJ) or unarchived.  So, if a new bbs
ad named HAPPY.BBS shows up, you can issue the command:

          ZDCSBA HAPPY.BBS

This will add HAPPY.BBS directly to the bbs ads database.

There are other ways of adding bbs ads to the bbs ads database, some
of which don't even require you to have the original nuisance files on
hand any more.  For more information about this, see the section in
the technical reference manual on UPDATING THE BBS ADS DATABASE.

The variety of ways you can add new bbs ads to the database makes it
easy to share information with other sysops and to catch new
annoyances as they are foisted upon us.



STEP 5 - Create the list of allowed duplicates
----------------------------------------------

This step is also optional.  You need to do this only if you want ZDCS
to recognize certain files as allowed duplicates.  Allowed duplicates
were discussed in the section on ZDCS options earlier in this walk-
through.

Allowed dupes are a special case.  They are not flagged as dupes when
they are encountered by ZDCS;  they are also not considered "original"
or non-dupe files.  They are transparent in any calculations,
completely and utterly ignored.  They aren't even counted when
reporting the number of files in the zipfile.

An example may make this clearer.  Suppose a zipfile had a total of 15
files.  Two of those were allowed dupes, three were regular dupes, and
one was a bbs ad.  ZDCS would report the following information:

          13   files in the zipfile
           1   bbs ad
           3   dupes
          25%  dupes

The number of files in the zipfile would be 13 and not 15 because the
two allowed dupes are excluded from all the calculations.  The
percentage of dupes would be 25% (100% * 3/12) because the bbs ad,
once identified, would not be part of the calculation of duplicate and
original files.

Whether or not the option to recognize allowed dupes is enabled is
controlled by the presence or absence of an ASCII text file called
ZDCS.ADN in the ZDCS directory on your system.  This file is one that
you create with any text editor to list all the files, one per line,
that are allowed duplicates.  You can designate an allowed duplicate
by either its file name or its CRC value.  You can even mix the two
styles in the same file if you want to specify some files by name
(like VALIDATE.DOC) and others by CRC value.

To specify an allowed duplicate by name, just type the dollar sign $
followed immediately by the name of the file with its extension - no
blank spaces in between.  This preserves a file with a distinctive
name (like OMBUDSMN.ASP) even if it undergoes some revisions.

To specify an allowed file by its CRC value, type the pound sign #
followed immediately by the CRC value for the file - no blank spaces
in between.  If you don't know the CRC value offhand, you can get this
information from either PKZIP or ARJ.  With PKZIP, type

          PKZIP -V <ZIP filename>

to give you the CRC values for each individual file inside the ZIP
file.  With ARJ, type

          ARJ L <ARJ filename>

to give you the same CRC information.

The ZDCS.ADN file may be up to 256 lines long, but must contain no
blank lines and no blank spaces.



STEP 6 - Set up the upload file check
-------------------------------------

Almost done!  Now that you have created the ZDCS duplicate file
database, all you need to do is get the bbs to check all uploaded
zipfiles and GIFs from now on.  This will be done by processing the
uploaded zipfiles and GIFs with the upload file checker ZDCSFC.EXE as
the files are received.

ZDCSFC.EXE is the real-time upload file checker.  If the uploaded file
was a GIF, ZDCSFC.EXE first calculates the CRC value for it.  Then it
compares the CRC value of the uploaded zipfile or GIF against the ZDCS
database and the bbs ads database (if you've created the bbs ads
database, that is).  The results of this comparison are displayed file
by file for the caller and logged into the ZDCSFC log file.  (Remember
back on line 7 of the configuration file, when you specified the path
and filename for this log?)

ZDCSFC.EXE calculates the actual percentage of duplicate files in the
upload.  Since a GIF is a single file, it will either be 0% (not a
dupe) or 100% (a dupe).  For zipfiles, this actual percentage can vary
anywhere between 0 and 100.  ZDCSFC.EXE compares this actual
percentage against the maximum percentage you set in line 6 of the
configuration file.  (Boy, that configuration file is a handly little
thing.)

If the actual percentage is lower than the maximum, the upload is
accepted.   If the actual percentage is equal to or higher than the
maximum you specified, the upload is declined.  The file is never lost
or dumped!  PCBoard keeps these declined files in your private upload
directory, where you can review them.

If you want to bypass this filter, set the percentage to 100.  This
permits a duplicated GIF, or a zipfile with nothing but duplicates, to
pass the filter and never be declined.  At the other extreme, you can
set the percentage to 0.  This effectively requires that the uploaded
GIFs and zipfiles have no duplicates at all, or they will be declined.

ZDCSFC.EXE also updates the ZDCS database with the CRC values of all
uploads to keep your database current.  You do not have to do anything
special to include this information in the database for comparisons
with future uploads.

ZDCSFC.EXE does not modify the bbs ads database at all.  It's still
not smart enough to recognize a bbs ad until you've pointed it out
first - but it does remember them next time it sees them.

To process the new uploads with ZDCSFC.EXE, it must be called by the
PCBTEST.BAT file of PCBoard or by your file integrity checker.

You may choose whichever file integrity checker you like best (or have
already installed) to work with ZDCS.  You might find the integration
between the two to be seamless, as with EXZTEST version 2.23 and the
coming Xtest 3.0.  Even if the two aren't integrated, you can still
call ZDCSFC after your file integrity checker has finished processing
the upload, and ZDCS will take its turn on the file.  There are a
couple of special utilities that smooth the meeting between various
file integrity checkers and ZDCSFC.EXE.  You'll find more information
on them in the technical reference manual, and the utilities
themselves on The Bloom Beacon-Picayune BBS.

To use ZDCS with the file integrity checker of your choice (even if
it's just PKZIP -T), there are a couple of things you need to include
in the PCBTEST.BAT file.

First, it's highly recommended that you include the following three
lines to clean out old copies of these files left over from processing
other uploads:

               @IF EXIST PCBFAIL.TXT DEL PCBFAIL.TXT
               @IF EXIST PCBPASS.TXT DEL PCBPASS.TXT
               @IF EXIST ZDCS-DEL.LST DEL ZDCS-DEL.LST

Second, call your integrity file checker to process the upload.  (Your
syntax may vary.)

Third, call ZDCSFC.EXE to check the upload.  The appropriate command
is:

               ZDCSFC %1 %2 %3

The three parameters %1, %2 and %3 are provided by PCBoard 14.5a.  If
you are using ZDCS on a different bbs or on a non-bbs application like
CD-ROM mastering, check out the third party interface file for more
information on how to handle this.  The file is available on The
Bloom Beacon-Picayune as ZDCS-TPI.ZIP.

If you are running a version of PCBoard older than the May 22, 1991
version 14.5a, PCBoard will provide only two parameters.  Since the
third one is needed for pre-testing (and for the disposition line),
that means you must not enable those features. Other than that, ZDCS
should be OK on those older versions.

At the end of processing by ZDCSFC.EXE, you either have a new file
called ZDCS-DEL.LST or you don't.  This file contains the list of
individual files within an uploaded zipfile that are targeted for
deletion.  If you haven't enabled any deletion of either duplicate
files or bbs ads, this file won't be created.  Even if you have
enabled one or both of these deletions, the file still won't be
created unless the upload that was just processed had something to be
deleted.

Fourth, if there are files to be deleted, then perform the deletion:

          IF EXIST ZDCS-DEL.LST PKZIP -D %1 @ZDCS-DEL.LST
          IF EXIST ZDCS-DEL.LST ARJ D %1 !ZDCS-DEL.LST

If the zipfile was made with PKZIP, then the first line will take care
of it and the second line will return an error message.  If the
zipfile was created with ARJ, then the first line will return an error
message and the second line will take care of the file.  The error
message can be ignored;  it will not lock up your machine or halt the
processing.

Note that the actual deletion is done by PKZIP or ARJ.  This is also
the only time an existing AV stamp on an upload can be affected by
ZDCS.

This concludes our tour of PCBTEST.BAT.

In addition to creating the required PCBPASS.TXT and PCBFAIL.TXT
files, ZDCSFC.EXE also sets the DOS error level when it exits.  These
levels are explained in the reference manual ZDCS-REF.TXT.



PRE-TESTING
-----------

Callers have been enthusiastic about duplicate checking on a bbs file
system - until they discover that the huge file they've just finished
sending your bbs at 2400 baud is a set of complete duplicates.  Many
of them asked for a way to pre-test an upload before actually
transmitting the entire file.  ZDCS gives them this capability without
requiring any ongoing handholding, explanations or help from the
sysop.

Pre-testing is most valuable when it's simple enough that callers
actually use it.  ZDCS pre-testing requires nothing that might tax a
relatively novice uploader's skills.  There are no special files to
download, no complicated operations to get right, no arcane rituals to
perform.  All that your callers need is PKZIP and enough skill to
upload a file.  (An alternate pre-testing method for systems using ARJ
is in development but awaits the completion of the current ARJ beta
cycle.)

There are four simple things for you to do so that ZDCS pre-testing
runs smoothly on your bbs.

1. Make sure that your board code is no earlier than PCBoard version
   14.5a from May 22, 1991.  This and all subsequent versions of
   PCBoard support ZDCS pre-testing.

2. Confirm that the PCBTEST.BAT file contains the following line:

                  ZDCSFC %1 %2 %3

   You already set up the PCBTEST.BAT file as part of the ZDCS
   installation.  Just check it one more time, OK?  If you've
   inadvertently left out the third parameter, the pre-test will work
   just fine for the first caller and will barf all over subsequent
   callers, who will come complaining to you that their ZDCSTEST.CHK
   is being called a duplicate!

3. Make sure that *.CHK files may be uploaded to your bbs.  This is
   done via the UPSEC file, part of PCBoard.  If you don't permit
   callers to upload files with the .CHK extension, you've
   effectively prevented them from using the pre-test feature.

4. When you're ready, post the sample bulletin included in this
   release of ZDCS to explain to callers what the pre-test feature is
   all about.  The bulletin holds your callers' hands through the
   whole process, which should take some of the load off you.  If
   your bbs permits the uploading of ARJ, SFX or GIF files, you might
   want to add those initials where you see ZIP in the bulletin.  A
   ZIP-only bbs can use the canned bulletin right from the package.

If you track your callers' uploads and downloads, you might like to
know that the ZDCSTEST.CHK will not count as an upload.  The bulletin
mentions this little fact just in case some of your callers are a bit
=too= enthusiastic about collecting upload credits.



THAT'S ALL, FOLKS!
------------------

That's all there is to it, and there ain't no more.  ZDCS is fully
installed on your system.  If you want to fiddle with the options, go
right ahead.  Most of them can be turned on or off with no more than a
line change in the configuration file.

If you have more questions or are just curious about parts of ZDCS,
take a look at the reference manual ZDCS-REF.TXT.  It has more
excruciating and technical details than this walk-through does.

You will also find information about registration (ZDCS is reasonably
priced shareware - $25.00), support, access to free utilities and
accessories and a host of other goodies in the reference manual - even
how to reach us with comments and suggestions.