Media Share 9

home *** CD-ROM | disk | FTP | other *** search

/ Media Share 9 / MEDIASHARE_09.ISO / pcboard / ulp099b.zip / ULP.DOC < prev next >

Wrap

Text File | 1993-05-25 | 84KB | 1,572 lines

┌───────────────────┐ │ │ ║ UpLoadProcessor │ ╥ ╥ ╥ │ ║ │ ║ ║ ║ ╓──╖ │ ║ Version 0.99ß │ ║ ║ ║ ║ ║ │ ║ │ ╙───╜ ╨ ║──╜ │ ║ (c) Copyright 1992-1993 - Stacy Smith │ ╨ │ ║ └───────────────────┘ ║ ════════════════════╝ Courtesy of: The Bloom Beacon-Picayune BBS Node 1: (804) 525-9760 (USRobotics Courier Dual Standard) Node 2: (804) 525-5372 (USRobotics Courier V.32bis) FidoNet 1:276/112 Intelec RoseNet (ID: BB-P) Stacy Smith Route 6 Box 189 Forest, Virginia 24551 ┌────────────────────┐ │ 1. Introduction: │ └────────────────────┘ This system was born out of a need for a universal upload processor. There are many alternative systems available, but they are limited to the ZIP format and perhaps one or two others. Few are able to handle self-extracting archives. Most are limited in the number of levels of archive nesting allowed in a file to be tested. All require the use of a third-party duplicate file checking system if you want to screen your uploads for duplicates. Tired of waiting for PKZIP 2.something-or-another, I converted my BBS files over to the ARJ compression format, due to its superior compression ratio over PKZip and its features over LHA. I do not intend to debate the merits of one archive format over another, but due to my decision, the need for a universal upload processor became apparent, so off I went. While I was at it, I decided to incorporate other technologies, such as duplicate checking, archive format conversion, information lines, support for internal description files, etc., into a single package. This software is the result of my efforts to allow my BBS to handle any archive that my users can throw at it. ┌──────────────────────────────────────────────┐ │ 2. Features of the UpLoadProcessor System: │ └──────────────────────────────────────────────┘ ∙ Identifies and processes ARC, ARJ, HYP, LZH, PAK, SQZ, ZIP, ZOO and GIF files, regardless of their file extensions (ideal for software distribution networks, like .SDN files). ∙ Identifies and processes ARJ, LZH, PAK, SQZ and ZIP self-extracting (SFX) archives. ∙ Scans ARC, PAK, ZIP and ZIP SFX archives for DOS reserved keywords to prevent hacking by hex-editing. (ARJ and LHA are resistant to these type of hacking attempts). ∙ Detects ARJ security envelopes and ZIP version 1.x and 2.x authenticity verification (-AV) stamps, and can be left alone after processing (configurable). ∙ Detects and processes ARJ and ZIP archives with imbedded paths. ∙ Detects and rejects encrypted ARJ and ZIP archives. ∙ Selected uncompressed files uploaded can be processed and compressed using your default format. ∙ Removes known BBS ads from archives; includes a BBS ads maintenance utility so sysops can update their BBS ads databases in real time. ULP can also insert a BBS ad, if desired. ∙ Updates the PCBoard DOWNLOAD.TXT file, if desired, with the correct archive extension to reflect the conversion process. ∙ Allows the use of up to 9 different archiving programs, all user- configurable. Any archiving program used that is not listed above will be identified using the file extension only, until it's signature is determined and incorporated into the program. ∙ Uses a recursive processing routine that will allow (theoretically) unlimited nested archives (the only limit is imposed by the DOS path). This routine has been tested to 5 levels deep as of this writing. ∙ Allows the use of up to 3 different file-checking programs, all user- configurable, for virus and trojan checking, 3rd party utilities, etc. ∙ Allows the use of a GIF file checking program, completely configurable. ∙ Rejects GIF files based upon image width, height, colors and compression. ∙ User-definable disposition (rename or delete) of corrupted, duplicate or other archives (not virus-related). ∙ User-definable disposition (rename or delete) of virus-infected archives. ∙ Incorporates it's own CRC-32 duplicate checking system, as well as the associated database processing software. No more executing (and registering!) third-party duplication-checking software, sacrificing speed and memory. This system is extremely fast and it's database is much smaller than other systems. Despite it's size, the probability of a false duplication is more than 4 trillion to 1! The database is self-validating, to quickly determine if a database has been corrupted or altered. ∙ The duplication determination system uses two filters to determine an archive's level of duplication, preventing false rejection by simply counting up the number of blatantly duplicate files. ∙ Converts all uploads into a default archive format of your choosing, or they may be re-archived in their original format (user-defined). Nested archives can also be converted to your default format, or re-archived using their original format. SFX archives can be archived using your default format, or optionally left alone after verification. ∙ Can utilize a user-defined time window (in months) for acceptance of new upload files, based on the most recent file date in the archive. ∙ Changes all archive file attributes to 'read/write' from 'read-only'. ∙ Supports the use of private and public upload directories. Moves files and upload descriptions from the private directory to the public directory. Single directory operation is also available. ∙ Honors the '/' identifier in the description marking the file as a private upload for the sysop by processing the file, but not making it public. ∙ Supports the use of DESC.SDI and FILE-ID.DIZ description files in an archive, user-configurable, for those not running PCBoard 14.5a+ and PCBDESC. ∙ Smart word-wrapping word-wraps descriptions or leaves them as entered, depending upon the presence of boxes, etc. ∙ Can optionally insert an archive or GIF information line in the file description that contains various information about the archive or GIF files. ∙ Two modes of online testing are available: normal mode, which fully unpacks the archive and tests each file individually; and fast mode, which scans a ZIP or ARJ archive directly for file CRCs and dates, and uses the archiving program's internal integrity testing. ∙ The online tester will accept a redirected ARJ or PKUNZIP archive listing file to pre-verify the duplication and age limits before a user uploads the actual archive, saving him wailing and gnashing of teeth. ∙ ULPTEST can generate COM port status output to inform the online user of the progress of testing. ULPTEST supports IRQ2-7, non-standard port addresses and baud rates to 115K in direct mode, and also supports FOSSIL drivers. The port information can be defined on the command-line or can be read directly from PCBOARD.DAT and PCBOARD.SYS. ULP can also be implemented in a "seamless" mode (when registered) to look like a part of PCBoard. ∙ Import of FWKCS CRC database is supported to ease transition to the ULP duplication system. No more rebuilding databases! ∙ Archives failed for exceeding duplication limits can be viewed using ULPDB (the database program), to allow easy determination of manual archive acceptance. ∙ User-selectable process logging to a disk file. Two logging modes are available: terse and verbose. ∙ Written mostly in C (and a little assembler) for optimal speed, using Microsoft C 7.0, Borland C++ 3.1 and Turbo Assembler 3.0. ∙ Lifetime registration; pay ONCE and your registration number will work on all future versions! ┌─────────────────────────────────────────────────────┐ │ 3. Files Included in the ULP Distribution Archive │ └─────────────────────────────────────────────────────┘ ULP.EXE Event upload processing program. ULPTEST.EXE On-line testing program for use in PCBTEST.BAT. ULPDB.EXE CRC-32 database creation and maintenance program. ULPCFG.EXE Configuration program for the ULP system. ULPADS.EXE BBS ads database maintenance program. ULPADS.DB BBS ads database file. ULP.CFG Sample text configuration file for the ULP system. EXCLUDE.LST Sample exclusion list for archivers. PCBTEST.BAT Sample PCBTEST.BAT file for using ULPTEST. ULP.DOC This file. HISTORY.TXT ULP revision history in reverse order. ULPBLT Archive verification bulletin for sysops to post. REGISTER.FRM Registration form for ULP. COMMENT.TXT Sample comment file for inclusion in archive headers. FILE_ID.DIZ Internal description file. When you unzip the distribution archive, you should see my PKZIP authenticity verification stamp, and a '-AV' after every file in the archive: # SSU301 The Bloom Beacon-Picayune BBS If there are any files missing or added, or the -AV stamp is missing, the archive has been tampered with. It would be advisable to call my BBS (listed at the top of this document) for the latest version of ULP. ┌───────────────────────────┐ │ 4. Program Requirements │ └───────────────────────────┘ To the best of my knowledge, this program will run on most any machine capable of running PCBoard 14.5+. My BBS setup is DESQview on a LANtastic network, but other sysops that I have been in contact with have successfully implemented ULP on setups with CD-ROMs and other varying hardware and software. ULP has been developed and tested using the following archiving and file checking software packages: ARJ 2.10 through 2.39f (by Robert Jung) HYPER 2.5 (by P. Sawatzki and K. P. Nischke) LHA 2.12 and 2.13 (by Haruyasu Yoshizaki) LHarc 1.13c (by Haruyasu Yoshizaki) PAK 2.51 (by NoGate Consulting) PKPAK 3.61 (by PKWare) PKZIP 1.10 through 2.04g (by PKWare) SQZ 1.08.2 (by Jonas Hammarberg) ZOO 2.01 through 2.10 (by Rahul Deshi) SCAN V82 through V104 (by McAfee Associates) GIFTEST 4.0 Beta 10 through Beta 14 (by Dave Navarro) The ULP system requires DOS 3.x or later, as it uses DOS SHARE-compatible file reads and writes, and can use the DOS PATH to find the archiving and other utilities. ULP's memory requirements are relatively small (about 80K or so, plus the disk buffer allocated), but all programs are spawned or shelled, which reduces the free memory for the program being executed. It would be a good idea to have as much free conventional memory as possible (ULP itself cannot use EMS or XMS memory), especially if you use the ARJ compression system, which requires in the neighborhood of 280K itself to run. ULPDB also should have as much conventional memory as can be afforded, as it performs its sort in memory for optimal speed; it may use from 128K to 576K of memory depending upon the operation being performed. If your duplication database is larger than 500K (over 50,000 file entries), ULPDB may utilize a segmented sort algorithm, which is slower but capable of completely compiling a database up to 8MB in size (over 800,000 file entries)! The memory sort will probably be fine for many systems, as 50,000 files would probably average out to be about 4,000 to 5,000 typical archives. Obviously, the 286+ and 386+ versions of these programs that is available to registered users will not run on a 8088, 8086 or 80186 (huh?) based computer. So don't try it (I haven't, so I don't have a clue what will happen...probably spontaneous combustion...). ┌───────────────────┐ │ 5. Registration │ └───────────────────┘ The ULP system is not free; nor is ULP is crippled to force registration. ULP is fully functional, and will always remain so. The only variation with the registered copies is no time delay and beg message. Why register? Besides a clean conscience, you will get a registration code that will work for all future versions of ULP, and will remove the delay and message at the end of execution of each program. In addition, registration will allow you to run ULPTEST in a "seamless" mode so that it looks like a part of PCBoard to the user. In addition, the registation code will allow use of the 286 or 386 specific versions of ULP that are made available only to registered users of ULP. These versions are slightly smaller and faster than the shareware 086+ version. The registration fee for your unique code is $20 for non-commercial BBSes. The registration fee for commercial BBSes, defined as running your BBS in the course of a commercial business or for profit, is $40. Other variations are available; refer to the file REGISTER.FRM for all registration options. Please print the file REGISTER.FRM and fill it out. You can print out the form by issuing the following command from the DOS prompt: TYPE REGISTER.FRM > PRN ┌───────────────────────────────────────┐ │ 6. License, Warranty and Disclaimer │ └───────────────────────────────────────┘ I'll keep this part short and sweet, and dispense with the legal-ese: License: You are allowed to use ULP for 30 days, after which you must either register ULP or stop using it completely. ULP registration is a license for your use of ULP; I retain ownership of the software. A single registration applies to a single BBS system, regardless of the number of computers used in the system. If you run two or more distinct BBS systems on the same computer or network (with different names), you require two or more ULP registrations. Refer to the registration form for the currect pricing structure. Warranty: There isn't one. The only thing I'll guarantee is that ULP will take up disk space, and will disappear when deleted. Disclaimer: I'm not responsible for anything bad that happens. ULP works here, but I cannot be held responsible for it not working on your computer or doing any damage to hardware or software. If these aren't agreeable with you, then the best thing to do is delete ULP right now. I'll do my best to help any user (registered or not) that wants to use ULP, and I'll act on bug reports quickly, but I simply cannot and will not be responsible for anything bad, like lost data, disk crashes, or whatever else you can think of. ┌──────────────────────────────┐ │ 7. Principles of Operation │ └──────────────────────────────┘ Since the ULP system is made up of several inter-operating programs, this overview and concept explanation should help you understand how ULP is designed to be used. ******************************************************************************* READ THIS SECTION VERY CAREFULLY, AS IT WILL MAKE LIFE MUCH EASIER!!! ******************************************************************************* I believe that all responsible BBS sysops verify all of their uploads prior to posting them, in order to protect both themselves and their users. ULP is designed with idea in mind. Most, if not all, sysops process uploads in one of two ways (listed with benefits and liabilities as I see them): 1) Make all uploads private, processing them during a system event. BENEFITS: ∙ Takes up very little on-line time on the user's part to process archives. ∙ Allows the conversion of all archives to a default format, so that the BBS archives are consistent. ∙ Allows the BBS to accept any archive format...face it, it's hard enough to get some of these weenies to upload, much less compress them the same way. LIABILITIES: ∙ Files are not available immediately for download. ∙ The 'U' subcommand of PCBoard's (F)ile Directories command is no longer available to the user. ∙ Does not catch duplicates or aged archives until after the user has uploaded them, and perhaps leads to abuse by clever (?) users. (It is assumed that these sysops still use the venerable 'PKUNZIP -T' in their PCBTEST.BAT...) 2) Process (test) each upload online after the user uploads them, and making them available for immediate download. BENEFITS: ∙ Catches duplicate, defective and aged archives while the user is online, denying him upload credits. ∙ Files are available immediately for download if they are not made private in the PCBoard setup. LIABILITIES: ∙ Takes up on-line time for a user, potentially adding to his long-distance phone bill, discouraging further uploading; this process is typically quite slow for large archives. ∙ The archive format cannot be converted (a limitation of PCBoard), leaving many different archive formats in your BBS directories. Because of this, many sysops force users to use a specific format, which may or may not be consistent with other BBS's the user calls. The ULP system takes the best of both of these methods of testing. ULP.EXE is an event-mode processor which will fully process every new archive found in the source upload directory, including archive format conversion, and optionally move the file and description to a destination directory. In the simplest ULP system setup, you MUST run ULP.EXE in your event to completely process all uploads to your system. ULPTEST.EXE is an on-line processor, with a twist: it doesn't recompress the archive. It does little more than thoroughly test the upload; this will greatly reduce the amount of on-line time the user is forced to waste waiting for file processing. The conversion process is handled by ULP during the system event. Pay attention to this part: PCBoard normally has two upload directories for each conference: a private and a public directory. When PCBoard invokes PCBTEST.BAT, the upload is in the private directory. If the archive fails the testing, it will remain there. However, if it passes, one of two things will occur depending upon your system setup; if you have made all uploads private, the file will remain in the private directory; if you have not made uploads private, it will be moved BY PCBOARD (not ULPTEST) to the public directory. If you have made all uploads private, then setup and configuration of ULP is a snap: the source directory is the private upload directory, and the destination is the public directory. However, if you want to allow users access to untested uploads, then your source directory is the public upload directory, and the destination information is left blank. To illustrate the operation: MAKE ALL UPLOADS PRIVATE │ ALL UPLOADS AVAILABLE AFTER TESTING ───────────────────────────────────┼───────────────────────────────────── 2 directories: C:\PRIVATE │ 2 directories: C:\PRIVATE C:\PUBLIC │ C:\PUBLIC │ User uploads a file, gets placed │ User uploads a file, gets placed in C:\PRIVATE by PCBoard │ in C:\PRIVATE by PCBoard │ ULPTEST.EXE tests it │ ULPTEST.EXE tests it │ PCBoard leaves file in C:\PRIVATE │ If it passes, PCBoard moves it to │ C:\PUBLIC; if it fails, PCBoard │ leaves it in C:\PRIVATE │ ULP.EXE processes all new uploads │ ULP.EXE processes all *new* uploads found in C:\PRIVATE since last │ found in C:\PUBLIC since last event event and moves all good uploads │ to C:\PUBLIC │ The following are PCBSETUP (14.5a) screen snapshots of what the Main Conference should look like when implemented as described above: MAKE ALL UPLOADS PRIVATE: ╔═════════════════════════════════════════════════════════════════════════════╗ ║ Main Board Configuration ║ ║ ║ ║ Make All Uploads Private : Y ║ ║ Make All Messages Private : N ║ ║ ║ ║ Number of Message Blocks : 2 ║ ║ Name/Loc of MSGS File : C:\PCB\MAIN\MSGS ║ ║ ║ ║ Name/Loc of Users Menu : C:\PCB\GEN\BRDM ║ ║ Name/Loc of Sysops Menu : C:\PCB\GEN\BRDS ║ ║ Name/Loc of NEWS File : C:\PCB\GEN\NEWS ║ ║ ║ ║ Sort Name/Loc Upload DIR File Location of Uploads ║ ║ Public Upld: 0 : : ║ ║ Private Upld: 2 : C:\PCB\GEN\PRIVDIR : C:\PRIVATE\ ║ ║ ║ ║ Menu Listing Path/Name List File ║ ║ Doors : C:\DOORS\DOORS : C:\DOORS\DOORS.LST ║ ║ Bulletins : C:\PCB\GEN\BLT : C:\PCB\GEN\BLT.LST ║ ║ Scripts : : ║ ║ Directories : C:\PCB\GEN\DLFILES : C:\PCB\GEN\DLFILES.LST ║ ║ Download Paths (listing only) : C:\PCB\GEN\DLPATH.LST ║ ║ ║ ╚══ 10:13:03 ═══ 09-30-92 ═════ F1 ═ help ════ caps: OFF num: OFF ins: OFF ═╝ MAKE ALL UPLOADS AVAILABLE IMMEDIATELY AFTER TESTING: ╔═════════════════════════════════════════════════════════════════════════════╗ ║ Main Board Configuration ║ ║ ║ ║ Make All Uploads Private : N ║ ║ Make All Messages Private : N ║ ║ ║ ║ Number of Message Blocks : 2 ║ ║ Name/Loc of MSGS File : C:\PCB\MAIN\MSGS ║ ║ ║ ║ Name/Loc of Users Menu : C:\PCB\GEN\BRDM ║ ║ Name/Loc of Sysops Menu : C:\PCB\GEN\BRDS ║ ║ Name/Loc of NEWS File : C:\PCB\GEN\NEWS ║ ║ ║ ║ Sort Name/Loc Upload DIR File Location of Uploads ║ ║ Public Upld: 2 : C:\PCB\GEN\PUBLDIR : C:\PUBLIC\ ║ ║ Private Upld: 2 : C:\PCB\GEN\PRIVDIR : C:\PRIVATE\ ║ ║ ║ ║ Menu Listing Path/Name List File ║ ║ Doors : C:\DOORS\DOORS : C:\DOORS\DOORS.LST ║ ║ Bulletins : C:\PCB\GEN\BLT : C:\PCB\GEN\BLT.LST ║ ║ Scripts : : ║ ║ Directories : C:\PCB\GEN\DLFILES : C:\PCB\GEN\DLFILES.LST ║ ║ Download Paths (listing only) : C:\PCB\GEN\DLPATH.LST ║ ║ ║ ╚══ 10:13:03 ═══ 09-30-92 ═════ F1 ═ help ════ caps: OFF num: OFF ins: OFF ═╝ ULPTEST.EXE also has two modes of operation: normal and fast. The normal mode decompresses the files, performs file, duplication and age checking, and then deletes the extracted files and returns to PCBoard, informing PCBoard of the test results. Fast mode DOES NOT decompress the file; it firsts performs an archive integrity check, then scans ARJ and ZIP archives directly for duplicate and age information, and then returns to PCBoard (if the archive is not ARJ or ZIP, then normal mode is forced). In fast mode, file checking (viruses, etc.) is left for ULP to do (which is why the above discussion regarding private/public directories is important). ULPTEST.EXE will also accept a redirected ARJ or PKUNZIP listing text file with the special name VERIFY.ULP as input to pre-verify an upload for a user, before the user actually spends his time uploading the file only to find out it won't pass the limits you set. ┌───────────────────┐ │ 8. Installation │ └───────────────────┘ GENERAL (REQUIRED!): ─────────────────── Make a subdirectory on your hard drive. For the purposes of this document, we'll call it "C:\ULP". Unarchive the ULP distribution archive into this subdirectory. You've more than likely already made it this far, if you're reading this file. <g> The ULP system opens several files at once for various reasons. I would recommend that you have a minimum of FILES=30 per node in your system CONFIG.SYS file, since ULPTEST is run in conjunction with PCBoard. If you are running PCBoard under a network or a multitasking operating system, you should already have DOS's SHARE.EXE loaded. You must have SHARE loaded in order to take advantage of the file sharing and locking methods used by the ULP programs to prevent data loss. (If you are running a single-node system without a multitasker, SHARE is not needed). Edit the configuration file to suit your needs. Proper configuration will require you to refer to this section, the previous section and the section titled "Configuration", which has an in-depth explanation of each configuration parameter and its function. It would probably be wise to fire up your printer for this document. For this document, we'll assume the name of your text configuration file is "ULP.CFG". After editing the configuration file, generate the binary version (called "ULP.BIN" in this document) using the ULPCFG.EXE program as follows: ULPCFG -TULP.CFG -CULP.BIN Note that for any of the ULP programs, you can obtain a limited program syntax screen simply by executing the program with no arguments. FOR EVENT-MODE UPLOAD PROCESSING (REQUIRED!): ──────────────────────────────────────────── As a minimum setup, you MUST run ULP as an event-mode processor, as ULP.EXE handles all of the CRC database updating, archive conversion, file and description moving, archive information line computation, and other features. Further, ULP requires that you define the upload disk directories and upload DIRxx files. They can be either a private/public setup or public setup, as described below (and in the 'principles of operation'section): - PRIVATE/PUBLIC SETUP: This setup will not allow users to get to any archive that has not been fully processed. In PCBSETUP, under Main Board Configuration (and any other conferences you want ULP to process), make all uploads private. Define a private upload directory and a private DIRxx file (which should have already been done, since all sysop-only uploads are placed here by PCBoard). Also create a public upload directory and public DIRxx file, and add it into the DLPATH files. This is how I have my system setup. - PUBLIC SETUP: This setup will allow users to download archives from the public upload directory, including partially testing archives. In PCBSETUP, under Main Board Configuration (and any other conferences you want ULP to process), define a private upload directory and a private DIRxx file, and a public upload directory and a public DIRxx file (this is where PCBoard will place all new non-private uploads since you did not make them private). To process your uploads during your system event, add the following lines to your EVENT.SYS file (be sure to change the drive and path as necessary): C: CD \ULP ULP -CULP.BIN ULP will lock the duplication database, preventing any other program from accessing them (ULPTEST and ULPDB, specifically). Therefore, I strongly suggest you have all nodes down when running ULP. Once you have ULP configured, before you run ULP for the first time, the process data file must be initialized using the -I parameter. The syntax is as follows, but please refer to the manual operation section below for more information: ULP -CULP.BIN -I This is only done ONCE, and never needs to be done again; ULP maintains this file as a course of action during normal operation. FOR THE INTEGRATED CRC-32 DUPLICATION CHECKER (HIGHLY RECOMMENDED): ─────────────────────────────────────────────────────────────────── Initially, you must create the CRC database from scratch. If you have mostly ZIP and ARJ files, then this should be very quick (on the order of 10 minutes per 1000 archives for my 10MHz AT clone on a hard disk; a CD-ROM will be somewhat slower). After you edit the configuration file, go to the ULP directory and this command will create and compile your database of your current downloadable file base: ULPDB -CULP.BIN -A@C:\PCB\GEN\DLPATH.LST -S The '@' symbol (a '!' may also be used) indicates to ULPDB that the argument is a list file, not a directory. The list file must have one path per line, and each path must end with a trailing backslash. In general, you may simply use your PCBoard DLPATH.LST file, unless you index your downloads. If you would like to include any off-line files in your CRC database (or have to add files originally not included in the database) you can temporarily copy your offline files to a temp directory (for the sake of argumentation, let's call it "C:\TEMP\"; note the trailing backslash). You can then add and compile them into the CRC database with the following command (no list symbol!): ULPDB -CULP.BIN -AC:\TEMP\ -S After you've added the offline files, just delete them from the disk, and if someone uploads a file that you already have, it will be rejected. * NOTE: The default method of database generation will be the fastest mode since ULPDB will scan ARJ and ZIP archives for CRCs without unpacking. This is not as accurate as recursively unpacking the archives since ULPDB can only 'see' one level deep into an archive; nested information will not be obtained. If this poses a problem, add the -U parameter to the end of the command lines above; this will force ULPDB to unpack all archives and compute the CRCs. This will be significantly slower than the default method, but 100% accurate. If you already have a FWKCS duplication database, you can import and translate it into the ULP database by using the -I command line parameter. ULPDB will prompt you for which database format it is, and will then proceed to import it into the ULP database defined in ULP.BIN: ULPDB -CULP.BIN -IC:\FWKCS\CSLIST -S Once you have your database built, you should also perform maintenance on the duplication database files regularly. This will compile any new data from the day's uploads into the main database, and remove any added temporary data from ULPTEST. This is not required to be done every day, but it will speed the duplicate checking in ULP and ULPTEST. To perform database maintenance, add the following line to your PCBoard EVENT.SYS after the event-mode processor has run: C: ──────┐ These lines are repeated CD \ULP ├───── for clarity. ULP -CULP.BIN ──────┘ ULPDB -CULP.BIN -S ULPDB also locks the CRC-32 database, preventing any other program from accessing them. I strongly suggest you have all nodes down when running ULP. IF YOU USE PCBOARD'S 'TESTFILE.EXE' UTILITY: ──────────────────────────────────────────── * NOTE: If you are unfamiliar with the operation of PCBTEST.BAT, you will need to refer to the PCBoard manual for an explanation of PCBTEST.BAT and TESTFILE.EXE. In general, the use of TESTFILE.EXE is not necessary with ULPTEST. ULPTEST is fully capable of handling most any situation that arises. However, if you must use TESTFILE.EXE for some reason, in order to take advantage of the upload pre-verification abilities of ULPTEST, you must add the extension 'ULP' to the TESTFILE.EXE extension list. This will allow the file 'VERIFY.ULP' to pass through the filter created by TESTFILE so that ULPTEST may scan it. You will also have to add 'EXE' to your extension list if you use TESTFILE.EXE, to allow SFX archives to be processed by ULPTEST. FOR THE ONLINE UPLOAD TESTING (OPTIONAL): ───────────────────────────────────────── Online testing of archives can greatly reduce the amount of garbage your BBS can pick up from user's uploading defective archives, ancient files, and duplicates. The theory that ULPTEST uses is to simply check the archive AS FAST AS IT CAN for archive integrity, duplication, and age, and then return the user to greater and better things. ULP.EXE will handle everything else during the system event. * NOTE: In order to use ULPTEST, you must set PCBoard to swap by adding the /SWAP switch to the SET PCB= environment variable in BOARD.BAT (in PCBoard 15.0, this is in PCBSETUP, not the environment variable). Otherwise, there will not be enough memory for ULPTEST to execute programs like ARJ, SCAN, etc. To use ULPTEST.EXE for on-line testing of archives, use the following command line in PCBTEST.BAT (refer to the sample PCBTEST.BAT included in the ULP distribution archive, and the PCBoard manual, to get the general logic of PCBTEST.BAT's operation). Note the explicit paths to the ULPTEST executable and the ULP configuration file: C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 ULPTEST will garner any other information that is required for operation from the PCBOARD.DAT and PCBOARD.SYS files. In most installations, ULPTEST will locate these two system files via the environment variables PCBDAT, PCBDRIVE and PCBDIR. If the environment variables are used, PCBDIR and PCBDAT are required; PCBDRIVE is required only if the BBS drop files exist on a different drive than PCBTEST.BAT is being executed from. * NOTE: Don't rely on PCBoard to create these variables; it only does so prior to executing a door. As a last resort, ULPTEST will look in the current directory for the files it requires. If you have the environment variables disabled in PCBoard (using the /NOENV switch), you can define the location of these files by adding the -P parameter: C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 -PC:\PCB\NODE1\ Or, or you can provide the port and node number information directly to ULPTEST by using the -N, -I, -A and -B command switches: C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 -N5 -I4 -A3F8 -B38400 where -N is the node number, -I is the port IRQ line, -A is the port base address (in hex) and -B is the baud rate of the port. If you are using the DSZPORT environment variable to define the port IRQ and address to DSZ, ULPTEST will use this information as well. In addition, regardless of the method used to inform ULPTEST of the port information, ULPTEST defaults to using hardware handshaking. If you want to use software (XON/XOFF) handshaking or both, use the -H command switch with either HARDWARE, SOFTWARE or BOTH as the argument: C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 -Hboth ULPTEST is also capable of using a FOSSIL driver, and automatically detects if you are running PCBoard /M and uses the appropriate FOSSIL port. However, if you wish to force ULPTEST to use a FOSSIL driver (for example, to access high IRQs), use the -X command switch with the FOSSIL port number as the argument: C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 -X1 During the course of testing, ULPTEST will output the current testing status to the user online. Note that this information is not displayed locally; it is being transmitted to the online user only. The program header can be disabled in registered versions of ULP, making ULPTEST seamless to look like a part of PCBoard itself. A sample follows: Verifying FILENAME.ZIP... ┌────────────────────────────────┐ │ UpLoadProcessor 1.23 │ Registered to: Bloom Beacon-Picayune │ Copyright 1992-93 Stacy Smith │ Serial number: 1000 └────────────────────────────────┘ Identifying file archive format...ZIP Screening archive for corruption...OK Unpacking archive...OK Testing files for viruses, etc...OK Checking for duplicate files...OK Checking average age of files...OK Archive passed. ULPTEST will return a DOS errorlevel of 0 if the archive is OK, and an errorlevel of 1-12 if there was an error (depending on the error). This allows the PCBTEST.BAT batch file to branch based upon the results of the ULPTEST. The appendix to this document lists all of the DOS errorlevels set by various ULP programs. ULPTEST also creates the requires PCBPASS.TXT and PCBFAIL.TXT for PCBoard to determine if the archive passed or not. ┌────────────────────┐ │ 9. Configuration │ └────────────────────┘ The sample configuration file included in the distribution archive is heavily commented, but some additional information is provided below: GENERAL PARAMETERS: ─────────────────── REG_CODE The registration code EXACTLY as provided to you on your registration letter. If this is an unregistered version, comment out this line. Note that this is case-sensitive, as the serial number, BBS name and sysop name are all encrypted into this code. BBS_NAME The BBS name EXACTLY as provided to you on your registration letter. If this is an unregistered version, comment out this lines. Note that this is case-sensitive! SYSOP_NAME The sysop name EXACTLY as provided to you on your registration letter. If this is an unregistered version, comment out this line. You get the picture by now... SWAP If you wish the ULP programs to swap themselves out of memory prior to executing external programs, set this parameter to YES. The ULP programs will attempt to swap to EMS, XMS and lastly, the hard disk. If you do not want this feature, enter NO or comment the line out. This feature will be of most benefit to users using ARJ to compress archives, networked systems and small DESQview windows. Use this feature ONLY if you have a memory crunch! INTERNAL_DEL If you wish the ULP programs to delete the extracted files themselves, versus using DOS, set this parameter to YES. If not, enter NO or comment the line out. In some cases, letting DOS do the work is faster, but foreign language implementations of DOS may prevent this ability. DISK_BUFFER The size of the disk I/O buffer you wish to allocate in bytes (valid values are between 2048 bytes and 16384 bytes). Optimal performance is obtained with a value that is a multiple of 2048 bytes, since this is the size of a typical hard disk cluster. 4096 seems to be a good number for me, although a larger size will improve performance somewhat. If you find yourself running out of memory when executing archivers or file checking programs, reduce this number to increase your available memory. If no value is specified, the disk buffer will default to 4096 bytes. LOG_FILE All of the ULP programs will write process and error information to a log file, if desired. This is the path/filename of the log file for ULP to record its activities. Comment out if you don't want any disk logging. LOG_MODE If you want ULP and ULPTEST to record their statistics and finding for each archive processed to the log file, use the VERBOSE keyword. If you only want errors and summary statistics logged to disk, use the TERSE keyword or comment the line out. VERBOSE mode will quickly produce a large log file, but gives much insight into what ULP is doing and why a particular upload failed. DIR_CFG_FILE The path/filename of the upload directory configuration file for ULP. This file is automatically created by ULPCFG.EXE from data contained within the ULP.CFG text file, and must be defined for ULP to operate. PROC_DAT_FILE The path/filename of the ULP process data file. This file contains information on previous ULP archive processing and is maintained automatically by ULP. This file must exist, or ULP will not operate; to intitialize the process data file, use the -I command line switch in ULP.EXE. WORK_DIR If you want ULP, ULPTEST or ULPDB to use a specific directory for storage while processing, enter the full pathname (including the trailing backslash). The drive letter MUST be included. If this directory does not exist at runtime, the ULP programs will attempt to create it. This can be set to a RAM disk for greater speed. If you do not desire to use a specific directory, comment it out (ULPDB will extract at its current location on disk). If a RAM disk is used (which will greatly speed processing), be sure that largest expected archives will fit on the RAM disk decompressed. If an archive will not fit onto a RAM disk, ULPDB will flag it as a bad archive in the log file and not record its CRCs. ERR_DISP This defines how ULP dispositions archives found to have an error other than a file check failure. These kind of errors take the form of ZIP CRCs, etc., and are not potentially system damaging. Under some conditions, these archives can be restored. If you want ULP to delete the archive and description, enter DELETE; if you want ULP to rename the archive to a .ERR extension, type RENAME or comment the line out. VIRUS_DISP This defines how ULP dispositions archives that fail a file check, such as a virus scan. These errors are (obviously) potentially system-damaging. (I wouldn't keep them if I were you.) If you want ULP to delete the archive and description, type DELETE; if you want ULP to rename the archive to a .CHK extension, type RENAME or comment the line out. VIEWER The command line for an ASCII file viewing (or editing) utility, such as Vernon Buerg's venerable LIST utility. This will be called by ULPDB to facilitate viewing of an archive information file generated for the purposes of archive review. Place the @FILES@ variable where the filename is to be entered on the utility's command line. UPLOAD DIRECTORIES: ─────────────────── DIR_CFG_FILE The path/filename of the upload directory configuration file for ULP. This file is automatically created by ULPCFG.EXE from data contained within the ULP.CFG text file, and must be defined for ULP to operate. Each pair of upload directories (one source, and one destination), is denoted by UL_DIR x, where x is a sequential number. An unlimited number of directory pairs can be processed by ULP. The next 5 keywords are REQUIRED after the UL_DIR statement; do not comment any of them out or insert other parameters or comments in between them: UL_DIR = denotes the beginning of a pair of upload directories definition. SOURCE_PATH = the directory where the uploads are placed by PCBoard. This must include the trailing backslash! NOTE: if you are using the 2 public directory setup, this parameter will be the INITIAL public upload directory; the same goes for the next two parameters). SOURCE_LIST = the source upload directory list file that contains the archive descriptions, including the path. SOURCE_HDR = the number of lines to ignore at the beginning of the source upload directory list file (for headers, detachable header filespecs, etc.) DEST_PATH = the destination directory where the uploads can be placed by ULP after successful processing. This must include the trailing backslash! (Leave blank if all uploads are public). DEST_LIST = the upload directory list file that contains the archive descriptions, including the path. ULP will append this file with the descriptions of successfully processed and move archives. (Leave blank if all uploads are public). For example, if you want all uploads to be private, you would define the upload directory as (noting that the SOURCE_HDR should be set to the appropriate value): UL_DIR 1 SOURCE_PATH c:\private\ SOURCE_LIST c:\pcb\gen\privdir SOURCE_HDR 1 DEST_PATH c:\public\ DEST_LIST c:\pcb\gen\publdir If you want all uploads to be public, you would define it as: UL_DIR 1 SOURCE_PATH c:\public\ SOURCE_LIST c:\pcb\gen\publdir SOURCE_HDR 1 DEST_PATH DEST_LIST AGE-LIMIT CHECKING: ─────────────────── AGE_CHK If you want ULP to fail an archive based on the average age of the files contained within, enter YES; otherwise, enter NO or comment out the line. AGE_LIMIT The maximum average age in months, that you will accept for an upload. (If the ERR_DISP is set to RENAME, it will be renamed to a .ERR extension). ARCHIVING CONTROL FLAGS: ──────────────────────── ARC_CVT If you want ULP to convert the archive to a default archiving format of your choice, type YES; if not, type NO or comment the line out. This only affects the 'outside' level of the archive. This flag has no affect on the conversion of internal, nested archives. ARC_NEST_CVT If you want ULP to convert any internal archives found inside the archive to your default archiving format, type YES; if not, type NO or comment the line out, and they will be rearchived using the original archiving method (using the original extension, even if it isn't that archiver's default extension). The benefit of converting archives is it may save some disk space; on the down side, it will potentially hose automatic installation programs/batch files that may be contained within the archive. SFX_CVT If you want ULP to convert a self-extracting (SFX) archive to your default archiving format, type YES; if not, type NO or comment the line out. If NO is selected, then the archive will be processed, and then left alone if it passes (NOT re-archived!). The reasoning for this is the LHA autostart batch files and other similar features; they are usually used for novice users and for automatic installation. By converting SFX archives, this may save some disk space, but can cause inconveniences. For example, if you don't have PKZ110.EXE on your BBS, a novice user has no way of obtaining the PKZIP utilities from your BBS. * NOTE: If there is a BBS ad in a SFX archive, and SFX_CVT is set to 'no', the ad will remain in the archive. Can't be helped... KEEP_AV If you want ULP to keep ZIP archives with authenticity verification stamps (-AV) regardless of the other archiving control flags, then set this to 'yes', or comment it out. If you set this flag to 'no', then the archive will be recompressed per the other control flags. * NOTE: If there is a BBS ad in a -AV stamped archive, and KEEP_AV is set to 'yes', the ad will remain in the archive. KEEP_ENV If you want ULP to keep ARJ archives with security envelopes (similar to a ZIP -AV) regardless of the other archiving control flags, then set this to 'yes' or comment it out. If you set this flag to 'no', then the archive will be recompressed per the other control flags. * NOTE: If there is a BBS ad in a secured archive, and KEEP_ENV is set to 'yes', the ad will remain in the archive. ARCHIVER DEFINITIONS: ───────────────────── Each archiver is denoted by ARCHIVER x, where x is a sequential number. Up to 9 archivers can be executed by ULP. ULP is capable of automatically detecting ARC, ARJ, HYP, LZH, PAK, SQZ, ZIP and ZOO archives. The next 7 keywords are required after the ARCHIVER statement; do not comment any of them out or insert other parameters or comments in between them. I would also advise against changing the command options I have set, unless you have need to do so. Some are there to ensure uninterrupted processing (e.g. /m1 in LHA). Any other archiver can be added, if desired; the only requirement is that the archiver must return a DOS errorlevel so that ULP can determine if it ran properly. If you do not explicitly define the path to your archiver(s), it must in a directory included in the DOS PATH environment variable. * NOTE: The ARJ and ZIP formats MUST be included for ULP to function properly! ARCHIVER = denotes the beginning of an archiving program definition. ARC_EXT = the extension for this particular archiving format. ARC_PRIMARY = if this is the default archiving format you use on your BBS, type YES; if not, type NO. ARC_PACK = the command line to pack archives, including all command-line switches desired. Place the variable @ARCHIVE@ where the filename is to be inserted in the command line and @FILES@ where the filespec is to be inserted. If you want a comment to be inserted, you must include it in the command-line switches. This cannot be a batch file! (DOS re-direction is allowed) ARC_COMMENT = the command line to comment archives. Place the variable @ARCHIVE@ where the filename is to be inserted in the command line. If the archiver does not support comments, or if you do not wish to add comments, leave the line blank. ARC_UNPACK = the command line to unpack archives, including all command-line switches desired. Place the variable @ARCHIVE@ where the filename is to be inserted in the command line. This cannot be a batch file! (No DOS re-direction allowed) ARC_TEST = the command line to test archives, including all command-line switches desired. Place the variable @ARCHIVE@ where the filename is to be inserted in the command line. This cannot be a batch file! (No DOS re-direction allowed) ARC_ERR = the errorlevel that the archiver returns upon success. This information should be available in the program's documentation. UNCOMPRESSED FILES TO PROCESS: ────────────────────────────── PROC_EXT ULP can process selected uncompressed uploads, using this option. Up to 10 uncompressed file types can be specified by extension. Do not use any compression format extensions (ARJ, ZIP, etc.)! Any files that are uploaded with one of these extensions and could not be identified by signature will be file-checked, duplicate-checked, and compressed using your default archiving method. If you do not want to use this feature, simply remove or comment out these lines. It probably is not a good idea to have EXE as an extension to compress, since ULP is not able to detect ARC, HYP or ZOO self-extractors, and they would be packed as an uncompressed archive. FILES TO NEVER PROCESS: ─────────────────────── NO_PROC_EXT ULP can be forced to NOT process selected uploads, using the NO_PROC_EXT keyword. Up to 10 file types can be specified by extension. THESE FILES WILL BE AUTOMATICALLY PASSED; be sure this is desired. This is intended for use with non-DOS uploads, such as the Unix Tar Z format, or Macintosh files. Note that these files will NOT be uncompressed, tested for viruses, duplication, age, etc.. Please note this fact to your users of these files. FILE-CHECKING (VIRUS SCANNING, ETC.): ───────────────────────────────────── Each file checker is denoted by FILE_CHECKER x, where x is a sequential number. Up to 3 file checking programs can be executed by ULP. The next 2 keywords are required after the FILE_CHECKER statement; do not comment them them out or insert other parameters or comments in between them. Any kind of file checker may be used, such as a virus checker, or a 3rd party CRC duplication checker, if desired. If you do not explicitly define the path to your file checking program(s), it must in a directory included in the DOS PATH environment variable. The only requirement is that it must return a DOS errorlevel, otherwise ULP has no clue if there was an error or not. Note that if you are running a network, the use of network-specific scanners may be necessary. * NOTE: If you are using McAfee's SCAN (most sysops do), be sure you are using the correct program. SCAN is fine for single node and multitasker systems and scanning a local drive from the network server, but NETSCAN is required to scan a network drive from a workstation. FILE_CHECKER = denotes the beginning of a file-checker definition. FILE_CHK_CMD = the command line to execute each file checker, including all command-line switches. Place the variable @FILES@ where the filespec is to be placed on the command line. This cannot be a batch file! (No DOS re-direction allowed) FILE_CHK_ERR = the errorlevel that the file checking program returns upon success. This information should be available in the program's documentation. GIF FILE CHECKING ───────────────── MIN_WIDTH If ULP is to reject GIF files based upon picture width, enter the minimum acceptable width in pixels. If no width limit is desired, set it to zero or comment out the line. MIN_HEIGHT If ULP is to reject GIF files based upon picture height, enter the minimum acceptable height in pixels. If no height limit is desired, set it to zero or comment out the line. MIN_COLORS If ULP is to reject GIF files based upon the number of colors in the picture, enter the minimum acceptable number of colors. If no color limit is desired, set it to zero or comment out the line. KEEP_GIFLITE If ULP is to keep uploaded GIF files if they have been compressed with GIFLITE, set this to YES. If not, set it to NO or comment it out. At this time, if you use GIFTEST, this parameter will have no effect. * NOTE: Accepting GIFLITEd GIF files can compromise the duplication system due to its variable compression ratio! I strongly suggest you do not accept GIFLITEd GIF files! Each external GIF file checker is denoted by GIF_CHECKER x, where x is a sequential number. Up to 3 GIF file checking program(s) can be executed by ULP. In addition, do not use the GIF file checking program's internal duplicate checking or file handling abilities, if any. ULP and ULPTEST will take care of these operations. The next 2 keywords are required after the GIF_CHECKER statement; do not comment them out or insert other parameters or comments in between them. GIF_CHECKER = denotes a new file checking/virus program definition. GIF_CHK_CMD = the command line to execute each GIF file checker, including all command-line switches. Place the variable @FILES@ where the filespec is to be placed on the command line. This cannot be a batch file! (No DOS re-direction allowed) GIF_CHK_ERR = the errorlevel that the GIF file checking program returns upon success. This information should be available in the program's documentation. DESCRIPTION PROCESSING: ─────────────────────── WORD_WRAP If you want ULP to word-wrap inserted descriptions, enter YES for smart word-wrapping, otherwise, type NO or comment the line out. Note that ULP now checks for boxes, etc. before attempting a word-wrap. INT_DESC ULP can extract and insert FILE_ID.DIZ and DESC.SDI internal description files into the DIRxx files. If you want ULP to insert internal description files, type YES; if not, type NO or comment the line out. DESC_LIMIT If ULP is to insert internal description files, the following parameter is the maximum number of lines that ULP will insert from the description file. This defaults to 10 if the line is commented out. Clark Development Company (the developer of PCBoard and FILE_ID.DIZ) has defined that a FILE_ID.DIZ file should have no more than 10 lines, but ULP will insert up to 15 lines. INFO_LINE ULP can add a line of archive information for the benefit of your users. The format is completely defined by you. If you want ULP to add a single line to the archive description, type YES; if not, type NO or comment the line out. Note that the statistics generated by ULP will not include any FILE_ID.DIZ or DESC.SDI files found in the archive. INFO_INDENT The column where the | marker goes in your directory listing setup. The standard PCBoard setup indents to column 32. ARC_INFO_FMT The format that you would like for the archive information line (do not include the '|' character or the space; ULP will add those). This format is completely configuration by the user; any ASCII character can be used. To include specific information compiled by ULP, just add the variables. The legal variables are (must be in all caps): @#@ = Number of files in the archive (and nested archives) @NEWEST@ = Date of the newest file in the archive @OLDEST@ = Date of the oldest file in the archive @BYTES@ = The total number of bytes (uncompressed) in the archive @DATE@ = Today's date @CR@ = Inserts a line break for multiline capability GIF_INFO_FMT The format that you would like for the GIF information line (do not include the '|' character or the space; ULP will add those). The legal variables are (must be in all caps): @WIDTH@ = Horizontal resolution (number of columns) @HEIGHT@ = Vertical resolution (number of rows) @COLORS@ = Number of colors in GIF @COMPRESS@ = Inserts "GIFLITE" or "None", depending upon compression @DATE@ = Today's date @CR@ = Inserts a line break for multiline capability DOWNLOAD.TXT UPDATING: ────────────────────── UPDATE_DLTXT If you want ULP to scan your DOWNLOAD.TXT file and update the file extension of the upload to reflect the correct archiving method used, enter YES. If not, enter NO or comment the line out. This feature will allow utilities that scan DOWNLOAD.TXT for file information to reflect correct information. * NOTE: If you are using a utility that creates an independent database from the DOWNLOAD.TXT file, such as PCBFX version 2.x, be sure ULP.EXE is run in the event BEFORE that utility so the independent database reflects the changes that ULP makes to DOWNLOAD.TXT. DLTXT_FILE The full path and name for the PCBoard DOWNLOAD.TXT file. ULPTEST TESTING: ──────────────── TEST_MODE This defines how ULPTEST processes an archive. NORMAL indicates a full test is to be performed, identical to the ULP event processing except that the archive formats are not converted or recompressed (this is a limitation of PCBoard). FAST indicates a fast test in which the archive is scanned for archive integrity, duplicates, and age without unpacking the archive (ARJ and ZIP files only). If this mode is used, it should be held for later processing by ULP so that it can be file-checked for viruses, etc. Also, FAST mode can only 'see' one level deep in the archive. If you have a slow computer and/or hard disk, FAST mode would be preferable for your users when large archives are uploaded. If you comment this line out, NORMAL mode is assumed. I recommend you use NORMAL mode, and then set a limit with the next parameter for when FAST mode will be utilized. TEST_LIMIT If NORMAL mode is used, the archive's file size will first be compared against this limit (in kilobytes). If it is larger than this number of kilobytes, it will be forced into FAST mode to speed the process and prevent wasting the user's online time. Entering 0, or commenting the line out will disable this feature, resulting in NORMAL mode being used at all times. If TEST_MODE is set to FAST, this parameter has no effect. For my system, a good value is 300 kilobytes, as anything much bigger than this takes quite awhile to decompress, test, and delete the files extracted. COMM_IO If you want ULPTEST to output status information to the user on-line, set it to YES; if you do not want ULPTEST to output status information, enter NO or comment the line out. ULPTEST supports both direct interrupt-driven I/O and FOSSIL driver I/O. SUPPRESS_HDR If you want ULPTEST to suppress the program header output to the user via the COM port (the program name, copyright notice, and registration information). This gives a seamless look for ULP into PCBoard. If this is desired, set this to YES; otherwise, set to NO or comment out the line. * * NOTE: This will work for registered users only! BBS AD HANDLING: ──────────────── ADS_CHK If you want ULP to remove known BBS ads from new uploads during processing, type YES; if not, type NO or comment the line out. ADS_DB The directory path and name for the BBS ads database. This database can be updated by the sysop by using the ULPADS utility (see section 11 of this document), freeing the sysop from being reliant on me to maintain the ads database. If you find a large number of new BBS ad files, please upload them to my board for inclusion into my master ads database. BBS_AD_FILE The directory path and filename for your BBS ad to be added to each and every archive processed by ULP. Note that you must provide a path, or it will not be copied. I STRONGLY recommend against them; BBS ads are aggravating to other sysops, so please be considerate and use the ARJ or ZIP comment instead. BBS ad files are the scourge of the BBS world. This parameter is commented out by default. CRC-32 DUPLICATE FILE CHECKING: ─────────────────────────────── DUPE_CHK If you want ULP to use its internal CRC-32 duplication checking, type YES; if not, type NO or comment the line out. If you want to use a third-party duplication checker, such as ZDCS, refer to Appendix B for installation information. I would strongly suggest using ULP's, as it is fast and compact, not to mention integrated, eliminating the time to shell to disk for each file in each archive to be tested. DUPE_LIMIT The minimum percentage of duplication allowed for a file to pass. Anything smaller will pass regardless of the executable duplication limit. I would suggest this number be set to around 50, as some program authors don't modify every file, especially with maintenance releases. EXE_DUPE_LIMIT The maximum percentage of duplication allowed for executable files and overlays to pass. This number should be higher than DUPE_LIMIT; I would recommend about 80. This will catch archives that are duplicates, but someone has changed enough files to normally pass it when uploaded (such as configs, high score files, etc.). This will also pass files where the author has changed nothing but the executables; they would have failed before. Executables are determined by extension; any file with one of the following extensions will be counted as an executable: EXE, COM, SYS, OVL, OVR, DLL. DUPE_DB The path and name for the CRC-32 database file. It's a good idea to keep the database, index, and new data files all together, just to make things easier. DUPE_IDX The path and name for the CRC-32 database file index. ┌────────────────────────┐ │ 10. Manual Operation │ └────────────────────────┘ While the ULP system is mostly automatic, there are occasions where some operations may have to be done manually: ULP: ──── During the course of operation, ULP will rename archives that have been found to be defective in some manner according to the following convention: .UNK Unknown archive format .DOS DOS reserved keyword found in archive .ERR Error occurred while unpacking archive (archive integrity failure) .CHK Error found while file checking archive file (virus, etc.) .DUP Excessive duplicate files contained in archive .PCK Error occurred while repacking archive file .AGE Age limit exceeded by archive file .ENC Encrypted file found in archive .BOM Bomb file detected in archive .BAD Error found while testing GIF file .RES Unacceptable GIF resolution .GLT GIF compressed with GIFLITE I would enter these extensions in your UPSEC file, as ULP will automatically pass over any file with these extensions, presuming that they are defective archives. Doing so will prevent a user from uploading a file with one of these "reserved" extensions. If you feel that these files are acceptable after reviewing them, you can force them to be accepted by adding the command-line switch -O (for override) to the ULP command line, e.g.: ULP -Culp.bin -O This will accept the archives regardless of duplication or age limits. It will not override unpacking, packing, integrity and virus errors, however, for the safety of your board and your users. To initialize the ULP process database, use the -I command line switch: ULP -Culp.bin -I ULP will scan your source upload directories as configured and generate a process data file. From then on, ULP will maintain this data file automatically. Three other command-line parameters are available in ULP. '-R' will retest all archives found in the subdirectory passed to it. It will not use duplication checking or age checking, nor will it repack the archive; it was installed to allow the periodic retesting of archives for viruses, etc.: ULP -Culp.bin -Rc:\temp\ Another, '-M' will retest and convert all archives found in the subdirectory indicated; this can be used for mass archive conversion: ULP -Culp.bin -Mc:\temp\ Note that the '-R' and '-M' switches cannot be used on a CD-ROM or other write-protected media at this time. ULPDB: ────── If a file has been failed for some reason, you can view the contents of the archive using the -V command switch with ULPDB. This will scan the archive, and generate a report of the results, which are indicative of why it failed. For example: ULPDB -Culp.bin -Vc:\privul\archive.dup This will produce a report of the archive ARCHIVE.DUP located in the specified path. This can be useful in determining if an archive that failed duplication should be removed or should be accepted anyway, and similar reasons. In addition, duplication databases can be merged using the -M command-line option. In order to merge a second database into your main database: ULPDB -Culp.bin -Msecond.db where 'second.db' is the full path and filename for the second duplication database to be merged with the main CRC-32 database. This will allow pre-built databases to be constructed for CD-ROM disks for easy implementation. Finally, database entries can be purged from the database using the file date contained in the database record using the -P command switch: ULPDB -Culp.bin -P60 where '60' is the maximum age (in months) for a file to remain in the database. Removal is based upon the file date represented by the entry, NOT the date the file entry was made into the database; these are not one and the same. In the example above, all records for files older than 60 months (5 years) will be removed from the database. As a protective measure, ULPDB will not perform a purge if the value specified is less than 6 months. ┌──────────────────────────┐ │ 11. BBS Ads and ULPADS │ └──────────────────────────┘ The ULP system includes a BBS ad removal feature based on CRC-32 calculation of the file contents and other data. In this fashion, ULP can detect a known ad file despite the file name and date. In order for sysops to be able to 'keep up' with new ads produced by the weenie sysops who insert the @!&*#%$ things, I have included a utility called ULPADS that can scan a BBS ad file, and update the BBS ads with it's information. Don't worry about duplication, as part of the process is to purge duplicate BBS ad info. The command-line syntax for the ULPADS.EXE program is simply: ULPADS -Ddatabase.ad -Fbbs.ad where 'database.ad' is the full path and filename for the BBS ads database, and 'bbs.ad' is the path and filename for the BBS ad file to be added to the ads database. If you have a subdirectory full of BBS ads, the following batch file provided by Ian Singer may be of help: rem DELADS.BAT rem this batch file written by Ian Singer 1 Jul 92 to take all rem the bbs ads in c:\ulp\ad and run them against the database rem in c:\ulp rem for %%f in (c:\ulp\ad\*.*) do ULPADS -Dc:\ulp\ulpads.db -F%%f I would greatly appreciate your uploading of any new BBS ad files that you may collect over time to my BBS so I can update the master listing that I include with the ULP distribution archive. Please refer to the top of this document for my BBS number. The latest version of my BBS ads database is included in the distribution archive. In order to add whatever new BBS ads information that may be contained, you can merge the two BBS databases: ULPADS -Dmaster.db -Mother.db where 'master.db' is the full path and filename for the master (your) BBS ads database, and 'other.db' is the path and filename for the other BBS ads datatbase to be merged with the master ads database. ┌───────────────┐ │ 12. Support │ └───────────────┘ If you require support for ULP, I can be contacted by any of the following means: ∙ Fidonet netmail (node: 1:276/112) ∙ Intelec PCBoard conference ∙ RoseNet routed mail (system ID: BB-P) ∙ The ULP support conference (#42) on Salt Air (PCBoard support BBS) ∙ The Support conference (#2) on my BBS (listed at the top of this document). ┌─────────────────────────┐ │ 13. The Future of ULP │ └─────────────────────────┘ ULP will be supported as long as I'm in the BBSing business (which will be quite a while...once it's in your blood, you can never shake it <grin>). The ULP system will be rapidly expanding it's features so it will be your first choice in BBS upload processors. Some current plans: ∙ Add the ability to preprocess files prior to file checking them; for example, decompress executables that have been PKLite'd. ∙ Support other BBS software directory listing formats. If you have any other suggestions, or want other archivers supported, please contact me via Email, U.S. snail-mail or on my BBS at the number at the top of this document. Thanks for giving ULP a try! ┌────────────────────────────────┐ │ Appendix A: DOS Errorlevels │ └────────────────────────────────┘ The errorlevels returned to DOS by the ULP system programs are consistent among all of the programs (primarily with the error conditions); the following is a list of the errorlevels than can be returned by the ULP programs, along with which programs return the codes: 0 Successful execution (ULPTEST, ULPDB, ULPADS) 0 Successful execution, archives were processed (ULP) 1 Successful execution, nothing to do (ULP) 1 Unknown archive format (ULPTEST) 2 DOS reserved keyword found in archive (ULPTEST) 3 Error unpacking archive (archive integrity) (ULPTEST) 4 Error file checking archive files (virus, etc.) (ULPTEST) 5 Error duplicate checking archive files (ULPTEST) 7 Error commenting archive (ULPTEST) 8 Age limit exceeded by archive files (ULPTEST) 9 Encrypted file found in archive files (ULPTEST) 10 Bomb archive detected (ULPTEST) 11 Bad GIF file (ULPTEST) 12 Unacceptable GIF resolution (ULPTEST) 13 GIF compressed with GIFLITE (ULPTEST) 99 Help screen (executing a program with no or an insufficient number of arguments) (All) 100 Could not close all open files (ULP, ULPTEST, ULPDB) 101 Unknown command line parameter (All) 102 Config file not found (All) 103 Invalid config file format (All) 104 Excessive number of command line parameters defined for file checker (maximum number of parameters is 10) (ULP) 105 Excessive number of command line parameters defined for archiver (max is 10) (ULP, ULPTEST, ULPDB) 106 Excessive number of words defined in archive information line (max is 10) (ULP) 107 No primary archiver defined (ULP) 108 Invalid disk buffer size (All) 109 Upload directory list file(s) not found (ULP, ULPTEST) 110 Unable to allocate heap memory (All) 111 Unable to open or lock disk logging file (ULP, ULPTEST, ULPDB) 112 Invalid registration code or attempting to run 286 or 386 versions without registering (ULP, ULPTEST, ULPDB) 113 Unable to open/lock CRC database file(s) (ULP, ULPTEST, ULPDB) 114 Unable to open BBS ads database file (ULP, ULPADS) 115 Invalid filename provided by user on command-line (ULP, ULPTEST) 116 Invalid path provided by user on command-line (ULPDB) 117 Invalid or no node number (valid values are 1 to 256) (ULPTEST) 118 CRC database usage not enabled (configuration file parameter DUPE_CHK is set to 'no') (ULPDB) 120 Invalid or old database format (All) 121 Unable to process com port/invalid port parameter (ULPTEST) 122 Unable to access PCBOARD.DAT/PCBOARD.SYS (ULPTEST) 123 CRC database failed validation test (ULPDB) 200 Undefined error (internal to ULP system program(s)) ┌────────────────────────────────────────────────────────┐ │ Appendix B: Using ZDCS vs. ULP's Duplication System │ └────────────────────────────────────────────────────────┘ The following message should provide the information required to integrate ULP and ZDCS: =========================================================================== Packet: SALTAIR Date: 05-11-93 (08:42) Number: 44 From: STACY SMITH Refer#: 1187 To: ALL Recvd: NO Subj: ULP/ZDCS PCBTEST.BAT 1/2 Conf: (44) ZDCS --------------------------------------------------------------------------- Between myself and another ZDCS user, we have come up with a simple way to use ZDCS and ULP together by running ZDCS after ULPTEST in PCBTEST.BAT. While I will continue to work on internal support for ZDCS within ULP, this will make the combination quite functional in the interim. =========================================================================== Packet: BB-P Date: 05-11-93 (02:55) Number: 1187 From: JIM CONDON Refer#: NONE To: ALL Recvd: NO Subj: ULP/ZDCS PCBTEST.BAT Conf: (2) Support --------------------------------------------------------------------------- After trying other Upload testing programs for PCBoard I think I have found one that suits my needs and is reliable. I like the ZDCS dupe checking system better then the one that comes with ULP. If you like what you got stop reading! If you would like to use ZDCS along with ULP below is my pcbtest.bat. It will not test file attaches (in PCBoard 15.0's message base). If you want to do that remove line 6. It will use ULPTEST via PCBoard's TEST command to check the integrity of files that are already online. It will also test uploads using ULPTEST first. If it fails for any reason it will go back to PCBoard without adding the bad upload to the ZDCS database. If ULPTEST passes the file then ZDCSFC is called to add it to the database. It will also test zdcstest.chk files without loading ULPTEST. You will have to edit it slightly to reflect where to look for the file being tested. Also your paths to ZDCSFC and ULP maybe different. Feel free to use or modify (like I could stop you if I wanted to!) it to suit your needs. I hope this helps. @echo off cls IF EXIST PCBFAIL.TXT DEL PCBFAIL.TXT IF EXIST PCBPASS.TXT DEL PCBPASS.TXT IF EXIST ZDCS-DEL.LST DEL ZDCS-DEL.LST if '%2 == 'ATTACH goto end if %1 == D:\PRIVATE\ZDCSTEST.CHK goto zdcs c:\ulp\ulptest -Cc:\ulp\ulp.bin -F%1 -M%2 -D%3 if exist pcbfail.txt goto end if %2 == TEST goto end goto ZDCS :zdcs c:\zdcs\zdcsfc %1 %2 %3 if exist d:\private\zdcstest.chk del d:\private\zdcstest.chk :end Line 1 and 2 are self explanatory I think! Lines 3, 4 and 5 are just house keeping (extra precaution) that was in the sample pcbtest.bat that came with ZDCS 2.0. Line 6 is a line David Terry (CDC) suggests people running 15.0 add to pass file attaches without testing. Line 7 Checks to see if this is a ZDCSTEST.CHK file and if it is skips past ULP and just runs ZDCS. Line 8 is the ULPTEST command line from the docs (or was it the included pcbtest.bat?) Anyway Stacy Smith's line. Line 9 tests to see if ULPTEST failed the upload for ANY reason. I think this way is better then errorlevel testing. If pcbfail.txt is found it exits back to PCBoard and NEVER runs ZDCSFC. I feel if the upload failed it shouldn't be added to the ZDCS database since someone else may try to upload a good copy that would then probably fail the dupe check. Line 10 checks to see if pcbtest.bat was invoked via the TEST command in PCBoard and if it is skip ZDCSFC and brings PCBoard backup without wasting time loading ZDCSFC just to have it pass control back to PCBoard. Line 11 Takes you to ZDCSFC. If it got this far it should be a valid upload and ZDCSFC should test it and add it to the database. Line 12 just a DOS label. Line 13 the actual ZDCSFC command. I believe it is from a sample pcbtest.bat that came with ZDCS 2.0. Line 14 Checks for the zdcstest.chk file and deletes it. For some reason I was sometimes getting one left over and callers couldn't upload a test file till I killed it off. It probably isn't even needed. Line 15 is just a Dos label. P.S. Don't forget to configure ULP and ULPTEST not to use ULPDB if you are using ZDCSFC instead. One dupe checker is enough! =========================================================================== Some comments of my own regarding the above message: Lines 3 and 4 are superfluous, but don't hurt anything. ULPTEST, as a matter of design, cleans up any stray PCBPASS.TXT and PCBFAIL.TXT files. Line 6 is not necessary when version 0.99ß is released in a week or two, as ULPTEST is/will be fully compatible with PCBoard 15.0, including attach mode. ULPTEST will test and log the results of testing an attachment, while always passing the attached file (there's no reason to reject an attachment, although it might be interesting to know if a virus is in one). Line 13 doesn't require the %3 parameter on the ZDCS command line. At this point, ULPTEST has already inserted the FILE_ID.DIZ and DESC.SDI file and ZDCS is just duplicating effort. Also note that you cannot use the VERIFY.ULP method of pre-verifying uploads; you will have to tell your users to continue using the ZDCSTEST.CHK method. If there are any questions, please don't hesitate to ask. I can't help with ZDCS questions, but I can help with everything else.