Beijing Paradise BBS Backup/PARADISE.ISO/software/BBSDOORW/SQUSER44.ZIP/SQUSER44.DOC

Text File | 1993-06-06 | 19.2 KB | 529 lines

 ╔═════════════╗
 ║ ╔═════════╗ ║
 ║ ║ SQUsers ║ ║
 ║ ╚═════════╝ ║
 ╚═════════════╝

  A Fidouser.lst generator for squish message bases amongst other things.

 ╔══════════════╗
 ║ Legal Stuff: ║
 ╚══════════════╝

  SQUsers is my copyright material. You can use it! You can give it to
other people to use. It is not, however, guaranteed to do anything. In
the event that anything goes wrong, my liability is limited to the
amount that you can prove you paid me directly for it - and I don't
charge for it! It should work as described below. It's not my fault if
it doesn't, but if you're polite I'll try and fix it. It works on my PC,
I do not guarantee it will work on yours.

  You may not distribute SQUsers without this document file. You may not
sell SQUsers. You may not charge in any way for the distribution of
SQUsers. This includes shareware libraries and BBSes that charge for
access. SQUsers must be distributed free of all charge. If you run a
software library and offer SQUsers, then if someone sends you a disk and
return postage you have to give them a copy. If you run a subscription
BBS, SQUsers must be in your free download area or nowhere on your
system. I'm sure you get the idea.

  In other words, I didn't write this program, which I am releasing free
to anyone that wants it, so that anyone else could make money out of
copying it!

 ╔═════════╗
 ║ Purpose ║
 ╚═════════╝

  SQUsers is a program designed to generate fidouser.lst style netmail
address look-up files from squish style message bases, fido *.msg style
message bases, nodelists, old fidouser.lst style files and msged/squish
style cc lookup files.

 ╔═════════════════╗
 ║ Historical Note ║
 ╚═════════════════╝

  I wrote SQUsers in TurboC 1.5 originally, and it was not bad, but it
had a few problems. I have recently upgraded my compiler to BorlandC
3.00++, and this seems to have got rid of some of the problems, it also
means I can compile 80286 code. The program is now up to version 4.10,
and this will probably be the first public beta version.

 ╔══════════════╗
 ║ Useage Notes ║
 ╚══════════════╝

  The archive contains two versions of the program, squservx.exe is an
8086 compiled version, squservx.286 is the 80286 code. It also contains
this doc file. "vx" are the major and minor version numbers. Released
versions will always have a minor version of 0. Anything other than
squserx0 is a beta version. The same applies to the docs (this file) and
the distribution archive.

  SQUsers will use as much memory as it can find to hold a linked list
of names and fido addresses, and if it fills the whole of available ram
up will then start using temp data files. Processing the full FidoNet
nodelist, 20 Mbytes of squish message files, a 1.3 MegaByte input file
(from an earlier run), several smaller files etc created a total of five
temp files, another 3 were created during the merger, and finally the
output file was created - on my 25 MHz 80386 with 6 Mb of disk cache
this took about 45 minutes - but it was a lot of data to process.

  The theoretical limit to the number of usernames is about 16 million,
but I haven't proved that, and it will probably be a bit higher. Someone
might reach it one day. That will be some address book!

  Note that SQUsers creates temp files of the form $SQUSER$.xxx where
xxx is a 3 digit hex number. Please make sure no such files exist in
the directory you run SQUsers from, it will check when it starts, and
refuse to do anything if it finds any!

  SQUsers can take a config file name as its only parameter, failing
that it will look for "SQUSERS.CFG" in the current directory. The config
file tells SQUsers a lot about how it should work, where to find files
and where to leave its output file. If SQUsers is unable to find a
config file, it will create a very short sample config and exit.

 ╔════════╗
 ║ Output ║
 ╚════════╝

  SQUsers will let you know what it's doing as it processes the config
file and the input files. It may occasionally reccomend that you run
sqpack or sqfix on a squish area, and will report any problems it has
internally. The output can be redirected to a log file if you start
SQUsers up with a redirection for stdout on the command line, eg:

SQUSERS > squsers.log

 ╔══════════════╗
 ║ Error Levels ║
 ╚══════════════╝

  SQUsers will exit with an errorlevel of 0 if it ends OK, or with 1 if
there's a problem - it will also print an error message and give an
internal error code. If the problem isn't obvious then the best thing to
do is netmail me, it may mean a problem with the program. (I hope not.)
At the moment the program does a lot of error checking on file open and
close, memory allocation, reads and writes etc. As I get more confident
with it, I may remove some of this to speed things up.

 ╔═════════════════╗
 ║ The Config File ║
 ╚═════════════════╝

  This is a sample SQUsers.CFG file:

;-------------------------------------------------------------------------
;
; Squsers.cfg file for 2:251/20
;
; Use buffers for textfile read / write
;
buffers    on
;
; Output file: squsers.lis in current dir
;
outfile     squsers.lis
;
; My address 2:251/20
;
address     2:251/20
;
; Old fidouser.lst style list to use
;
oldlist     c:\fd\squsers.lis
;
; And another old fidouser.lst file
;
oldlist     c:\fd\sysops.lis
;
; Squish files c:\sqbase\fido\*.sqd
;
squish      c:\sqbase\fido\*.sqd
;
; I want the adresses used in the UK sysop echo (region25) to be used in
; preference to any other addresses from nodelist or message bases.
;
squish      c:\sqbase\fido\region25.sqd
;
;  **** Beta Option Only ****
;
;  Suppose it crashes on your netmail area, turn on enhanced progress
;  reporting here .....
;
debug       on
;
; netmail messages c:\netmail\*.msg
;
; This is also used for echomail kept in FTS1-015 (Fido) *.msg format.
;
; The fido directory spec must end in a trailing backslash! This was not
; made clear in earlier versions!!
;
fido        c:\netmail\
;
;  **** Beta Option Only ****
;
;  Turn off the enhanced progress reporting used on the netmail area
;
debug       off
;
; Use nodelist c:\nodelist\nodelist.*, process the whole nodelist. You can
; process multiple nodelists. If a wildcard is used then it will look for
; file.nnn where nnn is a numeric sequence.
;
nodelist    c:\nodelist\nodelist.*    0    0     0
;
; Use nodelist c:\nodelist\nodelist.*, process zone 2 region 25 to make
; sure that where a name has more than one address I get the UK one.
;
nodelist    c:\nodelist\nodelist.*    2    25    0
;
; I want the adresses used in the UK sysop echo (region25) to be used in
; preference to any other addresses from nodelist or message bases, so I
; insert that next.
;
squish      c:\sqbase\fido\region25.sqd
;
; This fidouser.lst style file has some addresses that I want to over-ride
; any other addresses found for that person
;
oldlist     c:\fd\override.lis
;
; MsgEd/Squish Beta Team List
;
pvtlist     c:\fd\betamsq.lis
;
; My echomail links
;
pvtlist     c:\fd\links.lis
;
; My SQUsers beta testers
;
pvtlist     c:\fd\squser.lis
;
; End of squsers.cfg file
;
;-------------------------------------------------------------------------

  Lets look at the file. The comment lines are indicated by a semi-colon
in the first column - I know some people support a semi-colon on a
command line with comments after it, but I haven't got that far yet.
Maybe next week / month / year. Likewise all commands begin in column
one. All keywords are optional, but if you don't put in a sensible
selection the program may not do very much other than create a zero
length list file. If there is no config file, it won't even do that. It
is obviously a good idea to define at least one input file using either
squish, fido, pvtlist, oldlist or nodelist if you want the program to do
anything sensible.

  The file is parsed line by line, keywords can be used as many times as
is needed to get the desired result. This can be seen in the above file.

  The numbers of spaces dividing commands and parameters on each line
are not too important, as long as it's at least one, but each command
and the associated parameter(s) must be on a single line.

 ┌──────────────────┐
 │ buffers  <state> │
 └──────────────────┘

  The buffers command controls whether squish uses internal 16Kb buffers
when reading from and writing to text files. <state> may be "on" or
"off". Text files are the output file, any input files defined as
nodelist, fido, pvtlist or oldlist, and any temp files created due to
insufficient memory to hold all the usernames & addresses found as a
single group of lists. Buffers are allocated and deallocated on the fly,
the maximum overhead during data collection is 16 Kb, although this
rises to 48 Kb when merging temp files. (There is no linked list in use
while merging temp files though.) It is suggested that you either omit
the command (the default is off) or use "buffers off" if you have a read
ahead cache of any sort and / or a write cache, however it may be
worthwhile to experiment. If you have no cache at all, "buffers on" will
almost allways give a noticeable improvement in processing speed.

 ┌─────────────────────┐
 │ outfile  <filename> │
 └─────────────────────┘

  This tells SQUsers where to put its output, the default if this is
left out is to create "squsers.lst" in the current directory. If more
than one outfile keyword is used, the last one in the config file will
determine the output file. It does not make a lot of sense to use this
entry more than once.

 ┌────────────────┐
 │ debug  <state> │
 └────────────────┘

  New parameter for beta versions, state can be "on" or "off", and
enables / disables enhanced progress reporting, including procedure &
function entry with parameters etc.

 ┌────────────────────────┐
 │ address <fido address> │
 └────────────────────────┘

  This is your fidonet address. No-one will be included in the list if
their address matches yours, it is assumed that you don't need to use
netmail for local users. If you leave this out, SQUsers will just
include everyone it finds.

  Setting this to 0:0/0 disables address rejection, so you can have your
Fido address rejected processing one set of input files, then you
othernet address for your othernet files, and your thirdnet address for
the next set etc ......., then finally turn it off while you process the
local message area. This would make sure that you didn't send netmail to
other systems for people who used your local netmail area.

 ┌─────────────────────┐
 │ Oldlist  <filename> │
 └─────────────────────┘

  Filename must be a real filename, no wildcards.

  This indicates an old fidouser.lst style file to incorporate in to the
new list. Note that the only format requirements for the file are as
follows:

(a) It must have one entry per line.
(b) Entries must be: <lastname>, <firstnames><spaces><address>
(c) Spaces must have two or more spaces.

  Multiple lists can be combined using squsers, in the order that they
are included. The latest address found for an individual will be the one
that ends up in the output file. Although FIDOUSER.LST files are
normally sorted, and usually have the addresses right justified, SQUsers
isn't too fussy about this, instead it only requires that there be at
least two spaces between the username and the address. As the standard
for max name length is 36 characters in the FTSC1 message and packet
standards, and the Squish message base format, and as the address in a
fidouser.lst file is meant to start after column 40-something, this
should not be a problem.

 ┌────────────────────┐
 │ squish  <filespec> │
 └────────────────────┘

  This is the filespec of a squish area (or areas) to process. Again, if
you have multiple directories, or wish to specify an order for
processing your files instead of the order in which the directory
look-up routine finds matching files, you can use the keyword as many
times as you like.

  Filespec may be a full path and filename of a single *.sqd file, or a
wildcard of the form [drive:][\path\]*.sqd where drive and path are
optional, and * may be a partial or completely wildcarded name. The .sqd
is important.

  For example, supposing you have squish areas for fidonet, worldnet and
copsnet, and in addition want your fido regional sysops echo to be used
if a name crops up with more than one address:

squish     c:\squish\msg\fido\*.sqd
squish     c:\squish\msg\cops\*.sqd
squish     c:\squish\msg\world\*.sqd
squish   c:\sqbase\msg\fido\uksysop.sqd

  Note that in this example, if the same surname crops up in more than
one area, the address used will be the last one processed, so if Joe
Smith has entered messages in every message area on your system, the Joe
Smith who left a message in uksysop is the one who ends up in the
userlist.

 ┌───────────────────┐
 │ Fido  <directory> │
 └───────────────────┘

  The path to any fido (*.msg) areas you want to process, this is used
in a similar way to the squish <filespec> keyword, but you may only
specify one directory (echomail or netmail area) per line, with no
wildcards. The path must terminate in a backslash '\' character.

 ┌───────────────────────────────────────────┐
 │ nodelist <filespec> <zone> <region> <net> │
 └───────────────────────────────────────────┘

  If the filetype is wildcarded, as in the above example config file,
SQUsers will try and find a file matching filespec.nnn, otherwise it
will try and match the actual file. Zone, Region and Net work as
follows:

nodelist c:\nodelist\nodelist.* 0 0 0

  This will try and process the whole nodelist, it will (if this is a
Fido nodelist) take a lot of time and involve temp data files and lots
of disk access.

nodelist c:\nodelist\nodelist.* 2 0 0

  This will process all the nodes in zone 2. You can of course use other
zones, and to process all the nodes except zone 5 (for example) you
would include separate lines for zones 1, 2, 3, 4 and 6.

nodelist c:\nodelist\nodelist.* 2 25 0

  This will process only those nodes within zone 2 region 25.

nodelist c:\nodelist\nodelist.* 2 25 251

  This will process only those nodes in zone 2, region 25, net 251.

nodelist c:\nodelist\nodelist.* 2 0 0
nodelist c:\nodelist\nodelist.* 1 14 141

  This will process all of zone 2 and net 141, region 14 in zone 1.

  SQUsers can produce a very selective extract from the nodelist if the
config file is set up correctly. SQUsers can not process nodelists that
use "boss" or "point" keywords.

 ┌────────────────────┐
 │ Pvtlist <filename> │
 └────────────────────┘

  This is another way to incorporate lists of people in the final
fidouser.lst file, the format of a pvtlist is one entry per line, in the
form:

<name>!<address>

eg:

John Doe!1:222/333
Denis McMahon!2:251/20

etc ......

  Note that, if you happen to use MsgEd/Squish by John Dennis, this is
the same format as the netmail cc files use. This means that you can
incorporate the names and addresses in those files straight in to your
userlist.

 ╔═════════════════════╗
 ║ Processing Sequence ║
 ╚═════════════════════╝

  Note that in the above example config, the input files are processed
in the following order:

oldlist     c:\fd\squsers.lis
oldlist     c:\fd\sysops.lis
squish      c:\sqbase\fido\*.sqd
squish      c:\sqbase\fido\region25.sqd
fido        c:\netmail\*.msg
nodelist    c:\nodelist\nodelist.*    0    0     0
nodelist    c:\nodelist\nodelist.*    2    25    0
squish      c:\sqbase\fido\region25.sqd
oldlist     c:\fd\override.lis
pvtlist     c:\fd\betamsq.lis
pvtlist     c:\fd\links.lis
pvtlist     c:\fd\squser.lis

  Remember that the last address found is the one used, so, for
example, an address in the oldlist file c:\fd\override.lis would be used
in preference to one found in the nodelist if the usernames were the
same, and the pvtlist files take priority over the squish and fido
message areas etc.

  Note that this rule may not hold true if temp files are used, as they
may be merged in the wrong order.

 ╔═════════╗
 ║ History ║
 ╚═════════╝

  This is a precis of the revision history etc.

  Version 1.00 - First public beta - several problems, and I had a
compiler problem at the same time. However, I kept tinkering and it
slowly improved through Version 2.0. Version 2 would usually generate a
reasonable userlist from a group of squish areas, but not much else.

  Version 3.00 - This was the first one I compiled with the Borland C++
V3.00 compiler - and was a great place to start for the next stage. The
enhancements from here on in are numerous, bute here goes, this list
takes it up to 4.2, and is not in order of implementation:

        Improved list handling and io routines for spead.

        Implemented Nodelist, PvtList, OldList handlers.

        Got Fido handler to work.

        Detects Squish empty / bad frames instead of crashing.

        Temp File Merger now combines files in the right order.

        Comments in the config file.

        Make a sample config if not found.

        Doesn't write empty files any more. (It actually deletes them
        after writing!)

        Error exit clean-up improved.

Version 4.3

  Fixed a problem with the Fido *.msg handling - I was using the wrong
pointer when I filled a block of memory with 0x00 bytes - this was found
after Clay reported a lock-up.

  Fixed a piece of code in the Squish area handling - it wasn't always
processing the whole message base.

  Thanks to Clay Tinsley for pointing these ones out! :-)

  Added the "Buffers" config command.

  Realised that even with the modified code, SQUsers could not allways
be guaranteed to combine temp files such that the "use last address
found" principle was guaranteed to work all the time.

  I'm working on this, but I can't see an easy to implement and
foolproof solution at the moment.

Version 4.4

  Removed some code that I left in there while debugging. Whoops.

  Implemented "User Debug" option, on beta versions enhanced progress
reporting can be enabled at any point in the config file with "debug on"
and disabled with "debug off". Like address and buffers, this applies
while processing the input files listed after the switch.

  Compiled beta versions with Debugger info, if you have Turbo Debugger
and know how to use it you may be able to give me the line numbers in
the source file that cause the errors.

  A couple of cosmetic changes, reporting nets skipped while looking
through the nodelist for the required zone / region / net was generating
a lot of noise on the screen.

 ╔═════════════════╗
 ║ And Finally ... ║
 ╚═════════════════╝

  Thanks to Don Dawson and Johan Lindqvist for their help with testing
version 1, and (I hope) version 4.xx. Thanks also to all the others that
have been helping me with version 4.xx testing, to Scott Dudley who
developed the Squish message base and John Dennis who developed Jim
Nutts' MsgEd code in to a Squish compatible message editor.

  All trade marks copyrights etc are acknowledged.

  I think that explains it all. The author may be contacted at:

2:251/20@FidoNet

  Which is a mail only node accepting netmail from listed FidoNet nodes
between 00:00 and 06:00 UTC.

Denis McMahon
29th May 1993