Current Shareware 1994 January/SHAR194.ISO/email/dove0793.zip/DOVEMAIL.DOC

Text File | 1993-04-08 | 23KB | 399 lines

Documenatation for DoveMail
Copyright 1991, 1992, 1993 by Jack Decker

PLEASE NOTE:  The version of DoveMail shipped with this documentation is a
BETA TEST version.  It may not be totally bug-free at this point in time.

PLEASE NOTE:  This documentation is not intended to be a tutorial on how to
handle UseNet newsgroups.  If you do not understand how UseNet newsgroups
differ from Fidonet echomail, you may wish to obtain a feed to the UFGATE
echomail conference and ask appropriate questions there.

WHAT IS DOVEMAIL?

DoveMail is a program that will read an uncompressed UseNet batched newsgroup
file, extract the messages and send them on to other nodes according to a
control file.  It is written primarily for use in Fidonet, and other
Fidonet-technology networks.  DoveMail is Copyrighted software and may be used
only under the conditions set forth at the end of this document.

The reason for this program is this:  Currently, when a UseNet newsgroup is
gated into Fidonet, the usual practice is to convert it to Fidonet echomail
and then send it down the line in that format.  Unfortunately, no conversion
process is perfect, and some potentially useful control information is usually
lost in the conversion.  In addition, most Fidonet echomail processors don't
handle long messages well at all.

My idea is that if the newsgroups can be transported within Fidonet in
something resembling their native format, they can be converted to echomail
(if necessary) by appropriate software running at each system.  In other
words, the newsgroups will be transmitted through the network in something
pretty close to their original UseNet format, and only converted to echomail
(or some other message format) as necessary when a destination system is
reached.  While there may still be some errors in the conversion process, at
least they won't be cumulative... that is, you'll only have to worry about how
the conversion software you're running mangles the messages, and not about
what may have happened somewhere upstream.

Also, those messages cross-posted to multiple newsgroups will only be
transmitted once in UseNet format, whereas if the messages are converted to
echomail, a separate copy of the message is transmitted for each echomail area
that the message winds up in.

You can figure out most of what you need to know abut DoveMail by reading this
documentation file and DoveMail.Cfg, the sample configuration file included
with this package.  It is liberally commented and most of the options are
fully explained.  Actually, there aren't that many options to deal with, so
setup should be relatively easy for anyone who has experience with handling
echomail.

If you're only using DoveMail as a "passthru" for newsgroups (for example, on
an intermediate system between two other nodes) the DoveMail program will be
just about all you need.  However, if you have a need to convert UseNet format
mail to and from echomail, you may need another package such as Ufgate or
FredGate.  The difference is that those packages will (hopefully) only be
tossing mail for your system, not for other systems you may wish to feed.
Therefore, any errors you may have during the conversion process will only
affect you, not those you feed newsgroups to.  And, you'll be able to retain
only the control line information that you personally need in the messages you
convert to Echomail.

If your system can use messages stored in the *.msg (standard Fidonet single
message per file) format, then you may be able to get by with just the NewsToss
and NewsScan programs included in the DoveMail archive.  See the read.me file
included in the archive for more information.

As far as DoveMail is concerned, your system is just another system it's
feeding.  When it receives mail from other systems, it can create a mail
packet just for your system with only the newsgroups you want.  When your
system sends out mail, if it can create a batched newsgroup packet, DoveMail
can process it and send it to your uplinks and downlinks.

Here are some things DoveMail cannot do on its own:

1) Handle compressed mail... you need to uncompress it first via your batch
file (or some other mechanism) if necessary.  If you are receiving your feed
from another Fidonet system, I suggest using a program like GUS, POLYXARC, or
SPAZ that can determine the compression program used to create the archive,
and call the proper uncompression program.  If you are receiving a feed from
UseNet system, you may need to use the CUNBATCH.EXE program found in this
archive (see separate documentation file) and an MS-DOS version of the Unix
COMPRESS program (one is included in the archive, or you may wish to use
another version such as one you might find in the distribution archives of
programs like FredGate, UfGate, or Waffle) to decompress the files you receive.
Note that you may need to use the -i option of certain versions of COMPRESS
(see "PROGRAM LIMITATIONS" below).  Of course, if you wish to receive
newsgroups directly from a UseNet system, you may need to use certain other
utilities packaged with programs like FredGate, UfGate, or Waffle.

2) Handle UNbatched mail... DoveMail is a newsgroup processor only, and simply
doesn't understand unbatched mail (this capability may come later).  That also
implies that it will not handle mail (what we in Fidonet call "netmail")
messages.  There are considerations involved in handling mail messages that I
don't fully understand yet, so in the interim you'll have to settle for
routing netmail through the UUCP gateway that serves your net or region, or
the nearest node running a program like FredGate or UfGate.

3) Directly convert mail to or from *.msg, *.pkt, or any other common Fidonet
format.  It may do some of that later on as well, although I'd prefer to see
development of BBS's and message readers that could directly view and create
batched newsgroup files.  For now you can use FredGate or UfGate if you need
to convert to Fidonet formats, or if your system uses the *.msg file format,
the NewsScan and NewsToss programs found in this archive may be adequate for
the purpose.

Dovemail DOES maintain the "Path:" line of messages it passes, and it DOES put
outgoing newsgroup files in their proper outbound mail directories when
BinkleyTerm (or a similar compatible mailer program) is used.  It is Domain,
Zone, and Point aware when used with BinkleyTerm.

[Please note that I am only familiar with BinkleyTerm, so please don't ask me
questions about how to interface DoveMail with other mailers.  It SHOULD work
with just about any system, but no guarantees!]

THE DOVEMAIL.CFG FILE

This version of DoveMail uses a relatively short configuration file (if you
don't count the comment lines).  If you read the DoveMail.Cfg file, you'll
find that most of the options are explained fully.  Note that you can have
multiple DoveMail.Cfg files; just specify the name of the file you want to use
in the command line when you invoke DoveMail.  For example:

     DOVEMAIL DOVE2.CFG

Would use DOVE2.CFG as the configuration file.  If you don't specify anything
on the command line, it will look for the file DOVEMAIL.CFG.

You can specify an optional switch on the command line, -I.  If this switch
is used, it reverses the effect of the IgnoreFirst command in the configuration
file.  See the sample config file for more information.

A few lines found in the sample DoveMail.Cfg file might require some
additional explanation:

INFILES contains the filename mask (including path) of filenames that MIGHT
appear on your system containing batched newsgroups.  These should be the
filenames of the UNCOMPRESSED batched newsgroup files.  If you are being fed
by another system that uses DoveMail, the filename mask you'll probably use is
*.pku (prepend a pathspec if these files will not be in the current
directory).  On the other hand, if you get your feeds from a true UseNet
system, then the filenames might be something more cryptic, like fps????.d.
If you get batched newsgroups from several different systems and they each use
a different pattern, you may be further ahead to designate a particular
subdirectory for incoming newsgroup files only, uncompress all incoming
newsgroup archives into that directory, and use a statement like

     Infiles C:\NEWSGRUP\*.*

to cause DoveMail to examine every file in that directory.

OUTBOUND denotes the path to the Outbound directory for Fidonet mail for YOUR
zone and domain.  If you feed mail to other domains or zones, or to point
address, DoveMail will attempt to put the outbound batched newsgroup files in
outbound directories using the same naming conventions that BinkleyTerm would
use.  If you don't specify a zone or domain for a feed address, the outbouund
batched newsgroup files will be put into the directory specified by the
OUTBOUND statement, as if your own zone and domain were used.  If this is not
satisfactory, you can explicitly specify an output filespec (path and
filename) for a node you feed (you'd also do this to create a file for use on
your own system).

PKTEXT specifies the default extension to be used for outgoing batched
newsgroup files to be sent to other nodes.  Normally, uncompressed outgoing
batched newsgroup files will be placed into the appropriate outbound area in
files with the extension ".UUT", which is the batched newsgroup equivalent of
a Fidonet ".OUT" file.  In other words, if you are sending mail to a Fidonet
address, a mail packet will be generated in the proper outbound directory
using the filespec [net][node].UUT, where [net] and [node] are four digit
hexadecimal values (which your mail packer should change to a file with the
extension ".pku").  A .UUT file is treated exactly like an .OUT file for
Fidonet format mail except that it contains messages in UseNet format.

Consider the way that a normal echomail message is exported.  First, your
echomail processor (ConfMail, QMail, etc.) takes the messages from your
message base and exports them to a file in your outbound area with the
filename [net][node].OUT, where [net] and [node] are four digit hexadecimal
values as mentioned above.  Then, the processor or another external program
(such as oMMM) finds the .OUT file, renames it to a new file with an extension
of .PKT, and calls a compression program (such as LHARC or PKZIP) to add the
.PKT file to a mail archive file (which has an extension of .MO?, .TU?, .WE?,
.TH?, .FR?, .SA?, or .SU?).  It is expected that a .UUT file will be treated
in much the same way as an .OUT file, except it will be renamed to a file with
an extension of .PKU (rather than .PKT) before being archived into the mail
archive.  There is no reason I can think of that .PKU files cannot be archived
into the same mail archives (the .MO? ..... .SU? files) as .PKT files,
particularly if the recipient of the mail archive uses a program like GUS,
POLYXARC, or SPAZ to unarchive the mail archives before DoveMail and an
echomail processor is turned loose on the .PKU and .PKT files, respectively.

At present, programs like oMMM don't recognize a .UUT file, so you will
probably have to use some batch file magic or an external program (such as
DovePack, included in this archive with this program) to get the .UUT files
sent out.  It is hoped that authors of such programs will include support for
the .UUT/.PKU files in future revisions.  In the meantime, the PKTEXT command
gives you a way to have output files named with an extension other than .UUT,
if that will help you any.  For example, you might find it more convenient to
create .OUT files for "dummy" net/node numbers, IF you are EXTREMELY CAREFUL
to not send .OUT files created by DoveMail to nodes that might also receive
.OUT files created by your netmail or echomail processor (mixing .OUT files
created by the two programs could result in files unreadable by either
DoveMail or your echomail processor!!!).

OTHER CONFIGURATION FILE OPTIONS are explained in the sample DoveMail.Cfg file
included with this program.  If anything is unclear in regard to the use of
these options, please let me know and I will try to provide better
documentation in later revisions.

SQUISH USERS - If you are using Scott Dudley's SQUISH echomail processor,
please be aware that the default configuration WILL cause incoming *.PKU
files to be discarded without ever being unpacked from the mail archive.  To
avoid this, either use a program like GUS, POLYXARC, or SPAZ to unarchive
incoming mail archives before SQUISH can get at them, or else change
COMPRESS.CFG as follows:  In COMPRESS.CFG you will see a comment that reads
as follows:

  ; The Extract command tells Squish how to remove packets from an
  ; archive of the specified type.  "%a" will be translated to the name
  ; of the archive, and "%f" will be translated to the name of the
  ; file to extract.  (The "%f" specification may be translated into
  ; a wildcard!)

Then you will see "Extract" lines of the following format:

  Extract       pkunpak /r %a %f

Wherever you see %f in an Extract line, change it to *.PK? and SQUISH should
unpack your *.PKT and *.PKU files without problem.  Note, however, that if
you do this and you're also running the Maximus BBS, you'll need to use an
original COMPRESS.CFG for Maximus, and a modified COMPRESS.CFG for Squish
(that is, you'll need a separate COMPRESS.CFG for each program), or your
Maximus users will have problems!

ERRORLEVELS - DoveMail may exit with any of the following errorlevels.  Note
that anything above 1 should be considered a serious error:

0 - Batched newsgroup files found and processed,
1 - No batched newsgroup files were found
2 - Bad path was given on command line
3 - Unexpected error (probably a program bug, hope you never get this one)
4 - Serious config file error
5 - Error while opening/reading input files
6 - Error while opening/writing output files

PROGRAM LIMITATIONS (Please be sure and READ THIS SECTION):

Incoming batched newsgroup files may not be greater than 2,147,483,647 bytes
in length.  In other words, there is NO practical limit!  Nor is there a
practical limit on the size of a single message.

No more than 100 incoming batched newsgroup files may be processed in a single
run of DoveMail.  This can be modified by using the "MaxFiles" line in the
DoveMail.Cfg file.

There is no specified limit on the number of newsgroups that DoveMail can
process, but if too many are specified in the config file you could run out
of memory, although this is highly unlikely (however, this program has not
been tested with a large number of feeds).  A newsgroup specified with the
wildcard ending characters ".all" counts as only one newsgroup for memory
allocation purposes.

You can feed up to 32 nodes (including yourself) with this version of
DoveMail.  I hope that is adequate for everyone, since it would be a fair
amount of work to increase that number.  Please note that DoveMail opens a
file for each node receiving newsgroups (that are found in the packet
currently being tossed), plus the input file and the console device are open,
so if you feed a number of nodes you may need to increase the value associated
with the FILES statement in your CONFIG.SYS file.

No individual line in a message may be greater than 32,767 characters in
length.

This version of DoveMail expects to find linefeeds (ASCII 10 decimal
characters) ONLY at the end of each line in a message.  It may not (and
probably will not) be able to handle feeds that use other terminator(s) at the
ends of lines.  If your feed sends you batched newsgroup files that have other
terminators and DoveMail won't process them, send me a sample of a batched
newsgroup file that falls into this category and I'll see what I can do (no
promises, though).  PLEASE NOTE:  If you are receiving your feed from a UseNet
system and using one of the MS-DOS versions of COMPRESS to decompress the
files, you may first need to use the CUNBATCH.EXE program included in this
archive (see separate documentation).  If you are using a version of COMPRESS
OTHER THAN the one supplied in the DoveMail archive, and if it offers the
option to convert UseNet text files to MS-DOS format, DON'T DO IT!!!  For
example, with one version of COMPRESS that's making the rounds, you'll need to
use the -i switch to avoid adding Carriage Returns to the file.  That
particular version of COMPRESS defines usage of the -i switch as follows:

     -i: Image mode (defined only under MS-DOS).  Prevents conversion
         between UNIX text representation (LF line termination) in
         compressed form and MS-DOS text representation (CR-LF line
         termination) in uncompressed form.  Useful with non-text files.

Some versions of COMPRESS, INCLUDING THE ONE IN THE DOVEMAIL ARCHIVE, do not
attempt to convert text files to MS-DOS format (adding CR's) so this may not
apply to you.  Check the options for the version of COMPRESS you are using.
Again, DoveMail most likely WILL NOT WORK properly if there are CR's in the
file.  This is a design decision made in the attempt to keep message files as
close to native UseNet format as possible, in an attempt to retain some measure
of portability across platforms.

Messages may not have more than 30 header (control) lines at the top of the
message.  You can change this limit by using the "HeaderLines" line in the
DoveMail.Cfg file (however, any attempt to set it to a value of less than 20
will have no effect).

This version of DoveMail will never split a message header line (e.g. a Path:
line), no matter how long it is (I'm assuming that message header lines will
never approach the 32,767 character maximum... if one does, you've got other
problems besides DoveMail to worry about!).  I have been unable to find
anything referring to a maximum line length in the UseNet standards, but if
you find that this causes a problem with a system you feed (or with your
feed), please let me know what maximum line length is required and I may be
able to add something in.

DoveMail will attempt to process incoming batched newsgroup files in file
timestamp order, in an attempt to try and keep messages in some semblance of
order (the NewsToss and CUnBatch programs in the DoveMail distribution archive
will also do this).

LEGAL STUFF:

[PLEASE NOTE:  I give up.  I've had so many negative comments over the
"license agreement" in previous versions of this program that I've revised it
once more.  Now, almost everyone can use Dovemail freely, and hopefuly no one
other than bona-fide "control freaks" should find anything objectionable
here.]

This is preliminary documentation for DoveMail.  DoveMail is NOT public domain
software.  However, DoveMail users will NOT be required to pay for the use of
DoveMail, now or ever.  DoveMail is Copyrighted software which may be used
under the following conditions:

DoveMail is offered with no warranty, expressed or implied, of any kind.  If
you live in a place that disallows this type of limitation, you may not use
this software.  Under no circumstances is DoveMail warranted to do anything
other than take up space on your hard drive, and if by some chance it manages
to destroy some data that you consider valuable, I assume no responsibility
for that (though I'd certainly like to hear about it).  IF YOU USE DOVEMAIL,
YOU DO SO AT YOUR OWN RISK.  If you cannot accept this, or if the laws of your
area do not permit the author of software to absolve himself of all legal
liability that may result from the use of the software, then you may not use
DoveMail.

There is only one other remaining restriction on the use of DoveMail.  If you
are NOT a Fidonet sysop, the following restriction DOES NOT apply to you.
Otherwise, if you ARE a Fidonet sysop, you may NOT use DoveMail in any
newsgroup distribution scheme that refuses newsgroup feeds to other sysops
based solely on where they are grographically located.  In other words, you
are not required to feed newsgroups you receive to any other system, but if
you do, you may not refuse a newsgroup feed to a node solely because they are
not in your net or region.  Nothing in this paragraph should be construed as
requiring you to provide feeds to anyone at your expense, but if they are
willing to pay for the call, you have to give them a feed under the same
conditions you would give feeds to anyone in your own net or region (for
example, if you will only give feeds to nodes that can connect at 1200 bps or
higher in your own region, you are perfectly justified in refusing a 300 bps
pickup from any region).

Your duty to provide feeds to others is not diminished by the fact that
newsgroup feeds may be available at some other node (for example, one that is
closer to them).  It is and should always be their right to NOT receive their
feeds from the "nearest" node.  Fidonet echomail policy tries to force people
into relationships even though they simply may not be able to get along with
each other.  This should never be the case with DoveMail users.

UseNet newsgroups are NOT echomail, and should never be confused with
echomail.  UseNet does not have the stupid, idiotic, assinine (substitute a
stronger word if you can think of one) geographic restrictions that have
plagued Fidonet echomail for many months now.  The UseNet philosophy is best
summed up in a verse in a rather cute signature line I saw at the end of a
message in comp.dcom.modems (attributed to [e-mail]):

          A host is a host from coast to coast
          & no one will talk to a host that's close
          Unless the host (that isn't close)
          is busy, hung or dead!

(If this sounds familiar but you can't quite place it, think about the theme
song of an old TV comedy series where the horse had all the good lines...)

Just because a newsgroup is imported into Fidonet does NOT mean that Fidonet
echomail rules or policies apply.  So if you are a "control freak" that
insists on imposing geographic restrictions on others, you're just going to
have to find or write some other software to do it with.  The ONLY intent of
this section is to prevent sysops from refusing to feed newsgroups to other
sysops based on their geographic location, as is (unfortunately!) common
practice with Fidonet echomail feeds.

If the above creates some unusual problem for you, please write to me and I
may grant you partial relief from the above restriction, on a case by case
basis, considering the unique circumstances of your situation.

If you have questions or comments, please direct them to Jack Decker at
Fidonet address 1:154/8 (Internet address [e-mail]).
Please note that my node is private, unlisted since it is only accessible
during the late night hours.  If you have a need to send me something via file
attach, please send me a netmail message first and I'll get back to you with
further instructions, or if it's a short file just UUENCODE or XXENCODE it and
send it in a netmail message.