Il CD di internet

home *** CD-ROM | disk | FTP | other *** search

/ Il CD di internet / CD.iso / SOURCE / N / NN / _NN-SPOO.TAR / usr / doc / nn / NNTP < prev next >

Wrap

Text File | 1991-04-23 | 14.4 KB | 369 lines

NNTP SUPPORT ------------ This file describes the NNTP support available in nn release 6.4. The NNTP support was implemented by Rene' Seindal, seindal@diku.dk. PREREQUISITES ------------- First of all, you need read-access to an NNTP-server, and if you want to post, the server must allow that. If you have news on one of your systems, and want to run an NNTP server on that system to feed other local systems, you need to get and install the nntp-1.5 distribution with at least patches 1-3 (I think patch 8 is the latest). It is available from several ftp-sites in the USA. It is also available on freja.diku.dk (ip 129.142.96.1). However, just to run nn on you local system with or without NNTP, you don't need anything besides the nn 6.4 distribution!! The necessary modules to access a remote NNTP server is an integrated part of nn, so if you specify to use NNTP, the necessary code is automatically included. A recent version of the `mini-inews' program which is required to post articles via an NNTP server is also included for your convenience. See the INEWS section below. HOW IT WORKS ------------ NNTP is supported both in nn and nnmaster. When NNTP is used, the database with the header information used by nn is still maintained on the local system (because NNTP does not know about the nn database (yet?)). When the master is set up to use NNTP, it will connect to the NNTP- server in each iteration of the collection (the interval set with -r), get a copy of the active file, and incorporate the new articles into the database. To do this, the master will temporarily transfer one article at a time from the NNTP-server to the local system. When the articles are read with nn, it will use the local database to present the menus, and fetch the articles from the NNTP-server as they are requested by the user. It will connect to the NNTP server the first time it is necessary to fetch an article. Neither nnmaster, nor nn will use NNTP if they run on the NNTP-server itself (they will directly access the news files). Both nn and nnmaster access the server in reading mode. The master and all client MUST use the same server at all times, since the local database contains article numbers, that are only unique for each NNTP-server. SHARING THE DATABASE -------------------- You must also decide whether you want to share the database between your local news clients, and how you are going to do it. The database will take up some disk space, normally about 1Mb per 10.000 articles. There are several ways to manage this space. This simplest solution, is to let each client run it own master, i.e., have its own database. This means, of course, no sharing. Alternatively, one host can run the master, and distribute the database to the others via e.g., rdist. This doesn't save disk space, but saves load on the NNTP-server. Last, the database can be shared with NFS/RFS (see the description of NETWORK_DATABASE in the config.h file). The possibility of making a `nndb-server' stands open. It could be realized either as a separate server, running under inetd, or it could be incorporated into nntpd. It has not been implemented, but might be part of a future release (any volunteers?). CONFIGURATION ------------- To use NNTP in nn, you must edit the relevant parts of config.h: NNTP You enable the use of NNTP by defining the macro NNTP. NNTP_SERVER Both the master and the clients will look up their NNTP-server in the file given by the macro NNTP_SERVER. If the name is not an absolute path name, it is taken to be relative to LIB_DIRECTORY. The format of the file is compatible with the one used in clientlib.c in the nntp-1.5 distribution, i.e., the first non-blank line, not starting with '#' is taken to be the name of the NNTP-server. This file MUST be present, and must contain a valid host name. NNTP_POST If you use the NNTP based inews (part on the NNTP distribution) and you have hosts that are not allowed to post to the NNTP server, you should defined this. It will make nn check at connect time whether the NNTP server allows postings, and reject all attempts to post, if the server disallows posting. If you do not define this, users will be allowed to post by nn, but the posting will eventually fail. Again, this parameter is only relevant, if you use the NNTP based inews, and have hosts that are not allowed to post. NNTP_MINI_INEWS_HEADER According to RFC 977, the mini-inews that comes with nntp requires that the articles it receives contain a complete header including Message-ID, Date, and Path fields (which the normal inews generates itself). I consider this to be a bug in the RFC, and maybe some (future?) versions of mini-inews will provide these fields. However, if you use mini-inews for postings and your version of mini-inews does not generate these headers, you should define NNTP_MINI_INEWS_HEADER in config.h. Notice that the Message-ID and Path may be less than perfect (but the Message-ID should be unique)! NEWS_LIB_DIRECTORY & INEWS If either is defined, they specify the destination of the mini-inews program when installed below with INEWS being used if both are defined. If neither is defined, it will be installed in /usr/lib/news/inews. EXCELAN You can define this symbol if your tcp/ip is based on the EXCELAN implementation (no netdb.h, no getservbyname(), no gethostbyname()). (This belongs in the s- file!) You will (probably) also need to add -lsocket to NNTP_EXTRA_LIB. Normally it will use port 119 for nntp, but this can be modified through "#define IPPORT_NNTP 1150" (or similar) in the init file. NO_BZERO NO_RENAME You can define these symbols if your system doesn't provide the bzero() and/or rename() functions (these #define:s belong in the s- file). NNTP_EXTRA_LIB Here you can specify libraries and loader flags which are only needed when compiled with NNTP. The normal EXTRA_LIB is still used. TUNING ------ Both the server and each client maintains a cache of recently accessed articles, to minimize communication with the server (mainly to avoid fetching large digests continuously). The master needs the cache when it splits digests, and the clients need it, because nn has a tendency to reopen the articles several times. The master's cache is kept in LIB_DIRECTORY, and each client's cache are kept in the users .nn directory. The constant NNTPCACHE (defined in nntp.c but can be redefined in config.h) defines the size of the cache, whose optimal size depends on the amount of news kept on line on the NNTP-server. Values of 5-10 gives reasonable results. The effect is most striking when reading digested news. The location and size of the cache can also be changed on a per-user basis via the related nntp- variables (see nn.1). INSTALLATION ------------ Making and installing nn using NNTP does not differ from a non-NNTP nn installation, except for the differences in the configuration and the need to specify the NNTP server in the NNTP_SERVER file. Notice however, that the NNTP_SERVER file must be properly initialized before doing the 'make initdb'. If something goes wrong in the initialization of the database, you will have to run 'nnmaster -I' again by hand. INEWS ----- If you want to post news via the NNTP server, you will need to run a program named `(mini-)inews' which is included in the normal NNTP distribution. However, since this is the only part of the NNTP you need to run nn, a recent version (1.5.7) of the mini-inews program is included with nn in the inews/ directory. If you already have installed and maintain mini-inews on your system, you should not use the version that comes with nn - it would just be extra unnecessary work. However, if you don't run mini-inews already, using the version that comes with nn will be somewhat simpler to configure, because it takes advantage of the definitions you have already made for nn in config.h. Thanks to Steve Simmons who contributed the simplified inews configuration (which I took the freedom to simplify even further). To configure and install mini-inews, all you should need to do is (everything must be done with inews/ as current working directory): $ cd inews $ edit conf.h -- modify as described in the file $ make If things work, type: $ su # make install As far as I know, NNTP_MINI_INEWS_HEADER needs to be defined in nn's config.h for nn to work with this mini-inews version. See the files inews/README.NN and inews/README for further information. ERROR HANDLING -------------- The handling of errors have been improved since the initial release. The master will handle most errors by closing the connection, and returning to the main loop. All errors in the master are logged, with a code of `N,' so they can be inspected with the `n' command in nnadmin's Log menu. A few errors are considere fatal. If any of these occur operation will be discontinued. These errors are such as failure to find the NNTP server, failure to find the NNTP service, and responses from the NNTP server in the 500 range (ill-formed requests, access denied, ...) NNTP server timeouts are handled specially. If the NNTP server times out, both nn and the master will attempt to restart it (by connecting again). This shouldn't happen in the master (which won't leave sockets idle for that long), but it can easily happen in nn, if it is left suspended for too long. If the server responds with code 400 (Service discontinued), a reconnect is also tried. PROBLEMS -------- I am not certain what should happen if the server sends back responses in the 1xx range. I do not know whether a NNTP server is allowed to return one of these responses on its own initiative. If it is, nn should probably ignore (or display) the messages. Currently, nothing is done to treat these responses in any way. I have seen a strange thing happen to the master, which I have not been able to reproduce. The master ran on a Sun-4 running SunOS 4.0, and the NNTP server was a VAX 785 running MORE/bsd. The NNTP software was version 1.5.3. The master was stuck in a read from the NNTP server. A netstat on the Sun show an established connection to nntpd on the Vax, but a netstat on the Vasx did not show any NNTP connections. There was no nntpd running, and no messages on the console indicating any failures. [ It is now known that this problem is related to the socket not having the KEEP ALIVE flag set, but I have not got the necessary patches to fix it, ++Kim ] SPONTANEOUS NNTP ERROR 502 -------------------------- Sometimes nn or nnmaster may stop with the following message: NNTP 502 You only have permission to transfer, sorry. This particular case is probably the result of the NNTP server trying to turn your IP address into a fully qualified domain name (FQDN) so it can look you up in its access file. The NNTP server probably uses the domain name server (DNS) to map IP addresses into FQDNs. If the local DNS doesn't already know the answer, it has to go out over the network to find it. This can take a few seconds, and the library routine that does all this for the NNTP server might time out before the answer gets back to it. If this happens, the NNTP server doesn't know your FQDN, so it gives you the default access specified in the server's nntp_access file, which is usually "xfer" (article transfer only). In the time it takes for you to run nn again the DNS usuallu has its answer back, so things usually work the second time. One way to work around this problem is to specify the IP address of the client in the nntp server's access file; then it is not necessary to lookup the FQDN. Thanks to Tim Ramsey and Nick Sayer for this information. DEBUGGING NNTP CONNECTIONS -------------------------- If you want to debug the nntp connection, you can run the nnmaster with the option -D2 (or -D3 which also turns on the normal -D verbose output). In the nn client, you can turn on the nntp-debug variable in the init file. The debug output from nnmaster will be placed in $TMP/nnmaster.log while the output in the client will appear on the message line. POSSIBLE EXTENTIONS TO THE NNTP SERVER -------------------------------------- The new expire method used in release 6.4 is very efficient on local systems, because it will just read the spool directories to get a list of available articles in each group. However, with nntp, the only way I know of to get a list of available articles in a group with nntp is the XHDR request. However, this will open every article in the group to extract the desired field, but the only thing I am interested in is the article number itself. So I suggest to add a LISTGROUP request to the NNTP server to return the equivalent of ls $GROUPDIR | sed -n '/^[0-9][0-9]*$/p' (in any order - nnmaster will sort the list itself). Currently nnmaster will test whether this request works before using the XHDR request, so no changes to nnmaster will be required to take advantage of such a fix. Another possible performance increase would be if there was a request to get the current modification time of the ACTIVE file. This is the check nnmaster will do to see if there might be work to do on a local system, but with NNTP it has to read the active file from the server and compare it to a local copy to determine whether there is work to do. A simple ACTIVESTAT request returning the active file's age and size would fix this. Currently nnmaster is not prepared to use such a request, but it would be easy to add. ALTERNATIVES ------------ Alternative implementations can be conceived, especially in the master. The master normally collects articles by rereading the active file, looking for changed article numbers. For each group with new articles, it reads the new articles and adds them to the database. This scheme has been kept in the NNTP-based master, to keep the changes at a minimum. An alternative solution could be to use NEWNEWS to get a list of new articles since last collect, and fetch each article in sequence. The newsgroups and article numbers within each group could then be found in the Xref: field. This would probably improve efficiency, since the master would then generate fewer failing requests (for non-existent articles), and it would only read cross-posted articles once. This solution would, however, require some surgery on the current structure of the masters main loop. Suggestions and improvements are very much welcome. Send your ideas or contributions to nn-bugs@dkuug.dk.