Slurp is an advanced passive NNTP client for UNIX. It will connect to a remote NNTP server and retrieve articles in a specified set of Usenet newsgroups that have arrived after a particular date (typically the last time it was invoked) for processing by your local news system.
There are two configuration files used by slurp.
Entries in slurp.sys take the form
This format should be familar to people who have used the C News sys file. Entries for a particular host can be continued on more than one line by using a '\' at the end of the line. e.g.
Slurp is even more picky about the presence of whitespace than C News. It can only appear in comments. Comments begin with a '#' and continue to the end of the line.
Using distributions is not recommended - they're only really included for completeness. Under current NNTP implementations, setting distributions requires the server to open each article, search through for the distributions line and check it against the supplied list. This will not only increase the load on the server substantially, but increase the amount of time for the connection.
The file slurp.tim contains the time when slurp last connected to the NNTP server at each host. slurp can then use this time to pick up all the articles that have arrived at the server since the last session.
Entries in slurp.tim take the form
Where hostname is the name of the NNTP server host and time the retrieval start time. Unlike its use from the command line, time does not need surrounding quotes here.
Each time slurp is run and slurp.tim updated, the current slurp.tim will be backed up in the file slurp.tim.o.
When run, slurp will first retrieve the appropriate newsgroup list, distribution list and start time for the specified server, either from the configuration files or overriding those settings with the command line options. If the -l or -w options are not set, then it will connect to the remote host and retrieve the current time there to use as the start time for the next session. Slurp will now connect to the NNTP server at the host and issue a NEWNEWS request, asking for all the articles that have arrived in the specified list of newsgroups since the specified time. The server will respond with a list of article ids. If an article id is not already present in the local history file, then it will be stored in memory. If the number of newsgroups is too large to fit on one line (NNTP has a maximum line length of 512 characters) then a series of NEWNEWS requests will be carried out, adding futher article ids to the memory list if they are not already present.
Once this has been completed, slurp moves into an article retrieval stage. A pipe to the C News or INN rnews program is opened. Slurp will then go through the list of articles ids in memory and request them in turn from the server, adding them to the batch of articles being piped to rnews. If the batch gets too large or has too many articles, then the pipe will be closed, submitting the batch to the news system, and a new pipe opened for a new batch.
Once all the articles have been retrieved, the pipe to rnews will be closed, submitting the final batch of articles. If the -w option has not been set, then the previously obtained time to use for the next NEWNEWS will be written to slurp.tim.
Statistics on the connection will be logged to syslog (or stderr if syslog is not available). The new article count is the total number of articles that have been submitted to rnews. The duplicate count is how many message ids were found to already exist on the local system. If two NEWNEWS requests are necessary and an article id was returned by both requests, then it will be included twice in the duplicate count. The missing count is those articles which were in the server's history file but didn't exist as actual article files, usually because they have been cancelled. If configured, the speed of transfer of the article retrieval stage will also be logged.
Slurp returns a series of return codes which may be useful to controlling programs:-
Locking of slurp.tim is not currently carried out, so if two copies of slurp are running at the same time, then there is the possibility that they will clash while updating.
There is currently no support for the NNTP simple authorisation protocol.