Changi NNTP Server 1.0

[ Bottom of Page | Previous Page | Next Page | Table of Contents ]

EXPIRE.EXE
News Expiration

By default Expire scans the history file and uses the arrival time or an optional explicit expiration date recorded in this file to purge old articles. The program provides a large set of options to modify this default behaviour to your specific needs.

One of the main features of this program in conjunction with the Changi server is its ability to perform news expiration without interrupting connected clients. In a first step Expire instructs the server to queue incoming articles and starts purging outdated news. In a second step it rebuilds the history after forcing the server to freeze all connections. When at least the new history is available, Expire tells the server to continue in normal operation mode.

The next chapters will cover the following topics:

Unless option -C is used, Expire needs a text file sorting program, which is included in the Changi distribution and must be available in the CHANGIWORKDIR.

Usage Examples

When starting Expire manually from the keyboard, you should always add option -v to produce verbose output.


Daily Expiration

The prefered way to call Expire on a daily basis is with option -C.
expire -Cv
In this case all entries of purged articles in the history file are simple marked canceled. Expire doesn't need to rebuild the index. Therefore this option is also well suited while running out of diskspace. However, using Expire in this mode will not shrink the history files. From time to time (once a week), you should run
expire -v

Tuning Expiration Times

Normally Expire removes all articles one week after their arrival at the local system. Option -e can be used to globally change this default. The value given with this option is interpreted as a number of days. Different time units may be specified by any of the following trailing characters:

s seconds
m minutes
h hours
d days (default)
w weeks

It's possible to combine several units. The example below will remove all articles older than half a week (3 days and 12 hours).

expire -ve3d12h
Note, that articles may define explicit expiration dates in the header, which are normally honored by the program.

Most probably you'll want to apply different expiration times to different newsgroups. You can do so by creating a configuration file named expire.conf with any text file editor.

#
# Sample expire.conf
#
local.* 4w 12w
local.grep* 12h 12h
comp.os.os2*,de.comp.os.os2* 2w 6w
Lines with character # in front are ignored and could be used to include comments. All other lines must contain three items separated by one or more spaces and/or tabs. Obviously the first item is a comma separated list of newsgroups, where pattern matching is supported. The two following items define the minimum and maximum time to keep articles in those groups. The minimum time is the one which we set with option -e in one of our examples above. The maximum time to keep an article is the absolute deadline. Even if an explicit expiration date in the article's header states a later expiration, the article will be removed.

If you copy expire.conf into CHANGIWORKDIR and start Expire with

expire -ve8
then the program will expire all articles in the specified groups after the specified time periods. Articles of all other groups will expire after 8 days unless they contain an explicit expiration date beyond this period.


Late Duplicate Detection

Due to misconfigured sites, duplicate articles may arrive long after their originals had been distributed. To add some extra security, Expire maintains a so called remember period. By default this period is equal to the expiration time and all message-ids will be removed from history at the same time the article is deleted.
expire -ve2w -E4w
With these options Expire will remove articles which arrived two or more weeks ago. However, option -E4w forces the program to keep related message-ids in the history for four weeks.


Rebuilding

In case your history files get corrupted, which is may be the case after a system crash with Changi running in the background, you should call
expire -rv
before restarting Changi. This will rebuild the history from scratch and additionally update the active file. Option -xs may be added to instruct Expire not to contact the local server (which shouldn't be running with a corrupted history).

If new groups had been added before the crash, Changi may not have been able properly update the active file. Calling

expire -arv
scans all newsgroup directories. If Expire locates any article file in a directory below CHANGINEWSDIR without a corresponding active file entry, that newsgroup name will be automatically added to the list of active newsgroups.

In a worst condition you may even lose the complete active file. Run

expire -Arv
to rebuild a new one from scratch.

Pitfalls

Rebuilding the history file may occupy three to five times the diskspace used during normal expiration.

Removing a newsgroup will not remove the articles left in that group. This is no problem, because these articles are still recorded in the history file and will be expired. However, when rebuilding the history file with

expire -rv
Expire will only process groups named in the active file. Left over articles in removed groups will never expire.

Note also, that rebuilding the active file may silently recreate recently removed groups, if unexpired articles had been left.


Cleaning Up

It's recommended to run
chanco go expiring
Chanco oversync normal
after Expire has finished its task. The first call, Chanco go, provides extra security in case of internal Expire problems, and will ensure that Expire didn't leave the server flushed or paused. The second call, Chanco oversync, will start a background task within the server, which will update all overview files. This avoids long waiting times when a client accesses a newsgroup with expired articles for the first time.


[ Top of Page | Previous Page | Next Page | Table of Contents ]


URL: changi/manual/expire.html
Created: 25 August 1996
Revised: 21 November 1996
Author: harald@os2point.ping.de