Source Code 1992 March

home *** CD-ROM | disk | FTP | other *** search

/ Source Code 1992 March / Source_Code_CD-ROM_Walnut_Creek_March_1992.iso / usenet / altsrcs / 3 / 3146 < prev next >

Wrap

Internet Message Format | 1991-03-28 | 31.8 KB

From: dt@yenta.alb.nm.us (David B. Thomas) Newsgroups: alt.sources Subject: reap(8): expire news as space needed Message-ID: <1991Mar28.025957.1976@yenta.alb.nm.us> Date: 28 Mar 91 02:59:57 GMT This message contains the README, followed by the source: README for reap version 1.1 by David B. Thomas (dt@yenta.alb.nm.us). A Problem: News volume fluctuates wildly...sites with small disks must either expire aggresively, often deleting more than necessary, or worry that an unexpected huge dose of news might overfill the disk. A Solution: Implement a tool that expires according to a user-defined scheme until sufficient freespace is reclaimed, then stops, leaving as much juicy news online as is feasible. Reap does this. Expire Does Two Jobs: Both Bnews and Cnews expires really do two jobs: 1. trim history files 2. delete outdated articles Thanks to some inspired jootsing (acronym for "jumping out of the system") by Mike Murphy (mrm@sceard.com) and others, it is more than possible to separate those two functions. This is, of course, in keeping with the unix philosophy of one tool doing one job well! What Reap Does: Reap only takes care of the second job: deleting old articles. It works by checking freespace, and processing one line at a time from a list of expire functions, until the desired freespace is attained. Each expire function consists of an age limit in days (decimals okay) and a list of newsgroups to process or not process, sys file style. Ex: .5 alt.sex.pictures,talk,!talk.bizarre,junk 1 rec,!rec.games,!rec.humor This example would check freespace, and if more space is needed, expire to .5 days everything in alt.sex.pictures, talk (except for talk.bizarre) and junk. Then it would stop and check freespace again. If still more space is needed, it would expire to 1 day everything in rec except rec.games.* and rec.humor.*. It's that simple. Okay...So How Do I Arrange To Trim The History Files, Since Reap Can't: Included in this distribution is a shell script (mostly written by Mike Murphy) to handle Cnews history files. It shouldn't be too difficult to do something similar for Bnews, or you can give in and use the original expire utility with options that tell it to expire the history files only...but I think this will be slow. It just comes down to removing lines from ordinary text files, based on their contents. Murphy and I used awk. But Is It Fast: Yes, largely because it doesn't have to do much. Even "find | rm" is slower because find is repeatedly exec-ing rm. "rm -rf" has me beat, though, I'll bet! :-) Since the functions of deleting articles and trimming history are separate, I now run reap every six hours, but trim the history list just once a day. That effectively keeps my disk space up to snuff, but only thrashes at the history file in the middle of the night. Credits: I owe a lot to Mike Murphy for inspiring me with his "trasher" system. I also owe a lot to all of your netters who will flood me with more suggestions and improvements in the coming weeks (hint, hint!). little david dt@yenta.alb.nm.us ps - reap is freeware. No strings(1) attached. #!/bin/sh # This is a shell archive (shar 3.44) # made 03/28/1991 02:51 UTC by dt@yenta # Source directory /u/dt/src/reap # # existing files will NOT be overwritten unless -c is specified # # This shar contains: # length mode name # ------ ---------- ------------------------------------------ # 934 -rw-rw-r-- Install # 339 -rw-rw-r-- MANIFEST # 900 -rw-rw-r-- Makefile # 2918 -rw-rw-r-- README # 969 -rwxrwxr-x exphist # 3567 -rw-rw-r-- lib.c # 1611 -rw-rw-r-- list.sample # 2497 -rw-rw-r-- main.c # 6129 -rw-r--r-- reap.8 # 2845 -rw-rw-r-- reap.c # 1195 -rw-rw-r-- reap.h # # ============= Install ============== if test -f 'Install' -a X"$1" != X"-c"; then echo 'x - skipping Install (File already exists)' else echo 'x - extracting Install (Text)' sed 's/^X//' << 'SHAR_EOF' > 'Install' && X To install: X 1. Edit tops of Makefile and reap.h to fit your system. X [If you are installing this on a non-system-V machine, you will need X to rewrite the freeblox() function at the top of lib.c. This should X be very easy. Please email your working mods to me.] 2. make 3. Compose a sample function list file or use the sample provided to X test reap (use the -n option to avoid really removing files). 4. su and make install. 5. Arrange to trim your news system's history list regularly. If you have X Cnews, try the "exphist" script provided, run from cron. That X script requires that a tiny file, /usr/lib/news/histdays, containing X the number of days of history data to keep, or else you can hardcode X in your favorite number. If you have Bnews, I don't have a nifty X suggestion, other than using expire(8) with a ridiculously long X article expire time, but a sane history expire time. 6. Arrange to run reap from cron. SHAR_EOF chmod 0664 Install || echo 'restore of Install failed' Wc_c="`wc -c < 'Install'`" test 934 -eq "$Wc_c" || echo 'Install: original size 934, current size' "$Wc_c" fi # ============= MANIFEST ============== if test -f 'MANIFEST' -a X"$1" != X"-c"; then echo 'x - skipping MANIFEST (File already exists)' else echo 'x - extracting MANIFEST (Text)' sed 's/^X//' << 'SHAR_EOF' > 'MANIFEST' && Included with this distribution are: X README - general description and credits Install - hints on how to install reap and get it going Xexphist - history trimmer utility for Cnews list.sample - sample function script file for reap reap.8 - man page lib.c - source main.c - source reap.c - source reap.h - source Makefile - makefile SHAR_EOF chmod 0664 MANIFEST || echo 'restore of MANIFEST failed' Wc_c="`wc -c < 'MANIFEST'`" test 339 -eq "$Wc_c" || echo 'MANIFEST: original size 339, current size' "$Wc_c" fi # ============= Makefile ============== if test -f 'Makefile' -a X"$1" != X"-c"; then echo 'x - skipping Makefile (File already exists)' else echo 'x - extracting Makefile (Text)' sed 's/^X//' << 'SHAR_EOF' > 'Makefile' && #---------------------------- # parameters for installation #---------------------------- X # directories where manual and executable are to be installed BINDIR = /usr/lib/newsbin/expire MANDIR = /usr/man/man8 X # owner, group and file mode for manual and executable EXEOWNER = bin EXEGROUP = bin EXEMODE = 775 MANOWNER = bin MANGROUP = bin MANMODE = 664 X X # your favorite C compiler #CC = cc CC = shcc X # directory access functions library, if not in standard C library. LIB = -ldirent X X #------------- # boring stuff #------------- X OBJ = main.o lib.o reap.o X reap: $(OBJ) X $(CC) -s -o reap $(SHLIB) $(OBJ) $(LIB) X $(OBJ): reap.h X install: reap reap.8 X cp reap $(BINDIR) X chown $(EXEOWNER) $(BINDIR)/reap X chgrp $(EXEGROUP) $(BINDIR)/reap X chmod $(EXEMODE) $(BINDIR)/reap X cp reap.8 $(MANDIR) X chmod $(MANMODE) $(MANDIR)/reap.8 X chown $(MANOWNER) $(MANDIR)/reap.8 X chgrp $(MANGROUP) $(MANDIR)/reap.8 SHAR_EOF chmod 0664 Makefile || echo 'restore of Makefile failed' Wc_c="`wc -c < 'Makefile'`" test 900 -eq "$Wc_c" || echo 'Makefile: original size 900, current size' "$Wc_c" fi # ============= README ============== if test -f 'README' -a X"$1" != X"-c"; then echo 'x - skipping README (File already exists)' else echo 'x - extracting README (Text)' sed 's/^X//' << 'SHAR_EOF' > 'README' && X README for reap version 1.1 by David B. Thomas (dt@yenta.alb.nm.us). X A Problem: X News volume fluctuates wildly...sites with small disks must either Xexpire aggresively, often deleting more than necessary, or worry that an unexpected huge dose of news might overfill the disk. X X A Solution: X Implement a tool that expires according to a user-defined scheme until sufficient freespace is reclaimed, then stops, leaving as much juicy news online as is feasible. Reap does this. X X Expire Does Two Jobs: X Both Bnews and Cnews expires really do two jobs: X 1. trim history files X 2. delete outdated articles Thanks to some inspired jootsing (acronym for "jumping out of the system") by Mike Murphy (mrm@sceard.com) and others, it is more than possible to separate those two functions. This is, of course, in keeping with the unix philosophy of one tool doing one job well! X X What Reap Does: X Reap only takes care of the second job: deleting old articles. It works by checking freespace, and processing one line at a time from a list of expire functions, until the desired freespace is attained. Each expire function consists of an age limit in days (decimals okay) and a list of newsgroups to process or not process, sys file style. Ex: X X .5 alt.sex.pictures,talk,!talk.bizarre,junk X 1 rec,!rec.games,!rec.humor X This example would check freespace, and if more space is needed, expire to .5 days everything in alt.sex.pictures, talk (except for talk.bizarre) and junk. Then it would stop and check freespace again. If still more space is needed, it would expire to 1 day everything in rec except rec.games.* and rec.humor.*. It's that simple. X X Okay...So How Do I Arrange To Trim The History Files, Since Reap Can't: X Included in this distribution is a shell script (mostly written by Mike Murphy) to handle Cnews history files. It shouldn't be too difficult to do something similar for Bnews, or you can give in and use the original expire utility with options that tell it to expire the history files only...but I think this will be slow. It just comes down to removing lines from ordinary text files, based on their contents. Murphy and I used awk. X X But Is It Fast: X Yes, largely because it doesn't have to do much. Even "find | rm" is slower because find is repeatedly exec-ing rm. "rm -rf" has me beat, though, I'll bet! :-) X X Since the functions of deleting articles and trimming history are separate, I now run reap every six hours, but trim the history list just once a day. That effectively keeps my disk space up to snuff, but only thrashes at the history file in the middle of the night. X X Credits: X I owe a lot to Mike Murphy for inspiring me with his "trasher" system. I also owe a lot to all of your netters who will flood me with more suggestions and improvements in the coming weeks (hint, hint!). X X little david X dt@yenta.alb.nm.us X ps - reap is freeware. No strings(1) attached. SHAR_EOF chmod 0664 README || echo 'restore of README failed' Wc_c="`wc -c < 'README'`" test 2918 -eq "$Wc_c" || echo 'README: original size 2918, current size' "$Wc_c" fi # ============= exphist ============== if test -f 'exphist' -a X"$1" != X"-c"; then echo 'x - skipping exphist (File already exists)' else echo 'x - extracting exphist (Text)' sed 's/^X//' << 'SHAR_EOF' > 'exphist' && #! /bin/sh # exphist - expire history file X # =()<. ${NEWSCONFIG-@<NEWSCONFIG>@}>()= . ${NEWSCONFIG-/usr/lib/news/bin/config} X PATH=$NEWSCTL/bin:$NEWSBIN/expire:$NEWSBIN:$NEWSPATH ; export PATH umask $NEWSUMASK X days=`cat /usr/lib/news/histdays` X lock="$NEWSCTL/LOCKexpire" ltemp="$NEWSCTL/L.$$" Xecho $$ >$ltemp trap "rm -f $ltemp ; exit 0" 0 1 2 15 if newslock $ltemp $lock then X trap "rm -f $ltemp $lock ; exit 0" 0 1 2 15 Xelse X echo "$0: expire apparently already running" | mail "$NEWSMASTER" X exit 1 fi X cd $NEWSCTL rm -f history.n history.n.pag history.n.dir now=`getdate now` age=`expr 86400 \* $days` ago=`expr $now - $age` awk "{split(\$2,dates,\"~\");if(dates[1]>$ago)print \$0}" history >history.n mkdbm history.n [ -s history.n ] && mv history history.o && # install new ASCII history file mv history.n history && rm -f history.pag && # and related dbm files rm -f history.dir && mv history.n.pag history.pag && mv history.n.dir history.dir Xexit 0 SHAR_EOF chmod 0775 exphist || echo 'restore of exphist failed' Wc_c="`wc -c < 'exphist'`" test 969 -eq "$Wc_c" || echo 'exphist: original size 969, current size' "$Wc_c" fi # ============= lib.c ============== if test -f 'lib.c' -a X"$1" != X"-c"; then echo 'x - skipping lib.c (File already exists)' else echo 'x - extracting lib.c (Text)' sed 's/^X//' << 'SHAR_EOF' > 'lib.c' && X /* X * lib.c X * subroutines needed by reap utility X */ X #include "reap.h" X /* X * freeblox(devno) X * X * THIS WILL REQUIRE MODIFICATION TO WORK UNDER NON-SYSTEM-V UNIX. X * IT SHOULD BE EASY. PLEASE EMAIL WORKING MODS TO THE AUTHOR: X * dt@yenta.alb.nm.us X * X * assuming devno is the device number of a file system, X * this function returns the free space on that file system, X * in blocks (whatever that means for that filesystem) as an int. X * X */ X freeblox(devno) int devno; { X static struct ustat ust; X X if (ustat(devno, &ust)) X ouch ("%s: ustat() failed\n"); X X return (ust.f_tfree); X } /* freeblox() */ X X X X /* X * parse(cmd) X * execute a line from the script file X * X * valid command lines: X * a blank line is ignored X * # comment (ignored) X * days filespecs X * X * filespecs is a comma separated list of any combination of the following: X * 1. a directory (newsgroup) to include, without recursion. X * ex: alt.sources. X * 2. a directory to include, recursively. X * ex: alt.sources X * 3. a directory to exclude, without recursion. X * ex: !alt.sex. X * 4. a directory to exclude, recursively. X * ex: !alt.sex X */ X parse(cmd) char *cmd; { X register int l; X int recurseflag; X double days; X char *t, X *p, X *q, X *seps = ",\n "; X X /* skip initial whitespace */ X p = cmd; X while (isspace(*p)) X ++p; X /* ignore blank lines or comments */ X if (*p == '\0' || *p == '#') X return(1); X /* read the expire time from the line as a floating point number, X * then figure out what the timestamp on a file that old would be */ X age = now - (time_t) (atof(p) * SECINDAY); X /* skip ahead to the filespec list */ X while (!isspace(*p)) X ++p; X while (isspace(*p)) X ++p; X /* initialize exclusion list to null list */ X preclude(); X /* for each filespec in list */ X for (t = strtok(p, seps); t; t = strtok(NULL, seps)) { X X /* forbid starting with a slash */ X if (*t == '/') X *t = '.'; X X /* change dots to slashes except in column 1 */ X while ( (q = strchr(t+1,'.')) != NULL) X *q = '/'; X X /* for final dot or slash, remove it, and shut off recursion */ X recurseflag = 1; X if (t[l = (strlen(t) - 1)] == '/') { X recurseflag = 0; X t[l] = '\0'; X } X X if (*t == '!') X exclude (t+1, recurseflag); X else X include (t, recurseflag); X } X X reap(); X X return (0); X } /* parse() */ X X X /* X * newnode,exclude, include, preclude X * X * functions to maintain global linked lists: X * incl include this path in list of dirs to reap X * excl leave out this directory X (each entry can be recursive or not) X * X * the functions are: X * include (text,rflag) add to incl list X * exclude (text,rflag) add to excl list X * preclude() clear both lists X */ struct filspec * newnode(text) char *text; { X struct filspec *f; X X if ( (f = (struct filspec *) malloc (sizeof(struct filspec))) == NULL || X (f->name = malloc (strlen(text)+1)) == NULL) X ouch ("%s: out of memory\n"); X X strcpy (f->name, text); X return (f); } Xexclude(text, rflag) char *text; int rflag; { X struct filspec *f; X X f = newnode(text); X f->recurse = rflag; X f->next = excl; X excl = f; } include(text, rflag) char *text; int rflag; { X struct filspec *f; X X f = newnode(text); X f->recurse = rflag; X f->next = incl; X incl = f; } preclude() { X struct filspec *p; X X while (incl != NULL) { X p = incl; X incl = incl->next; X free (p->name); X free (p); X } X while (excl != NULL) { X p = excl; X excl = excl->next; X free (p->name); X free (p); X } } X X /* X * ouch(fmt, arg) X * outputs an error message and exits with error status X */ X ouch (fmt,arg) char *fmt, *arg; { X fprintf (stderr, fmt, progname, arg); X exit (-1); } SHAR_EOF chmod 0664 lib.c || echo 'restore of lib.c failed' Wc_c="`wc -c < 'lib.c'`" test 3567 -eq "$Wc_c" || echo 'lib.c: original size 3567, current size' "$Wc_c" fi # ============= list.sample ============== if test -f 'list.sample' -a X"$1" != X"-c"; then echo 'x - skipping list.sample (File already exists)' else echo 'x - extracting list.sample (Text)' sed 's/^X//' << 'SHAR_EOF' > 'list.sample' && # yenta's /usr/lib/news/reaplist function script file. .1 junk,alt.flame,control,comp.binaries,alt.sex.pictures.,alt.fractals.pictures .1 alt.desert-storm,alt.desert-shield,soc.motss,soc.culture 1 soc.singles,alt.activism,alt.romance.chat .5 talk 1 sci,!sci.skeptic 2 sci.skeptic .5 comp.text,comp.sys.atari,comp.windows,comp.os.vms .5 comp.unix.sysv386 .5 misc.jobs,misc.handicap 2 soc,!soc.singles,!soc.motss,!soc.culture .5 rec,!rec.arts.anime,!rec.arts.animation,!rec.arts.movies,!rec.games.hack,!rec.radio.amateur,!rec.autos,!rec.arts.tv,!rec.audio,!rec.video,!rec.music,!rec.skydiving 1 rec.music,!rec.music.synth 3 rec.music.synth,rec.skydiving 2 alt.tv 2 rec.arts.anime,rec.arts.animation,rec.arts.movies,rec.radio.amateur,rec.autos,rec.arts.tv,rec.audio,rec.video 3 rec.games.hack 1 misc.headlines,misc.consumers,misc.kids,misc.legal 1 alt.sex,!alt.sex.bondage,!alt.sex.pictures. 1.5 comp,!comp.dcom.telecom,!comp.binaries,!comp.text,!comp.sys.atari,!comp.windows,!comp.os.vms,!comp.sys.att,!comp.sys.3b1,!comp.sources.3b1 3 comp.dcom.telecom 1 comp.binaries 1 alt.sources 1 news,!news.announce.newusers 3 biz 3 misc 3 alt,!alt.sources,!alt.tv,!alt.desert-shield,!alt.activism,!alt.fractals.pictures,!alt.desert-storm,!alt.romance.chat 3 comp.sys.att 5 comp.sys.3b1 5 comp.sources.3b1 7 general # up to here is "normal expire"... now let's get desparate .1 alt.sex.pictures .1 rec.arts.anime,rec.arts.animation,rec.arts.movies,rec.ham-radio,rec.autos,rec.arts.tv,rec.audio,rec.video .1 biz .1 talk .1 news .1 soc .1 sci .1 comp .1 alt .1 misc # if we got here, we're in big trouble -- wipe it all! 0 . SHAR_EOF chmod 0664 list.sample || echo 'restore of list.sample failed' Wc_c="`wc -c < 'list.sample'`" test 1611 -eq "$Wc_c" || echo 'list.sample: original size 1611, current size' "$Wc_c" fi # ============= main.c ============== if test -f 'main.c' -a X"$1" != X"-c"; then echo 'x - skipping main.c (File already exists)' else echo 'x - extracting main.c (Text)' sed 's/^X//' << 'SHAR_EOF' > 'main.c' && X /* X * main.c X * main routine and globals for reap utility, version 1.1 X */ X X #include "reap.h" X X /* linked lists for included and excluded file specs */ struct filspec X *incl = NULL, X *excl = NULL ; char X *scriptfile = SCRIPT, /* path of function script */ X *progname, /* argv[0] for errors */ X *newsdir = NEWSDIR /* news root directory */ ; int X verbose = 0, /* verbosity (-v) flag */ X dryrun = 0 /* do-not-unlink (-n) flag */ ; time_t X age, /* unlink files older than this */ X now /* current time */ ; X X /************************ X * MAIN FUNCTION * X ************************/ X main(argc, argv) int argc; char *argv[]; { X int device, /* device # containing newsdir */ X c, X lineno = 0, X summary = 0, X thenfree, X errflag = 0; X long wantblox; /* desired free space */ X char *p; X static char line[MAXLINE]; X FILE *script; /* file ptr for function script */ X struct stat st; X X /* save argv[0] for error spewings */ X progname = argv[0]; X /* parse args */ X while ( (c=getopt(argc,argv,"vsnf:")) != EOF) X switch(c) { X case 'n': X ++dryrun; X break; X case 'v': X ++verbose; X break; X case 'f': X scriptfile = optarg; X break; X case 's': X ++summary; X break; X default: X ++errflag; X break; X } X X if (argc - optind != 1 || X sscanf(argv[optind], "%ld", &wantblox) != 1 ) X ++errflag; X X if (errflag) { X fprintf (stderr, X "usage: %s [-v] [-s] [-n] [-f funclist] freeblocks\n", X progname); X exit (-1); X } X X /* stat newsdir to find the number of the device it's on */ X if (stat(newsdir, &st)) X ouch ("%s: can't stat %s\n", newsdir); X device = st.st_dev; X /* open function script file for reading */ X if ( (script = fopen (scriptfile, "r")) == NULL) X ouch ("%s: can't read %s\n", scriptfile); X /* Record the current time, for deciding what gets the axe. */ X time (&now); X /* chdir to newsdir */ X chdir (newsdir); X /* record initial freespace if summary is desired */ X if (summary) X thenfree = freeblox(device); X /* main loop ... process until goal reached or end of script */ X X while (freeblox(device) < wantblox && X (p = fgets (line, MAXLINE, script)) != NULL ) X parse (line), ++lineno; X X fclose (script); X X if (summary) { X c = freeblox(device); X printf ("Freespace was %d, now %d. Cleared %d.\n", X thenfree, c, c - thenfree); X printf ("Stopped after line %d in %s\n", lineno, X scriptfile); X } X /* return 0 if freespace goal was met at exit, 1 if not */ X exit (freeblox(device) >= wantblox ? 0 : 1); X } /* main() */ X SHAR_EOF chmod 0664 main.c || echo 'restore of main.c failed' Wc_c="`wc -c < 'main.c'`" test 2497 -eq "$Wc_c" || echo 'main.c: original size 2497, current size' "$Wc_c" fi # ============= reap.8 ============== if test -f 'reap.8' -a X"$1" != X"-c"; then echo 'x - skipping reap.8 (File already exists)' else echo 'x - extracting reap.8 (Text)' sed 's/^X//' << 'SHAR_EOF' > 'reap.8' && .TH REAP 8 LOCAL .SH NAME reap - remove news articles as space needed .SH SYNOPSIS .B reap [-v] [-s] [-n] [-f scriptfile] freeblocks .SH DESCRIPTION .I Reap checks disk freespace and deletes netnews articles according to a flexible user-specified scheme until .I freeblocks minimum freespace is available. It does this by sequentially reading commands (expire functions) from a text file (function script), and executing them one at a time. .PP Each expire function consists of an age limit and a list of newsgroups to expire to that limit. Before executing Xeach new expire function, reap checks the freespace on the news spool device. Reap will exit when either the desired freespace is attained, or the end of the function script file is reached (the latter case is considered an unsuccessful exit). By carefully designing the function script, a news administrator can automate a wide variety of expiring policies. .PP The .B -n causes a dry run (as in make(1)). When this option is specified, reap merely reports which articles it would delete if the option were left off. This is useful for debugging function scripts. However, since no space is actually freed in this mode, it will either do nothing (there was enough space to begin with) or proceed through the entire function script file, indicating that reap would delete all applicable articles. .PP With the .B -v option, reap prints a verbose commentary on its progress to the standard output. .PP The .B -s option causes reap to print a brief summary of blocks freed to the standard output just before exiting. It also indicates how much of the script file was processed. This is independent of -v. .PP By default, reap looks in /usr/lib/news/reaplist for its list of functions. The .B -f option is used to specify an alternate function script file. .SH FUNCTION SCRIPT FILE FORMAT A function script file consists of a series of expire functions, one per line. Each expire function contains an age limit (in days, decimals okay), followed by a comma-separated list of newsgroup specifications, similar to the sys file format. .PP There are four types of valid newsgroup specifications that can go in the list: .PP (1) An ordinary newsgroup name (alt.sex) indicates that any articles in that newsgroup and any of its descendants (such as alt.sex.pictures), which are older than the age limit, should be expired. .PP (2) A newsgroup name with a trailing dot (alt.sex.) indicates that any articles in that newsgroup, which are older than the age limit, should be expired. However, recursion is not implied. Articles in descendants of that group (such as alt.sex.pictures) are not affected. .PP (3) A newsgroup name with a preceding exclamation point (!alt.sex.pictures) indicates that that newsgroup and all of its descendants should be excluded from the list. For example, "1 alt.sex,!alt.sex.pictures" means to expire everything subsumed under alt.sex to 1 day, but do not touch anything subsumed under alt.sex.pictures. .PP (4) A newsgroup name with both a preceding exclamation point and a trailing dot (!alt.sex.pictures.) indicates that that newsgroup should be excluded from the list, but its descendants should remain in the list. For example, "1 alt.sex,!alt.sex.pictures." means to expire everything subsumed under alt.sex to 1 day, including the contents of any subgroups of alt.sex.pictures (like alt.sex.pictures.d), but do not touch the articles in the newsgroup alt.sex.pictures. .PP Blank lines and lines beginning with a "#" are ignored. A sample function script file: .PP .nf X # my first function script file X 0.5 talk,junk,alt.sex.pictures. X 2 rec,!rec.games,!rec.humor X 2 misc .fi .PP In the example, if space is needed, reap Xexpires to .5 days the entire talk and junk hierarchies, and alt.sex.pictures (not touching any subgroups of alt.sex.pictures, such as alt.sex.pictures.d). If still more space is needed, the rec hierarchy (excepting all of rec.games.* and rec.humor.*) is expired to 2 days. If freespace is still short, reap then expires all of misc to 2 days. .PP Note that, while it would be possible to consolidate the 2-day lines, leaving them separate makes it possible for reap to stop in between them if sufficient space is cleared. .PP A practical function script file would directly or indirectly include every group carried on the system, with some age limit. Otherwise, the spool directory will inevitably overflow without operator intervention. .PP It is valid to specify a dot all by itself (.) in the newsgroup list field. This is taken to mean the entire spool directory hierarchy, so "14 ." means to expire all news to 14 days. Be warned that reap must search the entire news hierarchy when this feature is used. Still, it is probably a good idea to end all function script files with a "0 .", so that, in absolute desperation, all news would be removed. .SH FILES .TP 25 /usr/spool/news News spool directory .TP 25 /usr/lib/news/reaplist Default function list file .SH DIAGNOSTICS Returns zero if sufficient space was free at exit, 1 if the entire function script was executed without freeing enough space, and -1 on an error condition. .SH CAVEATS What constitutes a disk "block" is implementation-dependent. .PP History files are not updated. That must be performed as a separate operation. .PP For best performance, exclude recursively (without the dot) whenever possible. Otherwise, reap will have to search the excluded directory for any subdirectories which might not have been excluded, which is slow. .SH BUGS The use of the ustat() system call is not very portable, and your humble author isn't aware of its non-system-V equivalents. .PP Lines in the function file are limited to 1024 characters. .PP The "cleared" figure in the summary merely indicates the difference in freespace before and after the run, not necessarily the actual number of blocks liberated by reap. .SH AUTHOR Reap is freeware by David B. Thomas (dt@yenta.alb.nm.us). You are free to copy, distribute, staple, bend, fold or mutilate this package to your heart's content. Please email any enhancements or ideas for enhancements. SHAR_EOF chmod 0644 reap.8 || echo 'restore of reap.8 failed' Wc_c="`wc -c < 'reap.8'`" test 6129 -eq "$Wc_c" || echo 'reap.8: original size 6129, current size' "$Wc_c" fi # ============= reap.c ============== if test -f 'reap.c' -a X"$1" != X"-c"; then echo 'x - skipping reap.c (File already exists)' else echo 'x - extracting reap.c (Text)' sed 's/^X//' << 'SHAR_EOF' > 'reap.c' && X /* X * reap.c X * contains the reap() function. X * X * Once the global linked lists incl and excl have been stuffed, X * reap() actually scans the filesystem for files that meet the specs X * and unlinks them if they are older than the age global variable. X */ X #include "reap.h" X X reap() { X struct filspec *f; X X for (f = incl; f != NULL; f = f->next) { X if (verbose) X printf ("scanning %s ...\n", f->name); X dodir (f->name, f->recurse); X } X } /* reap() */ X X X dodir(name, rflag) char *name; int rflag; { X /* the following can be overwritten safely during recursion */ X static struct filspec *e; X static struct stat st; X static struct dirent *dp; X static char thisname[MAXFILENAME+1]; X /* the following must be preserved through recursion */ X char *fullpath; X int eflag = 0; X DIR *dirp; X X /* open directory for reading */ X if ( (dirp = opendir(name)) == NULL) X ouch ("%s: can't read directory %s\n", name); X X /* see if this directory is excluded. X * If it's excluded recursively, quit here. X * If it's excluded non-recursively, set a flag, so we won't consider X * deleting any files in it, but we'll still explore subdirectories. X */ X for (e = excl; e != NULL; e = e->next) X if (!strcmp (name, e->name)) X break; X if (e != NULL) { X if (e->recurse) X return; X else X ++eflag; X } X X /* loop for each directory entry */ X while ( (dp = readdir(dirp)) != NULL) { X X /* name might be exactly MAXFILENAME characters long, and thus X * might not be null-terminated. Some insurance: X */ X strncpy (thisname, dp->d_name, MAXFILENAME); X thisname[MAXFILENAME] = '\0'; X X /* skip dot and dotdot */ X if (!strcmp(thisname, ".") || !strcmp(thisname, "..")) X continue; X X /* build the full pathname of current object */ X if ( (fullpath = X malloc(strlen(name)+strlen(thisname)+2)) == NULL) X ouch ("%s: out of memory\n"); X X sprintf (fullpath, "%s/%s", name, thisname); X X /* try to stat the object */ X if (stat(fullpath,&st)) { X fprintf (stderr, "%s: can't stat %s\n", X progname, fullpath); X free (fullpath); X continue; X } X X /* maybe recurse if it's a directory */ X if ( st.st_mode & S_IFDIR ) { X if (rflag) X dodir (fullpath, 1); X free (fullpath); X continue; X } X X /* it's a file ... is this a non-recursively excluded directory? X * if so, there's nothing to do to this file X */ X if (eflag) X continue; X X /* leave it alone if this directory is excluded, or X * if it's new enough. X */ X if (eflag || st.st_mtime > age) { X free (fullpath); X continue; X } X X /* reap this file! */ X if (dryrun) { X printf ("Would unlink %s\n", fullpath); X free (fullpath); X continue; X } X if (verbose) X printf ("Unlinking %s\n", fullpath); X X if (unlink (fullpath) == -1) X fprintf (stderr, X "%s: cannot unlink %s\n", progname, fullpath); X X free (fullpath); X X } /* while */ X X closedir (dirp); X } /* dodir() */ SHAR_EOF chmod 0664 reap.c || echo 'restore of reap.c failed' Wc_c="`wc -c < 'reap.c'`" test 2845 -eq "$Wc_c" || echo 'reap.c: original size 2845, current size' "$Wc_c" fi # ============= reap.h ============== if test -f 'reap.h' -a X"$1" != X"-c"; then echo 'x - skipping reap.h (File already exists)' else echo 'x - extracting reap.h (Text)' sed 's/^X//' << 'SHAR_EOF' > 'reap.h' && X /* X * reap.h X * header file for reap utility X */ X /* set MAXFILENAME to the maximum number of characters in a filename for X * your system. Typically 14 or infinity, where infinity equals X * 256 characters. :-) X */ #define MAXFILENAME 14 X /* set NEWSDIR to the directory containing news on your system. X * Very commonly /usr/spool/news X */ #define NEWSDIR "/usr/spool/news" X /* set SCRIPT to the path of the default function script file X * Usually this is /usr/lib/news/reaplist X */ #define SCRIPT "/usr/lib/news/reaplist" X X X X #include <stdio.h> #include <string.h> #include <ctype.h> #include <time.h> #include <malloc.h> #include <sys/types.h> #include <sys/stat.h> #include <ustat.h> #include <dirent.h> X #define MAXLINE 1024 /* max len of line in script */ #define SECINDAY (3600 * 24) /* seconds in a day */ X /* structure for linked lists of included and excluded file specs */ struct filspec { X char *name; X int recurse; X struct filspec *next; }; X Xextern struct filspec X *incl, X *excl ; Xextern char X *progname, X *scriptfile, X *newsdir ; Xextern int X verbose, X dryrun, X optind ; Xextern char *optarg; Xextern double atof(); Xextern long freeblox(); Xextern time_t X age, X now ; SHAR_EOF chmod 0664 reap.h || echo 'restore of reap.h failed' Wc_c="`wc -c < 'reap.h'`" test 1195 -eq "$Wc_c" || echo 'reap.h: original size 1195, current size' "$Wc_c" fi exit 0 -- Bottom of stack = 0x40000 Stack pointer = 0x3fffe Don't push it!