OS/2 Shareware BBS: 35 Internet

home *** CD-ROM | disk | FTP | other *** search

/ OS/2 Shareware BBS: 35 Internet / 35-Internet.zip / dnew54c3.zip / MANUAL.TXT < prev next >

Wrap

Text File | 1996-07-27 | 77KB | 2,037 lines

This file has been extracted from the word 6 document manual.doc, because of this it's format is very bad, I suggest you print the word6 document if at all possible. DNEWS Management DNEWS: A fast, efficient, low management NNTP news server for Windows NT, VMS and UNIX systems. Version 2 NetWin World, P.O Box 27574, Mt Roskill, Auckland, New Zealand, Email netwin@world.std.com DNEWS Management DNEWS Features 5 Dynamic sucking feed. 5 Flexible expire settings 5 Flexible access restrictions 5 Binaries available for common platforms 5 Email confirmation of posts 5 Items stored in a database 5 Compiling and porting relatively easy 5 Installation easy, totally configurable without recompiling 5 Documented 5 Supported 5 Speed 6 XOVER implemented in software 6 Streaming support 6 Machine impact automatically limited 6 DNEWS Installation 7 Main installation steps for Windows NT 7 Main installation steps for VAX 8 Main installation steps for UNIX 8 TELLNEWS commands 9 Frequently Asked Questions (FAQ's) 11 Where is dnews.conf? 11 Where are the log files? 11 Creating local news groups 11 My news reader says the newsgroups don't exist? 12 I don't know if posts are working? 12 I'm still not convinced my posts are getting to the other side of the planet? 12 I have an old fashioned feed coming in but I'm not getting any items? 12 I deleted some item files by hand, is that ok? 12 What should I do now that I've deleted history.dat 13 Netscape shows items but I can't read them, why? 13 Can I restore some news files (*.itm, *.idx) from tape?1 3 Why doesn't the status report match actual diskspace used? 13 How do I setup DNEWS to take in a `normal' NNTP - IHAVE feed 13 How do I setup XMIT to send to other sites 14 Adding XMIT to Windows NT Services (startup) 14 Adding XMIT to UNIX startup 14 Adding XMIT to VMS startup 14 How do I move the spool area to a bigger disk 14 How do I make the updates run only at night 15 How do I test the server using TELNET 15 What files can I copy from an INN or CNEWS system 15 How do I rename a volume on UNIX 15 DNEWS goes very slow when users connect 16 My system runs out of memory? 16 DNEWS goes very slow 16 DNEWS locks up for brief periods (or permanently) 16 DNEWS Crashes 17 Large messages & Memory Use. 17 How can I cut out spamming (cross posts) 17 How can I make XOVER faster? 17 TELLNEWS says Can't BIND? 18 TELLNEWS fails or locks ups 18 TELLNEWS says remote node is unknown 18 TELLNEWS says "Cannot access tellnews.pass" 18 I see errors "NNTP, Sorry I can't talk to you" in the log file? 18 What does "Transfer permission denied" mean? 19 I still can't figure it out, how can I get help? 19 DNEWS Mailing List 19 To subscribe to the discussion list send to: 19 To subscribe to the announcement list 20 Configuration files 21 dnews.conf 21 access.conf 22 Notes: 23 expire.conf 23 Example: 23 newsfeeds.conf 24 moderators.conf 24 control.conf 24 Example: 25 How news gets in and out of your system 26 FEEDING news INTO your system 26 Feeding in using an NNTP IHAVE feed. 26 Feeding in using a SUCKING feed (default) 26 FEEDING news OUT OF your system 26 Sending news up line using a POSTING feed (default) 26 Sending news up line using an NNTP-IHAVE sender 27 Appendix A, What is USENET 27 What is USENET News? 27 News groups 27 How your article will get to the rest of the world 27 What to read first 27 What is an FAQ? 27 Be careful posting 27 How big is it? 28 Appendix B, Time Zone Names 28 Appendix C, NNTP Summary 29 authinfo [user Name]|[pass Password] 29 article [MessageID|Number] 29 body [MessageID|Number] 29 date 29 group newsgroup 29 head [MessageID|Number] 29 help 29 ihave <message-id> 29 last 29 list [active] 29 list active.times 29 list newsgroups 29 listgroup [newsgroup] (extension) 30 mode reader (extension) 30 mode stream (extension) 30 newgroups yymmdd hhmmss [GMT] [<distributions>] 30 newnews newsgroups yymmdd hhmmss [GMT] [<distributions>] 30 next 30 post 30 slave 30 stat [MessageID|Number] 30 check [MessageID] (extension) 30 takethis [MessageID] (extension) 30 xgtitle [group_pattern] (extension) 30 xhdr header [range|MessageID] (extension) 30 xover [range] (extension) 31 xpat header range|MessageID pat [morepat...] (extension) 31 xpath MessageID (extension) 31 Copyright & Disclaimer 31 Corrections & Additions. 31 DNEWS Features Dynamic sucking feed. A sucking feed may use as little as 10% of the normal disk space and network bandwidth. By only getting news items that people usually read very large savings are possible particularly for small sites & workgroups. A hundred users can be supported on 100-200 MB depending on their reading habits! I run a single user server on my NT as a background service, it's using less than 5MB. Flexible expire settings By allowing expiration by 'number of items' low volume groups become much more useful. Also by allowing settings which change with disk space the system is much more stable to large influxes of news. Flexible access restrictions Reading and Posting access can be given separately to different users and selections of news groups. Binaries available for common platforms On many platforms you do not need to build the binary yourself, e.g. VMS, (VAX & ALPHA), Solaris, Linux, Windows NT Email confirmation of posts DNEWS can be configured to send email confirmations as it sends items up to a main news server. This gives users a clear record of a posted item and it's successful transfer into the 'net'. Items stored in a database Items are stored in a database which puts items together in blocks of 100. This saves directory entries, inodes, and the empty space usually wasted due to blocking factors. Also file open and closes are minimized, this is essential on systems where this is a 'costly' operation, e.g. VMS. Compiling and porting relatively easy DNEWS was designed to be portable from the ground up, it does not rely on obscure UNIX features. Installation easy, totally configurable without recompiling DNEWS can be configured without recompiling and all the configuration scripts have been simplified where possible. Documented A user manual is available. Supported EMAIL support is available for help with installation and management. Speed On comparable systems DNEWS typically out performs other news software, (even the likes of CNEWS and INN in some instances), other VMS news servers don't even come close. This is partly because DNEWS uses a more advanced database to store the news items. The difference is also very notable on smaller systems (e.g. Linux) XOVER implemented in software No xover database is created, the normal DNEWS database is fast enough to create this information 'on the fly' Streaming support Streaming NNTP extensions are built in. Machine impact automatically limited DNEWS runs in a single process, this speeds up connect time as a process does not need to be created, and it self-limits the impact on the system. DNEWS Installation Always refer to install.txt for the most up to date installation notes and helpful hints etc. When upgrading use newver.cmd to copy the new binaries into place instead of install.cmd which would over-write your .conf files. Main installation steps for Windows NT · FTP the files FTP ftp.std.com ftp> cd /ftp/vendors/emagic/dnews ftp> binary ftp> get dnews_nt.zip ftp> quit · Extract the files to a temporary directory and then install them mkdir \tmp cd \tmp pkunzip ..\dnews_nt.zip install.cmd del \tmp\*.* · EDIT dnews.conf and carefully define the following symbols · NOTEPAD winnt35\system32\dnews.conf Symbol Example Description manager joeblogs@here. Your Email address this.place nntp_fe news.feeder.si The news site where you are going eder te to SUCK a feed from. You need NNTP READER access to this site. myname news.this.plac Your full internet name. e diskspa 100 Total disk space in megabytes that ce DNEWS can use confirm *.this.place Send email confirmation to postings from these addresses. (set this to match your local email addresses) · You may want to also redefine SPOOL in DNEWS.CONF if you want the news spooling area to be on another disk. · If your news feed stamps items with a name other than exactly what you put in DNEWS.CONF under NNTP_FEEDER, then you must add an exclude field to your newsfeeds.conf file to stop DNEWS from trying to echo every item it gets. See the section Adding an Exclude Field for more details on doing this. · Edit newsfeeds.conf and add an SMTP queue so DNEWS can send email when it wants to: NOTEPAD \dnews\newsfeeds.conf SITE mail.gateway.here.com SMTP · Edit access.conf to grant IP access to any machine which will read news from your system, you can give a wildcard ip name or number, names will only work if your DNS is configured correctly. NOTEPAD \dnews\access.conf *.here.com:Read,Post:::* 161.23.1.*:Read,Post:::* · Start the system. In Control Panel, double click on SERVICES and then click on DNEWS and click on START. · Try out the management command: TELLNEWS STATUS · Try a news reader · If it doesn't work check the log files and then check the section in this manual on Common Problems: TYPE \dnews\dnews.log Main installation steps for VAX 1. FTP ftp.std.com /ftp/vendors/emagic/dnews... 2. @extract Extracts the required files 3. $ edit register_user.com Change uic and disk to match your machine 4. $ @register_user Register user dnews$server on your machine. 5. $ edit dnews_start.com Define the disk(s) where DNEWS is installed 6. $ edit sys$manager:systartup_v5.com Add the startup line (see startup.tmp) 7. $ edit sys$sylogin: Add the symbol (see login.tmp) 8. $ edit dnews_top:news.conf Modify the file DNEWS.CONF to describe your system by defining the following symbols MANAGER, NNTP_FEEDER.... TIMEZONE, DISKSPACE, MYNAME 9. If your news feed stamps the path header with a name other than it's ip name as in `nntp_feeder' then edit newsfeeds.conf and add an exclude command to prevent your news system from `echoing' all messages back to your feeder. 10. $ edit access.conf Grant an ip range access to the system, e.g. *.here.comp.edu:read,post,:::* 161.23.4.*:read,post,:::* 11. $ @dnews_top:dnews_start Start the system 12. $ type dnews.info Type the log file, give it one or two minutes before panicking. 13. $ tellnews status Get a status report 14. Go to your favorite PC news reader software and try it out!!! Main installation steps for UNIX · FTP ftp.std.com /ftp/vendors/emagic/dnews/dnews_linux.tar.Z · Extract the files to a temporary directory # uncompress dnews_linux.tar.Z # tar -xvf dnews_linux.tar # cd dnews · EDIT dnews.conf and carefully define the following symbols Symbol Example Description manager system@here.th Email address of the news manager is.place nntp_fe news.feeder.si The news site where you are going eder te to SUCK a feed from. You need NNTP READER access to this site. myname news.this.plac Your full internet name, if left e out this may be picked up from the system. diskspa 100 Total disk space in megabytes ce confirm *.this.place Send email confirmation to postings from these addresses. (set this to match your local email addresses) df df -k Path and parameters for df command (Display Filesystem) sendmai /usr/lib/sendm Path and parameters for sendmail l ail -t %s · EDIT your startup script and add the following line, startup scripts are different for every version of UNIX: # emacs /etc/rc.d/rc.local Add a line: /usr/local/dnews/dnews_start.sh · If your news feed stamps the path header with a name other than it's ip name as in `nntp_feeder' then edit newsfeeds.conf and add an exclude command to prevent your news system from `echoing' all messages back to your feeder. · EDIT access.conf and grant ip access to ip numbers or names in your domain, names will only work if your reverse name translation is setup correctly. *.this.site.edu:read,post:::* 161.24.3.*:read,post:::* · If you have changed any of the default directories, then you must edit all the shell scripts, (*.sh) to reflect the change. · Run the installation script: # ./install.sh · Start the news server # ./dnews_start.sh · The system will now try and fetch a list of newsgroups from your feeder site, this may take 10 minutes the first time and the system will not respond to the tellnews command while this is happening. Use `ps - alx | grep news' to check it is running, and look at the log files: # more /usr/local/dnews/dnews.log # more /usr/local/dnews/nohup.out # ps -ef | grep news · When the system has had a chance to get going, try it out using the tellnews command # tellnews status · Go find a PC and try out the system! TELLNEWS commands The tellnews command is used to communicate with the server, you can find out what it's doing, modify some internal settings, reload the configuration files or shut down the server cleanly. To use the command you need read access to the DNEWS configuration directory. Try this command to find out what your server is doing: tellnews status Or this command to shut down your server tellnews exit Here is a table of all the other TELLNEWS commands: Command Description addlost Scans all newsgroups for articles that are not in the history file, and adds them to the history file. Use this if you have deleted your history file, or if you have restored a group from tape. This command will take many hours to run on several Gig's of news. cached wildgroup Force a group or groups to be cached for days the next so many days, if you give a wild card on UNIX you must put quotes around it to stop the shell from interpreting it. Example: tellnews cached comp.os.* 999 expire Start an expire run, use this if the system is getting short on space and the last expire didn't work for some reason: (response will be slow while this is running) exit Close down DNEWS fixgroups Fetch the complete list of groups from the feeder site again, if you have less than 300 cached groups then you will need to do this once every month or so to pickup new groups. fixrange Corrects the internal count of how many items are in each news group, if WinVN crashes when you read a group then this is a good thing to try. This command is instant and harmless so it's always worth a try. getgroups Get or update the active.dat file from the feed site. If you have more than 300 groups cached this is automatic, otherwise you will need to do it once a month getnames Get the active.names from our feeder. getold wildgroup Get older news from the feeder, use this days after deleting the entire system except the active.dat file, to reload some old news from the server. killitem grpname Delete a specific item, use with caution num killgroup Actually delete group wild.group.name newgroup name Add a new group by hand. y=Normal group, y|m creator m=Moderated the description is added to description the newsgroups file. purgelost Scans every newsgroup for items which are not in the history file and deletes them. This will take several hours to run on a large system. reload Reloads dnews.conf and other configuration information. You must use this whenever you change a configuration file or the changes will probably not take effect. rotate Renames all the log files and deletes the oldest one. dnews.log > dnews.log2 > dnews.log3 > dnews.log4 > deleted rmgroup Mark group as deleted wild.group.name setused Sets the amount of space currently used in MB. The change will take 2 minutes to take effect. This is needed if dnews has lost track of how much space it is actually using for some reason. tellnews setused 950 status Show what the server is doing stats Show statistics on NNTP command usage. update Start a suck update. These are normally automatic every 10 minutes but you can disable the automatic updates using the UPDATE_AT settings in DNEWS.CONF uncached Set a group or groups to uncached. Items wildgroup in the group will then expire normally, this does not prevent a user from fetching the group again. tellnews uncached comp.os.* Frequently Asked Questions (FAQ's) Where is dnews.conf? This is the main configuration file, and you can't move it from it's built in directory which is: Windows NT /winnt35/system32/dnews.conf (system dir) Unix /etc/dnews.conf All other configuration files will be in the config directory defined in dnews.conf Where are the log files? Look in dnews.conf for the symbol LOG_DIR, this will tell you which directory to find them in, there are three files: dnews.log The main log file dnews.in A one line log of each message that came in dnews.out A one line log of each item dnews sent out. The file dnews.log contains .in and .out messages as well. By default the logging level is set to INFO, you can set it in dnews.conf to one of, error,warn,info,debug. E.g.: loglvl warn In debug mode all error messages are sent to the log file, in info mode only error,warn and info messages are printed. In warn mode only error and warn messages are sent, and in error mode only serious errors are printed to the log file. Every night, or if the log files get more than DEBUG_ROTATE lines in them the files are rotated like this: dnews.log -> dnews.log2 ->dnews.log3 ->dnews.log4 - >(deleted) Creating local news groups If you wish to have local groups on your server, that are not on the server you 'suck' from, then you should remove the 'POSTING' flag from the feed to your feeder site (in newsfeeds.conf). But only if you can get ihave (or transfer permission) to send articles up stream rather than simple 'posting' access. You can still have local groups if you can't get ihave access, it's just not the `recommended' setup. You can add groups with the TELLNEWS NEWGROUP command, e.g. tellnews newgroup here.chat y myname Local group here.chat = name of group y = Normal group (m = moderated) myname = A record of who created the group (one word) Local group = description for active.names file. To stop your local group from being sent up stream, you would change the line in your newsfeeds.conf file: groups *,!here.chat,!here.secret.* NOTE: KNOWN BUG, with version 2.0 items will still be sent upstream if you have left the 'POSTING' keyword in your feed. This will probably be fixed in a future version. (ofcourse this isn't a major problem unless the items are very secret as your upstream site will probably just reject them anyway) My news reader says the newsgroups don't exist? Check access.conf, you might have access to get in, but no access to read the news groups so they will appear empty or give `no such newsgroup' errors. I don't know if posts are working? There are two main reasons for posts to fail: 1. You've messed up your newsfeeds.conf file. 2. Your provider is not allowing you to send messages via `ihave' Follow these instructions to figure out where the error is: Post an item to some local TEST news group, e.g. nz.test, then check the file DNEWS.OUT, this should give one line per message sent `out' of your system, if this file is empty, or gives an error, first consider if the error is from DNEWS or if it is from the site you are sending items to. Second, look in dnews.log, search for "out:" once you've found the failure message you will probably find more clues just above or below it as to what went wrong. I'm still not convinced my posts are getting to the other side of the planet? Post an item to alt.test, you should receive 10-15 messages from news robots all over the world. I have an old fashioned feed coming in but I'm not getting any items? There are two main reasons for incoming feeds to fail: 1. You've messed up your access.conf file (or your DNS does not work) 2. Your provider has not successfully setup the feed. In case 1), first try letting everyone in by putting a wildcard access line at the bottom of access.conf *:read,post,ihave:::* tellnews reload Then check the log files, or use tellnews status, and you will see the name and number of any machine sending you IHAVE messages, (in the list of connected channels) put this name in access.conf. If tellnews status shows the number twice, but no name, then your DNS is not working, use numbers in the access.conf file instead or fix your DNS. I deleted some item files by hand, is that ok? This is like editing a database with a binary editor and removing a few records by hand, it is not a good idea :-( However, DNEWS is written for people just like you, the database will correct itself with a little luck. :-) Please do not edit an item file and remove individual items or change them, this will corrupt the database and the items will simply dissappear. What should I do now that I've deleted history.dat First I advise you not to play with guns because you are in danger of shooting yourself in the foot. Before restarting dnews, delete history.idx and history.dat, and then start dnews and type: tellnews addlost Which will scan the entire database and rebuild the history file. This could take 12 hours or more to run on 4 Gig of news. Netscape shows items but I can't read them, why? This means the item is missing from the history database, this should never happen, but since it has, try this command: tellnews addlost Which will scan the entire database and rebuild the history file. This could take 12 hours or more to run on 4 Gig of news. Can I restore some news files (*.itm, *.idx) from tape? You have restored part of a database, to merge this new stuff into the indexes use: tellnews addlost Which will scan the entire database and rebuild the history file. This could take 12 hours or more to run on 4 Gig of news. Why doesn't the status report match actual diskspace used? DNEWS maintains an internal count of free disk space, this can be wrong, or possibly your setting for `diskspace' in dnews.conf could be wrong. Decide which is wrong, either fix the number in dnews.conf or use the `setused' command to change the internal count, e.g. to set the used value to 950MB, type in: tellnews setused 950 Then wait 2 minutes and do another: tellnews status How do I setup DNEWS to take in a `normal' NNTP - IHAVE feed In access.conf, give your feed site access to send you ihave messages: my.feeder.site.com:ihave:::* In dnews.conf, comment out the nntp_feeder definition. In newsfeeds.conf, replace: site $NNTP_FEEDER groups * builtin posting With: site my.feeder.site groups * builtin And in dnews.conf add a line: cachedall true (Also see the section on copying files from an existing INN or CNEWS system) How do I setup XMIT to send to other sites If you have more than one large feed coming into your site, then you should use XMIT to send to all sites and NOT use the builtin feeding system at all, (see the keyword builtin above, remove it!!) The builtin feeder will not run fast enough to `pass through' large feeds so you must use the XMIT feeder in all directions. (Leaf sites can still use a sucking feed quite safely as this is also very efficient) The XMIT feeder is a high speed, auto configuring streaming feeder with built in scheduling, so you don't need to use CRON or anything else tricky like that. First add entries to your NEWSFEEDS.CONF file, this creates the .feed file that XMIT will use: NOTEPAD newsfeeds.conf site other.site.here.com groups * Now modify your startup procedure to run the xmit job for each feed you have defined in newsfeeds.conf Adding XMIT to Windows NT Services (startup) On windows NT you need to setup each xmit job as a service, to do this use the program ADDSVC.EXE which came with the dnews distribution to add a service for each outgoing feed, e.g. (notepad setupsvc.cmd and then run it to create the feeds) addsvc -add dxmit1 xmit.exe "dxmit1 site.name.edu - at *5 *" addsvc -add dxmit2 xmit.exe "dxmit2 site2.name.edu -at *5 *" Then from CONTROL PANEL, SERVICES, and start the feed(s). Adding XMIT to UNIX startup Add lines to your dnews_start.sh procedure, e.g. cd /usr/local/dnews ./xmit send.to.site.name -at "*2" "*" ./xmit send.other.site.name -at "*7" "*" Adding XMIT to VMS startup Modify RUNXMIT.COM and add it to dnews_start.com How do I move the spool area to a bigger disk First shutdown DNEWS: tellnews exit Then move the entire SPOOL tree structure from it's current place to it's new place, for example on NT you might use file manager to move c:\dnews\spool... to e:\dnews\spool... Change the definition of SPOOL in DNEWS.CONF NOTEPAD \winnt35\system32\dnews.conf spool e:\dnews\spool Restart DNEWS: (NT) Control Panel, Services, Start (UNIX) su root; dnews_start.sh How do I make the updates run only at night Updates for a sucking feed normal run once every ten minutes, this is controlled in DNEWS.CONF with the command update_at *5 * This "*5" tells dnews to do an update whenever the minutes end in a digit `5' and the second "*" tells it to do updates during any hour. To do one update a night at 5 past 2 in the morning you would use: update_at 5 2 To do updates every 10 minutes from 6p.m. to 8a.m. you would use: update_at *5 18-8 How do I test the server using TELNET NNTP is a text based protocol, this means you can test it, and even read news without any software, just by using telnet and typing in commands by hand, this is not a seriously good way to read news, but it is a very good way to test your own server or your feeding server: NT Accessories, Telnet, Connect, Remote Service Enter the feeder.site and under PORT type in NNTP Under Terminal Preferences turn ECHO on Press Control J to end lines instead of ENTER VMS $ telnet feeder.site /port=119 UNIX > telnet feeder.site 119 Type in this sequence of commands to read an article by hand: group comp.os.vms (select a group) head (display the header) body (display the body) next (next item please) help (show valid commands) quit (exit the connection) What files can I copy from an INN or CNEWS system This is not necessary, but if you are replacing a system then there are some files that you may want to preserve so that news.rc files are not `too' confused by the new system. You can safely copy these files: active -> active.dat active.times -> active.times newsgroups -> active.names (may be changed in future) If you copy the active file, you must reset DNEWS with this special manager command: tellnews resetall How do I rename a volume on UNIX You may wish to rename a current volume /var/spool/news to be /var/spool/dnews, to do this: cd /var/spool/news mkdir OLD mv * OLD umount /var/spool/news emacs /etc/vfstab (or /etc/fstab) (change /news into /dnews) mount /var/spool/dnews When you are happy with the new system, or you need the space: rm -r /var/spool/dnews/OLD DNEWS goes very slow when users connect Check your DNS is working, try putting nodns true In dnews.conf, and fix access.conf to use numbers then type: tellnews reload Then see if that speeds things up, if it does then your DNS is faulty. My system runs out of memory? First check your pagefile settings, for a full feed you will need about 120MB of pagefile on Windows NT. See the section on large news messages. Send the output of tellnews status to the mailing list or NetWin and ask for comments. DNEWS goes very slow Check your DNS is working. Type in: tellnews stats (not status) This gives a listing of NNTP commands and how long dnews is spending on each one, this will show if one command is slow, and also if one command is being used more than is reasonable. Also consider adding memory. Send the output of tellnews stats and status to the mailing list or NetWin for further suggestions. Consider installing more memory or a faster disk drive. If the xover command is using most of the time then consider increasing the buffering of headers. (see dnews.conf options) DNEWS locks up for brief periods (or permanently) Panic! Check the log file dnews.log, work out from the time stamps what dnews was doing when it locked up. This can happen if you run an expire while lots of people are trying to use the system or while an ihave feed is sending stuff in. Use the tellnews stats command, this will also point to commands which are taking too long to process for some reason. Check your DNS and also that site you are sucking from is working and it's news server is working, either of these can cause a problem. If the site you suck from is down for a few hours, you may have to remove the feeder line from dnews.conf until it is up again. (normally you won't have to do this, it's just a thing to watch for) DNEWS Crashes Check the end of the file dnews.log, the only partly acceptable reason for a crash is if DNEWS runs out of memory, if the errors indicate this then see the section on memory problems above. If it gives an error `signal xxx causing exit' then set the debugging level to `debug', if the log files just ends suddenly, then set the debug level to `flush' loglvl debug (or flush) debug_rotate 3000 Now next time it crashes email the last few hundred lines of dnews.log to NetWin. We really don't accept crashes as acceptable behaviour and will go to some extreme lengths to fix this sort of problem. Large messages & Memory Use. Unless you have infinite memory on your system then it is necessary to limit the size of messages that can be processed. This also saves a lot of wasted bandwidth and disk space for you and every other news system. DNEWS limits messages to 1MB by default, (a fairly generous limit considering many news readers still die horribly with messages greater than 64K and anything bigger than this should not be sent via news but should be put on an FTP/WEB site and a pointer to it should be sent) To deal with messages of size 1MB dnews needs about 6MB of memory, so if you are short of memory you may want to reduce this. To limit messages to 100k you would add a line in DNEWS.CONF ITEM_MAX 100000 (this setting is not dynamic, you must restart DNEWS) How can I cut out spamming (cross posts) Spamming, is where people cross post to hundreds of news groups with information which is not relevant to the content of the news group. e.g. advertising pyramid schemes etc... Cross posting is a neat idea but enormously abused mechanism, you can limit the abuse by rejecting items that are crossposted to more than `n' groups, a value of 5-10 would seem reasonable to me. The default is 100 for DNEWS. You can chage the limit in newsfeeds.conf under the ME settings How can I make XOVER faster? By default DNEWS caches the last 200 item headers that it has read, if you have spare memory then it is well worth increasing this value: The upper limit is 9000, at approx 3K per header this would use about 27MB, but a value of 2000 would use 6MB which most systems could easily deal with. In DNEWS.CONF HEAD_CACHE 2000 TELLNEWS says Can't BIND? Can't BIND, and tellnews says "I can't talk to you" when you telnet to the port. On LINUX the slackware distribution comes with a dummy nntp daemon installed, you must remove this so dnews can bind to the socket. edit /etc/inetd.conf (comment out the NNTP line) kill -HUP <pid of inetd process> TELLNEWS fails or locks ups The DNEWS process may take 1-2 minutes to initialize, so wait a bit before panicking. Check if the process is running ps -axl | grep dnews (linux) ps -ef | grep dnews (SunOS) Check the log files more /usr/local/dnews/dnews.log On UNIX if it gets started up with the wrong PID you will get all sorts of protection problems, all the files in: /var/spool/dnews/... (including var/spool/dnews) must be owned by the same user, normally 'news'. Also all the files in: /var/local/dnews/* Also on UNIX it is possible to start dnews twice, this can cause real disasters, always check with ps and by looking at the log before restarting DNEWS. TELLNEWS says remote node is unknown Check the myname variable in DNEWS.CONF, also check your DNS is working, if its isn't then use the number in dnews.conf On VMS, if the symbol tellnews is not defined, it will try and run telnet instead, this can be confusing. You may have to log in again to get the symbol, or if you forgot to put it in your sylogin: script then you will have to do so, e.g.: $ tellnews :== $dnews_top:tellnews TELLNEWS says "Cannot access tellnews.pass" For some reason tellnews cannot find it's configuration files, on NT this could be caused by a file protection problem or more likely if the disk drive letter was not specified in the configuration file DNEWS.CONF. On VMS and UNIX you must be running the tellnews command from the usercode ROOT or NEWS or NEWS$SERVER. I see errors "NNTP, Sorry I can't talk to you" in the log file? Your news-host does not want to talk to you, this is because there is a security file which gives a list of hosts which are allowed to connect, the name of your machine must be in this list, and your machine's name must be in the Domain Name Server tables as well as the Reverse Translation Domain Name server tables. The access file may be called something like: CNEWS /usr/local/lib/news/nntp_access INND /usr/local/news/nnrp.access DNEWS /usr/local/dnews/access.conf DNEWS-NT /dnews/access.conf What does "Transfer permission denied" mean? This can come from one of two directions, DNEWS can give this error when a site tries to feed it new items. If you are using a sucking feed then you should stop the site that is trying to send to you from sending as DNEWS will ask for any items it wants. If you do want to allow incoming IHAVE messages, then add an IHAVE record for this ip address in the ACCESS.CONF file, and also remove the 'POSTING' flag from the NEWSFEEDS.CONF file for the ME site. The last possibility is that this error is coming from your feeding site, in which case you need to contact the system manager of that site and ask him to give you `ihave' access. I still can't figure it out, how can I get help? You have two options, the DNEWS discussion list has lots of expertise on it. So try that first. If that fails, or if you are a registered (not free) customer then as part of normal support NetWin provide email advice and help on setting up your system. In general try and give as much information as possible about your setup and what you want to do, and ALWAYS include a FAX or phone number just incase your mail system is broken as well :-) e.g. Hardware & OS: Windows NT - Intel (or NT-Alpha or Solaris 4.3 etc...) Version of DNEWS: 2.0b Type of feed: Sucking feed (or full ihave feed...) FAX: (include country code) And any relevant setup files, usually one or more of: dnews.conf, newsfeeds.conf, access.conf And maybe a few lines from dnews.log showing the problem (Please don't email 3MB of log files to us :-) And state your problem clearly: (a bad example) My feed doesn't work, why not? (a good example) The IHAVE feed coming INTO my system from my provider is being refused, he says it gives an error, `transfer permission denied', what should I do? DNEWS Mailing List There are two mailing lists for DNEWS, one is only used to make major announcements of new version or urgent patches. The other is for general discussion, you may want to join both. To subscribe to the discussion list send to: dnews-request@adam.com.au A message containing one word: subscribe Then to post to the list send to: dnews@adam.com.au To subscribe to the announcement list Send a message to: maiser@emagic.co.nz Containing the line: subscribe dnews This is a moderated list, it's only function is to provide timely information on new features, releases, bugs, installation gotcha's etc to interested people. You will not get 20 messages a day from this list, it will probably average 1-2 messages per month. Configuration files The following files define your news system in approx. order of importance: File Name Description news.conf Main configuration file access.conf Controls nntp access to your server expire.conf Specifies how long to keep news items newsfeeds.con Defines feeds out of your news system f up the line of news servers moderators.co Defines where to send email to a nf moderated news group control.conf Defines how to process control messages for auto creation and deletion of news groups dnews.conf cachedonly true (don't allow items into non cached groups) cachedall true (pretend all groups are cached) Symbol Example Description active_st true Stops new groups from being added atic by the sucking feed. Use this if your feed site has lots of groups you never want added into your active file. cachedall true Pretend all groups are cached, use this if you are taking a full IHAVE feed in. cachedonl true Only accepts items into groups y which are cached, this is used for the hybrid feed where items are sent by a feed but only taken if users are reading that particular group. confirm *.this.place Send email confirmation to postings from these addresses. (set this to match your local email addresses) config dnews_top: Directory for configuration files (*.conf) debug_rot 10000 Automatically rotates log files ate (renames and deletes the oldest one) if more than 10000 lines are written. This is good if you have turned on debugging as you always have the last few thousands lines if a problem develops but you don't run out of disk space due to the huge log file. diskspace 100 Total disk space in megabytes a dynamic feed needs 20-200MBs, a full feed needs approx. 1000MB expire_at 5 23,3 Run an expire job at 5 past eleven and 5 past 3 in the morning. feed_read true Sends the `mode reader' command er (default) when connecting to a feeding site, this is the default. You will probably never have to change this setting. history \dnews\spool Directory for history file, this could grow to 40MB for a full feed (80,000 blocks) history_m 5 Pause if less than `n' megabytes in free item_max 100000 Sets the maximum size for any article that DNEWS can deal with, the default is 1,000,000. A smaller number will save memory. life 30 Number of days to keep caching a news group after someone reads an item in it. logdir \dnews Directory to write log files logdir_mi 5 Pause server if less than 5MB is n left free on logging disk. loglvl info One of: error,warn,info,debug,flush There are four levels of error reporting, "error" includes serious errors, "warn" includes possible problems, "info" includes general information messages and "debug" which is used for tracing bugs. "flush" is the same as debug but the log file is flushed with every single line written, this is useful if the system is crashing but not providing a stack dump. loguse true Write a log of every item body read by users in the file used.log manager system@here. Email address of the news manager this.place mail smtp This is only used on VMS systems. The name of the foreign email system on VMS, typically in% or smtp% (don't include the %) myname news.this.pl Your full internet name, if left ace out this may be picked up from the system. It's best to put it in by hand to make sure it's right. nntp_pass mysecretword Use this if the site you are sucking from requires a username and password to access it. nntp_user jbloggs Use this if the site you are sucking from requires a username and password to access it. nntp_feed news.feeder. The news site where you are going er site to SUCK a feed from. You need NNTP READER access to this site. If you are not sucking a feed then comment this line out. If you need to specify a PORT other than the default one (119) then add it on the end of the name, e.g. feeder.site.edu:2021 nodns true Stops DNEWS from doing reverse DNS lookups every time a user connects, you will have to put ip numbers into access.conf if you use this feature, but if your DNS is faulty or slow then this can improve connection times. port 119 The TCP port which DNEWS should listen for incoming connections on. postonly true Stops items from being posted locally until the `echo' back from the feeder site, this gives a very clear confirmation that the message has been passed up stream. You cannot do this if you want to have local groups which are not on the site you are sucking from. report no Stops the daily email report from being sent to the manager after every expire run. timeout 30 Define how long to wait after a channel becomes idle before disconnecting it, the default is 120 minutes. timezone NZST Timezone hh:mm or timezone name Appendix B too_old 50 Refuse items that were created more than this many days in the past, this prevents old items from being re-distributed when a machine dumps old news items onto the network. spool \dnews\spool Disk name (or logical) for news articles spool_min 5 Pause if less than `n' megabytes free update_at *0,*5 * Check for new items every time the minutes end in a 5 or a zero and at any hour, e.g. every 5 minutes.. workarea \dnews\spool Directory for work files (active.dat, *.feed, etc) workarea_ 5 If less than `n' megabytes are min free on this disk then pause the news system, (stop accepting new articles) access.conf This file controls access to your system, you can control access by IP name, IP number or even by individual user using password protection. <host>:<perm>:<user>:<pass>:<groups> Paramet Example Description er <host> *.my.domain A wildcard name or number, this 161.29.4.* rule is only used if the connection matches this address <perm> Read, Post, Allow any combination of reading Ihave posting or ihave feeding from this site. <user> chrisp Some news readers can send a user name, if so then it will be matched here, and the password can be checked. <pass> rumplestiltski The password in plain text n <groups *,!*.erotic.* Allow access to groups matching > this wild card Notes: Unlike other news implementations, you can permit reading and posting to different combinations of news groups, for example you may wish to allow students to read all groups but only post to local news groups. expire.conf This file can be tricky to use, remember order is significant!! The expiration file is very different from INN and CNEWS implementations, Rather than a simple set of rules, the DNEWS expire rules match both news group names and current disk usage, so as use increases the rules can become more strict. In addition rules can specify minimum or maximum numbers of items, this allows a single rule to apply to groups which get 20 items a day, or 20 items a month. Each line of the file must have the following format: UPTO x GROUPS wild KEEP n DEFAULT n PURGE n MAXITEMS n MINITEMS n Symbol Example Description upto 20% This rule applies if less than 20% of the disk space is full. groups alt.*,comp.bin This rule applies if the group aries.* matches this list keep 2 Keep items for this many days even if they have already expired. (From the expire date in the article header) default 10 Keep items this long and then delete unless the expiration date says to keep it longer. purge 35 Delete after this many days even if the expire header says to keep it longer. This should be set long enough to keep FAQ's and no longer to prevent abuse. maxitem 1000 Delete old items if the group has s more than this many items in it. Good for controlling high volume groups. minitem 10 Don't apply any other rules if the s group has less than this many items in it, this is good for making `low volume' groups useable. Example: # Default rule upto 10000% groups * keep 0 default 1 purge 10 maxitems 0 minitems 0 # Start to panic a little upto 90% groups * keep 0 default 2 purge 33 maxitems 100 minitems 0 # Normal operation, put your meaningfull settings here. upto 60% groups * keep 2 default 5 purge 33 maxitems 1000 minitems 10 upto 60% groups comp.* keep 2 default 10 purge 33 maxitems 2000 minitems 10 # Lots of space, just stop things getting silly (e.g. 10000 items in...) upto 30% groups * keep 2 default 17 purge 33 maxitems 3000 minitems 100 upto 20% groups * maxitems 2000 # Keep any local groups forever (this line must be last to work) upto 1000% groups local.* keep 99999 minitems 10000 newsfeeds.conf This file controls feeds out of DNEWS to other sites, it also controls the feed `into' DNEWS via the same mechanism. Symbol Example Description site me The name of the site to feed, $NNTP_FEEDER usually it's full internet name. my.leaf.site The `me' feed must be the first one in the file. builtin (flag, no Specifies the builtin feeding value) mechanism should be used. This is only intended for sending local postings up stream, it is not fast enough to send a full feed on to another site. posting (flag, no Specifies that the nntp `post' value) command should be used to send items upstream. This gives your feeder site the job of deciding how to email postings to moderated groups. exclude feeder.real.na Use this if your feeder site is me,feeder.thir `name stamping' articles with a d.name name other than it's ip name. This MUST be right or DNEWS will attempt to echo everything it gets back to the feeder site. (check the path header) smtp (flag, no Specifies this is not really a value) feed, but on a vax it is actually the email queue used to send articles to moderated groups, this is only needed on VMS and then only if you are not using the `POSTING' mechanism to feed upstream. maxcros 50 Throw away items which are s crossposted to more than `50' news groups. This is good for preventing `spamming' which is where people post adverts and money making schemes to inappropriate news groups. maxsize 100000 Limit items to 100K, the default is 1MB, if you want to make it larger than 1MB then you must also increase ITEM_MAX in DNEWS.CONF. Personally I think 100K is too big, ftp sites and web should be used for larger items. moderators.conf You won't need to touch this file unless you want to add a local group and make it moderated. In that case add your groups at the top of the file as the first match is used. Format: newsgroup:email-address If the email address contains a "%s" then that is replaced by the news group name with the dots changed into dashes. Example: local.gossip.moderated:chrisp@marc.cri.nz gnu.*:%s@tut.cis.ohio-state.edu *:%s@uunet.uu.net control.conf This file controls the action taken when control messages are received. This is the secret to the administration of news. The following control messages are checked by this file: · newgroup · rmgroup · version · checkgroups · ihave · sendme · sendsys · senduuname However only the following three are implemented. · newgroup · rmgroup · version Blanks are not allowed, e.g. :: would be misinterpreted. For each type of control message, you can specify based on who sent it, whether or not to do what the message says, just log it, or email the message to the news administrator. The log file that DNEWS creates can be used to action the control messages at a later date as it contains valid commands. All control messages are logged to `control_(message).(action)' Format: message-type:email-address:action message-type rmgroup, newgroup, version email-address A wildcard email address, e.g. news@trusted.site action "drop", "log", "mail", "doit", "doit,mail" Example: all:*:*:mail ihave:*:*:drop sendme:*:*:drop sendsys:*:*:drop senduuname:*:*:drop newgroup:tale@*.uu.net:comp.*,misc.*,news.*,rec.*,s ci.*,soc.*,talk.*:doit rmgroup:tale@*.uu.net:comp.*,misc.*,news.*,rec.*,sc i.*,soc.*,talk.*:doit How news gets in and out of your system FEEDING news INTO your system With DNEWS you have two choices, you can push the news into your system using an NNTP IHAVE sending program or a SUCKING feed. A pushing ihave feed is run from the server which provides you with a feed. This is the standard way of sending news but it has two disadvantages: 1. An NNTP IHAVE feed comes as a random selection of items from various news groups, this prevents any optimization when creating indexes and storing items. 2. An NNTP IHAVE feed must be setup from the other end. This prevents auto configuration of what is fed, it also makes configuration difficult as you may have limited or virtually no access to the feeder site configuration. Feeding in using an NNTP IHAVE feed. If you use this method then you must take a full feed, or at least a fixed subset of feeds, you can't use the DNEWS dynamic system. To set this up simply modify the access.conf file to allow the site that is feeding you to have IHAVE access. news.feeder.site:ihave:::* In dnews.conf add one line cachedall true Then the job is done apart from setting the feed up from the other end which is entirely your problem! Feeding in using a SUCKING feed (default) This is extremely easy to setup, in the file NEWS.CONF define the variable NNTP_FEEDER to point to a site where you have NEWS READER access. Simple wasn't it :-) FEEDING news OUT OF your system Sending news up line using a POSTING feed (default) If you are a LEAF site. (A leaf site is where you don't feed news on to anyone else) then you have an easy option, you can pass the job of `posting' an item up stream to your feeder site. Using this system DNEWS essentially pretends to be a NEWS READER which is posting your item directly on your FEEDER site. Any error message will be emailed to you. In the file NEWSFEEDS.CONF define one feed as follows: site $NNTP_FEEDER groups * builtin posting If your feeder site is stamping each news item with a name different from it's IP name that you used in your DNEWS.CONF file then you will have to add an exclude line to this feeding definition to stop the feed from trying to send every item that comes in back to the feeder. exclude feeder.name.stamp The special commands `builtin' tells DNEWS to use it's own internal NNTP link to send any articles to this feed and the command `posting' tells DNEWS to POST articles rather than pass them up using the IHAVE mechanism. In order to get email confirmation of each post as it is really posted use the CONFIRM variable in news.conf to match all local email addresses. Sending news up line using an NNTP-IHAVE sender If you are a leaf site then just remove the `posting' keyword from your newsfeeds.conf file. However, if you want to pass a full feed on to someone else then you need to use the XMIT program as the builtin feed cannot deal with large volumes. See the section "How do I setup XMIT" in the FAQ section of this manual. Appendix A, What is USENET What is USENET News? USENET News is a world wide bulletin board. It has no management or central system. It is a large number of computers and networks connected together. News groups The news is divided into approximately three thousand groups. These groups are chosen by mutual agreement using a complicated voting system. How your article will get to the rest of the world When you post an article, it travels from your machine to all the NEWS servers which your machine is connected to, (usually just one other machine, but in the case of a university, you may be connected to three or four other `news' machines). Each of these servers in turn pass on your `article' to all the machines that they are connected to. In this way your message eventually gets to every other news machine in the world. There are some very clever built in systems to prevent an article from traveling in a loop but these need not concern you. The important point is there is built in redundancy, so once your message gets past your local system a single faulty system will not stop it from getting to the rest of the world. What to read first There are a number of postings in the news group "news.announce.newusers" that should be read and followed by all users using news for the first time. What is an FAQ? FAQ stands for Frequently Asked Questions. These are questions that every one asks when they first become interested in a topic, the `old timers' in a news group get sick of answering them again and again so they create an FAQ. This is a list of all the questions and answers. Always look for an FAQ before posting a question. Be careful posting Remember when you ask a stupid question on NEWS you have just annoyed approximately ten to one hundred thousand people, depending on the news group. So always consider very carefully before posting. How big is it? An item in a widely distributed news group will reach an estimated 24,000 sites world-wide hosting 1,000,000 potential readers. Most sites are educational institutions, but many research organizations, companies, and government departments are also part of USENET. This number is growing exponentially, as many as 20,000,000 users are on the internet and most `private' individuals will only begin to connect over the next couple of years. Appendix B, Time Zone Names gmt Greenwich Mean ut Universal utc Universal Coordinated cut Coordinated Universal wet Western European bst British Summer nst Newfoundland Standard ndt Newfoundland Daylight ast Atlantic Standard adt Atlantic Daylight est Eastern Standard edt Eastern Daylight cst Central Standard cdt Central Daylight mst Mountain Standard mdt Mountain Daylight pst Pacific Standard pdt Pacific Daylight yst Yukon Standard ydt Yukon Daylight akst Alaska Standard akdt Alaska Daylight hst Hawaii Standard hast Hawaii-Aleutian Standard hadt Hawaii-Aleutian Daylight ces Central European Summer cest Central European Summer mez Middle European mezt Middle European Summer cet Central European met Middle European eet Eastern Europe msk Moscow Winter msd Moscow Summer wast West Australian Standard wadt West Australian Daylight hkt Hong Kong cct China Coast jst Japan Standard kst Korean Standard kdt Korean Daylight cast Central Australian Standard cadt Central Australian Daylight east Eastern Australian Standard eadt Eastern Australian Daylight nzst New Zealand Standard nzdt New Zealand Daylight Appendix C, NNTP Summary authinfo [user Name]|[pass Password] Used to authenticate individual users. The command is first sent to define a user and then sent a second time to give the password. Older news readers may not allow the use of this feature. On the server side add entries to access.dat using an editor. article [MessageID|Number] Responds with the article, if no second parameter is given then the current article is sent. News readers should avoid using MessageID as it generates extra work for the server. body [MessageID|Number] Sends the body of the message. date Responds with the current time from the servers point of view, (not standard) group newsgroup Sets the current group head [MessageID|Number] Sends the header of a message. help Sends a list of valid commands. ihave <message-id> Used by another news system to `offer' new items to this news system. last Sets the current news item to one item lower. list [active] List on it's own or with the active qualifier responds with a list of known news groups. list active.times Responds with a list of news groups and the date and time they were created. This file may or may not be complete depending on how the system was installed and how long it has been running. list newsgroups Responds with a list of news groups and a short description of each one. This list may or may not be complete or even available depending on how your system is installed. listgroup [newsgroup] (extension) Gives a list of valid items in a news group. This is currently the only efficient way for a news reader to establish which items in a group actually exist. mode reader (extension) Does nothing, this command only exists for compatibility with INN. mode stream (extension) Responds with a success status to signify that the streaming extensions are supported. newgroups yymmdd hhmmss [GMT] [<distributions>] Lists news groups created since the given date. newnews newsgroups yymmdd hhmmss [GMT] [<distributions>] Lists message-id's of news items that arrived after the specified date. next Sets the current point to the next item. post Used by news readers to post a new item. slave Has no affect. stat [MessageID|Number] Checks an item exists, responding with it's message id. check [MessageID] (extension) Checks an item exists. This is one of the STREAMING extensions to NNTP. It is different from the stat command in that it will respond that it still has a message even after the message has expired. takethis [MessageID] (extension) This is the other half of the streaming extension to NNTP, this command behaves like the IHAVE command but doesn't have a refusal stage so it's return status doesn't need to be checked before the next item is sent. xgtitle [group_pattern] (extension) Extension used by ANU-NEWS to actually use the newsgroups file to respond with a list of newsgroups that matches the pattern, and gives their one line descriptions. xhdr header [range|MessageID] (extension) Lists the requested header of all items that fall within the given range. xover [range] (extension) Lists all the items within range giving a fixed set of headers in return. Under DNEWS this command is implemented in software, no actual database is required as the DNEWS database is efficient enough to produce this information on the fly. xpat header range|MessageID pat [morepat...] (extension) Weird extension, don't use this. In shell syntax it is roughly equivalent to: xhdr header range | grep pat xpath MessageID (extension) This command is not implemented, as their is no internal file name that can be returned and frankly the command should never have existed as it breaks every rule of information hiding and abstraction. But hey, that's UNIX :-) Copyright & Disclaimer The software DNEWS is Copyright (C) 1995 NetWin LTD, New Zealand, All rights reserved. The software remains the sole and exclusive property of NetWin at all times it may not be modified and resold or coppied in any way other than as described in the documentation. This software and the accompanying files are sold "as is" and without warranties as to performance of merchantability or any other warranties whether expressed or implied. No warranty of fitness for a particular purpose is offered. (DNEWS is an extremely complex piece of software. NetWin will make reasonable efforts to solve any reported problems but we must limit our legal liability for obvious reasons.) Corrections & Additions. Please let us know if you find errors in this manual or if you think some additional information should be added.