GOPHERD/GOPHERLS/GINDEXD

Section: Maintenance Commands (8)
Updated: Local
Index Return to Main Contents
 

NAME

gopherd, gopherls, gindexd - serve data using the gopher protocol.  

SYNOPSIS

gopherd [-DIc][-s securityfile] [-L loadavg] [-l logfile] [-u userid] [-U uid] Datadir Port  

OPTIONS

Datadir is a directory where the information for the gopher server resides

Port is the socket port to use (or if running under inetd, the port that the server is running under.)

Changing the name of the gopherd executable will change how gopherd will behave. If invoked as gopherls , the program will list out the Datadir to the screen, processing all the links and .cap things. If invoked as gindexd, the program can serve up the single index Datadir. We discourage the use of gindexd, since it adds an extra process that must be run. It is included for backwards compatibility.

-D Enable copious debugging output.

-s Restrict/Enable security from securityfile

The -s parameter is used to control access to the server. When a host connects to the server, the server checks for matches in the security file. First the server checks the first character of the line. If it's an exclamation mark, the server will deny access in the case of a match, otherwise it gives access if a match is made. The criteria for matching is:

If the entry is numeric (i.e. an internet #) then we get a match if the "hostname" matches the numeric string or the numeric string + anything else, i.e. an entry like "128.101" would match 128.101.95.1 but not 37.15.128.101.
      If the entry is alphabetic then we test in the other direction. i.e. umn.edu would match moocow.umn.edu but not umn.edu.com
    An asterisk matches anything, thus the line * would give access to everything, an entry like !* would deny access to everyone. By default the security file has an implicit '*' at the end of the file. If you want to restrict access make sure to put a '!*' at the end of your security file.

Matches are checked in the order they are in the security file. If you want it to work correctly go from the specific to the general. (i.e. don't put your * entries before specific domains.)

Here's an example security file that will allow access to all hosts that have umn.edu at the end or start with 128.101 or 134.84. The !* restricts access to everyone else.


 128.101
 134.84
 umn.edu
 !*

-L

If the server is compiled with the LOADRESTRICT option then this option sets the load average at which connections are turned away.

-l Log connections to logfile

If a logfile is specified, the server will log the time, host and transaction of each connection. If there isn't a -l option passed to gopherd then no logging is done.

-I Specify when running under inetd

This is also useful for testing server code changes.

-C Turns server side caching off

The caching mechanism works as follows: The server checks for a file called .cache when doing a directory lookup. If it finds a relatively new one, it blasts it across the connection, no sorting required. If it doesn't find one, or finds an old one it resorts through the directory and makes a new .cache file.

-c Do not chroot(2) before processing connections. (Waterlooism)

This relies on code that (tries to) ensure(s) that files outside the data directory cannot be retrieved. This has the potential to be a gigantic security hole. Use with care, and with the -u or -U options. Also, if you are using it with inetd, be sure to specify the -I flag *first*.

This is very useful if you wish to have symbolic links to files outside the data directory available through the gopher server. For example, you might make a symlink to some local man pages, to avoid duplicating the information.

-u Run as user userid. (Waterlooism)

The server is run with reduced permissions (i.e. other than root) which can ensure that only publically readable files are available from the server.

This option can be used with or without the -c flag. If you're running a gopher server on top of an anonymous ftp directory you'll want to run with the option "-u ftp" to preserve the same level of security as anonymous ftp.

-U Like -u, but takes a numeric uid instead. (Waterlooism)

 

DESCRIPTION

Gopherd serves data using the Gopher protocol. The gopherd server understands how to serve text and directory objects. (For indexes see gindexd(8)) The server also support the creation of "Links". Thus it is possible to represent Telnet sessions or CSO servers.

The gopher server also contains enough intelligence to represent mail spool files as directories.

 

ADDING INFORMATION TO THE SERVER

The gopher server gets its information from the files and directories in Datadir By making changes in this directory tree, you will change what the gopher clients see.

 

DEFAULT TRANSALTIONS

All files/directories that start with a dot, or are named etc, usr, bin, dev, core are ignored or processed specially.

The title of each object presented to the client is the filename of the file/directory.

Directories in the tree are served as Gopher directory objects.

Text files are represented as Gopher text objects.

Ulaw sound files are are represented as sound objects.

Compressed files are assumed to be text, and are have their titles changed to remove the".Z" extension.

Mail spool files are represented as directories. When the client attempts to view the mail-spool file, the Subject lines are presented as the title of each mail message within the file.

Executable shell scripts are assumed to be of text object type. When the client requests to view the shell-script, the script is run, and the results sent to the client.

Files that end with .html are assumed to be WWW hypertext files. They are served up with type 'h'.

Files that end with .src are assumed to be WAIS searches. If the server has WAIS indexing compiled in, then these will show up as a search engine and a descriptive file.

The server sorts directories it sends out alphabetically. The server does distinguish between upper and lower case when sorting.

 

LINKS

You can override the defaults by creating what are known as Links in the Gopher data directory tree.

The ability to make links to other hosts is how gopher distributes itself among multiple hosts. There are two different ways to make a link. The first and simplest is to create a link file that contains the data needed by the server. By default all files in the gopher data directory starting with a period are taken to be link files. A link file can contain multiple links. To define a link you need to put five lines in a link file that define the needed characteristics for the document. Here is an example of a link.


 Name=Cheese Ball Recipes
 Numb=1
 Type=1
 Port=150
 Path=1/Moo/Cheesy
 Host=zippy.micro.umn.edu

The Name= line is what the user will see when cruising through the database. In this case the name is "Cheese Ball Recipes". The "Type=" defines what kind of document this object is. The following is a list of all the defined types:


 0  -- Text File
 1  -- Directory
 2  -- CSO name server
 4  -- Mac HQX file.
 7  -- Full Text Index
 8  -- Telnet Session
 9  -- Binary File
 s  -- Sound

The "Path=" line contains the selector string that the client will use to retrieve the actual document. The Numb= specifies that this entry should be presented first in the directory list (instead of being alphabetized). The "Numb=" line is optional. If it is present it cannot be the last line of the link. The "Port=" and "Host=" lines specify a fully qualified domain name (FQDN) and a port respectively. You may substitute a plus '+' for these two parameters if you wish. The server will insert the current hostname and the current port when it sees a plus in either of these two fields.

An easy way to retrieve links is to use the Curses Gopher Client. By pressing '=' You can get information suitable for inclusion in a link file.

 

OVERRIDING DEFAULTS

The server looks for a directory called .cap when parsing a directory. The server then checks to see if the .cap directory contains a file with the same name as the file it's parsing. If this file exists then the server will open it for reading. The server parses this file just like a link file. However instead of making a new object, the parameters inside the .cap/ file are used to override any of the server supplied default values.

For instance say you wanted to change the Title of a text file for gopher, but don't want to change the filename. You also don't want it alphabetized, instead you want it second in the directory listing. You could make a set-aside file in the .cap directory with the same filename that contained the following lines:


 Name=New Long Cool Name
 Numb=2

 

ADDING COOL LINKS

One cool thing you can do with .Links is to add neato services to your gopher server. Adding a link like this:


 Name=Cool ftp directory
 Type=1
 Path=ftp:hostname@path/
 Host=+
 Port=+
 
 Name=Cool ftp file
 Type=0
 Path=ftp:hostname@file
 Host=+
 Port=+

Will allow you to link in any ftp site into your gopher. Make sure that there is a /tmp directory to store the files for the gateway.

Another neat thing you can do is to execute commands:


 Name=Execed command name
 Type={a type}
 Path=exec:"args":/scriptname
 Host=+
 Path=+

This is usually used by other types of gateway scripts. For instance, The first script might take a search and get a few hits. It could then generate "exec" scripts that would retrieve the actual document the hit referred to.

 

FULL TEXT INDEXES

As of the 0.8 release of gopher, full text indices can be accessed directly using gopherd. Just put the indexes someplace in the gopher-data directory and then create a link. For example, assume you have a directory that contains some indexes called "fred" in the top level of your Gopher data directory.

 

Making a WAIS link

To make the "fred" directory into a WAIS full text index searcher, add the following lines to .cap/fred


 Type=7
 Path=7/fred/index

This will change the type into a full text index (7). The Path= format for a WAIS index is "7{path}/{dbname}".

 

Making a NeXT Digital Librarian link

To make the "fred" directory into a NeXT full text index searcher, add the following lines to .cap/fred


 Type=7
 Path=7/fred

This will change the type into a full text index (7). The Path= format for a NeXT index is "7{path}".

 

WWW and HTML support.

Gopherd can be queried by WWW clients in the hypertext mode. The server caches the HTML pages it makes. You can include HTML stuff that will appear above the directory by making a file called ``.about.html'' The gopher server will send this before the menu.

References to Gopherd-WWW pages are as follows:


  http://hostname.domainname:port/gopherpath/

 

EXAMPLES

/usr/local/etc/gopherd /home/gopher-data 70

This command will serve data from the directory /home/gopher-data on port 70 The gopher daemon forks off and will then start accepting connections. You can easily test the server by using telnet to connect to the port specified on the command line. Once connected, type return. A list of things in the gopher-data directory should be returned. For instance let's say that the server was started on the machine mudhoney. To test it you'd do the following:

telnet mudhoney 70

Here is an example of running the gopherd server from inetd. This example assumes that you have the files /etc/inetd.conf and /etc/services. Other machines and architectures may vary. Assume that we have a directory /gopher-data and we want to run at the standard port for gopher (70).

In /etc/services add the following line:

gopher 70/tcp

In /etc/inetd.conf add the following line:

gopher stream tcp nowait root /etc/gopherd gopherd -I /datadir 70

 

FULL-TEXT INDEX EXAMPLES

An full-text index is used to rapidly find data in a set of files. The first step in making a gopher server with indexes is to build the index on your data files.

When using the NeXT digital librarian this command is ixBuild , when using the WAIS software this command is waisindex

Okay, how about a real world example? Suppose that you have a directory structure that contains your grandmother's favorite recipes located in /home/mudhoney/recipes/. Also assume that there are subdirectories for cakes, pies, and stews i.e. :


 % pwd
 /home/mudhoney/recipes
 
 % ls 
 cakes/ pies/  stews/ 

In this example we will be running a gopher server on the directory /home/mudhoney/recipes.

 

Building the index with the NeXT and ixBuild.

Go into the directory you want as the root level of your index. If you want to index all of the recipes you'd type the following:


 cd /home/mudhoney/recipes
 mkdir .index
 ixBuild -V

This will make an index in the file:


 /home/mudhoney/recipes/.index/index.ixif 

If you wanted to just index the pies subdirectory you would do the following:


 cd /home/mudhoney/recipes/pies
 mkdir ../.index
 ixBuild -V -i../.index/index.ixif

or...


 cd /home/mudhoney/recipes/pies
 mkdir .index
 ixBuild -V -i.index/index.ixif pies

It's important that the filenames that are generated by ixBuild have the same format that is given by the gopher server! You can easily test this by using ixFind path. Also make sure that when you're indexing a subdirectory that the previous directories up to the gopher data directory get added to the path by ixBuild.

 

Building the Index with waisindex

With waisindex it isn't necessary to be in the root directory of the gopher server. The WAIS indexer stores filenames with their absolute paths, this causes problems later.

If you want to index the whole recipe collection in /home/mudhoney/recipes you'd do the following:


 mkdir /home/mudhoney/recipes/.index
 cd /home/mudhoney/recipes/.index
 waisindex -r /home/mudhoney/recipes

The waisindex program will create a bunch of files starting with index. These files together comprise an index to your data.

If you wanted to just index the pies subdirectory you would do the following:


 mkdir /home/mudhoney/recipes/.index
 cd /home/mudhoney/recipes/.index
 waisindex -r /home/mudhoney/recipes/pies

 

Linking in the Sample indexes

Both of these indexes are in a directory that starts with a period. Thus the server will ignore these directories when sending out information. So now you need to make a link to these indexes so people can start searching them. For the NeXT you would put this in a file /home/mudhoney/recipes/.Link


 Type=7
 Name=Recipe Index
 Port=+
 Host=+
 Path=7/

For a WAIS based gopher server you would put the following into the file /home/mudhoney/recipes/.Link


 Type=7
 Name=Recipe Index
 Host=+
 Port=+
 Path=7/.index/index

 

Changing the defaults for full text indexes...

You may want to change what gets returned for a full text index. Especially if you are storing indexes on a machine that doesn't have the actual data. In this case you can use what is known as a "hostdata" file to change what the server returns given a full-text search.

The server will look for a file called "<indexname>.hostdata" to get information from. If it doesn't find this file it will try just "hostdata".

A hostdata file consists of a text file with three lines in the following order:


 <hostname>
 <portnum>
 <path to strip>

These fields are used to override the defaults. For instance, the following hostdata file would substitute the hostname "goober.micro.umn.edu" for the current server host, the port 8000 for the current server port, and strip off the path /home/goober from the pathname entries in the index:


 goober.micro.umn.edu
 8000
 /home/goober

 

SEE ALSO

gopher(1)


 

Index

NAME
SYNOPSIS
OPTIONS
DESCRIPTION
ADDING INFORMATION TO THE SERVER
DEFAULT TRANSALTIONS
LINKS
OVERRIDING DEFAULTS
ADDING COOL LINKS
FULL TEXT INDEXES
Making a WAIS link
Making a NeXT Digital Librarian link
WWW and HTML support.
EXAMPLES
FULL-TEXT INDEX EXAMPLES
Building the index with the NeXT and ixBuild.
Building the Index with waisindex
Linking in the Sample indexes
Changing the defaults for full text indexes...
SEE ALSO

This document was created by man2html, using the manual pages.
Time: 10:05:20 GMT, January 08, 2023