Sunfish

Blackbanded Sunfish, Enneacanthus chaetodon
A 30-60mm long fish with a very compressed, deep body, a thin pointed pectoral fin and round tail fin. Habitat: swamps, ponds and river pools.

Sunfish
An NFS client, implemented as an image filing system. Habitat: RISC OS.

Contents

License

Sunfish is copyright © 2003 Alex Waugh

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

Requirements

Sunfish requires a working network connection and an NFS server to connect to. The server should be running version 2 of the NFS protocol, version 1 of the Mount protocol, version 2 of the portmapper protocol, and optionally version 2 of the pcnfsd protocol. It has currently only been tested with the Linux kernel server present in Debian Woody, but it should work with other servers. Both UDP and TCP connections are supported.

Sunfish should run on any version of RISC OS from 3.11 onwards. It has been reported to work on RISC OS 3.7, RISC OS 4 (including under Virtual RPC), Select, and RISC OS 5. It is 26/32bit neutral.

Sunfish has been reported to work with the Linux 2.4 kernel server in various distibutions, a Solaris 7/8 server, the TrueGrid NFS server on Windows, and Netware 6.5SP1a. It partially works with Allegro NFSd 1.1.4 on Windows, but a bug in that version of the server can cause file not found errors when copying files onto the server.

The rest of this document assumes your server is correctly setup. You may wish to refer to Allan Curtis' notes on setting up a Linux NFS server, or the Drobe article on setting up the TrueGrid server for use with Sunfish.

Any updates will be available from http://www.alexwaugh.com/

Setting up a mount

Sunfish is implemented as an image filing system. You can either write a mount file by hand, or use the frontend to create one.

To use the frontend, run !Sunfish which will put an icon on the lefthand side of the iconbar. Select New mount... from the menu then fill in appropriate values for all the options. The options correspond with those described below, and the default values in the filenames, connection and ports sub-windows should not need altering for most simple connections. The mount leafname option is used to give a name to refer to the mount file. When you click on Create then a mount file with the specified options will be created, and a new icon will be added to the iconbar for that mount.

To manually write a mount file, use your favorite text editor to create a file with the appropriate options as described below. Save this, then set it's filetype to Sunfish (&1b6). Then double click on the mount file you have just created and the server will be mounted and you can navigate the filesystem as normal.

Due to the way image filing systems work, you may not be able to edit the mount file once it has been mounted and while Sunfish is still running. There are two solutions - either set the filetype of the mount file to Text, edit it, then set it back to Sunfish, or *RMKill Sunfish before editing the file.

You may also have to ensure that the connection is not blocked by a firewall. The default configuration on some RedHat systems is known to block NFS traffic.

Example mount files

This is a very simple mount file. It will use pcnfsd to map the username and password onto a uid and gid.

Protocol: NFS2
Server: mint.cp15.org
Export: /home/ajw498
Username: ajw498
Password: fiddlesticks

The same as above, but specifying the uid and gid explicitly. pcnfsd is not required in this case, and there is no need for the password to be stored in the file either.

Protocol: NFS2
Server: mint.cp15.org
Export: /home/ajw498
UID: 1001
GID: 50

A more complex example.

Protocol: NFS2
Server: mint.cp15.org
Transport: tcp
Export: /tmp
UID: 1002
GID: 51
GIDs: 52 53 105
umask: 066
MachineName: caramel
ShowHidden: 0
Timeout: 5
DefaultFiletype: FFF
AddExt: 2
MaxDataBuffer: 8192
Pipelining: 1

Filenames

Filenames on a unix machine may contain characters that are illegal in RISC OS filenames. In order to allow access to such files, any illegal characters are changed into escape sequences of ?xx where xx is the hexadecimal value of the character. Spaces are mapped onto hard spaces, dots map to slashes and vice versa, and other characters are unchanged.

Unlike most other RISC OS filing systems, filenames may be case sensitive (depending on the casesensitive option). However, it is not recommended to use filenames that differ only in their case, as some programs or RISC OS may not distinguish between them correctly.

Options

Any line beginning with a # is treated as a comment.

Mandatory options

These options must appear in the file for a working connection to be established. Either a uid and gid must be specified or a username and password.

Protocol

Specifies the protocol to use. An error will be generated in the current version if this is not NFS2.

Server

The domain name or ip address of the NFS server.

Export

The name of the exported directory to mount. This should match one of the entries in /etc/exports or equivalent on the server.

UID

The user id to use. If this is not specified, then a username and password should be specified instead.

GID

The group id to use. If this is not specified, then a username and password should be specified instead.

Username

The username to connect as. If specified, then pcnfsd will be used to map the username and password to a uid and gid. If not specified, or pcnfsd is not available, then the uid and gid should be specified instead.

Password

The password to give to pcnfsd with the username. This password has to be stored in the file in plain text, so if that is a concern then you should specify the uid/gid instead.

Optional options

These options can be specified if needed, but if they are not then default values will be used. For most connections, the defaults will be sufficient.

Transport

The type of connection to use as transport. Can be either tcp or udp. If omitted, it defaults to udp. Note that older servers tend not to support tcp connections.

MachineName

The name of the machine the client is running on. Some servers may use it to allow or deny access. If omitted, it defaults to the value of <Inet$Hostname> which should be sufficient in most cases.

MaxDataBuffer

The maximum number of bytes to use for the data buffer of a read or write request. A bigger buffer will generally be faster than a smaller one, but some network cards (In particular, the Castle 100BaseT podule) cannot handle larger packets when using UDP. The maximum value is 8192 bytes, and it is recommeded that you set it to this if your network card can handle it (You will probably get timeout errors when reading a large file if it is too big). If omitted, it defaults to 4096 bytes for UDP connections and 8192 for TCP connections.

Pipelining

If set to 0, then all operations wait for a reply before sending the next request. If set to 1, then for file reads and writes new requests are sent before the reply from the previous request is recieved. This can give an increase in speed in some circumstances. If ommitted, it defaults to 0.

PortMapperPort

The port number to use for the portmapper service. If omitted, it defaults to the assigned number 111.

MountPort

The port number to use for the mount service. If omitted, the portmapper service will be used to find the correct port.

NFSPort

The port number to use for NFS. If omitted, the portmapper service will be used to find the correct port.

PCNFSDPort

The port number to use for PCNFSD. If omitted, the portmapper service will be used to find the correct port.

LocalPort

The port number to use for the client. This option can either be a single number to use a specific port, or a two numbers spearated by a space to give a range of ports from which a free one will be chosen. If a port number of 0 is specified then any free port above 1024 will be used. If ommitted, the default range of 800-900 will be used.

Generally, it doesn't matter what port is used, but some servers will only allow connections from ports less than 1024 unless the insecure option is used in /etc/exports on the server.

GIDs

A list of additional group ids that the user is part of. There can be up to 16 gids in the list, separated by whitespace. If pcnfsd is used then the list returned by that will override this setting.

Logging

If set to 1 then all operations will be reported to syslog. This should only be enabled for debugging, as it significantly slows things down. Note: if one connection specifies logging, then activity on all connections will be logged while that connection is mounted.

umask

The umask to use, specified in octal. All operations that modify a files attributes will have the Unix mode bits modified according to the umask. If not specified, it defaults to 022. If pcnfsd is used then the umask returned by that will override this setting.

unumask

RISC OS does not have any concept like a umask - the default attributes of a file are decided by the application that creates the file. Usually the default chosen is private read/write set and public read/write clear. The unumask allows you to force particular unix mode bits to be set, and is specified in octal in the same format as the umask. If not specified, it defaults to 0.

The RISC OS attributes are first converted to unix mode bits, with the private attributes mapping to user permissions, and the public attributes mapping to the group and others permissions (if a public attribute is set then both the corresponding group and others permission bit will be set). This value is then ORed with the unumask, and then ANDed with the inverse of the umask to give the final unix mode bits.

ShowHidden

If set to 1, all files will be shown, if set to 0 then all files beginning with a . will be hidden from filer views. If not specified, then it defaults to 1.

FollowSymLinks

The number of levels of symlinks to follow. If set to zero or a sequence of symlinks greater than the number is encountered then the link will be shown as a text file that you cannot read or write. If not specified, then it defaults to 5. Symlinks can only be followed if they point to a file within the mount directory or its children.

CaseSensitive

If set to 1, filenames will be treated as case sensitive, and if the filename doesn't match exactly then you will get a file not found error. If set to 0, then if an exact match for the filename isn't found then a case insensitive match will be tried as well. If not specified, then it defaults to 0.

If case sensitivity is off, and you have two (or more) files in the same directory that differ only in case then trying to access one of them may result in the wrong file being altered.

Timeout

The time in seconds to wait for a reply from the server before retrying or giving up. Defaults to 3 seconds.

Retries

The number of times to resend a request if no reply is received. Defaults to 2 retries. If TCP is being used as the transport then this setting is ignored because TCP will do any necessary retries itself.

DefaultFiletype

The hexadecimal filetype to give a file if it doesn't have an ,xyz extension or a . extension that matches a mimemap entry. If not specified then it defaults to FFF.

AddExt

Controls adding of a ,xyz extension. When 0, no ,xyz extensions are added, and all files get the default filetype. When 1, a ,xyz extension is only added when necessary. If the file has a . extension that matches a mimemap entry then no extension is added. When 2, a ,xyz extension is always added. If not specified, the default is 1.

Improving Performance

There are three options that are likely to have an effect on performance: the transport, the MaxDataBuffer option and the Pipelining option.

Using TCP as the transport may give better results than UDP in some circumstances.

Normally, the bigger the data buffer the better (the maximum size is 8192), however some network cards (notably the Castle 100bT podule) cannot cope with large packet sizes when using UDP. I suggest you try setting it to 8192, and if you start getting timeout errors when transferring files then reduce it.

The pipelining option (which is disabled by default) allows file read and write requests to be pipelined which can give around a 20% increase in speed. However, with some servers or network setups it could make things slower. If, for example, your server is running on a 100bT segment while the client is the other side of a switch on a 10bT segment then it is likely to make things worse on a UDP connection, but if the server and client are of comparable speeds on the same segment then it may make things faster.

The server can also have a large influence on the speed. For example, if I read a 30MB file from the server it may take 14 seconds, but if I then read it a second time it take less than half that time because of the filesystem caching on the server.

The following table shows the maximum speeds I have obtained with various settings, and a comparison with LanManFS. If you are seeing significantly slower results then it may be worth experimenting with the above options. All the tests were copying a 68MB file using the filer with the faster option set. I think the server writing the file to disc is the limiting factor in the 100bT write tests.

LanManFS on Iyonix 100bTSunfish on SA RPC, Simtec 100bTSunfish on Iyonix, 100bT
Read2.4MB/s0.98MB/s5.6MB/s
Write1.8MB/s0.68MB/s1.8MB/s

Reporting Bugs

If you discover a bug, please report it to me, or better still fix it yourself and send me a patch. When reporting a bug, first load SysLog, then set the logging option in your mount file before doing the operation that triggers the bug. A log file should be generated in !SysLog.Logs.Sunfish. If you have access to a machine that can run a packet sniffer such as ethereal then it would be very useful to use that to save a dump of the network traffic along with the syslog file. Also, please provide a description of the problem and exactly what operation you were doing to trigger the bug, and details of what OS and NFS version the server is running would be helpful, and the version of the Sunfish module and RISC OS version.

Known Bugs

The leafname returned from an OS_File 0 call will be in Unix format and may still have a ,xyz extension. This is only used for printing *opt 1 info so is not crucial. The PRMs state that *opt 1 may not work correctly on RISC OS 3 onwards anyway.

Wildcards in filenames are not supported. This would be a lot of hassle to implement for little benefit, and in any case I believe that it really should be fileswitch's job to sort out wildcards, not each and every filing system.

Devices and other non regular files are not supported, as they don't map to RISC OS functionality.

If ShowHidden == 0 and you try to delete a directory that contains hidden files then it will fail with a directory not empty error.

Sunfish does not support programs accessing files from callbacks, and will give an error if this causes reentrancy. This could be fixed if the internet stack didn't require callbacks to be triggered before it notices new data has arrived.

Some operations are not as fast as they could be. For the first release I have concentrated on getting a functional and stable program, and there are some places where performance could be improved in the future. In particular, file attributes and mappings between filenames and NFS file handles should be cached, as this would reduce network traffic and delay for directory listings and many other operations.

Recompiling

Full source code is provided. To recompile the module, you will need Norcroft v5.53 or later, cmhg 5.42 or later, TCPIPLibs, and Perl. Earlier 26bit versions of Norcroft will not work because the code makes use of 64bit integers and some C99 functions.

For some unknown reason, if you use amu then it will give an error AMU: Don't know how to make 'o.pcnfsd-calls' after generating pcnfsd-calls.c, but if you rerun amu then it will work. GNU make does not give this problem.