The Datafile PD-CD 3/PDCD_3.iso/internet/tcpip204/TCPIP_Doc/Notes/FTPClient

Text File | 1995-03-03 | 20.5 KB | 632 lines

New facilities in FTP Client (v2.04)
=====================================

The FTP client in !TCPIP has undergone the first stage of improvements
aimed at sorted out UNIX -> RISC OS path convertion problems, and making
it a bit more automated.

These include the addition of:

  A highly configurable path name processor.
  A batch ftp facilty (source command).
  An "mget" command.
  A "reget" command.
  "quote" and "site" commands.
  {ftp_data} and {ftp_list} variables.

v2.04 also adds:
  Preset options, set from main command window, or Config file.
  Fully automatic login based on data in an FTP server accounts file.
  Automatic execution of command files.
  Selectable prompt types, showing current directory.
  Selectable information level.
  A new ftp command "aftp" that allow instant logins, and optional
  command file selection.


Pathname Processor
~~~~~~~~~~~~~~~~~~
The path handling is the main area of improvement, and now allows UNIX
style paths to be parsed for file type extraction from one or more file
name extensions and is able to work out appropriate processing based
upon the extensions present or elements in the basic file name.

On the basis of the file name and/or extensions, the ftp client can now
work out what file type to give the file, which directory to store it in
and whether to keep unconverted extensions, whether to keep directory
elements from the original remote path etc.

When there is more than one extension, it is able to sensible use one
extension to determine the file's type and another extension, or the
file's name, to determine in which directory the file should be placed.

These conversions are done according to a set of rules given in a new
!TCPIP resource file. (!TCPIPUser.PathEnt).

When downloading groups of files, for example, the entired contents of a
directory and subdirectories containing C source files, there are often
files which might end up being placed in a different directory to the
rest as a consequence of name/extension base placing. As this can be
undesirable, it is posible to switch off the fully automatic processing,
and force preset processing according to the type of download your are
doing. This is most useful when using the new batch facility.

Finally, just in case there are any problems that I have missed, the old
original path name handling is still available in a rather more useable
and improved form.

Whatever path processing options are in use, directories are
automatically created. The lack of this was one of the most annoying
problems with previous versions of the FTP client. Also filenames are no
longer actively truncated, allowing advantage to be taken of such
utilities as !LongFiles. With appropriate Risc OS configuration, overly
long filenames will still be truncated, but by Risc OS rather than the
FTP client.


PathEnt file
~~~~~~~~~~~~
The !TCPIPUser.PathEnt consist of a series of either options lines, or
name matching lines. When trying to match a name or extensions, three
types of matching are carried out. These are:

  1.  Match the file name to determine where to put the file.
  2.  Match an extension to determine where to put the file.
  3.  Match an extension to determine a file type.

The first match in each of the above categories is the one used. How
this effects the layout of the PathEnt file will become clear.

Each options line starts with a "$" character and is followed by
required options on the same line, each separated by a space.

There are three types of options line as follows:

$root <pathname>
"$root <FTP$Dir>"

This must be present, once only. The given pathname set the root
directory in which files are to be placed. Normally, this is set to
"<FTP$Dir>" which in turn is set to "!TCPIPUser.Downloads".

$default [<option>...]
Example: "$default nocase nfsext truncdir keepext in Misc type &fff"

This line must be present, once only. It sets the default path
processing options this are modified by options given later, or applied
directly to any paths that are not otherwise matched.

$ [<option>...]
Example: "$ rev1ext in Source"

This line contains options that are to be applied to subsequent
extension/filename decoding lines. Any options given are applied in
addition to those given in the default options line.

Lines starting with a "#" character, or blank lines are ignored.

Lines starting with a "." describe extensions matching. Other lines
describe filename matching.

Extension and file name matching lines are as follows:

<name>/.<ext>[|<name>/.<ext>...] <file type> [<options>]
Examples: ".c|.cpp|.c++|.h|.[1-8]|.s|.y|.l &fff"
          "rfc* &fff in RFCs"

"/" in the above is used to represent "or" in the syntax rather than the
usual "|" as "|" is literally used to separate extensions in a list.
There must be no spaces in the extensions part.

The use of wildcards is allowed in the name and extension as follows:

  "*"     matches any string of 0 or more characters.
  "?"     matches any single character.
  "[...]" matches a range of characters, for eg, [0-9] matches 0, 1, 2,
          3 ... 9.

The file type is a valid RISC OS file type, for example, "Text" "fff",
"&fff" etc.

Addition extension specific options may also be given by including the
option after the file type.


Options:

  nocase    Treat upper and lower case alike in matching

  keepext    Keep extensions as file/ext1/ext2 etc

  noext        Remove all extensions

  revext    Reverse extensions so that dir/file.a.b becomes
        dir.b.a.file

  rev1ext    Reverse 1st extension only, so that app/file.c.gz
        becomes app.c.file/gz

  nopath    Strip path prefix for remote name so.

  in <dir>      Places file in directory <dir>. Unless <dir> is a full
                path name, <dir> is assumed to be a sub-directory of the
                specified root directory.

  type <type>   Set the default type to be applied in the absence of any
                other information.

  nfsext        Enables the handling of Acorn NFS style ,xxx hex extensions.


Any option prefixed with "!" is switched off rather than on.

When a type is matched, the corresponding extension is allways removed.


An Example File


# Set the location of the downloads directory.

$root <FTP$Dir>


# Set default options as follows:
# case-insensitive matching, handling of NFS extensions, keep
# extensions as file/ext/ext unless told not to,
# place unmatched file in "<FTP$Dir>.Misc"
# set their type to Text (&fff)

$default nocase nfsext keepext in Misc type &fff


# These option are applied to the following type matches until
# another options lines is encountered. These options are applied in
# addition to those given on the defaults line above.
#
# Ensure directory elements in original remote path name are stripped.
# Ensure extensions are stripped, leaving just the filename.
# Place matched files in directory "<FTP$Dir>.Docs"
$ nopath noext in Docs
.txt         &fff
.doc         &fff
.asc         &fff
.ps          &ff5


# A new set of options to be applied. This completely replaces those
# given in the previous options line, and is applied to the following
# match lines.
#
# Basically as above, but put files in "<FTP$Dir>.Archives"

$ nopath noext in Archives
.arc         &ddc
.archive     &ddc
.spark       &ddc
.spk         &ddc
.tar.gz      &ddc
.tar.z       &ddc
.tar         &ddc
.gz          &ddc
.z           &ddc
.zip         &ddc


# A new set of options for Image files
# The files end of in "<FTP$Dir>.Archives"

$ nopath noext in Images
.jpeg|.jpg   &c85
.gif         &695
.tiff|.tif   &ff0


# A more complicate extensions set for dealing with source file downloads.
#
# this matches .c, .c++, .cpp, .h, .1, .2, .3, .4, .5, .6, .7, .8,
# .s, .y, and .l.
#
# The first extension and file name are swapped round so that you
# get c.*, h.* files.
#
# Also, as these options *replace* previous ones, directory elements in
# the original path name are retained. So applic/source.c becomes
# applic.c.source.

$ rev1ext in Source
.c|.cpp|.c++|.h|.[1-8]|.s|.y|.l &fff


# Any files with "rfc" at the start of the name are placed in
# "<FTP$Dir>.RFCs". Any files with "draft-" at the start of the name are
# placed in "<FTP$Dir>.IENs". Also path elements and extension are
# stripped.
#
# Note, that the following two match lines don't have a "." prefix, so
# they match the name rather than the extension. Also note the use of
# options in the match lines.

$ nopath noext
rfc*         &fff in RFCs
draft-*      &fff in IENs


Hopefully this commented example will now enable you to create you own
additional entries to suite your exact needs.


Path Command
~~~~~~~~~~~~
Syntax: path off | auto | type | <option> [<option>...]

"path" on its own tells you what state the path processor is in.
Separate state is maintained for each active FTP client session.

"path off"  switches the path processor off, result in the use of the old
path handling.

"path auto" switches the path processor on and causes it to use
information given in the "PathEnt" file as described above.

"path type" switches the type decoding only on and causes it to use
information given in the "PathEnt" file as described above.

"path <option> [<option>...]" presets a set of options independantly of
the PathEnt file to be applied to subsequence "get" operations.

The option given are exactly as per a PathEnt file options line, but you
dont need to prefix with the "$" and in fact you must not.

Example as used when fetching a the entire contents of a C sourrce
directory:

path nocase rev1ext in <FTP$Dir>.Source type Text

(See "batch" command for a real in-context example)

Notes:

The "in <dir>" option has a full name specified as the given directory
in this case is not treated as a subdirectory of a specified root
directory.

The path processor, when active, will allways be able to decode RISC OS
filetype name extensions as appended by !TCPIP's FTP server in unix
mode.

However, in order to ensure paths are kept exactly as on the server, use
the "path ros" command.


SOURCE command
~~~~~~~~~~~~~~
This causes command to be taken from a pre-prepared file rather than
from user input. Any command that can be entered manually, may be placed
in an FTP command file, except the source command itself.

Syntax: source <filename>

Various directories are searched for a file matching the name given. In
order these are:

  !TCPIPUser.Scripts
  !TCPIPUser
  !TCPIPUser.^
  $ on current drive
  @ (the current directory)

If it cant be found, then obviously an error is given.

Note: only one command file may be active. The command file may not
      currently conatin another source command.


MGET command
~~~~~~~~~~~~
This, like "get" is used for retreive files from an FTP server. Unlike
"get", "mget" can be given a short list of wildcarded names that are
each expanded into a file list, with each file in the resulting list
being fetched via "get".

Syxtax: mget <listspec> [<listspec>...]

Its operation is equivelent to entering ls <listspec> for each
<listspec>, and then doing an "get <file>" for each filename returned,
and in fact this is exactly what it does on your behalf, as you will see
from the information display during an mget operation.

For example, to fetch the entire contents of two directories on a server
"dira" and "dirb", you could use "mget dira dirb". To fetch the contents
of current directory, and immediate subdirectories, use "mget *".

Another way of using "mget" is to type ls at the "ftp> " prompt, and
then select some files from the resulting listing (upto around 50 or so)
and then apply the preset macro menu "Get file(s)".

Before using "mget", it is worth ensuring that suitable path processing
options are setup with the "path" command.


SITE command
~~~~~~~~~~~~
This command allows a the normal FTP client command interpreter to be
bypassed, ensuring that an entered command is passed directly to the FTP
server for execution.

Syntax: site <command>



QUOTE command
~~~~~~~~~~~~~

This allows a command to be sent directly to an FTP server, bypassing
the normal client pre-processing of commands, just like the site
command. The difference between quote and site is that quote sets up the
FTP data channel, and so must be used when the underlying command is
expected to return data via the data channel.

Syntax: quote <command> [<local file for listing>]

Note: as the command to be sent to the server will often need to contain
spaces, you will often have to enclose the command in quotes as above.

<listfile> should only be specified in situtations where output is
expected via a data connection.


REGET command
~~~~~~~~~~~~~
This command allows a get operation to be continued after a previous get
operation that was terminated early for some reason, for eg because your
internet connection was broken.

Syntax: reget <remote file> [<local file>]


WINOPEN and WINCLOSE
~~~~~~~~~~~~~~~~~~~~
These open and close the FTP session windows.

Syntax: winopen
        winclose

The main use of this is in FTP command files that may be triggered by an
automated FTP session initiated from another wimp application, in
particular, a FTP fetcher for WWW would use this.

When a session window is close, and the quit command is executed, the
window will be silently discarded after the session closes without
further user intervention.

Normally, upon completion of an FTP session, the user is required to
close the window manually.


TRACE command
~~~~~~~~~~~~~
This command switches on full display of server replies. Normally, since
v2.04, most replies are filtered so that you only see those that might
be of interest. Setting the trace level to 1 ensures that you see the
full information returned from the server.

Syntax trace [<level>]

  Level = 0 strips command codes so you only see the informatiom content
          of relevent returned messages, and dont see any messages during
          data channel usage.

  Level = 1 causes all reply messages to be displayed in full.

  trace on its own displays the current setting.


PROMPT command
~~~~~~~~~~~~~~
With version 2.04, you can one of three differnt type of prompt,
standard, short directory and long directory.

Syntax: prompt [nodir|short|long]

  nodir selects the standard "ftp> " prompt to be used.
  short selects the leaf name of the current directory path to be used.
  long  selects the full directory path. If the full directory path is
        longer than 16 characters, as it often is, then it is diplayed
        on a line on its own, followed by "> " on the next line.



Other New Features
------------------

It is now possible to present some option for the ftp client with
command that may be placed in the main Config file, or entered at the
"net>" prompt in the main command window. These options allow selection
of terminal window size, default pathname processing, default trace
level, default prompt type, default transfer progress indication (hash)
and default tranfer mode.


FTPOPT command
~~~~~~~~~~~~~~
Preset various FTP options.

  Syntax ftpopt <option> <values>


Set Default Path Processing

  Syntax: ftpopt path <path options)

  This sets the default path processing options for each new FTP session.
  For a description of path options and the path command, see "FTPClient".


Set Default Window Size

  Syntax: ftpopt winsize [<n1>] [<n2>] [<n3>] [<n4>]

  If no parameters are given, then the current size is shown.

  When one or more of <n1> to <n4> is given, they are interpreted as
  follows:

    <n1>                  n1 is terminal height in rows.

    <n1> <n2>             n1 is terminal width in rows
                          n2 is terminal height in columns

    <n1> <n2> <n3>        n1 is terminal width in rows
                          n2 is terminal height in columns
                          n3 is window height in rows

    <n1> <n2> <n3> <n4>   n1 is terminal width in rows
                          n2 is terminal height in columns
                          n3 is window width in columns
                          n4 is window height in rows

   "terminal" in the above refers the total window work area, "window"
   just refers to the visible part of the window.


Set Default Prompt

  Syntax: ftpopt prompt nodir|short|long.

  Values, exactly as per FTP prompt command.


Set Default Progress Indicator

  Syntax: ftpopt hash 0|1|<lump size>

  Values, exactly as per FTP hash command.

    0 is no indication.
    1 exact byte count.
    >1 a "#" is displayed for every n bytes, where n is the value
    specified. Suitable values of n are 512, 1024 (1Kbyte), 2048 etc.


Set Default Trace Level

  Syntax: ftpopt trace <level>

  Levels exactly as per FTP trace command.


Set Default Tranfer Mode

  Syntax: ftpopt type image|binary|ascii

  image and binary are the same. This is the mode that you should use
  for all file transfers unless you are sure that the file content is
  text, even then, it is worth using binary mode as it is optimsed for
  low overhead, and so is faster than ascii mode.

  ascii mode cause end of line characters to be matches for the local
  system. ie CRs are stripped for files.



AFTP and Accounts file format
-----------------------------

AFTP
~~~~
"aftp" is a new alternative command to ftp <site name>. The current ftp
command accept a collection of terminal switches, that with the new
preset, are not really needed. Also the new command file facility in FTP
means that dialogues are not really needed, though are still useful.

"aftp" is much simpler for everyday use, and the effect of many auto-login
dialogues appended to the ftp command can be very much more simply be
acheived using aftp and a suitably setup ftp accounts file. While very
much simpler, it is actually rather more useful, as it exploits the
newly added facilties.

  Syntax: aftp <hostname>|<alias> [<command file>]

<hostname> is of course a normal internet hostname, or IP address.
<alias> is the name of an entry in the FTP accounts file. This file is
searched for a matching record that contains the full hostname of the
host, the login name, password, name of menu to attach to the session
window, an optional command file to execute and the name of the
directory you want to be in.

Distinguishing between hostnames and aliases is very simple. An alias
has entry in the accounts file, a hostname does not, instead, it gets
its login information from a "default" entry in the accounts file.

This means that if the name entered is not found, then the default
record controls the login.

<command file> is the name of an optional command file you wish to have
executed upon login. If a command file is specified in the accounts file
for this alias, the file name given in the aftp command is executed
after the preset command file.


Accounts File
~~~~~~~~~~~~~
This file is found in !TCPIPUser.FTP, and has a very simply record
structure as follows:

<alias>:
  host <hostname>
  login <username>
  password <password>
  menu <menu file>
  directory <directory name>
  source <command file name>

<alias>:
  ...

The start of a record is identified by a line with text and NO preceding
spaces, followed by a colon. Fields within each record have a field name
and field value. Fields must have at least one preceding space or tab
character.

Valid fields are:

  host <hostname>
    <hostname> is the full nameof an internet host, for eg
    "ftp.demon.co.uk". You may put an IP address here if you wish.

  login <username>
    <username> is just that, your user name, for example "anonymous".

  password <password>
    This is the login password, for example, for anonymous ftp, you would
    use your email address. Note, that you may use !TCPIP internal
    variables in thif file, so variabe such as {ftpuser} and {host} that
    have been setup for macro menus may also be used here. For anonymous
    login passwords, I use the entry {ftpuser}@{host}.

  menu <file name>
    This is the name of a macro menu file. !TCPIP expect to find this
    file in !TCPIPUser.Menus.

  directory <directory name>
    This is the name of the directory you want to be placed in, as would
    be entered with a "cd" command to that specific host.

  source <command file.
    This is the name of a preset command file. This useful for actions
    that you will allways want to have done everytime time you connect
    to this host.

You can alse set of a record with the name "default:". Values specified
in this record are applied when the hostname given in the "aftp" command
does not match any given in the file. You should not include a "host"
field in this record as it will obviously prevent you accessing the host
specified in the aftp command.

Any field may be ommitted. In particular, you may wish to omit the
password field for reasons of security. If no user name or password is
given, then you will be prompted for these when they are required as
normal.