World of A1200/World_Of_A1200.iso/programs/compress/misc/xdata/xdata.doc

Text File | 1995-02-27 | 14KB | 336 lines

                                xData v1.1
                             xDataPrefs v1.1a

                         (C) Martin W.Scott, 1993.

                          (NB: Requires OS 2.0+)


xData  is a program that allows other programs to read xpk-compressed files
as  if  they were normal files.  It is similar to PP by Michael Berg, which
provided a similar facility for PowerPacked files.

However,  PP  caused  problems  on  my  system  (it  crashed),  and anyway,
PowerPacker  has  been  superseded  by the more powerful and extensible xpk
standard.   Unfortunately,  it takes a long time for data-handling programs
to  catch-up  and recognise the xpk standard, not to mention those programs
that can't read any compressed files.

The  xpk  system  is  distributed  with  a  handler,  xfh,  which  allows a
pseudo-partition  to  be  set  up  to  handle  xpk'd  files in a completely
transparent  manner.   Whilst  this  method  is  always  going  to  be more
system-friendly than the patch method employed by xData and PP, it isn't as
user-friendly -- all your packed files have to be kept in the one place.  I
didn't  want to install the xfh handler, so wrote xData.  And remember that
the  xpk  system handles the decompression of PowerPacked files too, giving
increased flexibility to xData.

I  have  not  included the xpk libraries.  If you haven't got them and like
what  you've  read  so far, you should obtain the complete xpk distribution
(by ftp or fish disks).


How does it work
----------------
xData  patches  a  few  dos  routines to allow named programs to read xpk'd
files.   These named programs are called clients.  This is different to PP,
which  affected  ALL programs, and caused many incompatability problems (on
my system at any rate), which is why I decided on the client model.

Clients are added using the xDataPrefs program; when a client opens a xpk'd
file, the file is decompressed to the t: directory and the client reads the
decompressed  file.  When the client closes the file, the temporary file is
deleted (but it's not quite that simple -- see technical details below).

If  a  client  examines  the length of a compressed file, the xData patches
actually  tell  it  the  uncompressed  length (in case the client allocates
memory for the whole file to load into).

This client arrangement might seem restrictive, but it guarantees (though I
don't  :) system compatability.  Most users only compress certain files and
use certain programs to read them, so these are the only programs that need
to use patched routines.  It allows you to use your favourite data-handling
programs  rather  than  xpk-ified versions, e.g.  you can use ViewTex where
before you had to use ShowIFF or PPMore (for PowerPacked files).


Starting xData
--------------
Best   done   at  startup.   Can  be  run  from  Shell  or  Workbench,  and
auto-detaches  if  run  from  a  shell.   You  can  put  it  in  WBStartup,
user-startup   or  startup-sequence.   Since  it's  sort-of  a  preferences
handler,  I  put  it after the IPrefs command in the startup-sequence.  The
only restriction is that it must be run after ENV: has been set-up.

xData  is  a  commodity,  so it may be deactivated and reactivated with the
Commodities  Exchange.   If the patches cannot be removed (because a client
has a compressed file open), the user is notified.  xData may be terminated
from the Exchange too (or by re-running it).

NB:  I've  noticed  one  or  two problems disabling xData while clients are
manipulating  files, because they open/examine them repeatedly, and turning
the  patches  off in the middle of all this can confuse them.  xData checks
that all its files are closed, but can't anticipate these sort of problems.
However,  there is no problem in disabling/enabling xData while clients are
not busy opening files.


Editing Preferences
-------------------
Preferences are edited using the xDataPrefs program, which is based closely
on standard Workbench 2.1 preference editors.  All the usual menu items are
there,  and  the  gadgets  work as you'd expect.  Only the CLI interface is
lacking.  From the CLI, xDataPrefs takes either no arguments (in which case
the edit window is opened) or one argument, a preferences file to use.

On  starting  xDataPrefs,  you will see two lists: the left-hand one is the
list  of  clients  xData currently recognises, and the other is the list of
processes running on your system.  If you wish to add as a client a program
that  is  already  running,  look for it in the process list, select it and
click on the Export gadget.  It will then be transfered to the client list.

The process list does not update regularly -- you must click on the Refresh
gadget to update the list.

Other  new  clients are added with the Add gadget (surprise); pressing this
results  in  'New  client' being added to the list.  Press right-amiga-x to
clear  the  string  gadget  and  enter the new client's name (see below for
details  on  names to choose).  You would use this method if the client you
wanted wasn't running.

A sample client list is

    DPaint
    ViewTex
    More
    MuchMore
    Display
    CygnusEd
    egrep

etc. Case is not significant.

The  way  names  are given to processes on the Amiga is non-trivial.  There
are three general classes:

1. Workbench Processes

    These  are  the  simplest, and the process name is the name of
        the  icon  used  to  start  it.   e.g.  double-clicking on the
        DPaint icon creates a process called 'DPaint'.

2. Shell Processes

    Process  names  are  generally  non-descriptive,  e.g.   Shell
        Process,  but the actual command a shell process is running is
        stored  elsewhere,  and  xData  looks there.  The names stored
        there  vary depending on how the command was invoked, and what
        Shell  you're  using,  but  all you need to worry about is the
        command  name,  i.e.   the file part, not the path part.  e.g.
        from the shell, the process started by the command

        'run work:gfx/dpaint'

    will  match  the  client entry 'dpaint', even if the full path
        and command is stored in the process.

3. Custom Processes

    These  are  processes created by programs which are not of the
        first  2  types.   For  example,  I  would type 'ced' to start
        CygnusEd  from  a  Shell, it actually creates a process called
        'CygnusEd',   so  to  enable  the  editor  to  load  xpk'd  or
        PowerPacked files, a client called 'CygnusEd' must be added.

So,  the  names  to  use  as  clients  are  just  the names of the programs
themselves,  with  no  paths. Using the process list, you can usually find
the correct process name to add to the client list.


Problem Examples
----------------
A  couple  of  examples I have come across, that may also be instructive in
other situations:

PROGRAM: ADPro
PROBLEM: Doesn't recognise xpk'd files when 'ADPro' is a client.
SOLUTION:  ADPro  uses sub-processes to operate on files, including loading
and  saving  them.  The name of the sub-process used to load a file depends
on  the  file's  type.  i.e.  when loading an IFF file, a process with name
'IFF'  is  created.  Therefore, adding a client called 'IFF' will allow you
to  load  xpk'd IFF files.  There is no need to make 'ADPro' a client, just
the relevant loaders.

PROGRAM: DirWork
PROBLEM: Seems to determine what xpk'd files are, but can't load them.
SOLUTION:  DirWork  program  spawns  separate  processes  to  view text and
pictures,  so as well as adding the DirWork command to your client list (so
that  DirWork recognises crunched files as IFF etc.) you must add the names
of  its  spawn  processes  (which  do  the loading).  A quick glance at the
process  list  shows  that these are called 'DW - More' and 'DW - Show', so
adding the clients

    DirWork
    DW - More
    DW - Show

makes DirWork a fully-working client.

PROGRAM: xyzprogram (a dummy example)
PROBLEM: Doesn't work as client (name is definitely right)
     OR: Crashes system when made a client
SOLUTION: Don't add it as a client.

That last one isn't meant to be a joke.  If a program doesn't work when you
think  it  ought  to, you'll just have to live with the fact, or contact me
with details and I'll see what I can do.


Notes and Caveats
-----------------

   o    When  a  client  saves over a compressed file, the new file is
        not  recompressed  (I may extend xData's capabilities, but for
        now, recompression is unimplemented).

   o    If  a  client  copies  a  file (e.g.  DirWork) the destination
        version  is  actually  uncompressed.   Non-clients  copy files
        without  this problem, of course.  This is a pain in the ar*e,
        but  there's  not  a  lot  I  can do about it (short of adding
        mind-reading  capabilities  to  xData).  If this really annoys
        you,  don't  make the offending program a client.  In the case
        of  DirWork,  if  you  have buttons to load text or view pics,
        selecting  these  will  start the subprocesses without DirWork
        needing  to know the file types, so you can remove it from the
        client  list,  just  leavine  'DW - More' etc.  You have to do
        without the auto-recognition if you do this though.

    I've  recently  been  using  the  new BrowserII, and it's type
        recognition  system  is powerful enough to detect not only xpk
        files, but also what they are, WITHOUT being made a client.

   o    Xpk'd executables will not be auto-decompressed when loaded by
        clients  (for  example,  binary  editors).   This  is to avoid
        conflicts  with xLoadSeg, but shouldn't be a problem (the only
        instance  I  can think of when a program would want to load an
        executable  is  the binary editor example).  However, xData is
        completely compatible with xLoadSeg.

   o    Remember  that  the  xpk  system handles PowerPacked files, so
        xData does too.  (So if you run PP, you can junk it).


Hints
-----
To temporarily remove an client from the list, start the preferences editor
and  prefix  the  client name with a ';' (or something like that).  It will
then fail to match program.


Credits
-------
xData and xDataPrefs are written entirely in C, compiled with SAS/C 5.10.

The gadget imagery was created using the excellent GadToolsBox.

The system patch method is based on a Commodore-supplied example (but done
in an cache-friendly way).

The xpk compression system is the creation of Urban Dominik Mueller.

PowerPacker.library is copyright Nico François.


Technical Details
-----------------
Here's  a brief explanation of what xData does.  When a program attempts to
Open()  a  file  for  reading,  xData  checks its client list to see if the
caller  is  a client.  If so, the file is checked to see if it is xpk'd (or
PowerPacked),  and  if it is, the file is decompressed to the t: directory,
under  a  unique  name.   Then xData opens the decompressed file and passes
that  handle  to  the  caller.   Normal calls to AmigaDos functions Read(),
Seek()  etc.   result in actions on the decompressed file; when the file is
Close()ed, xData checks whether or not the file was decompressed, and if it
was,  notes  that  the  file  is  closed; it will then be really closed and
deleted (at least) 5 seconds later, by the main xData task.  The reason for
postponing  the deletion is that many programs check what the file contains
(by opening it), close it and reopen it for use.  Strange though this is (a
Seek()  would  be  better  in many cases), it does lead to compressed files
being   decompressed   twice   (or  more),  which  slows  down  performance
significantly  in  the  case  of large files (and on slow processors).  The
postponement is for a period of 5 seconds, which seems a reasonable tradeof
between speeding up reopens and keeping memory use down. Examples of
programs that sometimes open twice are ViewTex and Deluxe Paint.

If a client calls Examine(), xData checks if the file is compressed, and if
so,  returns  its  decompressed length.  It determines the length by making
some assumptions about the format of an xpk'd (or PowerPacked) file, rather
than  decompressing the file.  This results in minimum overhead (though the
overhead  is greater than an ordinary Examine(), because the file has to be
opened  and  partially  read.   The assumptions about file format shouldn't
cause  problems  with  future  versions of the xpk system, as the format is
very unlikely to change.  However, I shall keep in touch with the author of
xpk, Urban D. Mueller, to avoid future compatability problems.


History
-------
 v1.1a    - xDataPrefs: fixed stupid memory bug (wasn't freeing task-list).

 v1.1    - xDataPrefs: major enhancement - now has task list so addition of
          new clients is simplified.

    - xData: now postpones deletion for 5 seconds (for reasons why, see
          technical details above).

 v1.0a    - xDataPrefs: fixed bug when entering name on empty list.

    - xDataPrefs: window now font-sensitive.

 v1.0ß    - initial release.


Legalese and Distribution
-------------------------
It  seemed  fitting (and safe) to use the same disclaimer as the xpk system
has, so here it is, pertaining to the xData package:

This  package  may  be  freely  distributed,  as  long as it is kept in its
original,  complete,  and  unmodified  form.   It may not be distributed by
itself or in a commercial package of any kind without my permission.

These  programs  are  distributed in the hope that they will be useful, but
WITHOUT  ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
or FITNESS FOR A PARTICULAR PURPOSE.

All  of  that  said,  I spend a lot of time writing and polishing stuff for
release,  and would appreciate any monetary contributions if you find xData
useful.   None  of  my  work  is  shareware,  but  do believe people should
acknowledge others' work if they rate and use it [end of plea].


Contact
-------
I can be reached with comments, suggestions, bug reports, praise, money etc.
at the following addresses:

BEFORE END OF JULY 1993:

    Martin W. Scott,
    23 Drum Brae North,
    Edinburgh  EH4 8AT
    United Kingdom

AFTER END OF JULY 1993:

    Martin W. Scott, c/o
    557 Great Western Road,
    Ground flat right,
    Aberdeen  AB1 7PA
    United Kingdom

OR BY EMAIL UNTIL END OF JULY 1993: [e-mail]