Shareware Overload/ShartewareOverload.cdr/database/datamage.zip/CODE.ZIP/ISAM.TXT

Text File | 1990-08-20 | 16KB | 303 lines

                              D I S C L A I M E R


The  code included is provided AS IS, with NO WARRANTY WHATEVER.  If you  chose
to  place  it  into your programs YOU are responsible for  verifying  that  the
overall  program,  including  code furnished by HCWP, works.  I  will  bear  no
liability  OF ANY KIND for this code.  And I will MOST CERTAINLY NOT  guarantee
it to work in YOUR program!  All I can say is:  It works here, in mine.

DATAMAGE

The  draw_border function included with your sample places the DATAMAGE  banner
atop  the screen.  DATAMAGE is a registered trademark of Monte Ward  and  HCWP.
You may NOT use it in your programs!

THE CORE LIBRARY

The  module  of code called HCWP_LIB.C is the core library, and  has  it's  own
docs in the file: HCWP_LIB.TXT.

THE ISAM LIBRARY

The  module  of  code called MAGEISAM.C (and MAGEISAM.H)  are  the  C  language
interface to DATAMAGE files.  So far.  This module was compiled with the  QUICK
C compiler (2.5) and, save a few of those dern long/short mis-matches, compiled
perfectly.   If  you try to use another compiler, GOOD LUCK.  MIGHT  work,  but
you'll find dieeetomsbin and dmsbintoieee functions, which turn BASIC formatted
double-floats into ieee formatted double-floats don't exist in many  compilers,
so be prepared to do it yourself.

The  module  called MAGEDEMO.C uses SOME of the functions to  create  a  simple
program.   It's purpose is to provide a practical demonstration of the  use  of
the  ISAM,  and to provide a platform for your programs.  It is  NOT  a  slick,
finished product.  Merely a template.

WHAT IS AN I.S.A.M. ?

What we have, here, folks, is the GUTS of a database.  ISAM stands for  Indexed
Sequential  Access  Method/Manager.  In this module is the code  to  read/write
records, find records, load DATAMAGE files - the basics.

The DATAMAGE ISAM was designed for maximum speed, low disk/memory overhead,  to
make  optimum use of MS-DOS and, most importantly, to be easy to program  with.
This is FAR easier/simpler than dBase methodology, for instance.

DATAMAGE ISAM PARTICULARS

The files are flat and assessable to the BASIC interpreter.  So, you CAN  write
hybrid programs with it, if you must.  But, PLEASE do yourself a favor and  get
QUICK BASIC if BASIC is your thing.  See POWRMAIL.BAS.

For  those  of us who prefer C, the format of the files on  disk  makes  little
difference.    Whatever   it  is,  we  can  handle  it.   But   simplicity   of
design/operation  and  the  aforementioned speed and  LOW  OVERHEAD  are  still
valuable.   And,  of course, the accessibility of your files  to  the  DATAMAGE
system  adds  many features to your programs.  The ISAM is NOT designed  to  be
state-of-the-art, just to WORK (fast!) and to be used by ordinary human beings.

DEFINITION FILES

One BIG difference between DATAMAGE files and ordinary files produced by  BASIC
is that they have definition files.  In a "dedicated" program definition  files
aren't necessary - we know the details of the file and hard-code them into  our
program.  This necessary with some of the big-name ISAMs, like B-TRIEVE.

Other  files,  like  dBase, have the file definition "built  in"  to  the  main
datafile.  This makes it MURDER to process the file and almost completely  dis-
allows  BASIC  from  processing the files.  But, it DOES make  it  possible  to
transport  the file more easily.  To transport a DATAMAGE file you simply  copy
all the files in the sub-dir.  That's harder.

DATAMAGE  files have their definition in two files:  HEADINGS.SAD and  KEY.SAD.
The  headings  file has strings, maximum 25 chars, and the key file  has  three
numbers  in  it's 33 byte records, ascii format.  These files are read  by  the
open_file function, and the DATAMAGE file structure is filled.

From then on, the information about a particular field in a particular file  is
at  hand in the structure.  To find out if field four in file four  is  numeric
you might go:  if (fi [4].key [4] -> type > 1).  fi is the file-info structure,
subscript  gets  us file four (the FIFTH file loaded), .key gets  us  into  the
array  of far pointers to structures defining the fields, -> type gets  us  the
field type, which is defined in the code.

So, the DATAMAGE isam provides you with the power to load and process ANY  file
created by/accessible to DATAMAGE.  It's loaded with PRACTICAL features the big
boys  just don't have, like read/writing ANY individual file/field, instead  of
having  to  move seven thousand bytes to/from disk just to change one.   And  a
window that allows the user to select a field in a file, etc, etc, etc.

DATAMAGE ISAM - FORMAT AND PARTICULARS

SPECS

     RECORDS IN FILE:  32,000
     FIELDS IN FILE:  200
     MAX STRING FIELD LENGTH:  35
     ALL NUMERICS DOUBLE PRECISION, MBF FORMATTED 8-BYTE STRINGS
     MAX INDEXES: 5
     INDEX MODES:  UNIQUE, NON-UNIQUE, CROSS
     RECORD NUMBERS: MANDATORY, UNIQUE
     MAX FILES OPEN: 8

RECORD NUMBERS

Each record in a DATAMAGE file has a number.  It may be 1 - 32,727.  The number
is  stored  in the file CTRLFILE.RAD.  If the number is zero, the record  is  a
"hole" and is available for use.  No record number can be duplicated.

FILE INDEXES

DATAMAGE  indexes  are simple NUMBERS.  The targeted string is  turned  into  a
signed  word  and recorded in the file:  FILEINDX.RAD.  The  indexes  have  the
capacity  to  hold five two-byte integers, so the records in the file  are  ten
bytes  long.   This  SIMPLE method offers INCREDIBLE  speed  in  searching  and
updating, and is the easiest to understand BY FAR.

SEQUENCING

The  first record in YOURDATA.RAD, the main datafile, is numbered by the  first
record in CTRLFILE.RAD and indexed by the first record in FILEINDX.RAD.  So, if
we were searching in FILEINDX and found something we liked, we would know  that
the  record  to read in YOURDATA.RAD would be the SAME sequence  as  the  index
record  we  now had.  We could get a field out of it to compare,  or  even  the
whole record if we needed it.

Likewise,  if we went searching in the CTRLFILE.RAD and found a  record  number
that struck our fancy, we would know that it's record was the SAME sequence  in
YOURDATA.RAD as the record number we now had from CTRLFILE.RAD.

CUSTOMIZATION

The ISAM is provided as SOURCE CODE.  If you have a program to do that requires
only  two  files open you may find it convenient to open the  three  associated
files in each DATAMAGE file and just LEAVE them open during the program.   Feel
free to do so.  The ISAM is set up to process eight files, and thirty-two files
open  is  more than most folks allot for in their config.sys  file,  and  would
waste memory most of the time.

In a program you have you want it to process one file, and ONLY one file.   So,
you  won't need the show_datafiles function.  You will pass a constant  to  the
open_file  routine.   Feel  free to REMOVE unused  functions  from  your  final
version.  There's no sense having un-used code.

LAN OPERATION

You  will  need  to add your own record locking.  May I suggest  you  read  the
file's  record  from  the CRTLFILE.RAD file, multiply it by -1,  and  write  it
back?   That  way,  the main record could be "read-only."   Locked,  but  still
readable.  This bypasses the locking on the lan, and a LOT of grief!

PROGRAMMING EXPERTISE

I  can  but  assume that you are a better C programmer  than  my  humble  self.
I  am  just a hacker from Indiana who likes to punch these keys.   Anyhow,  you
SHOULD  understand  C  (if anyone REALLY does?) and  have  written  random-file
programs of your own, possibly incorporating an ISAM.

If  you are just learning C (aren't we all?) this AINT the place to start.   If
you  are  wanting to learn random-file processing and already know C  you  will
find the code invaluable.

Whatever your level of digital dexterity you are going to have to THINK to  get
this.   There are remarks in the code, but they are "slim."  You will  need  to
read the function's code to figure out what global variables it takes as input,
and changes as output.  If I listed this you wouldn't read the code.

NOTHING NEW

After  programming for a number of years I realized that the  really  BEAUTIFUL
solutions  all had only one common ingredient:  SIMPLICITY.  And  the  DATAMAGE
ISAM,  pretty  or not, has a liberal dose of that.  This is the way  it's  been
done for YEARS, folks, on many a multi-million dollar mainframe computer.

All  I've done is to simplify it, adapt it to a PC, and add my indexing  scheme
which  I thought about FOR YEARS while using convoluted and  overhead-intensive
alternatives.   My  way may be described as brute-force computing, but  I,  for
one, can do without the errors that "finesse" often generates.

DATAMAGE INDEXES - THE BRUTE FORCE THEORY

As  the  hardware  progresses  BRUTE FORCE works  better  and  better.   Modern
machines  running at 20 Mhz with RLL controllers on FAST hard drives,  possibly
cacheted  with smartdrv etc, might as well be SLOW ram.  While other ISAMs  are
half-way  up  their B-trees I will have checked a thousand records  with  brute
force.  They say:  "I'll find the last record as quickly as the first!"   Isn't
that like saying:  The first record takes as long to find as the last?

You  may  have noticed that the RAM_BASE program within DATAMAGE  will  find  a
record  in a hurry.  That's because it can load up to eight  thousand  record's
indexes  into 80K of ram.  Then, it does index searches without  bothering  the
disk drive.  In this case, I will have checked FIVE thousand records with brute
force before the B-tree is clumb.

In  either case, all I need to do to update an index is to decode  the  string,
turn it into a number (routines supplied) then write it into the index file  in
the  proper sequence.  The write_MAGE_record function does this for you.   And,
from it, you can easily see how to do it yourself.

BRUTE  FORCE programming provides both the programmer and the computer  a  rare
opportunity  to run flat out.  You don't spend a lot of time reasoning;  that's
not what a computer does well, and reasoning introduces the opportunity to err.

Just target 10 bytes, gotten off disk or in ram, compare one or more of then to
search values, repeat if not equal and not end of file.  THAT's what a computer
does:  many FAST repetitions of some MINDLESS operation.

BRUTE FORCE, when well-directed, often results in the lowest overhead to do the
job.   DATAMAGE  indexes  are just ten bytes per record,  and  can  cover  five
fields; any/all of which can be unique or cross.

I  have often seen data "sets" (consisting on one or more files) in  which  the
indexes  took  up  FAR  more disk space than the  data.   And  the  PROGRAMMING
overhead  often  equaled/surpassed  the actual procedure  of  the  program  and
entailed linking in object files for which I had no code or using code I didn't
understand and couldn't have fixed if my life depended on it.  And, for what?

GETTING STARTED

In  the first couple screens of the MAGEISAM file the structures: fi  (defining
DATAMAGE files) and finfo (defining fields within DATAMAGE files) are  defined.
Copy these definitions to a piece of paper, then load the MAGEISAM.C file  into
your  compiler.  Search for open_file.  Read the function, while  referring  to
the  paper.  Dig it.  All the information on the loaded file is there, but  YOU
will have to write the if statements to check it, and your understanding of the
ISAM depends on "C-ing" these structures.

Once you "C" the open_file function the rest should become crystal clear.

The open_file function uses two of it's own FILE *s.  Such is not the case with
the majority of ISAM functions.  In any environment that has up to eight  files
open,  requiring the use of three FILE *s each, there's going to be a  shortage
of IOCBs.  And, it's FASTER to test a file to see if it's open than to (attempt
to) open it.  LOTS faster.

So,  then, HOW to test a file for open status?  LOTS of ways to do  that,  too,
Many of which involve linking in and calling yet another function.  Got lots of
THEM,  now.   I decided to set the file pointers in the fi structures  to  NULL
during the load, then set them to NULL every time I closed them.  That's  EVERY
TIME, hear?  The first time you don't, you'll be SORRY!

The  ISAM functions test the files, then open/close them if they  were  closed.
If you don't set the FILE * to NULL after closing a file the next ISAM function
you call on that file will "think" it's open.  NOT good.  Not to mention trying
to close it, when it's not open.

A "cheapie" way around this is to use your own FILE *s.  NOT recommended.   Use
mine.  Like I said, they're in short supply.  Let's say you have the main  file
open and you call write_MAGE_rcd.  Stipulate further that there are no  "holes"
in  the file, and a record is ADDED.  But, it was added "to" the FILE * in  the
fi structure, which was closed.  YOUR file pointer doesn't "know" it's there.

What  will actually happen is dependent on your DOS version,  your  controller,
etc,  etc.  It's a chance you DON'T need to take.  I have seen it  corrupt  the
fat, (yeah, I make mistakes, too) and maybe it could do worse?

THE files_loaded VARIABLE

Every  time you load a file you MUST increment files_loaded.  And decrement  it
when  you close it.  It's defined in the first screen of code.  If you wish  to
add this to the open_file function, by all means.  But I sometimes read it  off
disk, then use it as comparitor for a for loop when loading files.

If  the files_loaded BYTE is set to a value not equalling the number  of  files
open,  expect trouble.  The files are opened/closed IN ORDER.  You  COULD  open
file #8 first, but WHY?  I never have, so I don't know if it will work.

COMPILING

The ISAM is contained in a module of code.  Most all of it's functions are FAR,
so that they can be called from other modules.  This means you will need to use
the medium memory module when compiling.  NO, huge won't work.

The  ISAM  calls many of it's own functions.  These functions are in  the  same
module  as the calling function, but their definition requires them to  be  far
calls.  Far calls take longer than near calls, and occupy more code space.

To rectify the catche-22 above, the linker has a /F option.  This converts  all
far calls to functions that occur in the same module to near calls.  Saves time
and memory.  To use it, type /F after link/qlink.

WRAP-UP

Yeah,  I could have compiled it all to object files and simply made a  list  of
functions,  their parameters and returns.  I always like to be different.   YOU
may  have to do a little work, but you will actually understand the ISAM  after
you do.  You can even change it, if you DARE.

It's  true  that only a real programmer can understand the code.   And,  in  my
opinion,  just  as well.  If you can't read and understand the code  you've  no
business writing this kind of program in C.

If  you think you might, but avoid dis-appointment by not playing -  you  LOSE.

If you're just plain too lazy to read the code and, thereby, understand it  and
be,  therefore,  able to modify it - you LOSE, too.  But there  are  PLENTY  of
high-tech, object-file (and even tsr) ISAMs to fill your "need."  And, all  you
have to do is to link them in, call them like THIS, and they return like  THAT.
You won't be able to access their files with DATAMAGE, though.

It's fine, until you need to "call" them a way they don't call, or get a return
they  don't return.  If you are satisfied with limiting your programs, in  both
scope and performance, to their capabilities then your standards are lower than
mine.  I like to take PRIDE in MY work.