C!T ROM 5/ctrom5b.zip/ctrom5b/PROGRAM/BASIC/BLTQ121/!README

Text File | 1995-01-05 | 29KB | 598 lines

--------------------------------
Read me first (sharware release)
BULLET v1.2

--Documentation (read this before using CZ)

The documentation is online in the CZ.COM/CZ.HLP files. To get started just
make sure CZ.HLP is in the current directory and load CZ:

   C>cz

It can also be loaded high but make sure you have at least 15K of UMB space.
It will load high in as little as 4.5K but will not operate correctly in UMB
without at least 15K. Check out the Using_CZ in the TUTORIAL_INDEX--press
Alt-F1 to hotkey into CZ. Press END (cursor moves to TUTORIAL_INDEX). Press
ENTER. Move the cursor to under Starting_CZ. Press ENTER. Use the END/ENTER
combo to continue until you get the idea. A mouse makes using CZ even easier.

If you want to have a hardcopy of the CZ.HLP use the HLP2TXT.EXE program to
strip off the header info. Load it into your favorite WP or text editor.

This package is documentation/source-specific to QuickBASIC 4.5 and BASIC 7 in
that much of the source examples deal with using BULLET under these compilers.
Because of this, this version is called BLTQxxx.ZIP.


--Installation

BULLET comes with the BULLET.LIB in compressed form (BULLET.LI_). It must be
installed using the INSTALL.EXE program. INSTALL performs one function: it
decompresses BULLET.LI_ (to BULLET.LIB) and writes installation information
to the BULLET.LIB file. It does nothing else.

To install:

   C>install

The installation process asks two questions and then decompresses the file so
that BULLET.LIB is ready for use. BULLET.LIB is fully-functional and not
crippled. You may use BULLET.LIB and the BULLET package for evaluation only and
for an evaluation period not to exceed 21 days from the date of installation.
BULLET.LIB works with just about every DOS compiler on the market.


--Registration

BULLET is a shareware product. This means that you may try BULLET during the
evaluation period free of obligation. However, you may not use BULLET after
this evaluation period without registering--you must purchase a license to
use BULLET if you wish to continue to use BULLET or any programs produced with
BULLET. See the !ORDER.FRM for ordering information by check, cash, or money
order. Use !ORDER.CC for ordering by credit card.


--Upgrades and Updates, Site licensing

All updates and upgrades are available from the support BBS free-of-charge.
See the PRODUCT_SUPPORT heading under TUTORIAL_INDEX in CZ for more.

Site licensing is available at reasonable rates. Contact me for details.


--Distribution

BBS Sysops may carry BULLET on their BBS provided that all files listed in
the !PACKLST file are included in the archive. This file must be provided
free of charge, i.e., this file cannot placed in an area that requires users
to pay a fee for per-file download. A general subscription fee BBS is exempt
from this limitation, provided that no specific charge is assessed to this
particular file.

Shareware distributors/catalog vendors may carry BULLET on their inventory
provided that: the BULLET package be the only software on the diskette, that
it be made clear that the purchase of the disk does not constitute purchase
of the product, and that no charge greater than a reasonable amount for
distribution services be charged (for example, not more than 5 dollars).
CD-ROM vendors may include BULLET and may charge more than $5 for the CD,
provided that the CD is not for the sole or primary purpose of distributing
Bullet.

Only the shareware version of Bullet may be distributed without explicit
permission from the author.

In any case, the author of BULLET may at anytime request (and must be granted)
that the BULLET package be no longer advertised/distributed in the vendors
catalog and that within a reasonable time after request, advertising of BULLET
will end (within 6 months and distribution will end immediately). (Basically,
this means that you as a shareware vendor may include BULLET in your
advertising/distributing but that that I, as the author and owner of BULLET,
have the option of having you stop advertising/distributing BULLET, for
whatever reason (such as it becoming a non-shareware commercial product).)


--------------------------------------------------------
Status value return code with transaction-based routines

The transaction-based routines, InsertXB, UpdateXB, ReindexXB, and all LockXB
routines, do not return the completion status. The return value is the
pack index number that failed. The following example assumes that you are
using multiple index files. If you're not, replace AP(x).y with AP.y.

   AP(1).Func = InsertXB   'assume other AP()s setup with .nextPtr
   pid = BULLET(AP(1))
   IF pid = 0 THEN
      IF AP(1).stat = 0 THEN
         'all okay
      ELSE                      'error while adding the data record is returned
         stat = AP(1).stat      'in AP(1).stat if and only if pid=0
         DoErrorWithDataRecord stat
      ENDIF
   ELSE
      stat = AP(pid).stat       'pid is a 1-based value (1...)
      DoErrorWithIndexFile pid, stat
   ENDIF

For complete information consult the above named routines in CZ. Note that
ReindexXB does not operate on a data file separately so if its return
code=0 (pid=0) then all operations succeeded. However, with the LockXB
routines, if pid > number_of_packs then the lock on the data file failed.
The following example assumes that a single index file and data file are
to be locked.

   packs = 2                    '2 packs (two index files per data file)
   AP(1).Func = LockXB          'assume other AP()s already setup
   pid = BULLET(AP(1))
   IF pid = 0 THEN
      'all okay
   ELSEIF pid <= packs          'error while locking index file and error code
      stat = AP(pid).stat       'in AP(pid).stat
      DoErrorWithIndexFile stat
   ELSE                         'error while locking data file and error code
      stat = AP(2).stat         'also in AP.stat (in last AP pack)
      DoErrorWithDataFile stat  '(i.e., if here then pid=3)
   ENDIF


---------------
Troubleshooting

Common problems encountered by new users of Bullet are:

P: Data fields are skewed, with some fields containing
parts of other fields' data.

S: Most of these problems relate to the way C strings are
0-terminated, while dBASE DBF fields are fixed-length,
and so need not have the \0.  If you are going to use
strcpy, or any other C run-time library that requires
null-terminated strings, then be sure to provide for this
when you define your DBF record.  For example, if you
want two fields, LASTN and FIRSTN, to each have 15
characters, but will be using C strings, define the
fields as 16.  This will allow room for the \0 on your
strings, and give 15 true character places.

Often, the case is such that all appears normal when the
DBF file is accessed (i.e., the data fields don't appear
to be skewed), but indexed access is not correct.  This
is because your program code has templated the data
record consistent with what was written (and is read) at
the DBF level.  However, it is not consistent with the
DBF header specifications (written in CreateDXB) and so
any external access to the DBF (including Bullet keyed
access) will not be correct.

Skewed data also results when you change either your
program's data structure template of the DBF record, or
your DBF field list definitions, but don't do the same to
both.  The physical record buffer in your program must
exactly match that of the DBF record.  Off the subject a
bit, but possibly on your mind:  yes, you can create all
this on the fly (i.e., not hard-coded fieldnames, etc.),
but it does require that you manage all offsets yourself,
rather than have the compiler do it for you.  A job for a
C++ wrapper, I suppose.  Certainly do'able as I''ve done
it using a BASIC compiler (see IDEMO.*)

Another reason for skewed fields is that the compiler is
modifying the alignment of your program's DBF record
structure.  It is imperative that you instruct your
compiler to not modify any Bullet-used structure:
#pragma pack() is used by many compilers.  Consult your
compiler documentation.  For non-Bullet structures,
alignment doesn't matter to Bullet.  Bullet structures
are any of the Bullet packs, and any DBF record
template/buffer/structure that your program uses to
communicate with Bullet.

The main point is to ensure that your physical data
record (structure) you use in your C/C++ program exactly
matches, field for field, that of the DBF record on disk.


P: Error 240 occurs when attempting to index or reindex
when all else seems to be right.

S: dBASE specifications require that fieldnames of the
DBF be 0-filled.  Some programs create DBF files that do
adhear to this specification, but instead just null-
terminate the fieldname.  These files must be fixed
before they can be used by Bullet.  When creating Bullet
DBF files, you must 0-fill the fieldnames (11 bytes, 10
bytes data -- byte 11 always=0).  Memset() can be used to
quickly 0-fill the entire fieldlist.


P: Find an exact match on a GetEqualXB.

S: The problem usually stems from index files created to
allow non-unique keys.  Personally, I don't create index
files where the primary key is not unique, but the demand
was there and so Bullet provides for this.  However,
since there's no real way to offer indexing on physically
indentical keys, Bullet adds an enumerator word at the
end of each key (only if the index file is not using
unique keys).  This two enumerator is in high byte/low
byte order, and is incremented for each duplicate key in
the index file.  For example, if KINGM is added as a key,
and is found to be the first such key by Bullet, the
actual physical key is KINGM\0\0, where the enumerator is
the last two bytes (in Motorola-word order for proper
collating).  Upon the next insert of a "KINGM" key,
Bullet uses KINGM\0\1.  And so on.

To search for this key, which by the way is 7 bytes long,
and so reported by StatKXB, though logically it is only
the 5 bytes (as in strcpy(keyExp,"SUBSTR(LASTN,1,5)")
with non-unique keys permitted), you must account for
this in your keybuffer.  For example, in your global
scratch keybuffer (or you can use a separate buffer for
each index file, up to you)  you must specify all 7
bytes, not just the first 5.  This because the data at
bytes 6/7 may not equal \0\0 or \0\1, and so these keys
will not be found (error 201).  In this particular case,
where duplicate keys exist, and you want to locate a
KINGM record, you really have no choice but to start with
strcpy(keybuff,"KINGM\0\0", be aware of the "extra" \0
put by the strcpy function) and GetEqualXB, check the
data record returned, and if not the one wanted, do
GetNextXB until the desired record is located (or not).
Alternatively, you could specify "KINGM\255\255" and use
GetEqualXB followed immediately by GetPrevXB (enumerator
\255\255 is the very last possible, and so will never be
found, but will setup the internal gears so that the
GetPrevXB will locate the very last existing "KINGM??").


All this is really only a concern if you permit duplicate
keys.  The above could be made unique by adding expanding
the key expression.  For example, a key based on
SUBSTR(LNAME,1,5)+SUBSTR(SSN,6,4) would be pretty unique
(first 5 characters of last name followed by last 4 SSN
numbers (used as characters)).


Q: How do I specify the key I want to find?

A:  The keybuffer (AP.keybuffptr locating it) is filled
with whatever data type you want to locate (hence,
keybuffer is of void type).  If you want to locate a
character key, simply specify the key, such as
strcpy(keybuffer,"whatever") (see "Find an exact match on
GetEqualXB", above).  If you are locating a binary key
value, place the binary value, with the appropriate cast,
into the keybuffer: e.g., *((long *)keybuffer) = 5L;.


These problems are the most frequently expressed to me.
If you are having other problems related to getting
Bullet going, feel free to contact me at the locations
listed under Product Support.  My ability to quickly
respond to your requests is directly related to how
thorough you are in expressing the exact cicumstances of
any problem.

Complete sample programs are in short supply.
Experienced C or C++ programmers that would like their
"Bullet wrappers" possibly included in future releases
are requested to contact me.  Additional samples will be
made available on the BBS, available to all.  Of course,
registered Bullet users will have access to the best of
the best, and always have free access to the current
versions of Bullet.

When you've done everything and it's still not working,
it's time to read the documentation!

-------------------
Questions & Answers

Q: What is BULLET?

A: BULLET is a library of program modules that together let the programmer
   develop and create software that can manage huge amounts of data on
   disk using the industry-standard DBF data file format. It also uses high-
   speed b-tree index files to manage keyed data file access.


Q: What compiler is BULLET for?

A: BULLET can be used by nearly all DOS compilers. It's written entirely in
   assembly language. Because of this, it does not require any particular
   programming language or compiler vendor. The 4 requirements are listed
   in the !WHATIS.TXT file.


Q: Why do I need BULLET when all I need to handle are small amounts of data?

A: BULLET can deal with a database as small as 1 record or as large as several
   million. While your current needs may be small, your future needs are bound
   to expand. BULLET can work with you now, and later, even if you switch
   development platforms by moving to another compiler. And BULLET is fast,
   it can deal with a database with millions of records as easily as it can
   with just a few.


Q: But b-tree stuff, isn't that hard?

A: Everything associated with maintaining the database, its data files and its
   index files, is done behind-the-scenes by BULLET. You just specify how the
   data record is to look by specifying the number of fields, their lengths and
   types, and then specify how you want your index files to be based.


Q: So how do I design my data record?

A: You probably have a pretty good idea, already. A good way to determine what
   should go into a database is what you want to come out of it. For example,
   if you're doing a mailing list program, you'll want to have at least the
   name, perhaps broken into first name and last. Also you'll need the mailing
   address--4 lines is usually enough, so you'll want 4 separate address line
   fields. Then there's the State and ZIP, possibly even country. That's the
   minimum. It would look something like this:

   FIRSTNAME field of 15 characters; LASTNAME field of 19 characters; ADDR1
   field of 34 characters, ADDR2, ADDR3, ADDR4 as ADDR1; State field of 2
   characters, and ZIP a field of 5 (or 9 if ZIP+4) characters. You could
   specify the ZIP as a numeric field if you wanted.

   You'll notice that the longest field is 34 characters. Why? Because most
   mailing labels are 3.5 inches, about 34 characters across. Since the first
   name and last are usually put on the same line, their total should be 34.

   You'll probably want to add more fields like telephone number, last time
   written to, oh, just about anything that you'd think would be important to
   know. There you go, you've just designed your data record.


Q: Then what?

A: Then decide how you need to access this data. You'll want to access it at
   least by name, so one index you'll want is on the name. While you could
   specify the entire name be used as a key, say LASTNAME+FIRSTNAME, this is
   a bit of overkill. Instead, you may want to use just a portion of the name.
   A good candidate would be SUBSTR(LASTNAME,1,5)+SUBSTR(FIRSTNAME,1,1). This
   sets up a key that's only 6 bytes long. The first method, using all the
   name, would be 34 bytes long. By keeping your keys short you'll keep your
   index files small and your index performance high. And yes, you can also
   mix numeric field types with character field types in your key expressions.


Q: But what if I have two or more names that are identical, or very similar
   but have these parts of the names the same?

A: BULLET lets you specify if your index files allow only unique key entries or
   whether duplicate keys are permitted. When keying on a name you should have
   your index file allow duplicate keys. What BULLET does is number these
   identical keys by adding a suffix to each key (called an enumerator). This
   enumerator allows the index algorithm to treat each key as a different key.
   If you search the index for all matches in the first 6 characters of the
   key (the enumerators will always be different) these similar names will
   be found in consecutive order. To find out if the key you've just accessed
   is the actual person you had in mind, you'd scan the data record associated
   with that key for other information, such as middle initial, address,
   anything that would make that person recognizable from another with a
   similar key. If it isn't what you're looking for, get the next key and data
   record, and so on until the first 6 characters of the key no longer are the
   6 you're looking for.


Q: So I've got my data record designed and also my primary index file. What
   else should I do?

A: Now that the database is designed most of the "unknown" is taken care of.
   What comes next depends on how you, yourself, program. What I often do
   next is detail exactly what I want the output to be. That way, I've got
   the front and the back and just need to do the middle. The middle is where
   the fun's at. You'll be amazed at just how few of your in-the-middle coding
   is spent on managing the database. BULLET takes care of all the little
   details. You just need to give it the data and tell it what to do with
   it. Or you tell it what to get and it comes back with what you requested.
   You then do whatever you want with that data.


Q: I've looked at the header file and it sure has a lot of commands. You're
   telling me that this is simple?

A: Yes. Once you've created your data and index files, those mid-level
   routines are not often used. Almost everything you do in your in-the-middle
   coding will use the high-level routines. The InsertXB and UpdateXB routines
   handle adding new or changing existing data, and the GetFirstXB, GetNextXB,
   etc., routines handle getting the data. 90% of the time these are the
   routines your program will be using.


Q: What about all those packs? How can I keep them straight?

A: The good thing about modern programming langauges is that they let you
   build reusable code. The ideal way to use BULLET is to build reusable
   code objects in your programming langauge of choice and hide the down-and-
   dirty aspects of dealing with the various packs in those objects. For
   example, a create key routine could be written once and that object used
   for all your other programming projects:

      DECLARE FUNCTION CreateNewIndex% (filen$, kx$, kflags%, XBlink%)

      FUNCTION CreateNewIndex(filename$, kx$, keyflags, XBlink)

      DIM CKP AS CreateKeyPack
      DIM TmpFile AS STRING * 80
      DIM TmpExp AS STRING * 136

      TmpFile = filename$ + CHR$(0)
      TmpExp = kx$ + CHR$(0)

      CKP.Func = CreateKXB
      CKP.FilenamePtrOff = VARPTR(TmpFile)
      CKP.FilenamePtrSeg = VARSEG(TmpFile)
      CKP.KeyExpPtrOff = VARPTR(TmpExp)
      CKP.KeyExpPtrSeg = VARSEG(TmpExp)
      CKP.XBlink = XBlink
      CKP.keyflags = keyflags
      CKP.CodePageID = -1
      CKP.CountryCode = -1
      CKP.CollatePtrOff = 0
      CKP.CollatePtrSeg = 0
      stat = BULLET(CKP)
      CreateNewIndex = stat

      END FUNCTION

   The TmpFile and TmpExp were used in this QuickBASIC example so that the
   string data is assigned to a fixed-length string. VARSEG/VARPTR operate
   on fixed-length strings identically in both BASIC 7 and QuickBASIC 4.5.
   If you want to forgo this compatibility you could use a var-len string
   and the BASIC SADD() function instead of VARPTR. I find using the
   temporary fixed-length strings just as easy since a CHR$(0) needs to
   be appended to the string in any case. And they are temporary strings--
   as soon as the CreateNewIndex() routine exits, the memory assigned to the
   Tmp strings is released.

   So, once you write this create index object, you can reuse it over and
   over again without needing to recode it every time. Just a simple

   stat = CreateNewIndex(filename$,keyexpression$,keyflags,XBlink)

   is all you need to put in your in-the-middle code. Oh, the key expression,
   keyflags, and XBlink are all explained under CreateKXB in CZ.


Q: I think I'm getting the hang of it. What's next?

A: Jump in and start coding. You may want to look over, maybe even print out,
   one of the example programs. The BB_CAI10.BAS is straight forward, try
   that. If you have any questions, just pop-up CZ. It's all in there.


Q: Sounds good. But, tell me, what is the first thing I'm likely to muff?

A: Nothing serious. Just make sure that your record structure in memory
   reserves the first byte for the delete tag. Also, make sure that the field
   descriptors you assigned when you created the data file match your in-memory
   structure, i.e., if you've created a data file (using CreateDXB) with say, 3
   fields, each 25 characters long, make sure that your in-memory structure is
   also 3 fields, each 25 characters:

      TYPE ExampleRecordTYPE
      tag AS STRING * 1
      Field1 AS STRING * 25
      Field2 AS STRING * 25
      Field3 AS STRING * 25
      END TYPE

   Note: it is allowable to alias your physical fields but the total length
   must match the total length of the DBF data record. (Alias meaning that
   instead of using Field1 AS STRING * 25, you use Field1a AS STRING 13 and
   Field1b AS STRING 12. Be sure you know what you're doing!--the IDEMO.BAS
   program uses this technique (not in the strict sense of alias) by using a
   single 4000-byte in-memory record since it can't know beforehand what
   structure a random DBF file has.)

   Also, the transaction routines (InsertXB, UpdateXB, ReindexXB, and the
   LockXBs) return a transaction index rather than a completion code. Be sure
   to check the documentation for the routines.

Q: I'm off.

A: So am I, but be sure to...

   ...Read the manual! Until you do you can't take full advantage of BULLET.


---------------
What is BULLET?

If you've been wanting to create the ultimate database program, or just a simple
mailing list to handle your holiday cards, BULLET is the programming library for
you. BULLET is a library module for programmers developing in MS-DOS. What
language you ask? Would you believe most any? That's right, BULLET is designed
to operate as-is with most compilers* available for DOS (DOS 3.3 or above is
required). This is possible because BULLET is written entirely in assembly
langauge. Everything it needs it has built in, so no specific compiler run-time
library is required. Your compiler simply needs to be able to:

 1. Perform a far call to an external routine using the Pascal calling
    convention.
 2. Build a parameter pack, as in a C struct or BASIC TYPE, for example.
 3. Pass a single far pointer by value (of the parameter pack) to the external
    routine.
 4. Accept a return integer value from the external routine (optional).

BULLET uses high-speed b-tree indexes and the industry-standard DBF data file
format to quickly and easily move data to and from disk. The BULLET API includes
over 65 functions to perform tasks from low-level direct DOS file I/O to
high-level transaction-based updates to network routines that let you control
who has access to your files and when. National Language Support (NLS) is built
into each key file. This allows you to properly sort mixed-case and/or foreign
language alphabets. BULLET supports character keys up to 64 bytes as well as
16-/32-bit integer keys (signed or unsigned). Index files specify if duplicate
keys are allowed or whether unique keys are to be enforced. And although dBASE
DBF data file compatiblility is standard, you can create non-standard DBF files,
such as having fields contain binary data. Key expressions are specified using
text, as in kx = "SUBSTR(Fieldname,1,10)+Fieldname+SUBSTR(Fieldname,10,3)". Keys
may be composed of up to 16 separate fields, located anywhere within the record.

The transaction nature of BULLET greatly simplifies what is required of the
programmer. For example, let's say you have a 100,000-record data file with 12
active index files and need to insert a new record. With BULLET, you simply
build the data record and call the InsertXB routine. InsertXB adds the record to
the data file, builds all 12 keys and inserts each into the appropriate index
file. If an error occured, say, while inserting the 11th key, BULLET
automatically backs-out all changes just made to the key files and data file. As
another example, say, updating an exisiting data record, you'd make any
modifications to the data record and call the UpdateXB routine. UpdateXB updates
the record in the data file and automatically updates all 12 index files. If an
error occured, say, with the 5th key, BULLET automatically backs-out all changes
made to the key files and data file.  In addition, transaction-based network
routines are available. These high-level lock/unlock routines also automatically
handle the necessary reloading and flushing of file headers. And what  about
performance? Read on.


BULLET is fast! The ReindexXB routine indexes your DBF data files in no time
flat: that 100,000 record .DBF can be completely reindexed in under 30 seconds
on a fast computer system. The InsertXB routine can add 1,000 new records and
keys to that 100,000 record database at a rate of over 50 new inserts per
second. This isn't inserting into an empty database, it's inserting into a
database that's already 100,000 records large. UpdateXB can update
1,000 records in that database at a rate of 20 updates/second. And what about
access speed? How fast can you get to your data once it's in the database? Real
fast! To access from first key to last all 100,000 keys takes less than
25 seconds (4500+/sec). That shows how fast you can find keys. To access the
keys and also access each data record associated with that key takes a bit
longer. Accessing all 100,000 keys and records takes just over a minute
(1400+/sec). This is in-order access. Incredible. And if you wanted to access
from the last key to the first (reverse-order), the times are just as fast.
What do you get with the BULLET package?

 - The BULLET library module, 19K of code and static data space (small indeed)
 - Sample programs in BASIC and C plus an interactive demo for browsing any DBF
 - QuickBASIC .bi include and C .h header files
 - Over 200K of documentation, source examples, and a tutorial, all online
 - CZ, the 15K online, context-sensitive, hyper-text help manager
 - Royalty-free use of BULLET in your programs

Routines by category:

SYSTEM: InitXB, ExitXB, AtExitXB, MemoryXB, BreakXB, BackupFileXB, StatHandleXB,
GetExtErrorXB

MID-LEVEL RECORD/KEY ACCESS: CreateDXB, OpenDXB, CloseDXB, StatDXB, ReadDHXB,
FlushDHXB, CopyDHXB, ZapDHXB, CreateKXB, OpenKXB, CloseKXB, StatKXB, ReadKHXB,
FlushKHXB, CopyKHXB, ZapKHXB, GetDescriptorXB, GetRecordXB, AddRecordXB,
UpdateRecordXB, DeleteRecordXB, UndeleteRecordXB, PackRecordsXB, FirstKeyXB,
EqualKeyXB, NextKeyXB, PrevKeyXB, LastKeyXB, StoreKeyXB, DeleteKeyXB,
BuildKeyXB, CurrentKeyXB

HIGH-LEVEL ACCESS: GetFirstXB, GetEqualXB, GetNextXB, GetPrevXB, GetLastXB,
InsertXB, UpdateXB, ReindexXB

NETWORK: LockXB, UnlockXB, LockKeyXB, UnlockKeyXB, LockDataXB, UnlockDataXB,
DriveRemoteXB, FileRemoteXB, SetRetriesXB

LOW-LEVEL DOS ACCESS: DeleteFileDOS, RenameFileDOS, CreateFileDOS,
AccessFileDOS, OpenFileDOS, SeekFileDOS, ReadFileDOS, ExpandFileDOS,
WriteFileDOS, CloseFileDOS, MakeDirDOS