NetNews Usenet Archive 1992 #18/NN_1992_18.iso/spool/comp/lang/pascal/4861

Internet Message Format | 1992-08-14 | 11.2 KB

Path: sparky!uunet!munnari.oz.au!ariel!ucsvc.ucs.unimelb.edu.au!lugb!lux!9125113g
From: [e-mail] (Mitch Davis)
Newsgroups: comp.lang.pascal
Subject: Re: Arrays > 64K
Message-ID: <[e-mail]>
Date: 15 Aug 92 04:03:11 GMT
References: <[e-mail]> <1992Aug02.170808.2053@crash> <[e-mail]>
Sender: [e-mail] (USENET News System)
Organization: La Trobe University
Lines: 322

In article <[e-mail]> [e-mail] (William C. Thompson) writes:

>Here is a file you can include with your source code.  You need to
>define a type before you {$I} it.
>
>  HugeDataType is.  It follows that another restriction would
>  be you can only have one type of huge array. }

Heap big code deleted.  Here's my offering.  Note that because it uses
objects, you can have more than one array of whatever type you want.

Note this code was written by me for the International Pascal
Newsletter.  I'd suggest you study the supporting documentation in that
if you want to know more.

The file you want is PNL010.ZIP and is widely available.  Check archie.

[DOSMEM.PAS]

unit DOSMem;

interface

function Alloc (paras:word):word;

procedure Free (p:word);

function Largest:word;

implementation

function Alloc; assembler;

asm
  mov  ah, $48
  mov  bx, paras
  int  $21
  jnc  @1
  xor  ax, ax
  @1:
end;

procedure Free; assembler;

asm
  mov  ah, $49
  mov  es, p
  int  $21
end;

function Largest; assembler;

asm
  mov  ah, $48
  mov  bx, -1
  int  $21
  mov  ax, bx
end;

end.

[BIGARRAY.PAS]

unit BigArray;

{Program to accompany article in issue #10 of the Pascal NewsLetter.     }
{Author: Mitch Davis, [e-mail]                        }
{                     [e-mail]            }
{                     Fidonet: 3:634/384.6                               }
{                     [phone].                                    }

{ from public domain ideas by Trevor J. Carlsen, 3:690/644 }

{This unit lets you implement whopping great big arrays, larger than the }
{64k offered by Turbo Pascal.  It does this by offering a large array    }
{object.                                                                 }
{The array can have elements of any size, and the array can take all free}
{DOS memory.  Thus, you could conceiveably have an array of reals taking }
{some 400k or more.  Each array can take a subscript in the range of 1 to}
{the parameter you hand the Init method - ie, the array is one-based.    }

{Note this code has NOT been optimised - it needs it very badly!         }
{A future version of this unit (to be published in the PNL) will be      }
{heavily optimised, will offer multidimensional arrays, and will offer   }
{arrays that spill over onto disk, as well as specific support for sparse}
{arrays.  Keep your eyes out for it!                                     }

{This program has NOT been proven correct.  I know it works when used as }
{part of the MDP programs published in PNL #10, but I can't vouch for its}
{operation if and when used in different situations.                     }

{I suggest compiling this in 286 mode, at least until I can get around to}
{optimising it!                                                          }

{$R-,S-,N+}

{$DEFINE OnA286} {Remove this line if you're not on a 186 or higher.}

{$IFDEF OnA286} {$G+} {$ELSE} {$G-} {$ENDIF}

interface

{$IFDEF OnA286}
uses Test186;
{$ENDIF}

type BigDOSArray = object
                     procedure SetElemSize (newsize:word);
                     function GetMaxSize:longint;
                     procedure Init (Elems:longint);
                     function Elem (elemnum:longint):pointer;
                     procedure Done;
                   private
                     base:word; {paragraph of base}
                     linear:longint;
                     size:word; {size of each element}
                   end;

implementation

uses DosMem;

procedure BigDOSArray.SetElemSize;

begin
  size := newsize;
end;

function BigDOSArray.GetMaxSize;

var paras:word;
    bytes:longint;
    elemss:longint;

begin
  GetMaxSize := pred ((longint (DosMem.Largest) shl 4) div size);
end;

procedure BigDOSArray.Init;

var bytes:longint;
    paras:word;

begin
  base := DosMem.Alloc (succ((longint(Elems) * size) shr 4));
  linear := longint (base) shl 4;
end;

function BigDOSArray.Elem;

var offset:longint;

begin
  offset := pred(elemnum) * size + linear;
  Elem := ptr (offset shr 4,offset and $f);
end;

procedure BigDOSArray.Done;

begin
  DosMem.Free (base);
end;

end.

[Here's the article, and a sample (for "sample" read "silly"!) program
which shows you how to use it.]

BigArray - you've never seen arrays this BIG!  by Mitch Davis, (3:634/384.6)
============================================================================

 *** Error 22: Structure too large.

 *** Error 49: Data segment too large.

 *** Error 96: Too many variables.

 *** Runtime error 201 at XXXX:YYYY. {Range check}

 *** Runtime error 203 at XXXX:YYYY. {Heap full}

How many times have you been frustrated by these little beauties?  How many
times has it been because you had an array that just wouldn't stay down?
There are two ways to fix this - you could put your thinking on a diet and
work out some other way that doesn't consume quite so much memory
(recommended!) or you could do it the lazy way, and slot in a unit called
the (drum roll!) *** BigArray ***.

BigArray will give you arrays which can be as big as will fit in
conventional memory, ie, they aren't limited to 64k.  The arrays can be of
any type, and are of a single dimension.  (A version being written now lets
you have arrays that spill over to disk or EMS, as well as multi
dimensionals.)

The large arrays are implemented as objects.  You use the .SetElemSize
method to tell it how big your elements are, and you can subsequently find
out via .GetMaxSize how many elements the largest array can be.  Then you
call the Init method with the number of elements you'd like, and voila,
your array is done!  When you're finished with the array, call the .Done
method.

How are elements accessed?  You pass the element subscript to the .Elem
function, and it returns a pointer to that element.  The idea is that you
then assign that pointer to a pointer variable of the type you're storing.
Then by deferencing the pointer (following it by a caret [^]) you can then
access any fields, bits, WHATEVER that you would with a plain-jane
variable.

Let's look at a sample of how you'd use them:

program BigArrayTest;

{Program to accompany article in issue #10 of the Pascal NewsLetter.     }
{Author: Mitch Davis, (3:634/384.6) [phone].                      }

{$G+,M 16384,0,0} {Make SURE you tell TP not to steal all the DOS memory }
                  {for a heap!  If you're not using dynamic variables,   }
                  {set both numbers to 0.                                }

{This program provides a nonsense demonstration of how you use the tools }
{in the BigArray unit.                                                   }

uses Test186, BigArray; {I run my programs on a i286.  If you have an XT,}
                        {Remove the "G+," from above and the "Test186"   }
                        {from the uses section.                          }

type PigeonType = record
                    value:real;
                    changed:boolean;
                  end;

var PigeonHole:BigDosArray;
    PigeonPtr:^PigeonType;
    PHnum, MaxSize:longint;
    GlobalChanged:boolean;

begin
  writeln ('Welcome to the pigeon-hole.');
  with PigeonHole do begin {This sets up the big array.}
    SetElemSize (sizeof (PigeonType)); {Tells it how big each element will be}
    MaxSize := GetMaxSize;
    Init (MaxSize); {Make it as big as possible.  This is not compulsory.    }
  end;
  writeln ('There are ',MaxSize,' pigeon-holes, numbered from 1 to ',MaxSize,'.');
  write ('Please wait while I clear them... ');
  for PHnum := 1 to MaxSize do begin
    PigeonPtr := PigeonHole.Elem (PHnum); {Get the address of the element}
    PigeonPtr^.Changed := false; {Reference the changed field within that}
  end;                           {element.                               }
  writeln ('Done.');
  GlobalChanged := false; {This will save us search time later if no changes}
  repeat
    write ('Which pigeon hole? (1-',MaxSize,', 0 to quit): ');
    readln (PHnum);
    if (PHnum > 0) and (PHnum <= MaxSize) then begin
      PigeonPtr := PigeonHole.Elem (PHnum);
      with PigeonPtr^ do begin
        case Changed of
          true :writeln ('That pigeon-hole has the value of ',Value:3:2);
          false:writeln ('That is a new pigeon hole.');
        end;
        write ('Change it to? ');
        readln (Value);
        Changed := true; {means this ph will be shown in the end summary}
      end;
      writeln ('Pigeon-hole changed.');
      GlobalChanged := true;
    end;
  until PHnum = 0;
  writeln ('-------------------------------------------------');
  case GlobalChanged of
    false:writeln ('You didn''t change any pigeon holes.');
    true :begin
            writeln ('The pigeon holes you changed were:');
            write ('Wait..',#13);
            for PHnum := 1 to MaxSize do begin {scan thru the ph's}
              PigeonPtr := PigeonHole.Elem (PHnum);
              with PigeonPtr^ do if changed then writeln (PHnum,': ',value:3:5);
            end;
          end;
  end;
  writeln ('Thanks for using the pigeon-holes!');
  PigeonHole.Done;
end.

If you understand this (in conjunction with the "interface" section of the
BigArray unit), then you should have no problem working out how to use it.

A few usage notes:

  o The Big Arrays are 1-based, that is if you call BigDOSArray.Init(10),
    then the element subscripts run from 1 up to 10.
  o There is NO checking to ensure that the element you pass to
    BigDOSArray.Elem is within the defined range.  If it isn't, then you'll
    get garbage results for the pointer result.  Beware!
  o The code for BigDOSArray hasn't been extensively tested.  Caveat
    Emptor!
  o BigDOSArray needs optimising BADLY!  I plan to write it in optimised
    assembler throughout; in the meantime, you'll have to sacrifice speed
    for convenience.
  o I am aware that Turbo Power markets code which implements large
    arrays (and probably MUCH better than I've managed here!).  I haven't
    looked at their code.  Trevor Carlsen's OverSize unit also performs a
    similar function.  I looked at his code, and couldn't understand it.
    That's not meant as a slur on Trevor (3:690/644), who writes code so
    good it should be framed.  I just thought I'd take a different
    approach. Upshot:  If any of the other large-array products fit your
    bill better, use them!
  o The BigDOSArray unit currently compiles in i286 mode.  If you have an
    XT or similar, chop out the G+ and Test186 bits, in the same way as the
    comments in the demo program mention.

Well that about wraps it up.  I'd be more than happy to answer any queries
you have about the unit.  Next issue, time and space permitting, I'm
planning on having a followup article which goes into optimisation,
multi-dimensional arrays, etc.  Till then?  Make merry with your Pascal!

[End of article]

I hope this helps.

Mitch.