home *** CD-ROM | disk | FTP | other *** search
Text File | 1993-03-27 | 51.4 KB | 1,134 lines |
- (10U
-
-
- Clipper Memory Management
-
- by Roger Donnay
-
-
- A. Linking Techiques for Memory Management
-
- 1. Dynamic Overlaying of Clipper Code
- 2. Dynamic Overlaying of C/ASM Code
- 3. Using EMS or UMB Ram for Overlays
- 4. Static Overlays
- 5. Reloadable Overlays
- 6. Overlaying Library Modules
- 7. Speed Optimization
-
- B. Programming Techniques for Memory Management
-
- 1. Symbol Management
- 2. Memory effects of Database usage
-
-
- ╔════════════════════════════════════════════╗
- ║ LINKING TECHNIQUES FOR MEMORY MANAGEMENT ║
- ╚════════════════════════════════════════════╝
-
- The Clipper compiler has the responsiblity of converting source
- code to "machine-readable" code. This .OBJect code can not
- be "executed" by the machine however until it is "linked" to
- other .OBJect code segments in the Clipper libraries. This
- is the responsibility of the linker. Linking or "fixing up"
- references made by your code to external variables and functions
- is necessary to insure that your compiled program will address or
- call other programs, including functions in the Clipper libraries
- and DOS functions.
-
- A linker's basic job is to combine objects which have been
- compiled separately. Objects contain symbols of three types:
-
- a. PUBLIC symbols - those symbols which are callable from
- other objects
-
- b. STATIC symbols - those symbols which are callable only
- from within the same object
-
- c. UNDEFINED symbols - symbols which exist in other objects
- and are referenced or used by this object
- (also referred to as EXTERNAL symbols).
-
- The Role of Linkers in developing large applications has taken
- a new turn in the past few years. In addition to providing
- segment-linking, they are now also expected to provide the
- best "memory image" possible. This means that the linker must
- be sophisticated in it's management of code segments during
- the running of an application. The physical size of a
- Clipper application is no longer representative of it's memory
- image, because the linker "overlays" the code segments to
- use the smallest possible amount of memory. Some type of
- code (such as Clipper P-CODE) is easy to overlay while other
- types (ASM) may be more difficult. Applications that are
- almost all Clipper-compiled code will usually produce the best
- memory image regardless of the physical size of the .EXE.
-
- The subjects of this seminar are the two most commonly-used
- linkers available on the market today for linking Clipper 5.0
- applications. They are:
-
- RTLINK - A Clipper-5.0 specific linker developed by
- Pocketsoft, Inc. This is the linker that is
- bundled with Clipper 5.0.
-
- BLINKER - A Third-Party Clipper-specific linker published
- by Blink, Inc. for Summer 87 and 5.0 applications.
-
-
- ╔══════════════════════════════════════╗
- ║ DYNAMIC Overlaying of Clipper Code ║
- ╚══════════════════════════════════════╝
-
- Very large Clipper applications can consist of more code than
- can be loaded into the available amount of DOS memory and may
- not be able to execute due to insufficient memory. Clipper
- 5.0 has a "Dynamic Overlay Manager" actually built into the
- Clipper libraries that helps prevent this problem by loading
- the code into memory only as it is needed by the application
- that is running. The linker helps prepare the application
- to use this capability of 5.0 by splitting the Clipper-compiled
- application code into fixed-size "pages" that share a pre-
- allocated area of memory. These pages of code are then
- written to a seperate .OVL file or to the end of the .EXE
- file as determined by your linker options. When a procedure or
- function is called, the "dynamic-overlay manager" first checks
- to see if it is in memory. If it is, then it jumps to the
- called function. If it is not in memory, it loads the function
- from disk into any "free" location available in the overlay
- pool. If no free space can be found in the overlay pool, the
- overlay manager discards pages that have not been called
- recently to free space to load the called procedure.
-
- The larger the overlay pool, the less often the overlay manager
- needs to load from disk and the faster your program will run.
- The size of the dynamic overlay pool needed for your application
- is automatically determined by Clipper.
-
- The great thing about Clipper 5.0's dynamic overlaying system is
- that you, the programmer, don't need to do anything to create
- overlays. It is all done automatically by the linker. Only
- Clipper-compiled modules, not C/ASM modules, are overlayed
- "dynamically", although since much of the Clipper libraries has
- been compiled in Clipper, even large portions of the Clipper
- libraries will be dynamically overlayed.
-
- Application libraries and third-party libraries written in
- Clipper will also be automatically overlayed by RTLINK and
- BLINKER. In many cases it is more beneficial to write functions
- in Clipper (using LOCAL memvars) than in C or ASM because it
- will not increase the executable memory model. Even Clipper
- code that has been linked into to Pre-link libraries will be
- dynamically overlayed.
-
-
- INTERNAL/EXTERNAL Overlays
- ---------------------------
-
- INTERNAL overlays are dynamic pages which are written to the end
- of the .EXEcutable program file. During the running of the
- application, the dynamic overlay manager reads the disk and loads
- dynamic pages from this file. This is the default overlay
- method of RTLINK and BLINKER and it is often advantageous to use
- internal overlays to reduce the number of file handles being
- opened by your application. In some cases however, you may
- find that using multiple overlay files will increase the
- speed performance of your application because extracting code
- from several small files which are all opened at the same time
- is often faster that extracting code from one large file.
-
- EXTERNAL overlays are sets of dynamic pages which are written to
- a seperately designated file more commonly referred to as an
- overlay file. The positional command /DYNAMIC[:<ovlfile>] or the
- freeformat command DYNAMIC INTO <ovlfile> will tell RTLINK to
- write the dynamic pages to a file named <ovlfile>. You may
- wish to create a seperate overlay file when your application
- .EXE is too large to fit on a distribution disk.
-
- Examples:
-
- Rtlink> DYNAMIC INTO myprog.ovl
- Blinker> SECTION INTO myprog.ovl
-
- Rtlink> DYNAMIC INTO myprogs.ovl MYPROG1, MYPROG2
- DYNAMIC INTO myfuncs.ovl MYFUNC1, MYFUNC2
- Blinker> SECTION INTO myprogs.ovl MYPROG1, MYPROG2
- SECTION INTO myfuncs.ovl MYFUNC1, MYFUNC2
-
-
-
- ╔══════════════════════════════════════╗
- ║ DYNAMIC Overlaying of C / ASM Code ║
- ╚══════════════════════════════════════╝
-
- BLINKER also provides the ability to dynamically overlay code
- written in C or Assembly. This is the most common code found
- in many third-party libraries. RTLINK also provides for
- overlaying of C/ASM but it is more complex and requires
- understanding more about the code being overlayed. See
- STATIC OVERLAYS and RELOADABLE OVERLAYS for more information
- on overlaying C/ASM code with RTLINK.
-
- In the previous chapter, we discussed that application code
- compiled with the Clipper-compiler is dynamically overlayed
- using the overlay manager built inside the Clipper libraries.
- This Clipper-specific internal overlay manager is incapable
- of overlaying C/ASM code, so it is the responsibility of the
- linker to provide this capability. Each linker has it's own
- methods for overlaying this kind of code. BLINKER uses an
- overlay pool that is allocated at the start of the application.
- The size of this pool is determined at link-time. The larger
- the overlay pool, the faster the application will run because
- overlays will not be reloaded into memory from disk so often.
-
- The dynamic-overlay capabilities of BLINKER require that the
- objects being overlayed are "well-behaved". After much
- experimentation with our own libraries and many third-party
- libraries, it is now more clear what constitutes well-behaved
- modules:
-
- a. Well-behaved code uses the Clipper EXTEND interface,
- registers or the stack for parameter passing and does
- not use undocumented features of Clipper.
-
- b. All Clipper-compiled .OBJects are dynamically-
- overlayable.
-
- Modules which CANNOT be dynamically-overlaid are:
-
- a. Routines which handle interrupts.
-
- b. Modules in which the DATA area is changed during
- runtime rather than allocating memory in the root area.
-
- Dynamic overlaying of third-party libraries can be an
- aggravating learning experience if you don't get support from
- the third party vendor in trying to accomplish this task.
- Most of the third-party vendors now break their libraries
- into separate overlayable and non-overlayable .LIB files to
- make the task much easier.
-
- Once you have determined which .OBJ files and/or .LIB files
- are acceptable for overlaying, the task of telling the
- linker to do this is quite simple. A BLINKER link-script
- will look similar to an RTLINK script except there is always
- only one (1) overlay area because all code uses the same
- overlay "pool".
-
- Example of a BLINKER script for overlaying C/ASM:
-
- # Allocate 50k to the overlay pool
- BLINKER OVERLAY OPSIZE 50
- BEGINAREA
- # Clipper-compiled application code
- FILE myprog1.obj, myprog2.obj
- # C/ASM files
- FILE ccode.obj, asmcode.obj
- # C/ASM libraries
- LIB ccode.lib, asmcode.lib
- ENDAREA
-
-
- ╔══════════════════════════════════════════╗
- ║ Using EXPANDED or UMB RAM for OVERLAYS ║
- ╚══════════════════════════════════════════╝
-
- BLINKER provides a several very useful feature for increasing
- the amount of available memory for Clipper applications, i.e.,
- the use of Expanded (EMS) or Upper Memory Blocks (UMB).
-
- -- EMS --
-
- Many computers have EMS memory. EMS memory managers such as
- QEMM, 386MAX and DOS 5.0 create a memory area above
- conventional memory referred to as the "page frame". This 64k
- block of memory is mapped to an area of ram that can execute
- 8086 code. Blinker takes advantage of this feature by
- allocating the 64k EMS page frame rather than conventional DOS
- memory for their runtime overlay pool. This frees an additional
- 64k of memory for use by Clipper because the overlays are loaded
- into the page frame rather than conventional ram. If no EMS is
- present when the application starts, then the overlay pool will
- be established in normal (conventional) ram. To enable this
- feature, use the Blinker command:
-
- BLINKER OVERLAY PAGEFRAME ON
-
- -- UMB --
-
- Expanded memory managers will also create an area between the
- 640k and 1 meg memory address called UMB (upper memory blocks).
- This is the area where you would normally load your memory-
- resident programs such as DOS, network-drivers, etc. If your
- enviroment gives you about 40k - 60k of contiguous memory
- available in the UMB area, Blinker will load the overlay pool
- into the UMB area rather than conventional area, thus freeing
- more memory for your Clipper application. To enable this
- feature, use the Blinker command:
-
- BLINKER OVERLAY UMB ON
-
-
- ╔═══════════════════╗
- ║ STATIC Overlays ║
- ╚═══════════════════╝
-
- STATIC overlays are supported by RTLINK only. They provide a
- method of overlaying C and ASM code which cannot be overlayed by
- the dynamic-overlay manager. C and ASM code is not as easily
- relocatable as Clipper compiled PCODE (pseudo-code) therefore it
- cannot be overlayed "dynamically". A function which is
- dynamically loaded and unloaded from a pool is usually found in
- different places in memory at different times during the running
- of the program. When a C or assembly routine is called by an
- application linked with RTLINK it must be loaded in memory at
- the same address every time. This is accomplished by
- designating overlay AREAS where many different modules will
- occupy the same memory space, but only one-at-a-time.
- Designing Static overlays requires a more thorough understanding
- of the structure of your program.
-
- Static overlay segments usually look like this in your link
- file:
-
- # area 1
- BEGINAREA
- SECTION FILE A
- SECTION FILE B
- SECTION FILE C
- ENDAREA
-
- # area 2
- BEGINAREA
- SECTION FILE D
- SECTION FILE E
- SECTION FILE F
- ENDAREA
-
- You must be sure that you properly place your objects in the
- overlay segments to prevent computer "lock-up" or "crashing"
- at runtime. In the above example, any procedure or function in
- FILE A may call any procedure or function in FILE D, E or F
- because they are in different areas, however if a function in
- FILE A calls a procedure or function in FILE B, then your program
- will crash as soon as the program returns to FILE A because FILE A
- was removed from memory to load FILE B.
-
- Overlay management with "static overlays" is the most time-
- consuming because it requires extensive analysis of the
- structure of the program to insure that the modules are placed
- properly in the overlay areas.
-
-
-
- ╔═══════════════════════╗
- ║ RELOADABLE Overlays ║
- ╚═══════════════════════╝
-
- The following overlaying technique may be considered
- controversial because it is an "unsupported" and "undocumented"
- feature of the RTLINK linker supplied with Clipper-5.0. I feel
- however, that it is important to cover this subject because
- without an understanding of how to use this feature it may be
- impossible to develop certain types of Clipper applications.
- dCLIP is an example of a Clipper application that would not have
- been possible if I could not use "reloadable" overlays. Please
- be aware that this subject material is not supported by
- Nantucket and is provided as information to be used at your own
- risk.
-
- I have been RELOADING objects from the Summer 87 Clipper libraries
- for years, so when I received my copy of Clipper-5.0, the first
- thing I attempted to do was to overlay some of the larger C/ASM
- modules in the Clipper libraries using the RELOAD command in
- my linker script file. The Nantucket development team had the
- foresight to insure that their C and ASM compiler assigned a
- UNIQUE name to each Clipper C/ASM object in the Clipper libraries,
- thus allowing, you, the Clipper programmer to use both the
- RELOAD command and MODULE command in your .LNK script files.
-
- So what are "Reloadable Overlays"? These are a form of STATIC
- overlay in which segments of code are placed into overlay sections
- that occupy the same memory space at runtime, however the "calling
- module" is automatically "reloaded" into memory when returning
- from the "called module".
-
- Reloadable overlay segments usually look like this in your link
- file:
-
- RELOAD FAR 200
- # area 1
- BEGINAREA
- SECTION FILE A
- SECTION FILE B
- SECTION FILE C
- ENDAREA
-
- # area 2
- BEGINAREA
- SECTION FILE D
- SECTION FILE E
- SECTION FILE F
- ENDAREA
-
-
- Not all Clipper applications are going to need to use these
- reloadable static overlays, but if your application relies
- heavily on third-party libraries written in C/ASM, then it is
- recommended that you learn this technique.
-
- The main advantage of "reloadable overlays" over conventional
- "static overlays" is that you are never in danger of "lockup"
- in the event that a procedure or function in FILE A calls a
- function or procedure in FILE B because FILE A will be reloaded
- into memory on return from FILE B. Make sure when you use the
- RELOAD command that you always use it as follows:
-
- RELOAD FAR <stack size>
-
- where the <stack size> is the amount of memory in hex bytes to
- use for saving addresses. My experience is that most
- applications will run just fine with a stack size of 200. If
- your application does recursive nesting, however, you may need
- to increase the stack size. Recursive nesting is described as
- follows: Procedure A calls Procedure B which calls Procedure A
- which calls Procedure B, etc, etc, etc. Each time a procedure
- is called, its return address is "pushed" onto the stack, and
- each time you return to the calling procedure the address is
- "popped" from the stack. In the above example, if this
- recursive condition were allowed to continue for a large number
- of iterations, the stack would be overrun and the program may
- crash.
-
- Although your application will not crash using RELOAD, it may
- slow down a bit if overlay segments are not structured properly,
- because each time an overlay is reloaded the application must
- go to disk. In the above example if a function in FILE A
- repetitively calls a function in FILE B then your application
- will be very "disk intensive" and will run slowly. If a
- function in FILE A repetitively calls a function in FILE D there
- will be no slowing at all because both modules will remain in
- memory.
-
- Overlay management of C/ASM code with "reloadable overlays" is
- less time-consuming and more reliable than simple conventional
- "static overlays" and provides an additional advantage of
- allowing more modules to be overlayed, thereby saving additional
- memory usage.
-
-
-
- ╔════════════════════════════╗
- ║ OVERLAYING LIBRARY MODULES ║
- ╚════════════════════════════╝
-
- The MODULE command provides the important capability of
- overlaying the larger modules of CLIPPER.LIB or any other
- third-party library. With previous versions of these linkers,
- the overlaying of libraries was restricted to ALL modules in a
- library or NO modules in a library. This is ok for libraries
- which were designed for overlaying, but in most cases it is
- simply not practical to overlay an entire library due to speed
- performance problems. There are many modules in CLIPPER.LIB
- which can be overlayed without crashing the program, yet runtime
- performance can be disastrous. Determining the proper "set" of
- modules and the optimum "overlay pool size" is simply a matter
- of trial and error. For example, overlaying the MACRO module in
- one application may have very little affect on speed performance
- because of the design of the program and it's probable negligible
- On the contrary, it was completely unacceptable to overlay MACRO
- in the dCLIP engine because dCLIP relies heavily on the MACRO
- compiler for many of it's interpretive operations.
-
- The MODULE command is very useful in that it allows your linker
- to place "modules" from libraries into overlays. You can
- organize your projects by placing all your C/ASM objects in
- libraries with a library manager such as Microsoft's LIB.EXE
- then by using the SECTION MODULE <module> command in your link
- file you can place any module into any overlay area.
-
- Rtlink:
-
- LIB mylib
- BEGINAREA
- SECTION MODULE myfileA,myfileB
- SECTION MODULE myfileC
- SECTION MODULE myfileD
- ENDAREA
-
- The MODULE <module> command in RTLINK requires that the <module>
- exists in one of the declared libraries and that the name is
- "unique" to that module otherwise the command will be ignored and
- the module will be linked into the root memory area. This is not
- a problem with the Clipper libraries because the modules are given
- unique names.
-
- BLINKER supports the MODULE command differently and, in my
- opinion, better than RTLINK. Blinker allows you to define a
- specific module from a specific library to be linked into the
- application, thus you don't need to worry about the module being
- unique to one library. For example, if you have two libraries,
- both containing an ERRORSYS module, you can decide which ERRORSYS
- you want linked into the program.
-
- Blinker:
-
- LIB mylib
- BEGINAREA
- LIB grump
- MODULE errorsys FROM grump
- MODULE myfileA,myfileB
- MODULE myfileC
- MODULE myfileD
- ENDAREA
-
-
- DETERMINING THE MODULE NAMES IN A LIBRARY
- ------------------------------------------
-
- Every ".OBJect" file placed into a library is also assigned a
- "module" name. This module name is stored in the "COMENT"
- record of the object in the library. The "module" name of an
- object is assigned by the compiler of the object and is usually
- given the name of the original source code. The Clipper
- compiler assigns the same name as the .OBJ file, however many
- compilers will assign the name of the SOURCE file as the module
- name including drive letters and directories. For example, let's
- say we want to place the Funcky FINDATTR() function into an
- overlay. First, we must figure out what "module" name was
- assigned to the FINDATTR() function. There are library manager
- utility programs which can help you figure this out, but I'm
- going to show you how to do it simply with your linker.
- During VERBOSE linking the "module" name of each object being
- linked into the program is displayed on the screen as follows:
-
- FUNCKY50.LIB(C_MAXCHO) <- Clipper code
- FUNCKY50.LIB(C_PUTKEY) <- Clipper code
- FUNCKY15.LIB(findattr.C) <- "C" code
- FUNCKY15.LIB(finddate.C) <- "C" code
- FUNCKY15.LIB(findfirs.C) <- "C" code
- FUNCKY15.LIB(chrfound.ASM) <- "Assembly" code
- FUNCKY15.LIB(strcente.ASM) <- "Assembly" code
-
-
- You must make sure that you insert the command VERBOSE into your
- link script file to insure that this information is displayed.
- Next, when you perform the link, make sure route the dislay
- output to a file so you can have the information in text file
- format:
-
- RTLINK @MYPROG > MYPROG.TXT
-
- MYPROG.TXT will contain the complete list of modules linked into
- your program. Now, we must take this information and use it to
- develop an overlay strategy. The name of each module linked into
- the program is shown (in parenthesis) and must be referred to
- "exactly" as it is spelled. For example, to place the function
- FINDATTR() into an overlay, we must first determine the "module"
- name which would contain the FINDATTR() function. Fortunately,
- the FUNCKY15 library uses the same prefix name as the actual
- function, so in perusing the list of linked objects in MYPROG.TXT
- we find a module named "findattr.c". To place this module in an
- overlay section, simply use the command:
-
- SECTION MODULE findattr.c
-
- BLINKER's MODULE command is more useful than RTLINK's because
- it searches both the THEADR record and the COMENT record for the
- name of each module. This means that you don't need to
- reference the MODULE name as precisely with these linkers as with
- RTLINK. For example, if the module name in a library is
- FINDATTR.C, BLINKER will overlay it by with the following
- command: MODULE FINDATTR, because it will determine that what
- you are trying to overlay is FINDATTR.OBJ.
-
-
- OVERLAYING THE CLIPPER LIBRARIES
- ---------------------------------
-
- Overlaying the Clipper libraries is accomplished quite
- differently with each linker, therefore I have included
- examples for each linker.
-
- ----------
- RTLINK
- ----------
-
- RTLINK uses Static reloadable overlays with and the MODULE
- command to overlay much of the Clipper libraries and thereby
- reduce memory usage in your Clipper-5.0 applications. The
- below link script example creates an overlay area for overlaying
- the larger modules in the Clipper libraries which are not likely
- to call each other recursively, therefore you will probably
- notice very little difference in speed performance yet you will
- get up to 50K more memory overhead depending on how much of the
- Clipper libraries your application uses.
-
- You may get a "warning" message during link time if your
- application does not call one of the modules referenced in an
- overlay area. In the event this happens, simply remove that
- module from the link file. For example, if you are not using
- MEMOEDIT() in your application remove the line:
-
- SECTION MODULE 50\MEMOEDIT
-
-
-
- FI <my files>
- LIB <my libs>
- OUTPUT <my .EXE>
- RELOAD FAR 200
- BEGINAREA
- SECTION MODULE 50\MEMOEDIT, 50\MEMOTRAN, 50\MEMOREAD
- MODULE 50\MEMOWRIT, 50\MEMOLINE, 50\MLCOUNT
- MODULE 50\MLPOS
- SECTION MODULE 50\IS, 50\EXAMPLEC, 50\HARDCR, 50\PAD
- MODULE 50\PADL, 50\PADC, 50\PADR, 50\ASCAN
- MODULE 50\ASORT, 50\DIRECTRY, 50\AEVAL, 50\ACOPY
- MODULE 50\ADEL, 50\AINS, 50\COPYFILE, 50\TYPEFILE
- MODULE 50\SCROLL, 50\GETE, 50\DISKSPAC
- SECTION MODULE 50\SDF1, 50\SDF0, 50\SDFDYN, 50\DLM1
- MODULE 50\DLM0, 50\DELIMDYN, 50\DBCREATE
- MODULE 50\DBJUNCT, 50\DBSTRUCT, 50\ACHOICE
- SECTION MODULE 50\NET, 50\ACCEPT, 50\DATE, 50\OSDATE
- SECTION MODULE 50\FGET, 50\OLDBOX, 50\OLDCLEAR, 50\RUN
- MODULE 50\SEND, 50\JOINLIST, 50\SORTOF, 50\SORTBLOC
- ENDAREA
- BEGINAREA
- SECTION MODULE 50\TBROWSE0
- SECTION MODULE 50\TBROWSE1
- SECTION MODULE 50\EXAMPLEA, 50\PHILES, 50\DBF0, 50\DTX0
- MODULE 50\INITEXIT, 50\TXOPEN, 50\BOX, 50\STRTRAN
- ENDAREA
-
- When placing modules into overlays, you must use extreme care to
- not overlay functions which are called by "interrupts" or may be
- called "recursively". Code which is called by interrupts is
- called by a direct vector rather than an overlay vector this can
- cause computer lockup if the code segment is not currently in
- memory. Code which is called recursively would be something like
- special string handling routines that might be called in a
- do..while loop. If this function were placed in an overlay, you
- may experience considerable slowing and/or disk access.
-
- Many third-party library developers now distribute their product
- in two libraries: a "resident" library and an "overlayable"
- library. Most of the work has already been done for you in
- determining which modules can be overlayed. If the module is part
- of the "resident" library, don't make any attempt to try to
- overlay it or you will experience runtime problems. Also, don't
- make an attempt to overlay "Clipper code" because this code is
- automatically overlayed into "dynamic-pages".
-
-
- -----------
- BLINKER
- -----------
-
- Overlaying the Clipper libraries with Blinker is simpler than
- Rtlink because only one overlay area needs to be defined for
- all overlayable modules, files and libraries. Like Rtlink,
- Blinker will automatically overlay any code in any library
- that is compiled with the Clipper compiler.
-
- BLINKER OVERLAY OPSIZE 50
- BLINKER PROCEDURE DEPTH 50
- FI <my root files>
- LIB <my root libs>
- OUTPUT <my .EXE>
- BEGINAREA
- FILE <my overlayable files>
- ALLOCATE <my overlayable libraries>
- # Modules from CLIPPER.LIB
- MOD accept,acopy,adel,aeval,ains,appinit,atail,box,cmem,date
- MOD dbcmd2,dbcmd3,dbcmd4,dbcmd5,dbcreate,dbf0,dbfdyn,dbjunct
- MOD dbstruct,dtx0,dtx1,dtxdyn,dynina,errsys0,errsys1,fget
- MOD getenv,gets0,gets1,gets2,gx,joinlist,lupdate,memory
- MOD mrelease,msave,net,oldbox,oldclear,philes,run,saverest
- MOD sdf0,sdf1,seq,setcurs,sortbloc,startsym,tb,txopen,vall
- MOD vblock,vdb,vdbg,version,vmacro,vnone,vops,vpict,vterm
- MOD workarea,xmacro
- ALLOCATE extend
- ENDAREA
- LIB clipper, terminal ,dbfntx
-
-
- ╔══════════════════════╗
- ║ SPEED OPTIMIZATION ║
- ╚══════════════════════╝
-
- Most features of the new generation of linkers are designed
- for memory optimization. Unfortunately, improving memory
- performance "almost always" works against speed performance
- when the job is being handled by the linker. Overlaying of
- code requires that code segments be swapped in and out of
- memory and that "overlay vectors" rather than "direct vectors"
- are used to access the code. This increased overhead will
- slow down the application and often-times create unacceptable
- runtime performance.
-
- Simply overlaying an entire library is not always the best
- choice when speed performance is critical. For example, a
- function that skips a record pointer through a database must
- be called once for each record. If this function is in an
- overlay then the speed of browsing a database, locating a
- record, or reindexing a file can be greatly affected because
- the function must be called via the overlay manager. Speed
- is not usually a problem when overlaying code that has been
- compiled by Clipper, because the Clipper-code overlay manager
- is actually part of the Clipper libaries and is very efficient
- in the manner in which it dynamically overlays pages of P-CODE
- (Clipper .OBJs). C and ASM native code on the other hand must
- be overlayed by the linker's overlay manager and can greatly
- affect performance if an often-used function is not in the
- "root" memory area.
-
- In my opinion, there is no good rule-of-thumb for determining
- which modules should be overlayed other than the time-tested
- method of "trial and error". I have spent hours and hours
- testing different overlay strategies to end up with the
- just-right balance of speed vs memory optimization. This
- investment of time has always payed me great dividends because
- from then on I was confident that I done my homework and that
- my application was "perfectly tuned".
-
- The linkers included in this discussion offer some other fine
- features that can provide improved speed performance without
- sacrificing memory performance, however, they require that you
- utilize some of the advanced features of your computer
- hardware to get the benefit.
-
- 1. OVERLAY CACHING
-
- Both RTLINK and BLINKER (release 2.0) support the feature of
- "caching" overlays to improve runtime speed on computers with
- EMS or XMS memory. A "cache" is an area of memory that has
- been pre-allocated for storing code segments that have been
- overlayed and need to be swapped out to make room in memory
- for new code. Rather than discarding the code segment and
- then reloading from disk each time it is needed, it can be
- reloaded from the "cache" memory. Obviously, memory to memory
- loading is much faster than disk to memory loading.
-
- Blinker
- --------
-
- Blinker 2.0 uses the following commands for caching with XMS
- (2.0 and higher):
-
- BLINKER CACHE XMS niMax[%], niMinLeave[%]
-
- For example the command BLINKER CACHE XMS 50%, 1024 will use
- a maximum of 50% of all available XMS, but leave at least
- 1024KB available for other programs.
-
- Even faster caching is available by using EMS (extended memory)
- drivers because memory is handled in large contiguous blocks
- rather than the smaller 64k "pages" such as the XMS (expanded
- memory) drivers. Use the following command for caching using
- EMS (4.0 and higher)
-
- BLINKER CACHE EMS niMax[%], niMinLeave[%]
-
-
- Rtlink
- -------
-
- Rtlink's caching system is only useful when using STATIC
- overlays. In a previous chapter, I showed you how to overlay
- the Clipper libraries and third-party libraries using the
- RELOAD command, BEGINAREA..ENDAREA, and the MODULE command.
-
- To improve overlaying performance of C/ASM code in STATIC
- overlays, use the following Rtlink caching commands:
-
- CACHE EXPANDED <memory in K> | <% of available>
- CACHE EXTENDED <memory in K> | <% of available>
-
-
- 2. OVERLAY POOL SIZE
-
- Blinker provides for control of the overlay pool size to
- allow more code segments to remain in memory and prevent
- excessive "swapping" in and out of the pool. Increasing
- the size of the overlay pool will usually have the affect
- of improving speed performance, but not always. This is
- another of those "trial and error" options. The disadvantage
- of increasing the pool size is that the application memory
- will be reduced, so don't use more overlay pool than is
- needed for the application.
-
- Blinker: BLINKER OVERLAY OPSIZE <nMemoryK>
-
-
- 3. FORCING SEGMENTS TO ROOT
-
- Many times when overlaying a library, it is desirable to
- "exclude" code segments from being overlayed to improve
- speed. Most often, the reason for the exclusion is
- because the segment is used globally by many routines or
- in loops that perform thousands of operations. Sometimes,
- a segment is just so large that overlaying causes it to
- be swapped in and out of memory much too often. There are
- several methods to help you with moving code from an
- overlayed library to the "root" memory.
-
- Rtlink
- -------
- All modules in libraries that are Clipper-compiled .OBJs
- will be automatically overlayed when the library is declared
- with the LIB command in your link script. To force a
- Clipper module from a library to link to the root memory area,
- use the RESIDENT command and the MODULE command as follows:
-
- # MYLIB.lib contains my clipper code
- LIB MYLIB
- RESIDENT
- # force MYFUNCS.obj and MYPROCS.obj to the root
- MODULE MYFUNCS, MYPROCS
- # everything from now on is overlayed
- DYNAMIC
-
- Blinker
- -------
- Specified modules in Clipper-compiled libraries and C/ASM
- libraries which have been overlayed can be forced to root
- memory by using the MODULE command "outside" the BEGIN...
- ENDAREA. Example:
-
- BEGINAREA
- # overlay my Clipper application library
- LIB MYLIB
- # overlay my C library
- LIB CFUNCS
- ENDAREA
- # force MYFUNCS.obj and MYPROCS.obj to the root
- MODULE MYFUNCS, MYPROCS
- # force CFUNCS.obj to the root
- MODULE CFUNCS
-
- Blinker 2.0 also supports a handy feature for automatically
- forcing segments that are smaller than a specified size to
- the root area. Very small rountines (less than 32 bytes)
- take up more root space when they are overlaid, due to the
- overlay manager table entries, than they would if they
- were simply left in the root. Additionally, very small
- overlaid routines execute disproportionately slower, since
- the time overhead in loading / calling them may be large
- in comparison with the actual execution time of the routine.
-
- BLINKER OVERLAY THRESHOLD <nBytes> will allow you to force
- all segments that are smaller than <nBytes> into the root.
-
-
-
-
- ╔═══════════════════════════════════════════════╗
- ║ PROGRAMMING TECHIQUES FOR MEMORY MANAGEMENT ║
- ╚═══════════════════════════════════════════════╝
-
-
- ╔═════════════════════╗
- ║ SYMBOL Management ║
- ╚═════════════════════╝
-
- For a dynamic-overlay manager to work effectively, it is
- important that it load overlays at the smallest possible code-
- segment level, i.e., the procedure/function rather than the
- entire object. This requires managing the symbol table
- seperately from the code and data, therefore, these linkers
- place the symbol table into the root memory area and each
- function into a separate overlay segment.
-
- Symbols cannot be overlayed, therefore it is important that
- you program in a manner consistent with producing the smallest
- number of symbols in your Clipper-compiled objects. Here are
- some tips for reducing the symbol table size in your
- applications.
-
-
- 1. COMPILE SMALLER SEGMENTS ( C/ASM )
-
- If you are writing code in C or Assembly, then the overlay
- manager must bring an entire code segment into memory at one
- time. If you have many large code segments, then the size
- of the overlay pool may need to be increased. Normally, the
- requirements of the application dictate the size of the code
- segments, but it should be noted that a little extra time
- in optimizing the size of your code will usually pay off if
- you have a large library of routines.
-
- Clipper-compiled code is loaded "dynamically" so this is not
- a requirement when programming in Clipper.
-
-
- 2. USE CONSTANTS INSTEAD OF MEMVARS
-
- Clipper memory variables are always treated as "symbols".
- Refrain from using a memory variable if a constant is
- sufficient. For example, an unnecessary symbol can be
- eliminated by changing the code:
-
- escapekey=27
- DO WHILE INKEY()#escapekey
- *clipper code
- ENDDO
-
- to:
-
- DO WHILE INKEY()#27
- *clipper code
- ENDDO
-
- or:
-
- #define K_ESC 27
- DO WHILE INKEY()#K_ESC
- *clipper code
- ENDDO
-
-
- 3. USE ARRAYS INSTEAD OF MEMVARS
-
- Every different Clipper memvar name creates a "symbol", whereas
- an array name creates only ONE symbol. The following example
- shows how to save considerable memory in a clipper application
- by reducing the symbol count with an array.
-
- This code produces 10 symbols:
-
- mname = name
- maddress = address
- mcity = city
- mstate = state
- mzip = zip
- @ 1,1 SAY 'Name ' GET mname
- @ 2,1 SAY 'Address' GET maddress
- @ 3,1 SAY 'City ' GET mcity
- @ 4,1 SAY 'State ' GET mstate
- @ 5,1 SAY 'Zip ' GET mzip
- READ
- REPL name WITH mname, address WITH maddress,;
- city WITH mcity, state WITH mstate, zip WITH mzip
-
-
- This code produces 6 symbols:
-
- PRIVATE gets[5]
- gets[1] = name
- gets[2] = address
- gets[3] = city
- gets[4] = state
- gets[5] = zip
- @ 1,1 SAY 'Name ' GET gets[1]
- @ 2,1 SAY 'Address' GET gets[2]
- @ 3,1 SAY 'City ' GET gets[3]
- @ 4,1 SAY 'State ' GET gets[4]
- @ 5,1 SAY 'Zip ' GET gets[5]
- READ
- REPL name WITH gets[1], address WITH gets[2],;
- city WITH gets[3], state WITH gets[4], zip WITH gets[5]
-
-
- The following code also produces 6 symbols, however the 5.0
- pre-processor is used here to create both DEBUG and NON-DEBUG
- versions of compiled object code. In the following example
- the pre-processor is used to convert the 5 memvars to an array
- when compiling the final version.
-
- To compile the below code WITHOUT symbol substitution for
- debugging purposes, compile as follows:
-
- CLIPPER <my app> /dDEBUG
-
- To compile the below code WITH symbol substitution for your
- final application, compile as follows:
-
- CLIPPER <my app>
-
- The /dDEBUG switch has the same effect as the comand:
-
- #define DEBUG
-
- in your source code, however, it is much more convenient
- because it allows you to make debug or non-debug versions of
- your code simply from the DOS command line or .BAT files.
-
- #ifndef DEBUG && compiling non-DEBUG version
- LOCAL gets[5]
- #define mname gets[1]
- #define maddress gets[2]
- #define mcity gets[3]
- #define mstate gets[4]
- #define mzip gets[5]
- #endif
- mname = name
- maddress = address
- mcity = city
- mstate = state
- mzip = zip
- @ 1,1 SAY 'Name ' GET mname
- @ 2,1 SAY 'Address' GET maddress
- @ 3,1 SAY 'City ' GET mcity
- @ 4,1 SAY 'State ' GET mstate
- @ 5,1 SAY 'Zip ' GET mzip
- READ
- REPL name WITH mname, address WITH maddress,;
- city WITH mcity, state WITH mstate, zip WITH mzip
-
-
-
- 4. USE THE SAME NAME MEMVARS WHENEVER POSSIBLE
-
- Again, every "different" Clipper memvar in a module creates a
- symbol. If an object contains several procedures, use the
- same name for memvars even though they may not perform the same
- or similar functions. For example, procedure A and procedure
- B both need 5 memvars. If procedure A declares its memvars with
- 5 unique names and procedure B declares its memvars with 5
- unique names, then 10 symbols are used in the linked
- application. To eliminate 5 symbols, make sure that procedure
- B assigns the same name to the memvars as procedure A. This
- is not possible of course, if the memvars need to be public to
- both procedures and perform different functions, only if they
- are private.
-
-
- 5. USE COMPLEX EXPRESSIONS INSTEAD OF MEMVARS
-
- The following three lines of codes represents the method that
- most Clipper programmers choose to accomplish most programming
- tasks. It makes sense to code this way for readability, but
- if you are writing a very large application, this technique
- can be very symbol-intensive. The following three lines of
- code will read the disk file READ.ME into a memvar named
- READ_FILE, save the changed code into a file named EDIT_FILE,
- then write the changed code back to the disk file READ.ME.
-
- read_file=MEMOREAD('READ.ME')
- edit_file=MEMOEDIT(read_file)
- MEMOWRIT('READ.ME',edit_file)
-
- These three lines of code can be replaced by one complex
- expression which uses no symbols at all.
-
- MEMOWRIT("READ.ME",MEMOEDIT(MEMOREAD("READ.ME")))
-
- An additional advantage to coding this way is that less
- free-pool or VMM memory is used because the process of
- temporarily storing the text in READ_FILE and EDIT_FILE is
- completely eliminated. If you find that creating complex
- expressions such as this are unreadable and hard to maintain
- then use the 5.0 pre-processor to accomplish the same task as
- follows:
-
- # DEFINE read_file MEMOREAD('READ.ME')
- # DEFINE edit_file MEMOEDIT(read_file)
- MEMOWRIT('READ.ME',edit_file)
-
- The above code is just as readable as the original three lines
- of code but will not create any symbols at compile time. Try
- compiling this code with your Clipper 5.0 compiler and use the
- /P switch to write the pre-processed code to a .PPO file, then
- look at the .PPO file to see what is actually compiled.
-
-
- 6. USE LOCALS AND STATICS INSTEAD OF PRIVATES AND PUBLICS
-
- Sometimes it is just not possible or practical to write code
- without using symbols, so if you find yourself in this
- situation, Clipper 5.0 provides the new feature of
- "LOCALIZING" symbols to the code segment which is currently
- being executed rather than placing the symbol in the main
- symbol table. Local symbols are effectively "overlayed"
- because they are treated as part of the code segment rather
- than given a place in the main symbol table.
-
- Not only does this save valuable memory but it also improves
- speed performance because the public symbol table does not need
- to be searched each time a STATIC or LOCAL symbol is referenced
- in your code. Of course, if the symbol you are referencing is
- needed for the entire application or is used in a macro, then it
- must be declared as PRIVATE or PUBLIC. Symbols which are not
- declared at all are automatically assumed to be PRIVATE, so make
- sure you use the LOCAL declaration for all symbols in your code
- which you do not want to end up in the main symbol table.
-
- The following example shows how to save considerable memory in
- a clipper application by reducing the symbol count with LOCAL
- declarations.
-
- This code produces 6 main memory symbols:
-
- PRIVATE mcity,mstate,mzip
- mcity = city
- mstate = state
- mzip = zip
- @ 3,1 SAY 'City ' GET mcity
- @ 4,1 SAY 'State ' GET mstate
- @ 5,1 SAY 'Zip ' GET mzip
- READ
- REPL city WITH mcity, state WITH mstate, zip WITH mzip
-
-
- This code produces 3 main memory symbols and 3 local symbols:
-
- LOCAL mcity := city, mstate := state, mzip := zip
- @ 3,1 SAY 'City ' GET mcity
- @ 4,1 SAY 'State ' GET mstate
- @ 5,1 SAY 'Zip ' GET mzip
- READ
- REPL city WITH mcity, state WITH mstate, zip WITH mzip
-
-
- ╔══════════════════════════════════╗
- ║ MEMORY EFFECTS OF DATABASE USAGE ║
- ╚══════════════════════════════════╝
-
- Many programmers have taken an affection to "data-driven"
- programming to speed up the development of custom applications.
- As a developer of programming tools, rather than custom
- applications, I find that data-driven programming techniques
- don't serve me well due to the impact on both speed and memory
- performance. This seminar is dedicated to memory issues so
- we won't discuss the speed issues. Since Clipper evolved as
- an X-Base type language, the common approach to data-driven
- programming is to create an "engine" or "kernel" .EXE program
- that retrieves "custom" information about the application
- from a set of database files. The more sophisticated the
- system, the more data files are required for configuring the
- application. It is important to understand the impact of
- databases on memory usage when designing applications that use
- many databases.
-
- When a database is opened with the USE command, all the field
- names are placed into the public symbol table. This allows
- database field names to be used in expressions in the same
- manner as memvars. Because this symbol table is PUBLIC, the
- field names will remain in the symbol table, even after the
- database is closed. Clipper employs no mechanism to remove
- symbols from the symbol table, only to add them. As each
- database is opened the symbols are added, thereby reducing
- the remaining available root memory ( MEMORY(0) ). It is
- conceivable that an application could run out of memory if
- many databases are opened and closed. Fortunately, symbols
- of the same name are "reused" in the symbol table, so if you
- open and close the same database many times, the symbol memory
- space will not be reduced. Keeping this in mind, it is a
- good practice to use databases with fields of the same name.
-
- To improve speed operation, some programmers will open data-
- dictionary databases at the start-up of the program, load
- the custom program configuration into arrays, then close the
- databases. This action will reduce the amount of VMM memory
- needed for file buffers and lower the number of DOS handles
- required, but the symbols will still be added to the symbol
- table even though they may never be accessed by the program.
- A data-driven application in which the databases are used
- only during the start-up of the application could be re-designed
- to convert the database information to a text file to generate
- a "runtime" version of the application that will load the
- array(s) from a text file rather than databases, thereby
- eliminating the symbol-table problem.
-
-
-