Club Amiga de Montreal

home *** CD-ROM | disk | FTP | other *** search

/ Club Amiga de Montreal - CAM / CAM_CD_1.iso / files / 568a.lha / MemMan / MemMan.doc.pp / MemMan.doc

Wrap

Text File | 1991-11-10 | 19.3 KB | 413 lines

MemMan Low-Memory manager Copyright (C) 1991 Bryan Ford MemMan is freeware, NOT public domain. See MemMan.asm for details on distribution and use in your own programs. The MemMan distribution must contain the following files, complete and unmodified: MemMan.doc 19778 MemMan.asm 7073 MemMan.i 521 MemMan.h 933 MemMan.o 700 Macros.i 1405 MemMan is a short assembly language module which allows your application to be called whenever there is a memory shortage during an AllocMem() call. This allows your application to free up any memory it can do without so another program can use the memory. Resident libraries and devices use a similar system to automatically unload them if the system runs out of memory. However, MemMan provides two very important features that the standard expunging system doesn't. First, MemMan doesn't require an application to free everything it possibly can all at once. The application can free a little bit of memory, then MemMan will try the allocation again, and if the allocation still fails, the application can free more memory, and so on. Second, to go along with this, the application can tell MemMan how important a given piece of memory is. This allows the application to free some types of buffers or caches "at the drop of a hat" while keeping other more valuable items until the last minute. To use MemMan, you will need to link the "MemMan.o" file into your program. (MemMan was made to be used with SAS/C 5.10 or assembly language - you may need to make some adjustments to use it with other languages or compilers.) In program modules that use MemMan, you will need to include the header file "MemMan.h" or MemMan.i" as appropriate. At the very beginning of your program you must call the MMInit() function to initialize MemMan. At the very end, you must call MMFinish() to shut it down. During your program, you use the MMAddNode() and MMRemNode() functions to tell MemMan about things that can be freed during a memory crunch. All of these functions are described in detail later in the document. If you would like to make your own modifications to MemMan, you will probably need the A68k assembler by Charlie Gibbs. It may only work with a version LATER than version 2.71. This is because MemMan makes use of some enhancements to A68k which I have made just recently, and as of this writing, the new version has not been released yet. You can try it though - it may only require a few changes to get it to work with an older A68k or other assemblers. Also, MemMan.asm references the include files "memman.i" and "macros.i" in a subdirectory called "bry". You will either need to create this subdirectory on your system and put these files into that directory, or change MemMan.asm to reference them where you want them. I'd like to give a big thanks to Michael Sinz of Commodore-Amiga for his valuable suggestions for MemMan - it is now a whole lot more solid and system-friendly than it was when I sent it to him originally. (In fact, it's been basically rewritten since then...) The most reliable way of contacting me is at my parents' address. It may take a while for me to get something sent there, but it WILL get to me. I tend to move around a great deal, so mail sent directly to me sometimes has a hard time catching up. Send mail to: Bryan Ford 8749 Alta Hills Circle Sandy, UT 84093 I can be reached more quickly (for the time being anyway) on the phone or through one of the electronic mail addresses below: (801) 585-4619 bryan.ford@m.cc.utah.edu baf0863@cc.utah.edu baf0863@utahcca.bitnet If you want to get something to me through the mail reasonably quickly, FIRST call or E-mail me to make sure I'm still here, then send it to this address: Bryan Ford 27104 Ballif Hall University of Utah Salt Lake City, UT 84112 Enjoy! Functions ~~~~~~~~~ This section describes in detail the functions that MemMan makes available to the the application. These functions are defined as both assembly-style functions without any name prefix (such as '_' or '@') as well as names prefixed with '_' to allow calls from SAS/C. To call these functions from assembly language, all you need to do is 'xref' the names and put the appropriate arguments in the appropriate registers before 'bsr'- or 'jsr'ing to the function. For SAS/C 5.10, you just need to include the header file "MemMan.h" which prototypes these functions and tells the compiler which registers to use for arguments. For other compilers or languages, you may have to do a little adaptation. int Success = MMInit(void) D0 Initializes MemMan. You must call this before calling any other MemMan function except for MMFinish(). You must test the return code - if it is zero, it means that there wasn't enough memory to initialize. (The first time MemMan is initialized in the system, some code is made permanently resident. Other invocations of programs that use MemMan then use this code without allocating anything else.) You must NEVER call MMInit() more than once in your program! void MMFinish(void) Closes down MemMan. This never actually causes memory to be freed (the resident code remains in memory until reboot), but it puts the AllocMem() patch "to sleep" so it uses a minimum of extra CPU time while MemMan is not in use. It is not dangerous to call MMFinish() more than once, or to call it without ever calling MMInit(), or to call it after MMInit() failed. However, after you call MMFinish(), you may not call ANY other MemMan functions before your program terminates. Before calling MMFinish(), ALL MMNodes you own MUST have been taken off the list! Remember that the MMList is not private to your application - it is used by other applications that are running MemMan at the same time, and it remains in memory even after all applications using MemMan have terminated. (The same list will be used again the next time some program starts using MemMan.) If you leave any MMNodes on the list, chances are that they void MMAddNode(struct MMNode *node) A1 Adds an MMNode to the systemwide MMList. An MMNode represents an item that can be freed on demand. (See the 'MMNodes' section below for details on creating these structures.) This function has protection against adding a node twice, so as long as you don't mess with things you're not supposed to, you can call MMAddNode() any time you want to "make sure" a particular MMNode is on the list. This function is very simple and quick, especially if the node is already on the list, so you don't need to worry about calling this function often. void MMRemNode(struct MMNode *node) A1 Removes an MMNode from the MMList. As with MMAddNode(), you don't have to worry about calling this function with an MMNode that wasn't already on the list (as long as you call it with a valid MMNode). Also like MMAddNode(), this function executes very quickly, so there's no need to worry about calling it often. MMNodes ~~~~~~~ To use MemMan, you will need to create structures called MMNodes. This structure is defined in C as follows: struct MMNode { struct MinNode Node; /* Link into systemwide MMList */ char Linked; /* Flag (private - initialize to 0) */ char Pri; /* Priority, in line with ln_Pri */ long __regargs (*GetRidFunc)(long size,long memtype,void *data); void *GetRidData; /* Data to send to GetRidFunc */ }; Each MMNode structure is generally associated with one particular piece of memory (or several pieces closely tied together) that may be freed on demand when the system needs more memory. MMNodes are added to a single system-wide (one for all applications currently using MemMan) list by MMAddNode(), in order of priority. When a memory crunch occurs and some AllocMem() call is about to fail, MemMan starts traversing the system MMList. It first calls the MMNode with the highest priority, then retries the AllocMem() call once, then if it still fails, it tries the next lower MMNode in priority and tries again, and so on. If one of the calls succeeds in freeing up enough memory to allow the AllocMem() to succeed, MemMan will stop traversing the list immediately, so the lower-priority nodes aren't affected. If MemMan gets all the way through the MMList and still can't get enough memory to satisfy the AllocMem() request, it will finally fail and return NULL to the original caller. You can create an MMNode as either a static/global variable in your program, or allocate it dynamically with AllocMem(). If you use AllocMem(), you MUST either use the MEMF_CLEAR flag, or explicitly initialize the Linked field to zero before calling MMAddNode() or MMRemNode() with it. (Static/global variables always get initialized to zero before the program starts, so you don't have to worry about it in this case.) You should set the Pri field to an appropriate priority level. (See the 'Priorities' section for guidelines on how to select a priority level.) The GetRidFunc pointer should point to the function in your application which will be called during a memory crunch, in order to free some memory. (See the 'GetRidFuncs' section below for information on creating these functions.) The GetRidData pointer can be anything you want, and is simply passed to the GetRidFunc whenever it is called. Once you have created and initialized an MMNode, you can add it to the system list at any time with MMAddNode(), and remove it later with MMRemNode(). To reiterate what I said before, ALL nodes you add MUST be removed from the system list before you call MMFinish() and terminate your program. GetRidFuncs ~~~~~~~~~~~ The GetRidFunc pointed to by a MMNode is the function that MemMan will call whenever there is a memory crunch and it needs you to free some memory. It will be called with the following arguments: void GetRidFunc(long MemSize, long MemType, void *GetRidData) D0 D1 A0 The first two arguments are simply the arguments from the AllocMem() call that is causing the memory crunch. In general, you shouldn't need to look at MemSize, but it's there just in case you find some use for it. You can look at the MemType parameter to see if giving up your memory will actually benefit the caller at all. For example, if the caller specifies MEMF_CHIP and you know for *sure* that the memory you can free is *not* chip memory, then you don't need to free your memory. (Remember, though, that AllocMem() may have given you chip memory even though you didn't specifically ask for it.) The GetRidData argument is simply a copy of the GetRidData pointer you supplied in the MMNode. You can use it as a pointer back to the MMNode, or to some other related data structure or something. Since your GetRidFunc may be called by any Task or Process in the system, at any time AllocMem() can be called, you must be very careful when writing it. Here are some things to be careful of: You must not depend on any registers, except for the specified arguments. For SAS/C, this means you must use the __saveds qualifier when defining the function. Also, standard Amiga calling conventions must be used - you must save all registers except D0, D1, A0, and A1. The GetRidFunc must NEVER break the Forbid() state by calling Wait() or any function that might indirectly call it. If another task is permitted to execute while MemMan is in the middle of the MMList, things could get very interesting. Since it can be called from Tasks as well as Processes, it must NEVER call the dos.library, or any functions that might call the dos.library. (Sorry, no swapping memory out to disk.) It must use VERY LITTLE stack space. This means no big Un*x-style auto arrays! Ideally, the GetRidFunc should use no more (actually a little less) stack space than the actual AllocMem() function uses, which is very small. (I haven't checked exactly.) It must NEVER call AllocMem() or any function that might call AllocMem(). Sorry, this means that at this point you may not swap chip memory buffers to fast memory (unless of course you've already pre-allocated the fast memory buffer). I *may* add a feature to MemMan later that would allow you to call the previous AllocMem() vector. However, this would bypass other programs such as Mungwall that add themselves to the AllocMem() chain, possibly causing interesting problems. We'll see how things turn out. The GetRidFunc must NEVER call MMAddNode() or anything that could call this function. This could cause the MMList to be changed while MemMan is traversing it. The GetRidFunc MAY call MMRemNode() on the CURRENT MMNode (the MMNode that caused this call to be made) but NEVER on any OTHER node. Of course, the GetRidFunc may call FreeMem() as much as it wants. After all, that's the whole point of this system. To summarize, you should use __saveds or equivalent when programming in a high-level language, and keep your GetRidFunc as simple and direct as possible. If possible, limit your calls to FreeMem() and MMRemNode() only. Choosing Priority ~~~~~~~~~~~~~~~~~ MemMan traverses the MMList in order of priority from highest priority to lowest priority. Therefore, MMNodes with highest priority will be called first. You should assign your MMNodes with high priorities for pieces of memory that aren't needed very much or can be recovered easily, and assign lower priorities for those that you'd really like to keep if at all possible. Although this may be the opposite of what you would normally expect (having nodes of the lowest priority being the most "important" to keep in memory), there is a good reason for this: When several MMNodes have the same priority, the first one to be added will be called first, creating a kind of least-recently-used (LRU) caching system. This results from how Exec's Enqueue() function operates. If the list were to be traversed from the tail to the head, then MOST recently added nodes would be called first, which is not generally a good algorithm for memory management. It is a good idea to choose priority for your nodes carefully, since priority not only affects order of calls to your application, but also affects the order of calls to different applications. Remember that your MMNodes may be mixed in with the MMNodes of many other applications using MemMan at the same time. Priority should be chosen according to the penalty incurred for freeing that memory (time delays for restoring the data, disk reloads, etc.). Here are some general guidelines for selecting priority: 100 and up: Use these for pieces of memory that are mostly unnecessary or that can be recovered with almost no effort or time delay. You might create large "speed-up" hash tables and such with this priority - things that benefit system performance if enough memory is available, but are not necessary to correct functioning, and can be easily recovered later. 50 to 100: Use these priority levels for things that may require time delays or cause other small nuisances in order to restore them later. For example, precalculated data tables and such would fit into this category - things that aren't really needed at the moment, but may be a slight pain to get back later when they *are* needed. -50 to 50: These priority levels are intended for memory that isn't needed immediately, but contains cached data that will need to be reloaded (as opposed to simply recalculated) if needed later. For example, my overlay supervisor "Bovs" uses priority 0 for all overlay nodes not currently in use. -100 to -50: This range is for data that can still be recovered, but only at great expense. For example, large data tables which take a while to recaulculate, or data that must be reloaded *and* processed or decompressed if it is needed again. Below -100: These levels should be used for data that CANNOT be recovered at all. For example, you could put an application's undo buffers at this level - it's better to lose some possible undo capability than to not be able to use the program at all. Final Warnings ~~~~~~~~~~~~~~ MMNodes should remain private to a particular task or process. In other words, don't MMAddNode() in one task and MMRemNode() in another, on the same node. While the MemMan functions know how to deal with a shared MMList, they do NOT know how to deal with shared MMNodes. The only exception to this rule is with MMRemNode() called from within a GetRidFunc, as explained elsewhere. Always MMInit() before using MMAddNode() or MMRemNode(). (You may call MMFinish() without calling MMInit().) Never call MMInit() more than once in your program. Never call AllocMem(), MMAddNode(), or the dos.library from a GetRidFunc. Never call MMRemNode() on any MMNode OTHER than the one that caused this particular call. Never allow a GetRidFunc to break the Forbid() state by calling anything that might cause a Wait(). Use as little stack space as possible in your GetRidFunc. Finally, to reiterate one more time: ALWAYS remove ALL MMNodes you added BEFORE calling MMFinish()! This can't be stressed enough. If you fail to do this, you will most likely NOT see any immediate problems. The problems will only show up the next time you run out of memory, and even then only if the MemMan patch is currently "awake" (when an application is using MemMan). You should test your program VERY thoroughly for bugs in this area, as outlined in the 'Hints' section below. Be VERY careful to follow these rules. Test your program thoroughly, bang it to its limits, and subject it to every known program terror. MemMan and the operating system are not very forgiving when it comes to low-level programming like this. Hints ~~~~~ Don't keep MMNodes on the system list unless they actually represent free-able data. When you want to lock something in memory, use MMRemNode() to prevent it from being expunged. Then when you unlock it, use MMAddNode() to put it back on the list. A GetRidFunc should be able to simply FreeMem() and return, without having to check locks or anything. When debugging your program, use the 'Avail FLUSH' command in 2.0 (if you have 2.0 - if you don't, it would be a good time to get it) to test your program's handling of memory crunches. Play around with your program a little, get some caches loaded, etc., then use 'Avail FLUSH' to force MemMan into action. Make sure the system doesn't crash, and make sure memory is being freed as expected. Run your program several times in a row, in each case "exercising" MemMan, to make sure your program is not leaving "stale" MMNodes on the global list. Finally, run several copies of your program simultaneously to make sure they don't interfere with each other. If you happen to have another program that also uses MemMan (such as my MultiPlayer program), load up a copy or two of that alongside as well. In other words, really bash it out.