NetNews Usenet Archive 1992 #26/NN_1992_26.iso/spool/alt/msdos/programm/2700

Internet Message Format | 1992-11-10 | 9.4 KB

Path: sparky!uunet!snorkelwacker.mit.edu!ai-lab!zurich.ai.mit.edu!pshuang
From: [e-mail] (Ping Huang)
Newsgroups: alt.msdos.programmer
Subject: Re: QEMM/386
Date: 10 Nov 92 09:16:00
Organization: M.I.T. Artificial Intelligence Lab.
Lines: 149
Message-ID: <[e-mail]>
References: <[e-mail]>
NNTP-Posting-Host: rolex.ai.mit.edu
In-reply-to: [e-mail]'s message of 9 Nov 92 05:44:05 GMT

In article <[e-mail]> [e-mail] (Jeff Silverman) writes:

 > What does QEMM/386 do, from a programmers point of view?

From a VMS perspective, or from the perspective of a programmer used to
UNIX or OS's with flat virtual memory, QEMM/386 doesn't do very much for
you directly. What QEMM/386 does is to act as a EMS, XMS, VCPI, and DPMI
server. What does this mean? Jump to a couple of paragraphs below. But
first, keep in mind that from the 80386 processor and up, Intel's CPU's
support multiple views of memory which can be confusing. In real mode,
the chips emulate the 8086 which cannot address more than one megabyte
of memory. Addresses are generated as a 16-bit segment and a 16-bit
offset, which are combined by shifting the former 4 bits to the left and
adding, to create a 20-bit physical address. This is a pain for compiler
writers and assembly programers because segments are 64Kb in size and
you often need to use different sizes of pointers (16-bit near pointers
and 16:16-bit far pointers). (Note that this description more or less
still applies in V86 mode, which these chips have in order to be able to
emulate an 8086 while still running a real operating system in protected
mode, except that the 20-bit address formed is a linear and not physical
address -- the next paragraph explains the difference.)

In protected mode (note I omit specific discussion of the 80286 here),
the chips deal with memory in a slightly more complicated way. Instead
of 16-bit segment values which translate directly to the physical
address, you load 16-bit segment selectors into the segment registers.
You can then use offsets up to 32 bits in size (each selector has
information associated with it in a table, indicating the 32-bit linear
address which the segment starts at, whether the contents are data or
code and how large the segment is to allow memory protection, and other
information), so a pointer is 16:32 bits, or 48 bits (making the virtual
address space of the chips some pretty large number:
281,474,976,710,656). However, commonly code which takes advantage of
the 80386 simply set the segment registers to selectors which point to a
common place in linear memory, set the limits to the full 4Gb, and
thereafter never twiddle the selectors again, thereby implementing a
flat memory address space with 32-bit "pointers".  In addition to this
segmentation organization, the chips are capable of paging. Once the
chips resolve a memory reference from a 16:32 pointer to a 32-bit linear
address, they go through page translation (standard 4Kb page size) to
the 32-bit physical address, e.g. actually goes out on the memory bus.

OK, back to QEMM/386:

(a) EMS and XMS: QEMM/386 takes charge of as much of the conventional
    and extended memory as you direct (defaults to all). It takes charge
    of conventional memory so that if you run Desqview, it will be able
    to page different collections of memory pages into the conventional
    memory space so that DOS programs can be multitasked. QEMM/386
    allows programs which understand how to make use of expanded (EMS)
    or extended (XMS) memory to do so, with the advantage that it will
    freely interconvert between the two as necessary because it acts as
    the EMS and XMS servers. For example, if you have 7 megabytes of
    memory free, you can sequentially run program A, which uses six
    megabytes of expanded memory, and then program B, which uses six
    megabytes of extended memory, without having to tell QEMM/386 to
    shift memory from the EMS pool to the XMS pool: it is automatic.

(b) VCPI and DPMI: in the late 1980's, a number of software applications
    were starting to take advantage of the protected mode features of
    the 386+ chips (which include more than just changing the memory
    model). However, these programs would get into each others way by
    clobbering each other's data because there was no protocol to manage
    the chip's resources. VCPI (Virtual Control Program Interface) was
    created by Quarterdeck in conjunction with a number of other
    companies who were writing system-level software (386 DOS extenders,
    multitaskers like Desqview) or applications so big that they needed
    to use protected mode to fit to allow multiple programs which used
    the 80386 to co-exist. VCPI is simple and mostly allows programs not
    to get into each other's way. DPMI (DOS Protected Mode Interface)
    was created by Microsoft, Intel, and a committee of other companies.
    A program which is compatible with DPMI calls the DPMI server
    software to access the special features of the 386+ chips. I believe
    only one of the most recent revisions of QEMM allows it to act as a
    DPMI server. Windows 3.x is the best known and commonest DPMI server.

 > How is this possible?  If something, say, a TSR, is moved out of low
 > memory and into either High Memory (By high memory, I mean the region
 > between 640K and 1 Mbyte) or extended memory, how does it keep working?

TSR's and device drivers which are loaded in high memory or the HMA (an
area of memory slightly less than 64Kb which isn't in the first megabyte
of memory but which can be accessed by opcodes executing in real or V86
mode due to an artifact of creating the 20-bit address: if you take the
segment value to be 0xFFFF, you can access 65536-16 bytes of memory past
the first megabyte because adding offsets greater than 16 when using
registers which are 32-bits yields an address with the twenty-first bit
turned on, two) don't really notice anything different than if they were
loaded normally. Yes, they have been relocated in the first megabyte
address space, but the normal DOS .COM, .SYS, and .EXE formats are
designed so that they can be loaded anywhere in that first megabyte.

If you are really asking how programs like Desqview which can multitask
DOS programs can work, Desqview at every task switch remaps the sets of
memory pages which contains the applications being multitasked into the
first megabyte of linear address space, either by calling on QEMM/386 or
alternatively by calling on any EMS 4.0 servers loaded which has the
hardware to remap memory pages.

 > How does QuarterDeck's OPTIMIZE program work?  Please don't say it works
 > well, I KNOW it works well, I've been playing with it, I can see that it
 > works, but don't know what it's doing or how it does it.

OPTIMIZE tries to figure out how much memory all the TSR's and device
drivers you use on your machine takes up, and which ones would fit into
high (640-1024Kb) memory and the HMA. Note that various memory managers
have come up with tricks to make more high memory available, most of
which plays tricks with mapping memory pages into the linear space that
would otherwise be taken up by ROM-BIOS code that isn't being used any
more in the operation of the computer.

 > Why does DOS have a 640Kbyte limit?  Why would anybody ever want to use
 > expanded RAM if extended RAM is available?  If I have a "C" program
 > which runs properly under VMS, but needs, say, 3 Mbytes of RAM to store
 > all its data structures, what is the best way to run it under DOS?
 > Under Windows?

IBM made a reasonable decision back in 1981 to reserve some of the
one-megabyte address space of 8088/8086 chip for future use and
expansion purposes. The correct limit to complain about isn't the 640Kb
limit, it's the one megabyte limit imposed by the 8088/8086 CPU, and it
is unfortunate that so much effort and software has been written to try
to wring the last tiny bit out of the 384Kb difference between 640Kb and
1024Kb when the horizons could be so much larger.

If you have to write programs which needs lots of memory space, want
excellent performance and don't need to worry about whether your
customers will be able to run your programs (e.g. your code won't be
redistributed or is an in-house application), your best bet is to look
at the 32-bit C compilers that support going past DOS and 8088/8086
limitations -- you get much better compatibility with code which makes
evil assumptions like sizeof(int)==sizeof(pointer) and that of course it
is OK to do a malloc(1048576). There are a couple of commercial C
compilers for DOS which support use with a 386 DOS extender, but there
are also freely distributable ports of the GNU gcc/g++ compiler for DOS
(DJGPP) and for OS/2 2.0 (actually two slightly different ports). The
reason that a number of commercial applications support using expanded
or extended memory rather than using a DOS extender, even though it
takes more effort to write such code, is that expanded memory can be
added to *ANY* PC-compatible, including 8088/8086 machines, and extended
memory can be added to 80286 machines. Therefore, by writing to these
lower denominators, the applications will work passably on a larger
number of machines than if you were to limit their use to 386+ CPU's.

--
| Ping Huang (INTERNET: [e-mail]) speaking for himself.
| "One Thing to name them all, One Thing to define them,
|  One Thing to place them in environments and bind them..."