NetNews Usenet Archive 1992 #30/NN_1992_30.iso/spool/bit/listserv/sasl/5364

Text File | 1992-12-14 | 11.5 KB | 238 lines

Comments: Gated by [e-mail]
Path: sparky!uunet!elroy.jpl.nasa.gov!usc!howland.reston.ans.net!paladin.american.edu!auvm!UNC.BITNET!UNCSM1
Message-ID: <[e-mail]>
Newsgroups: bit.listserv.sas-l
Date:         Mon, 14 Dec 1992 09:22:00 EST
Reply-To:     Sally Muller <[e-mail]>
Sender:       "SAS(r) Discussion" <[e-mail]>
From:         Sally Muller <[e-mail]>
Subject:      Memory under MVS/ESA summary of responses
Lines: 226

----------------------------------------------------------------------
SUMMARY:  Summarizing Responses Memory under MVS/ESA
E-ADDR:   [e-mail]
PLATFORM: MVS/ESA SAS 6.07
NAME:     Sally Muller
PH/ADDR:  919-962-6501  OIT CB#3455, UNC, Chapel Hill NC 27514
----------------------------------------------------------------------
Hi SAS-Lsters!
Many *THANKS* to:
                  Melvin Klassen      Ben Conner
                  David Ehresman      Tim Berryhill
                  Lee Herman          "Horse's mouth" from SI
for answering 1) what is the difference between REGION and MEMSIZE?
2) is there any reason not to specify REGION=0M and MEMSIZE=0M?
and also for their tips on memory in general (attached to bottom of
this).

                REGION VS. MEMSIZE
and you thought PC expanded memory was confusing!  Some of the
responses conflicted.... seems IBM has managed to confuse alot of us
:-).

Ben Conner offered an historical perspective explaining that IBM
mainframe memory is structured and has evolved much like memory
on IBM PC clones.  Initially IBM mainframe memory had 24 bit
addressing scheme and was limited to 16 meg (analogous to DOS
640K limit).  JCL was invented during this time so REGION= parm
in your JCL refers to memory "below the line."  When 16 meg
wasn't enough IBM went to a 31 bit addressing scheme and created
"above the line" memory which takes a different set of
instructions to access and programs have to be specifically
written to access it.  Therefore the MVS memory above 16 meg is
analagous to DOS "above the line" which is anything above 1 meg
boundary.  To repeat, the JCL REGION= refers to "below the line"
memory and there IS NO JCL parameter now defined to control
"above the line" memory usage.

Lee posted his response to SAS-L.  To summarize, "REGION is a JCL
parameter to tell MVS (the operating system) how much memory the
batch job should be given to use. In our case, SAS is the program
which will be run in the batch job address space (or region).
MVS will note the parameter as a limit on how much memory to give
the batch job but not allocate actual memory to the job until it
is used or explicitly requested by the running program (in our
case the SAS run time system).

The SI person said that Lee's response was correct but that one
could delve alot deeper......  "One nitpicky detail: REGION=
specfies the amount of storage that MVS will return in a SINGLE
VARIABLE-LENGTH GETMAIN REQUEST.  It is NOT a total limit on
memory; the actual limit is 64K greater (see below).

Another extraordinarily confusing thing about REGION= is the way it
works vis-a-vis extended storage (ie, > 16meg).  The confusion is
compounded in that each site CAN change the defaults, but as IBM
ships it:

    if REGION= is less than 16Meg,
    the maximum storage available below the line is REGION= + 64K,
AND the maximum storage available above the line is 32meg.

    if REGION= is greater than 16Meg,
    the maximum storage available below the line is everything,
AND the maximum storage available above the line is REGION=.    "

It was because of this default 32meg of available storage above
the line that we added the MEMSIZE= SAS option, which limits the
total amount of storage that SAS uses.  Otherwise, a job running
with a REGION=4M could in fact use up to 36Meg, resulting in huge
impacts on system performance!"

David's posted note on REGION conflicted somewhat with what SI
source said.  David agreed with what happens with below the line
and above the line (he calls "extended region") when you request
region < 16M.  But David says:
    if REGION= is between 16Meg and 32Meg,
    the maximum storage available below the line is everything
and the maximum storage available above the line is REGION=32M.
The SI source had said that the latter (max storage available
above the line) would amount to the number specified in REGION=.)

Melvin offered a SAS perspective of REGION and referred to page
485 of the MVS Companion, "In order to speed execution, the SAS
System has been designed to keep as much of the program and data
in memory as possible."

REGION according to Tim is JCL, "and you can code it on both
your job and step cards.  Like TIME, if you code REGION in both
places you get the smaller value.  Assuming you have a cataloged
procedure (//DOITTOIT EXEC OURSAS6 ), the proc may include a
REGION specification (check your job log, maybe set
MSGLEVEL=(1,0) to get more).  If you do not specify REGION on
your job card or your step card, and it is not specified in your
cataloged proc, you get the installation default region size.  If
you specify REGION=0, or the installation default is REGION=0,
then your job is authorized for all available memory.  The memory
is not allocated to you yet, just (sorta) available.  As the job
requires memory, it will issue GETMAINs.  If the memory is no
longer available (someone else, also authorized, got it first),
you will get a GETMAIN FAILED message in the log."

                     MEMSIZE

Compared to REGION, MEMSIZE is a piece of cake.  In his posted
note, Lee said  MEMSIZE is a SAS parameter to tell the SAS run
time system how much memory within the amount specified in REGION
it can/should use.   As Ben and SI source note, REGION currently
is defined to control below the line memory and it is the "wild
west" as far as memory above the line is concerned which may
result in huge impacts on system performance.  "Ergo the
MEMSIZE= option, it tells SAS when to quit being greedy."

              REGION=0 and MEMSIZE=0

SI source was vehemently opposed -- "Specifying REGION=0 IS A
VERY VERY BAD IDEA!  For the same reason, specifying REGION= >
16M  IS A VERY VERY BAD IDEA!  In both situations, SAS may use up
ALL the available private area storage below the line, resulting
in memory failures when MVS tries to allocate some LSQA (which
grows from the top of the region down).  This was more important
with V5 and earlier, which did not used extended storage, and
which tried to manage the layout of storage by frequently
allocating all that was available.  With V6, we no longer try to
manage the layout, plus we preferentially allocate extended
private storage, using storage below the line only when we must,
or when we run out of extended storage."

Ben seconds SI:  "The drawbacks to using MEMSIZE=0?  A couple--if
several jobs running concurrently allocate it, you may run out.
Secondly, I've found that SAS, like children, doesn't do well
when given no limits <grin>."

Melvin calls it a double-edged sword.  "The good news is that
specifying an adequate REGION avoids extra I/O, e.g.,
   DATA; ...;
   PROC PRINT;
   DATA STEP2; SET; ... ;
   PROC PRINT;  /* SAS will **not** need to read the 'PRINT' procedure */
   RUN;         /* from disk for the **second** time. */

The  ***potential*** bad news is:

 - what if you specify REGION=32M, and the **real**
   computer-hardware has only 24M of storage?  MVS will spend a
   lot of CPU time doing paging, because not everything SAS is
   loading into **virtual** storage will fit into **real**
   storage at the **same** time.
 - what if you specify REGION=32M, and 7 other users also specify
   it, and all 8 of you submit a SAS job at the **same** time?
   MVS will see that the 8 of you are using 256M of virtual
   storage.  Hopefully, the real hardware will have 256M of real
   storage available.  If not, then MVS will have to do "paging".
   Is any paging bad?  Not necessarily, especially with an IBM
   3090 or ES/9000, when "expanded storage" is an available
   option on the machine.

Tim sees dangers but also advantages, "As nearly as I can tell,
an individual user never suffers from setting REGION=2047M (or
however much main memory their system has, 2047M is the IBM
limit) and MEMSIZE=0.  The problem is, SAS does not release
allocated memory until it approaches the MEMSIZE limit (it may
release memory when a GETMAIN fails due to the REGION limit).
Each procedure you invoke (MEANS, REG, UNIVARIATE) stays in
memory and some buffers remain allocated.  Rather than free
memory SAS is finished with, it allocates more.  Thus a job which
could run just fine in 8M may occupy 80M, and other jobs will
suffer GETMAIN failures. However, if your system programmer said
it was OK, and you do not get many GETMAIN failures, why worry? "

Lee says it can be done with careful monitoring, "With REGION=0M,
you give users what they need without limit other than the
absolute maximum address space configured by your MVS systems
person. If a SAS job keeps gobbling memory due to imprudent
coding of a DATA or PROC step used with large numbers of
observations, the potential for affecting other users is
increased. If your systems and performance people are comfortable
with the SAS workloads, your users will certainly appreciate
having two less parameters to be aware of and maybe your life
will be simpler as a result. If you are running MVS on a small
and/or heavily loaded machine, careful monitoring of real and
expanded storage use and paging should accompany the change over.
So the only down side I know of is possible physical memory
shortages. You apparently haven't had these so far so it sounds
like it's working as it should!"

                TIPS

Tim says, "these considerations are not unique to MVS - any
multi-tasking, virtual memory computer and operating system can
encounter these issues - VAXes, super computers, large
workstations and servers with multiple users".

Melvin says "Perhaps, adjusting the SYSLEAVE and PROCLEAVE
parameters will prevent these aborts, i.e., tell SAS to
"grab-it-all", but then give 100K back to MVS, so that MVS can
open files, talk to RACF, etc."

Tim points out a documentation error on page 486 of MVS Companion
-- "the default value MEMSIZE=0 limits the memory to 8Mb".
Earlier, page 293, same manual correctly states, "A value of 0
indicates that there is no limit, except the operating system
limit. "

MINSTG option for avoiding GETMAIN failures (i.e memory
shortage).  Tim says, "the MINSTG option will prevent this
(GETMAIN failures) but that has performance implications,
particularly if you call the same proc repeatedly."  Ben says.
"if my job bumps up against the system defined default (or flat
runs out), it sometimes abends.  I then have to code MINSTG to
try to avoid that condition (MINSTG says chuck anything you're
not using and brush between meals...I mean procs). "

Hiperspace:  Ben says, "The term hiperspace--that refers to a
different (and cheaper) kind of memory that IBM developed.
(Documented on pgs. 9-11 SAS Technical Report 218 - Changes and
Enhancements to MVS.)  Rather than being able to point to 1 byte
with a single address, hiperspace points to a 4K block.  Not too
great for program instructions, but wonderful for putting blocks
of data away safe that can be retrieved (again with special
instructions ).  Version 6.07 of SAS can now allocate files, etc.
using hiperspace.  Again, much faster than disk access.  It is
not unusual to find large IBM systems with 1 or more gigabytes of
hiperspace memory.  Some of the systems here have 3 gig
installed(!).  SAS can use it, the IBM SORT can use it, and a few
other utilities.  This also has a crude parallel in the PC
world--expanded memory."