NetNews Usenet Archive 1992 #19/NN_1992_19.iso/spool/vmsnet/internal/1187

Internet Message Format | 1992-08-25 | 8.6 KB

Path: sparky!uunet!paladin.american.edu!darwin.sura.net!wupost!usc!cs.utexas.edu!swrinde!network.ucsd.edu!mvb.saic.com!macro32
From: [e-mail]
Newsgroups: vmsnet.internals
Subject: RE: VMS Tuning
Message-ID: <[e-mail]>
Date: 25 Aug 92 19:51:47 GMT
Organization: Macro32<==>Vmsnet.Internals Gateway
Lines: 146
X-Gateway-Source-Info: Mailing List

Dick Joltes ([e-mail]) says:

>As most folks have pointed out, tuning isn't absolute.  You often can't take
>the rationale used on one machine to another, especially if the second system
>has a different workload than the first.  Tuning a single-user workstation is
>vastly different from tuning an 80-user 6000-400...

There are some things that can be generalized with success.  As you say, some
not.

>One method I've used in the past is to take a new application, install it
>on the system, then set up an account with a massive WSEXTENT.  Run the app
>from this test account and see how high the page count goes.  If, like some
>s/w I've tested in this way, the app keeps eating pages up to the WSEXTENT
>value, revise the figure down and try again until you see a performance hit.
>Other s/w will peak at some figure, which can then be used as the WSEXTENT
>for users requiring that app (unless the figure is ridiculous, in which
>case you revise it down to the performance hit mentioned above).

I like the first part of what you say here, not the second part.  There are
many things that affect the working sets of processes on a system: physical
configuration, sysgen params, process (sysuaf) params, VMS version, 'other'
activity, ...  For a process, the big three are WSDEF, WSQUO and WSEXT, as
modified by the PQL_MWSxxx sysgen values.  My philosophy regardless of VMS
version is generally to give small values for WSDEF and WSQUO and the largest
reasonable value for WSEXT.

Reasoning:

For all cases, a single process on the system that is faulting heavily enough
to flush the FPL/MPL caches is doing so on a system-wide basis.  In other
words, a username/process with improper WS characteristics impacts the entire
system, not just that process.  If you are a system manager with a mean
streak, taking out revenge on some weasel on your system by giving him/her
itty bitty quotas is not just killing him/her but your entire system whenever
he/she logs in and tries to do something.

Traditional tuning studies center on image-based activity as the source of
pagefaults (that is, memory management activity while an image is being
executed as opposed to image activation/image termination).  This is not
always true - DCL example below.

WSDEF is the default number of pages given to a process, and the number of
pages the process has at image termination.  The actual ws size can go up or
down from this value.

WSQUOTA is many things.  It is the number of pages a process can 'ask' for and
is guaranteed to get from the system.  If you make this too large for a large
number of processes, you can run out of 'real' memory.  For all versions of
VMS up to V5.4-2, processes that don't really need memory can 'ask' for it and
hang on to it whether or not they need it.  VMS V5.4-3 introduced a proactive
memory reclamation algorithm that improved this situation, but it is still a
minor problem.

WSEXT is the maximum number of pages that the system can decide to give to you
at its discretion.  If some other process needs the pages, the system takes
them back.  WSEXTENT is limited on a system-by-system basis by sysgen
parameter WSMAX.  I have never seen a situation where WSEXTENT should be less
than the maximum possible value.  Attempts to 'tune' a system by optimizing
WSEXTENT always seem to be beyond the useful part of the curve in the 'law of
diminishing returns'.  Set this value up and leave it up!

I have always been convinced that 'the system' does a better job of page
assignment than a user process.  Set WSQUOTA up, set WSEXT down!

WSSIZE is the number of pages in the process's working set.  Not all of the
pages are necessarily valid.  Seems to me SHOW SYSTEM gives you WSSIZE in the
WS field.

WS list length is the number of valid pages in WSSIZE.  This is the same or
less than WSSIZE, equal to process page count plus global page count.  Seems
to me f$getjpi (ppgcnt + gpgcnt) and SDA give you WS list length for a process.

Swapping and SWPOUTPGCNT:  I recommend that you set SWPOUTPGCNT to a
relatively large value of WSQUOTA+1 [This implies that WSQUOTA is the same for
all processes on the system].  If you do this, several things happen:

(1) if swapping is triggered, the first thing the swapper does is trim all
processes back to WSQUOTA (seems to me this is 'primary swapper trimming'
[?]). (2) If more memory is needed, the secondary swapper trimming algorithm
kicks in.  The Swapper picks a victim, trims it back to SWPOUTPGCNT (secondary
trimming?) and moves the process out to disk.  Various things happen with
global pages, FPL and MPL etc. that I won't deal with here.  If SWPOUTPGCNT is
really small, this secondary trimming is a high priority painful process.  If
SWPOUTPGCNT is WSQUOTA or above, virtually nothing happens system wide besides
moving the victim process to disk; SWAPPER goes back to sleep.  A relatively
large number of pages are freed up.  Generally the time you need to swap is
when a new process is being created on the system.  Whaddyaknow - one process
swapped out allows another to be created without more swapper trimming (can't
do this if SWPOUTPGCNT is too small).  If/when the swapped out process becomes
computable again, it is brought back from disk and immediately given a
'usable' number of WS pages without paging them in (although admittedly by now
any faulting might be hard faults).  Larger SWPOUTPGCNT inspires greater, more
efficient use of the swapfile(s) and less problems with pagefile
fragmentation.

In some mixed system clusters, it is useful to monkey with some of the
PQL_MWSxxxxx parameters to get what you want out of the WS characteristics.
This family of sysgen parameters is system specific, and specifies the minimum
values for any of the process parameters on the system.  For example, in a
mixed cluster, if you have a large system with plenty of memory clustered with
a small system that is memory-poor, it may be of interest to give all users in
the cluster small WSDEF/WSQUO values in sysuaf and larger values of PQL_MWSDEF
and PQL_MWSQUO on the big system.

To minimize traffic back and forth on the FPL/MPL lists, it is helpful to make
WSDEF close to, or the same as WSQUO for processes that do a lot of DCL and do
a lot of short quick activation of a lot of images (SYSTEM, OPERATOR, ...).
The WS is flushed back to WSDEF every time a process terminates an image.
This can be a lot of flushing if WSDEF << WSQUOTA.  If you have a large
command procedure doing a bunch of DCL, you are probably doing a lot of
FPL/MPL movement (VMSINSTAL, AUTOGEN, massive file delete, 'homegrown'
procedure, ...).

The unfortunate part of the VMS implementation of memory management in my
opinion is that the WS values (sysuaf) are username specific in a cluster,
and the sysgen parameters are system specific in a cluster.  These are
orthogonal concepts in my opinion.  Different images should be capable of
different working set sizes on the same machine with the same users.  I would
like to see INSTALL enhanced so that as well as characteristics of privileges,
open, shared, header resident, we could also see such things as WS
characteristics and process priority associated with an image (I have
submitted this as an SIR several times since 1984 and it has always been
munged somehow).

>Tests with ALL-IN-1 (yuch!) 2.1 gave me a WSEXTENT of 4096 (it wanted more
>and would keep eating pages up to the WSMAX of 20480...we trimmed it down
>to the above after repeated testing).

Why trim it?  Set it up and leave it up...

>Tests with Interleaf TPS 4.3.x gave a WSEXTENT of 8192.  The process stopped
>growing at this point.  With a memory-rich system we were able to assign
>this figure to WSMAX and give users the memory they needed to run properly.

It is my opinion that WSMAX as set by autogen keeps numerous things working
correctly.  WSMAX indeed puts an upper limit on WSEXTENT for any process on
the system, but if you manually set it to something in sysgen or autogen, you
may booger other things on your system such as pool expansion.  I don't
recommend setting WSMAX to anything other than what Autogen wants to use.

HTH
_______________________________________________________________
Harvey Brydon         | Internet:   [e-mail]
Dowell Schlumberger   | P.O.T.S.:   [phone]
    Ever have a cured ham and wonder what it had?