home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
The World of Computer Software
/
World_Of_Computer_Software-02-387-Vol-3of3.iso
/
r
/
rxridg93.zip
/
RXRIDGE.DOC
< prev
next >
Wrap
Text File
|
1993-03-01
|
94KB
|
1,540 lines
┌─────────────────────────────────────────────────────────────┐
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ ASCII Documentation File for ... RXridge.EXE │ │
│ │ │ │
│ │ Version 9301 │ │
│ ├─────────────────────────────────────────────────────────┤ │
│ │ │ │
│ │ a softRX SHAREWARE module from │ │
│ │ │ │
│ │ Bob Obenchain, CompuServe User [72007,467] │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
softRX is a trademark of The Software Prescriptions Company.
"Personal computing treatments for data analysis infirmities."
Copyright(c) 1989-1993 by Bob Obenchain, Ph.D.
ALL RIGHTS RESERVED
*******************
The author of RXridge, Bob Obenchain, is a member of
the Association of Shareware Professionals <ASP>.
┌───────────────────┐
─────────────────────────┤ Table-of-Contents ├─────────────────────────
└───────────────────┘
softRX User Registration . . . . . . . . . . . . . . . . . . . . . . 1
Introduction to RXridge. . . . . . . . . . . . . . . . . . . . . . . 2
Shrinkage Regression Background Information. . . . . . . . . . . . . 3
Computing System Requirements for RXridge. . . . . . . . . . . . . . 4
softRX Graphical Design Philosophy . . . . . . . . . . . . . . . . . 5
RXridge File Naming Conventions. . . . . . . . . . . . . . . . . . . 5
RXridge Walk-Through . . . . . . . . . . . . . . . . . . . . . . . . 6
Guidelines for Interpretation of Ridge Trace Plots . . . . . . . . .18
Shrinkage Regression REFERENCES. . . . . . . . . . . . . . . . . . .24
TECHNICAL APPENDIX: "Extent" and "Shape" of Shrinkage. . . . . . . .26
RXridge Update History . . . . . . . . . . . . . . . . . . . . . . .28
ASP Hub Network. . . . . . . . . . . . . . . . . . . . . . . . . . .29
┌─────────────────────┐
────────────────────────┤ USER REGISTRATION ├────────────────────────
└─────────────────────┘
RXridge.EXE, its ASCII documentation files [RXridge & RXreview.DOC],
and the included batch input (numerical example *.ARX) files are user-
supported electronic information that we encourage you to examine and
to share with others, provided you have not modified them in any way.
All softRX SHAREWARE modules are distributed in fully functional form
and will remain so without any usage time limits. softRX systems do
not display annoying registration "reminder" screens or messages. And
RXridge, version 9301 . . . . . . . . . . . . . . . Page 2 of 30
no (non-commercial) user who is unable to correspond with softRX in
English is expected to register.
**************************************
**************************************
Only YOU know whether softRX systems are useful to you.
And, if you do use them, you must register!
**************************************
**************************************
The registration fee for RXridge is $25 for use either by a single
individual or for installation on one single-user-at-a-time system.
Three softRX modules (say, RXridge, RXtraces, and PathProj) may be
registered together for $50. softRX offers MAIL-IN SUPPORT
SERVICES (e.g. answers to questions on the theory, application and/or
computation of statistical "shrinkage" regression techniques) only to
its REGISTERED users.
However, registration fees are mandatory for use of softRX modules in
any commercial or government environment. These users must register
and pay for their copies within 30 days of first actual use or else
their license is withdrawn. Site-License arrangements (of at most $15
per module) may also be made by contacting softRX.
Make checks payable to... Bob Obenchain
and mail them to... 5261 Woodfield Drive North
Carmel, IN 46033-8795, USA
!!! Thank You for Supporting the softRX SHAREWARE Concept !!!
┌────────────────────────────┐
───────────────────┤ INTRODUCTION to RXridge ├──────────────────────
└────────────────────────────┘
RXridge performs maximum likelihood estimation of generalized ridge
regression models. RXridge supports both keyboard and ASCII
batch-file data entry, tabulates ill-conditioning summary statistics,
makes inferences about the appropriate "form" as well as the "extent"
of ridge shrinkage, and generates interactive graphical "trace"
displays. RXridge also outputs ASCII files of the exact form needed
for input to the softRX ridge regression "training" and "review"
modules, RXtraces and PathProj.
You do not need to read this documentation before trying RXridge out.
There is very little damage you can do by giving "wrong" choices for
menu items and responses to prompts. However, first of all, please
RXridge, version 9301 . . . . . . . . . . . . . . . Page 3 of 30
> > > MAKE BACK-UP COPIES < < <
of the distribution files and/or of the original softRX shareware
archives. RXridge does not modify its own input files, but it writes
MANY output files for input to RXtraces, to PathProj, or to your
favorite Editor/WordProcessor system. These outputs may overwrite
[obliterate] previous files (in the current working directory) with
the same names; you should consider using unique (8 character) initial
filenames for each different shrinkage parameterization you try.
Each RXridge prompt has a [default] value that you may accept simply
by pressing the ENTER key; RXridge menus require you to press a single
key, where any differences between upper and lower case are ignored.
┌─────────────────────────────────────────────┐
────────────┤ Shrinkage Regression Background Information ├────────────
└─────────────────────────────────────────────┘
RXridge provides a user-friendly text-mode Windows/Prompts/Menus
interface to linear algebra computation and interactive graphical
display routines that:
...display TRACES of fitted regression coefficients,
scaled mean squared error estimates,
excess (OLS-ridge) eigenvalues,
inferior direction cosines, and
shrinkage factor patterns.
...make classical maximum likelihood inferences about appropriate
choices for the "Q-shape" parameter that
controls the "curvature" of a generalized
ridge regression shrinkage path.
...monitor 3 different types of likelihood ratios (classical,
empirical Bayes & random coefficient) that
identify optimal stopping points along a
ridge shrinkage path.
and...tabulate ANOVA statistics not only for least squares estimates
of Beta Coefficients but also for their
(Principal Axis) Uncorrelated Components.
RXridge displays results of ridge regression computations via "trace"
plots. In a trace, P quantities (several estimated coefficients,
risks, shrinkage factors, etc.) are plotted vertically against a
horizontal indicator of the extent of shrinkage (from "no" shrinkage
at the left-hand, least squares estimates all the way to "total"
shrinkage at the right-hand extreme, where all coefficient estimates
are zero.) Here, P denotes the number of non-constant predictor
variables in the regression model; RXridge accepts values of P from 2
to 20. RXridge uses a measure of the extent of shrinkage called the
"Multicollinearity Allowance," MCAL. See the APPENDIX to this
RXridge, version 9301 . . . . . . . . . . . . . . . Page 4 of 30
documentation for the definitions not only of MCAL but also of Q, the
parameter that controls the "shape" (curvature) of the ridge path.
RXridge attempts to identify ridge regression coefficient estimates
that are either "good" in the sense that they dominate least squares
estimates in every (multivariate) Mean Squared Error sense or are
"optimal" in one well-defined (univariate) MSE sense. Definitions for
"good" or "optimal" ridge shrinkage factors are based upon risk
(expected loss) calculations that apply to all forms of statistical
distributions. But the MAXIMUM LIKELIHOOD inferences for one and
2-parameter generalized ridge regression provided by RXridge use
standard, "normal" distribution theory.
WARNING: Your CONFIG.SYS file should specify at least...
files=20
Matrix decomposition functions bound into RXridge.EXE are compiled
from C code, Copyright 1987, 1988 by Numerical Recipes Software, and
are used with permission of the authors.
┌───────────────────────────────┐
────────────────────┤ Computing System Requirements ├──────────────────
└───────────────────────────────┘
RXridge.EXE requires your personal computer to be fully IBM-compatible
and to be running IBM PC-DOS or Microsoft MS-DOS version 2.0 or higher.
RXridge uses the screen handling capabilities of a CGA, EGA, or VGA
graphics card/monitor. If you system has only a Monochrome Graphics
Adapter (MGA) card or Hercules Graphics Card (HGC), you could "simulate"
CGA graphics capabilities in order to view trace displays and review
ASCII files from within RXridge. For example, if you have SIMCGA.COM
(version 4.0, July 1987) by C. P. Guzis, you could make the SIMCGA
system "memory resident" before invoking RXridge.
DOS Prompt> simcga 3
On the other hand, simulating CGA graphics will slow RXridge execution
down considerably. Thus a wise alternative for users without CGA+
hardware might be to run RXridge.EXE to perform all calculations without
making SIMCGA memory resident. Then, after invoking SIMCGA, use
RXtraces.EXE and PathProj.EXE to read the output files from RXridge.EXE
to regenerate graphical displays.
RXridge, version 9301 . . . . . . . . . . . . . . . Page 5 of 30
┌─────────────────────────────┐
────────────────────┤ Graphical Design Philosophy ├────────────────────
└─────────────────────────────┘
RXridge uses CGA (medium resolution, 320x200 pixel) graphics because
all true graphics hardware for IBM-compatible personal computers has
at least this (minimum) capability. Besides, the (8x8) text font for
this resolution resides in ROM within your system. But using CGA also
means that there isn't enough resolution to draw "nice" labels, tick
marks, and numerical values along axes without greatly reducing the
size of the trace plot window. I certainly did not want to use any
sort of reduced-size plot; rather, I decided to emphasize INTERACTIVE
(dynamic, on-screen annotation) methods for high-lighting / reading
numerical values directly off of shrinkage TRACE screen displays.
Once in RXridge graphics mode, you can view a "help" screen by
pressing either the SpaceBar or function key F1. The resulting HELP
screen tells you which keys to press to activate and move the
on-screen cursor and/or how to dynamically annotate, save, or print
the screen display. And you can always return to the Menus simply by
repeated pressing of the ENTER or Escape keys.
┌─────────────────────────┐
───────────────────────┤ File Naming Conventions ├─────────────────────
└─────────────────────────┘
RXridge accepts either keyboard or ASCII batch file data entry. If you
use keyboard data entry, RXridge will output a (data/parameter) archive
(filename.ARX) file containing the data you entered. The next time you
run RXridge, you could specify batch input from that filename. And, you
can also use any text editor to make changes and/or correct data errors.
RXridge performs ridge calculations and writes results to a series of
ASCII (text) output files. These files have a common FileName (of at
most 8 characters) and a 3-character DOS extension, ending with the
two letters "RX". The 3-character DOS extensions used by softRX
software systems for generalized ridge regression analysis are:
.ARX ...RXridge (data/parameter) input Archive,
.PRX ...Parameters and variable names,
.CRX ...generalized ridge Coefficients,
.RRX ...Risk (mean-sq-error) estimates,
.ERX ...Excess Eigenvalues (OLS-Ridge),
.IRX ...Inferior direction cosines, and
.DRX ...shrinkage Delta factors.
.XRX ...regressor matrix coordinates.
.YRX ...response vector coordinates.
RXridge, version 9301 . . . . . . . . . . . . . . . Page 6 of 30
.QRX ...Q-shape selection information.
.MRX ...MCAL shrinkage extent selection info.
.DTL ...Detailed RXridge Output
(correlations, coordinates, etc.)
.LRX ...squared error Loss (from RXsimu scenario)
.TRX ...True risk (from RXrisk analyses when
signal/noise ratios are known)
RXtraces and PathProj cannot regenerate graphical displays for a data
set unless they can find the corresponding *.PRX file generated by
RXridge. RXtraces uses the information from the first six lines of
this file to recreate ridge trace displays, while PathProj uses the
first three lines to construct 2-dimensional projections of the
shrinkage path. For example, the "mpg0.PRX" file contains...
Line 1: 4 = P, number of regressors
Line 2: 32 = N, number of observations
Line 3: 8 = STEPSPU, steps per unit change in MCAL
Line 4: 0 = QSHAPE of the shrinkage path
Line 5: XNAM= CYLNDS CUBINS HPOWER WEIGHT
Line 6: YNAM= MPG
By proposed convention, the 8-character filenames associated with each
fixed "Q-shape" curvature for a generalized ridge path end with, say,
1 or more trailing NUMERICAL characters, such as "0" or "-2". (By the
way, the "+" character cannot be part of any PC/MS-DOS filename.) For
example, the 4-character FileName "mpg0" could be used label all files
associated with the Hoerl-Kennard(1970a,b) "ordinary" ridge shrinkage
path, Q = 0, when applied to the example (gas mileage) data, MPG.ARX,
of Hocking(1976).
┌──────────────────────────┐
│ ┌──────────────────────┐ │
═════════════════════════╡ RXridge Walk-Through ╞══════════════════════
│ └──────────────────────┘ │
└──────────────────────────┘
RXridge is envoked by entering its .EXE filename at your DOS> prompt:
rxridge
The following start-up screen will then appear...
RXridge, version 9301 . . . . . . . . . . . . . . . Page 7 of 30
╔══════════════════════ softRX ridge ══════════════════════╗
║ ║
║ *** 2-Parameter Generalized Ridge Regression *** ║
║ ║
║ Memory available for Dynamic Allocation: 305.5 Kbytes. ║
║ ║
║ ╔════════════════════════════════════════════════════════════════╗ ║
║ ║ At colon Prompts : ...simply press ENTER to get the [default]. ║ ║
║ ║ ║ ║
║ ║ Are you using a COLOR monitor ? [Y|n] : Y <------------Default║ ║
║ ║ ║ ║
║ ║ Do you have a CGA (or better) GRAPHICS card, or Default║ ║
║ ║ Is CGA simulation software currently active ? [Y|n] : Y <------║ ║
║ ╚════════════════════════════════════════════════════════════════╝ ║
--- ---
--- ---
║ ╔════════════════════════════════════════════════════════════════╗ ║
║ ║ Do you wish to use K = Keyboard Data Matrix Input ? ...or ║ ║
║ ║ B = Batch Input from .ARX File ? ║ ║
║ ║ ║ ║
║ ║ Press K or B ...or H for Overview HELP --> H ║ ║
║ ╚════════════════════════════════════════════════════════════════╝ ║
╚════════════════════════════════════════════════════════════════════╝
RXridge uses dynamic RAM memory allocation for handling matrices. My
own personal computer usually has about 584 Kbytes of RAM free (out of
640 Kbytes) before I invoke RXridge. The start-up screen for RXridge
shows that only 305 Kbytes still remain free, so RXridge occupies
about 180 Kbytes all by itself.
When you press the H key, as shown above, a small "overview help"
window will "pop" out...
╔═══════════════════════════════════════════════════════════════╗
║ RXRIDGE.EXE...Overview Help ║
║ ║
║ RXridge is a *** softRX shareware *** system for performing ║
║ 2-parameter Generalized Ridge Regression computations. ║
║ ║
║ RXridge accepts either keyboard or ASCII batch file input of ║
║ (1) the matrix of regressor/response data values, and ║
║ (2) regression model/parameter specifications. ║
║ ║
║ RXridge also draws CGA ridge TRACE displays and writes out ║
║ ASCII files designed for input to other softRX shareware ║
║ systems, RXtraces and PathProj. ║
║ ║
║ Press any key to return to Data Matrix Entry... ║
╚═══════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 8 of 30
Let us suppose that, upon return to the RXridge start-up screen, you
press B ...for Batch input from a (data & parameters) .ARX file. In
fact, let us suppose that you want to use Hocking's gasoline mileage
data. So, fill in the next screen as follows...
╔══════════════════════ softRX ridge ══════════════════════╗
║ ║
║ ╔FileNames╗ Arrow Keys: Move Highlight Bar Up / Down. ║
║ ║AIRPOLUT ║ ║
║ ║HALDCEMT ║ Return Key: Selects the Highlighted File. HighLight
║ ║MPG██████║ <---------------
║ ╚═════════╝ Escape Key: Abandon BATCH RXridge Input. MPG and
║ press ENTER
║ The Batch Matrix .ARX input file is to be: mpg.arx ║
║ ║
║ Specify filename (max 8 characters) Take default here by ║
║ to save Detailed Output [mpg.dtl] : <----------------------- ║
║ simply pressing Enter ║
║ ║
║ The RXRIDGE Output Save file is to be: mpg.dtl ║
║ ║
║ Press Q to QUIT now...Other Key to Continue... ║
╚════════════════════════════════════════════════════════════════════╝
The next two screens will appear as variable names and numerical values
are read from the "mpg.arx" file...
╔═══════════════════════ softRX variables ══════════════════════╗
║ Number of Variables = 5. ║
║ Number of Observations = 32. ║
║ Column No. 1 Name = CYLNDS ║
║ Column No. 2 Name = CUBINS ║
║ Column No. 3 Name = HPOWER ║
║ Column No. 4 Name = WEIGHT ║
║ Column No. 5 Name = MPG ║
║ Data Values are to be entered in the Next Window. ║
║ Press Q to QUIT now...Other Key to Continue... ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 9 of 30
╔═════════════════════ softRX data entry ═════════════════════════╗
║ 6 160 110 2620 21.0 ║
║ 6 160 110 2875 21.0 ║
║ 4 108 93 2320 22.8 ║
║ 6 258 110 3215 21.4 ║
║ 8 360 175 3440 18.7 ║
║ 6 225 105 3460 18.1 ║
║ 8 360 245 3570 14.3 ║
║ 4 146.7 62 3190 24.4 ║
║ 4 140.8 95 3150 22.8 ║
║ 6 167.6 123 3440 19.2 ║
║ 6 167.6 123 3440 17.8 ║
║ 8 275.8 180 4070 16.4 ║
║ 8 275.8 180 3730 17.3 ║
║ 8 275.8 180 3780 15.2 ║
║ 8 472 205 5250 10.4 ║
║ 8 460 215 5424 10.4 ║
║ 8 440 230 5345 14.7 ║
║ -- More -- ║
║ 4 78.7 66 2200 32.4 ║
║ 4 75.7 52 1615 30.4 ║
║ 4 71.1 65 1835 33.9 ║
║ 4 120.1 97 2465 21.5 ║
║ 8 318 150 3520 15.5 ║
║ 8 304 150 3435 15.2 ║
║ 8 350 245 3840 13.3 ║
║ 8 400 175 3845 19.2 ║
║ 4 79 66 1935 27.3 ║
║ 4 120.3 91 2140 26 ║
║ 4 95.1 113 1513 30.4 ║
║ 8 351 264 3170 15.8 ║
║ 6 145 175 2770 19.7 ║
║ 8 301 335 3570 15 ║
║ 4 121 109 2780 21.4 ║
║ Memory still available for Dynamic Allocation: 289.1 Kbytes ║
║ Press Q to QUIT now...Other Key to Continue... ║
╚════════════════════════════════════════════════════════════════════╝
Note that even a "smallish" data set like this one can easily grab 15
Kbytes of RAM.
RXridge, version 9301 . . . . . . . . . . . . . . . Page 10 of 30
The RXridge "main menu" will appear next. But you don't have any
options on this first visit. Item "M" is mandatory right now!!!
╔══════════════════ softRX menus ══════════════════════════╗
║ ║
║ ╔════════════════════════════════╗ ║
║ ║ Main Menu ║ ║
║ ║ ║ ║
║ ║ M = Model Parameters ║ ║
║ ║ ║ ║
║ ║ S = Shrinkage Calculations ║ ║
║ ║ ║ ║
║ ║ R = Review Previous Results ║ ║
║ ║ ║ ║
║ ║ X = eXit RXridge ╔════════════════════════════════╗ ║
║ ║ ║ Item "M" from the softRX ║ ║
║ ║ choice --> M ║ Main Menu... ║ ║
║ ║ ║ ║ ║
║ ╚═════════════════════║ M = Model Parameters ║ ║
║ ║ ║ ║
║ ║ is mandatory at this time. ║ ║
║ ║ Press a Key... ║ ║
║ ╚════════════════════════════════╝ ║
╚════════════════════════════════════════════════════════════════════╝
First of all, some administrative details. Answering "N" for NO to the
first question generates the next three prompts...
╔═════════════════════ RXmodel Vars and Parms ═════════════════════╗
║ Y is ║
║ Use Default Screen & Pause Settings ? [Y|n] : N <------- ║
║ Default ║
║ Specify Level for On-Screen Detail: ║
║ ║
║ C => Complete Details.......Level = 1. ║
║ A => Abbreviated Detail.....Level = 2. ║
║ N => No Details Displayed...Level = 3. ║
║ Default is C ║
║ choice --> N <--------------- ║
║ ║
║ Screen Pausing: 0 => NO, 1 => YES Default is 1 ║
║ Set Screen Display PAUSE Parameter [1] : 0 <--------------- ║
║ ║
║ Screen Scroll Length: 10 to 40 lines. ║
║ Set Screen Scroll Length [22] : <---------------------- ║
║ Accept Default of 22 ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 11 of 30
Respond to each of the following variable "status" prompts by pressing
the P, R, or I key...or simply press Enter to get the [default] value...
╔═══════════════════ RXmodel Vars and Parms ═══════════════════════╗
║ ║
║ You will be prompted to specify a STATUS for each RXridge variable.║
║ ║
║ Specify... P ...to indicate a Predictor variable. ║
║ R ...to indicate the single Response. ║
║ I ...to ignore a variable in RXridge computations. ║
║ ║
║ Variable No. 1, Name = CYLNDS ... Status [P] : p ║
║ Variable No. 2, Name = CUBINS ... Status [P] : p ║
║ Variable No. 3, Name = HPOWER ... Status [P] : p ║
║ Variable No. 4, Name = WEIGHT ... Status [P] : p ║
║ Variable No. 5, Name = MPG ... Status [R] : r ║
║ ║
║ Number of Predictors = 4; Response Index = 5. ║
║ Take N Default ║
║ Use only a Subset of the Observations ? [N|y] : <----------------- ║
╚════════════════════════════════════════════════════════════════════╝
A response of Y to the "Subset of Observations" prompt would generate
the following sorts of additional prompts...
╔═════════════════════ RXmodel Vars and Parms ═════════════════════╗
║ Use only a Subset of the Observations ? [N|y] : Y ║
║ ║
║ Select a RANGE of Observations, n1 <= obs <= n2. ║
║ ║
║ First Observation [ 1 to 24 ] : n1 = 3 ║
║ First Obs. Number = 3 ║
║ ║
║ Last Observation [ 9 to 32 ] : n2 = 29 ║
║ Last Obs. Number = 29 ║
║ ║
║ Number of Observations in this Range = 27 ║
╚════════════════════════════════════════════════════════════════════╝
The "Vars & Parms" window continues to scroll upward revealing...
╔════════════════════ RXmodel Vars and Parms ══════════════════════╗
║ ║
║ Specify Variable Standardization Option: ║
║ ║
║ C => Center only (Mean at 0)..............Code=0 ║
║ R => Rescale (and center) variables.......Code=1 ║
║ E => rescale but Express on given scale...Code=2 ║
║ ║
║ choice --> R <<< Code=1 is the Default Here >>> ║
║ ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 12 of 30
Choice of scales along the given regressor axes is an extremely
important "initial phase" of every ridge regression analysis. Choice
R puts all variables into what Hoerl and Kennard(1970a,b) call
"correlation form." Choice C uses the given scales, which is a
reasonable choice as long as those scales are not too different.
Choice E rescales as in R but the re-expresses coefficient and risk
estimates as in C, using a rescaling invariance assumption.
Next...
╔════════════════════ RXmodel Vars and Parms ══════════════════════╗
║ Forming Response Vector and Predictor Matrix... ║
║ ║
║ Specify filename (max 8 characters) for ║
║ Regressor .XRX and Response .YRX Output [mpg] : <-------------- ║
║ Defaults here! ║
║ Use Default Screen & Pause Settings ? [Y|n] : <-------------- ║
╚════════════════════════════════════════════════════════════════════╝
If you hit some "wrong" keys while responding to prompts, you may not
be able to "correct" them right away...especially if you have already
pressed the ENTER key. However, when the main menu reappears, as
below, you can always re-run item "M" as often as necessary to get
each prompt response EXACTLY want you want!
On the other hand, let us suppose that you made no Model/Parameter
specification blunders. In that case, one usually proceeds by selecting
item "S" next...
╔════════════════════════════════╗
║ Main Menu ║
║ ║
║ M = Model Parameters ║
║ ║
║ S = Shrinkage Calculations ║
║ ║
║ R = Review Previous Results ║
║ ║
║ X = eXit RXridge ║
║ ║
║ choice --> S ║
║ ║
╚════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 13 of 30
ANOVA statistics will now appear for the principal axis rotation that
yields a canonical form...with "Uncorrelated Components"...
╔═════════════════════ RXridge Calculations ═══════════════════════╗
║ ║
║ PRELIMINARY CALCULATIONS: ║
║ ║
║ Memory still available for Dynamic Allocation: 273.2 Kbytes. ║
║ ║
║ ║
║ Component Summary Statistics: ║
║ ║
║ Singular Uncorrelated Principal t- ║
║ Values Components Correlations Statistics ║
║ 10.31 -0.4872 -0.9025 -12.05 ║
║ 3.35 -0.1325 -0.0797 -1.06 ║
║ 2.084 0.1535 0.0575 0.77 ║
║ 1.429 -0.6094 -0.1564 -2.09 ║
║ ║
║ Uncorrelated Component Std.Error = 0.074874 ║
║ R-squared Statistic = 0.8486 ║
║ Press a Key... ║
╚════════════════════════════════════════════════════════════════════╝
ANOVA statistics for the given regressor axes appear next. Note that
attention is drawn to any "sign conflicts" between Beta coefficient
estimates and the corresponding marginal correlation between regressor
and response coordinates...
╔═════════════════════ RXridge Calculations ═══════════════════════╗
║ ║
║ Ordinary Least Squares Summary: ║
║ ║
║ Marginal Regression Relative t- ║
║ Correlations Coefficients Std.Errors Statistics ║
║ -0.8522 -0.3832 0.4662 -1.97 ║
║ -0.8476 <--> 0.2385 0.5785 0.99 ║
║ -0.7762 -0.2336 0.3315 -1.69 ║
║ -0.8677 -0.6257 0.3955 -3.80 ║
║ ║
║ <--> ...Sign Conflict, Corr. vs Coeff. ║
║ Residual Variance = 0.17379 ║
║ Press a Key... ║
╚════════════════════════════════════════════════════════════════════╝
When the "shrinkage calculations" MENU appears, as below, you have a
choice between several possible tactics/strategies. For example, if you
only want to perform "ordinary" ridge regression, you could use item "S"
to confirm the default path shape [Q=0] and enter a custom filename for
output files. (In fact, you could even go directly to item "T" in this
case and write all output to the default file, "rxridge".)
RXridge, version 9301 . . . . . . . . . . . . . . . Page 14 of 30
But, here, let us select item "C" to let our data suggest an optimal
"Q-shape" via classical (normal theory) maximum likelihood...
╔════════════════════ RXridge Calculations ════════════════════════╗
║ Singular Uncorrelated Principal t- ║
║ Values Components Correlations Statistics ║
║ 10.31 -0.4872 -0.9025 -12.05 ║
║ 3.35 -╔════════════════════════════════════╗ ║
║ 2.084 ║ Shrinkage Calculations ║ ║
║ 1.429 -║ [Path Shape: Q= 0.0] ║ ║
║ ║ ║ ║
║ Uncorrelated Co║ C = Classical Max.Likelihood Shape ║ ║
║ R-║ ║ ║
║ Press a Key... ║ S = Specify Shrinkage Shape, Q ║ ║
║ ║ ║ ║
║ Ordinary Least ║ T = TRACE Computations ║ ║
║ ║ ║ ║
║ Marginal ║ M = Monitor 3 Extent Likelihoods ║ ║
║ Correlations ║ ║ ║
║ -0.8522 ║ R = Return to Main Menu ║ ║
║ -0.8476 <-->║ ║ ║
║ -0.7762 ║ choice --> C ║ ║
║ -0.8677 ║ ║ ║
║ ╚════════════════════════════════════╝ ║
║ <--> ...Sign Conflict, Corr. vs Coeff. ║
║ Residual Variance = 0.17379 ║
║ Press a Key... ║
╚════════════════════════════════════════════════════════════════════╝
The following table appears in the next window...
╔═════════════════════ RXmodel Vars and Parms ═════════════════════╗
║ ║
║ Classical Maximum Likelihood Shape Selection: ║
║ ║
║ Q-shape MCAL Konst CRLQ Chi-Sq Best ║
║ ║
║ 2.00 2.717 0.314 0.2980 57.91 ║
║ 1.50 1.839 0.267 0.4284 55.00 ║
║ 1.00 0.733 0.224 0.6492 46.25 ║
║ 0.50 0.413 0.303 0.8681 27.77 ║
║ 0.00 0.611 1.01 0.9670 9.94 ║
║ -0.50 1.340 7.42 0.9881 4.00 ║
║ -1.00 2.121 74.1 0.9881 3.97 <<< ║
║ -1.50 2.601 783 0.9853 4.86 ║
║ -2.00 2.850 8.27e+003 0.9830 5.54 ║
║ ║
║ NOTE: The Q-shape most likely to be MSE optimal ║
║ maximizes CRLQ and minimizes Chi-Squared. ║
║ ║
║ Press a Key... ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 15 of 30
So, now that we know our data favor the Q = -1 shrinkage pattern, which
Q-shape are we going to actually use? We still have several options!
Let us return to the "shrinkage calculations" MENU, and use item "S" to
specify a Q-shape...
╔══════════════════════ RXridge Calculations ══════════════════════╗
║ ║
║ Shrinkage Shape Specification... ║
║ ║
║ Q = Ridge Path Shape parameter can range from -5 to +5. ║
║ ┌─────┐ ║
║ Specify Q Shape [ 0.0] : │ -1 │ <----------------------- ║
║ └─────┘ Enter your choice here! ║
║ Q Shape = -1.0. ║
║ ║
║ Specify filename (max 8 characters) Accept Default ║
║ for Trace Matrix Output [mpg-1] : <---------------------- ║
║ ║
║ The Trace Matrix Output files will have filename: mpg-1 ║
║ ║
║ Press Any Key to Continue... ║
╚════════════════════════════════════════════════════════════════════╝
We could have chosen the "ordinary" ridge value of Q = 0 above, but we
will (first) test drive the Q = -1 value suggested by the data!
The following screens will now appear...
╔═════════════════════ Shrinkage Pattern Type ═════════════════════╗
║ ║
║ Shrinkage Pattern: Declining Deltas...Q < 1. ║
║ ║
║ Delta shrinkage factors for principal axes will ║
║ be in the same numerical order as the observed ║
║ spreads in regressor coordinates along those axes. ║
║ ║
║ This will shrink the relatively imprecise ║
║ components of the regression coefficient vector ║
║ more than the relatively precise components. ║
║ ║
╚════════════════════════════════════════════════════════════════════╝
╔══════════════════════ RXridge Calculations ══════════════════════╗
║ ║
║ RIDGE TRACE CALCULATIONS: ║
║ ║
║ [Path Shape: Q=-1.0] ║
║ Take ║
║ Specify No. of Steps Per Unit Increase in M [8] : <---------- ║
║ Default ║
RXridge, version 9301 . . . . . . . . . . . . . . . Page 16 of 30
║ Steps Per Unit = 8 implies a total of 33 Steps ║
║ along the TRACE from M = 0 to M = 4. ║
║ Confirm ║
║ Is that number of Total Steps satisfactory ? [Y|n] : <-------- ║
║ ║
║ Memory still available for Dynamic Allocation: 256.8 Kbytes. ║
║ ║
║ Now Computing...Step = 33, MCAL = 4.000 ║
║ ║
║ Press a Key... ║
╚════════════════════════════════════════════════════════════════════╝
Next the CGA interactive graphics TRACE displays will appear in the
following order...
1: Ridge Coefficient Trace
2: Estimated "Scaled" Risk (Mean-Squared-Error) Trace
3: Excess EigenValues Trace
4: Inferior Direction-Cosine Trace
5: Shrinkage Factor Pattern Trace
You will see the following HELP screen if you press the Space Bar
or function key F1 when in this graphics mode...
╔══════════════ RXridge.EXE - softRX (tm) shareware ══════════════╗
║ To view this HELP screen, press............. SpaceBar or F1 ║
║ To view the NEXT type of TRACE, press....... Enter or Escape ║
║╔═ Dynamic On-Screen Annotation ═╗╔═════ Special Effect Keys ══════╗║
║║ ║║ ║║
║║ View VALUE at Cursor, press V. ║║ Press the S key first, then ║║
║║ ...then move the Cursor with ║║ press a Second, Special Key... ║║
║║ ArrowKeys, Tab & BackSpace. ║║ ║║
║║ ║║ L or F10...Dump screen to HP ║║
║║ To HIGHLIGHT coordinates for a ║║ LaserJet/DeskJet. ║║
║║ Variable/Axis, press........H. ║║ I or F9....Dump to IBM dot ║║
║║ ║║ matrix graphics. ║║
║║ Move On-Screen Messages to.... ║║ E or F8....Dump to Epson FX. ║║
║║ Top of Screen, press..PgUp. ║║ O or F7....Dump to Okidata. ║║
║║ Bottom of Screen,.....PgDn. ║║ ║║
║║ ║║ S ...Save Screen in PCX form ║║
║║ NAMES and NUMERICS, press....N ║║ to specified filename. ║║
║║ & scroll left/right arrows. ║║ ║║
║╚════════════════════════════════╝╚════════════════════════════════╝║
╚══════════════════ Press Any Key to Continue... ══════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 17 of 30
The following is a much more complete discussion of the "interactive"
graphical options you have whenever RXridge displays a trace plot...
ENTER or ESCape key...will remove all on-screen annotation from a
trace and/or display the next trace.
V key....Displays an On-Screen Cursor that you can move with the "arrow"
keys (on your numeric keypad) as well as an On-Screen Listing
of the numeric values at the Cursor (see PgUp and PgDn, below.)
Use the up and down arrows to jump between the different curves
on each trace. Use the left and right arrows to decrease and
increase, respectively, MCAL by one "step", usually 0.125. Use
the BackSpace and Tab keys to decrease and increase,
respectively, MCAL, by 4 steps, usually 0.500. Finally, press
either... S (special key) to Save/Print the annotated screen
or....... ENTER key, ESCape key, or space bar ONCE to eliminate
the On-Screen Cursor and Values.
PgUp key...Move On-Screen Messages to the TOP of the Screen.
PgDn key...Move On-Screen Messages to the BOTTOM of the Screen.
N key....Replaces the trace plot with a "NAMES and NUMERICS" screen.
Use the right arrow, left arrow, Tab, and BackSpace keys to
change MCAL (by 1 or 4 steps)...same as with the V key, above.
(Up and down arrow keys are not used here.) Finally, press
ENTER just once to return to the trace plot.
H key....HIGH-LIGHT one of the curves on the trace plot by changing its
color from cyan to magenta and display the NAME of the
corresponding regressor variable or principal axis. Use PgUp
and PgDn to reveal trace details under the on-screen message.
You may Highlight curve "0" to return to a "plain" trace plot.
S key....SPECIAL KEY signal. A message will appear to indicate which
keys have been activate for screen saves, dumps, etc. Press
any other key to deactivate the special keys...
L or F10 dumps screen to HP Laserjet
I or F9 dumps to IBM Graphics Matrix
E or F8 dumps to Epson FX, JX, or LQ
O or F7 dumps to Okidata Microline
S = SAVE screen...as filename.PIX
NOTE: If you invoke the DOS "graphics" command before entering
RXridge, you could also use the Shift-PrtSc key combination
for screen dumps to your printer. With some dot-matrix
printers, this may work better than the I, E, O Special Keys.
RXridge, version 9301 . . . . . . . . . . . . . . . Page 18 of 30
╔═════════════════════════════════════════════════════════╗
║ GUIDELINES on the Interpretation of Ridge Trace Plots ║
╚═════════════════════════════════════════════════════════╝
┌─────────────────────────┐
│ Ridge Coefficient Trace │
└────┬────────────────────┴────────────────────────────────────────────┐
│ This trace shows how regression coefficient point estimates │
│ change as shrinkage (along a path of shape Q) occurs. Any │
│ coefficient estimate that is numerically "stable" will plot │
│ close to the straight line from its least-squares estimate at │
│ MCAL=0 to zero at MCAL=P. Unstable coefficient estimates │
│ will change more quickly, possibly switching numerical sign, as │
│ soon as MCAL starts increasing from zero. Super-stable │
│ estimates will change only very little initially, finally │
│ approaching zero only as MCAL approaches P. │
└─────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────┐
│ Estimated "Scaled" Risk (Mean-Squared-Error) Trace │
└────┬───────────────────────────────────────────────┴─────────────────┐
│ This trace gives normal distribution theory, "modified" maximum │
│ likelihood estimates of "scaled" risk (mean-squared-error │
│ loss) as shrinkage of shape Q occurs. │
│ │
│ The risk is "scaled" by dividing it by an estimate of the │
│ error (disturbance term) variance. In other words, scaled │
│ risk expresses imprecision in fitted coefficients as a │
│ multiple of the variance of a single observation. │
│ │
│ Maximum likelihood scaled risk estimates are "modified," │
│ first of all, so as to be unbiased. Then they are adjusted │
│ upward, if necessary, to have correct range relative to a │
│ known lower bound on scaled risk, which re-introduces bias. │
└─────────────────────────────────────────────────────────────────┘
┌───────────────────────────┐
│ Excess EigenValues Trace │
└────┬──────────────────────┴──────────────────────────────────────────┐
│ This trace plots the EigenValues of the estimated difference in │
│ Mean Squared Error matrices, ordinary least squares minus ridge.│
│ As long as all EigenValues are zero or positive, there is good │
│ reason to hope that the corresponding ridge estimators yield │
│ smaller MSE risk than Least Squares for all directions in │
│ P-space (i.e. all possible linear combinations.) As shrinkage │
│ continues, at most one negative EigenValue will appear. │
└─────────────────────────────────────────────────────────────────┘
RXridge, version 9301 . . . . . . . . . . . . . . . Page 19 of 30
┌──────────────────────────────────┐
│ Inferior Direction-Cosine Trace │
└────┬─────────────────────────────┴───────────────────────────────────┐
│ This trace plots the Direction Cosines (normalized EigenVector) │
│ corresponding to any negative EigenValue of the difference in │
│ MeanSquaredError matrices, OLS - ridge. This direction gives │
│ that single linear combination of ridge regression coefficients │
│ that not only fails to benefit from ridge shrinkage of shape Q │
│ but probably actually suffers increased risk due to shrinkage. │
└─────────────────────────────────────────────────────────────────┘
┌────────────────────────────────┐
│ Shrinkage Factor Pattern Trace │
└────┬───────────────────────────┴─────────────────────────────────────┐
│ This trace plots the Delta Shrinkage-Factor Pattern as shrinkage│
│ of shape Q occurs. All deltas are equal when Q=1; the trailing │
│ deltas are small when Q < 1; and the leading deltas are small │
│ when Q > 1. │
└─────────────────────────────────────────────────────────────────┘
So what did we learn from the traces about Hocking's MPG data? Well,
first of all, this certainly is an ill-conditioned regression situation!
The ridge regression coefficient for CUBINS goes negative (as it
"should" be) for MCAL of 0.75 or larger. But an "inferior direction"
also pops out with this much (or more) ridge shrinkage. Anyway, the
solution at ( MCAL = 1.0, Q-shape = -1 ) looks reasonable to me. What
do YOU think?
For more information on appropriate stopping-points along a ridge
shrinkage path, return to the "shrinkage calculations" MENU, and use
item "M". This option allows us to monitor the classical, empirical
Bayes & random coefficient likelihood ratio statistics associated with
any fixed Q-shape...
╔═════════════════════ softRX ridge ═════════════════════╗
║ ║
║ Monitor 3 Types of Likelihood Extent Selection: ║
║ ║
║ [Path Shape: Q=-1.0] ║
║ Take ║
║ Specify No. of Steps Per Unit Increase in M [8] : <-------- ║
║ Default ║
║ Steps Per Unit = 8 implies a total of 33 Steps ║
║ along the TRACE from M = 0 to M = 4. ║
║ ║
║ Is that number of Total Steps satisfactory ? [Y|n] : <----- ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 20 of 30
╔═════════════════════ softRX ridge ═════════════════════╗
║ ║
║ Now Computing...Step = 33, MCAL = 4.000 ║
║ ║
║ MCAL CLIK EBAYS RCOEF ║
║ 0.000 +infinity +infinity +infinity ║
║ 0.125 4.16e+003 22.3 22.4 ║
║ 0.250 1.82e+003 19.7 19.9 ║
║ 0.375 1.06e+003 18.3 18.5 ║
║ 0.500 686 17.3 17.6 ║
║ 0.625 467 16.6 16.9 ║
║ 0.750 327 16 16.3 ║
║ 0.875 232 15.4 15.8 ║
║ 1.000 164 GUP 15 15.4 ║
║ 1.125 116 14.6 15 ║
║ 1.250 80.5 14.2 14.6 ║
║ 1.375 54.6 13.8 14.2 ║
║ 1.500 35.8 13.5 13.9 ║
║ -- More -- ║
║ 1.625 22.6 13.1 13.6 ║
║ 1.750 13.5 12.9 13.3 ║
║ 1.875 7.85 12.6 13 ║
║ 2.000 4.84 12.4 12.8 ║
║ 2.125 3.97 <<< 12.3 12.7 ║
║ 2.250 4.84 12.2 <<< 12.6 <<< ║
║ 2.375 7.12 12.3 12.7 ║
║ 2.500 10.6 12.6 12.9 ║
║ 2.625 15.2 13.2 13.4 ║
║ 2.750 20.9 14.6 14.5 ║
║ 2.875 27.7 17.4 16.7 ║
║ 3.000 34.9 23.5 20.9 ║
║ 3.125 41 34.2 27 ║
║ 3.250 45.4 48.4 33.6 ║
║ 3.375 48.5 64.4 39.6 ║
║ 3.500 50.9 81.2 44.9 ║
║ 3.625 52.9 98.5 49.5 ║
║ 3.750 54.7 116 53.6 ║
║ 3.875 56.6 134 57.2 ║
║ 4.000 60.4 151 60.4 ║
║ ║
║ NOTE:The shrinkage-extent MCAL value most likely to be MSE optimal ║
║ minimizes a "normalized" (minus 2 log) likelihood ratio... ║
║ ║
║ MCAL= 1.0 is marked "GUP" above because this is the estimated ║
║ UPper limit on extent of shrinkage that is GOOD [dominates ║
║ least squares in the multivariate (matrix) MSE risk sense.] ║
║ ║
║ Press a Key... ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 21 of 30
CLIK is the asymptotic chi-square statistic for testing that the form
and extent of shrinkage at each step is MSE optimal. The solution with
the smallest entry in the CLIK column is thus the one which is most
likely to be the optimally biased estimate of Beta. However, if the
chi-square statistic is significantly greater than zero (compared to
quantities of chi-square with df = P-1 when QPAR is input or df = P-2
when QPAR is fit to data), then evidence is strong that the form (or
"shape") of the ridge path you are considering is not the most
appropriate one.
EBAYS is the minus-two-log-likelihood factor for the empirical Bayes
formulation of Efron and Morris(1977) in their discussion of Dempster
et.al. (1977).
RCOEF is the minus-two-log-likelihood factor for the random coefficient
formulation of Golub, Heath, and Wahba(1979) and Shumway(1982). This
third likelihood criterion behaves more like the empirical Bayes
criterion of Efron-Morris when M is small and more like the classical
Obenchain criterion as M approaches P.
According to the "2/R-ths rule" of Obenchain(1978), the MCAL shrinkage
extent for "good" (multivariate) ridge estimates is roughly a factor
of (two divided by R) times the "optimal" (univariate) MCAL values.
Since rank R = 4 in the MPG example, we again see evidence that ridge
shrinkage of at least MCAL = 1 is appropriate here.
*** RXreview of Output Files Saved to Disk ***
Next, we use item "R" on the "shrinkage calculations" MENU to return to
the MAIN MENU. And then item "R" there also to access file "Reviews"...
╔══════════════════════ RXridge Calculations ══════════════════════╗
║ ╔════════════════════════════════╗ ║
║ ║ Main Menu ║════════════════════════╗ ║
║ ║ ║kage Calculations ║ ║
║ ║ M = Model Parameters ║h Shape: Q=-1.0] ║ ║
║ ║ S = Shrinkage╔════════════════════════════════════╗ ║ ║
║ ║ R = Review Pr║ Review of Results ║ ║ ║
║ ║ X = eXit RXri║ [Filename:mpg-1 ] ║ ║ ║
║ ║ ║ ║ ║ ║
║ ║ choice --> R ║ H = Help on Review Keys & Menu ║ ║ ║
║ ╚════════════════║ ║ ║ ║
║ ║ R = Return to Main Menu ║ ║ ║
║ Now Computi║ ║ ║ ║
║ ║ S = Summary Files, for all Shapes ║ ║t write║
║ ║ ║ ║ ║
║ Press a Key║ T = TRACE Files of fixed Q-Shape ║ ║ ║
║ ║ ║═══╝ ║
║ ║ choice --> H ║ ║
║ ╚════════════════════════════════════╝ ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 22 of 30
The HELP item here first allows you to "move around" within the
RXreview Documentation Window (warning: CGA+ required here)...
╔══════════════════════ RXreview: rxreview.doc ══════════════════════╗
║12345678901234567890123456789012345678901234567890123456789012345678║
║ 1 2 3 4 5 6 ║
║Row 3 ║
║Row 4 RXreview.DOC...Browse text files stored on disk. ║
║Row 5 Window displays 23 lines of text. ║
║Row 6 ║
║Row 7 KEYS: ║
║Row 8 Esc, X or Q - terminate browse window display ║
║Row 9 ║
║Row 10 Up Arrow - move highlight line up one row ║
║Row 11 Down Arrow ║
║Row 12 or ENTER - move highlight line down one row ║
║Row 13 ║
║Row 14 Left Arrow - move window 20 characters left ║
║Row 15 Right Arrow - move window 20 characters right ║
║Row 16 ║
║Row 17 PgUp & PgDn - move highlight one page up or down ║
║Row 18 ║
║Row 19 Home - move highlight to the top,left of the file ║
║Row 20 End - move highlight to the bottom, left of file ║
║Row 21 ║
║ 1 2 3 4 5 6 ║
║12345678901234567890123456789012345678901234567890123456789012345678║
╚════════════════════════════════════════════════════════════════════╝
Then items "S" and "T" on the "review menu" are explained...
╔══════════════════════ RXreview Keys & Menu ══════════════════════╗
║ ║
║ Practice Using Review Window KEYS ? [N|y] : y ║
║ ║
║ The *** SUMMARY *** Review-Menu ITEM gives you access ║
║ to existing RXridge files associated with ridge paths ║
║ of all possible Q-shapes. ║
║ ║
║ The filename EXTensions for these files identify... ║
║ ║
║ .ARX => Archive (data/parameters) input for RXridge ║
║ ║
║ .XRX => Regressor X Matrix........input for PathProj ║
║ .YRX => Response Y Vector........input for PathProj ║
║ ║
║ .QRX => Classical Max.Likelihood Q-shape Selection ║
║ ║
║ Press a Key... ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 23 of 30
╔══════════════════════ RXreview Keys & Menu ══════════════════════╗
║ ║
║ The *** TRACE *** Review-Menu ITEM gives you access to ║
║ existing RXridge files associated with shrinkage along ║
║ a ridge path of fixed Q-shape, usually indicated by the ║
║ trailing NUMERICAL character(s) of the filename. ║
║ ║
║ The filename EXTensions for these files identify... ║
║ ║
║ .PRX => RXridge Parameters File...input for RXtraces ║
║ .CRX => ridge Coefficient Trace...input for RXtraces ║
║ .RRX => relative Risk Trace.......input for RXtraces ║
║ .ERX => Excess MSE Trace..........input for RXtraces ║
║ .IRX => Inferior Direction Trace..input for RXtraces ║
║ .DRX => shrinkage Delta factors...input for RXtraces ║
║ ║
║ .MRX => Monitor 3 Likelihoods for Extent-Selection ║
║ ║
║ Press a Key to Return to Review Menu... ║
╚════════════════════════════════════════════════════════════════════╝
When we return to the menu, let's try item "T"...
╔═════════════════════ RXreview TRACE Files ═══════════════════════╗
║ ║
║ This *** TRACE *** Review-Menu ITEM gives you access to ║
║ existing RXridge files associated with shrinkage along ║
║ a ridge path of fixed Q-shape, usually indicated by the ║
║ trailing NUMERICAL character(s) of the filename. ║
║ ║
║ The filename EXTensions for these files are always ║
║ .PRX, .CRX, .RRX, .ERX, .IRX, .DRX, or .MRX ... ║
║ ║
║ Existing Filename (max 8 characters) Take ║
║ for TRACE Review [mpg-1] : <----------------------------- ║
║ Default ║
║ All TRACE files will have filename: mpg-1 ║
║ ║
║ NOTE: Press Q Key to QUIT Review... ║
║ ║
║ Press a Key to Review : mpg-1.prx ║
║ ║
╚════════════════════════════════════════════════════════════════════╝
Here's what that file contains (warning: CGA+ required here)...
╔════════════════════════ RXreview: mpg-1.prx ═══════════════════════╗
║ 4 = P, number of regressors ║
║32 = N, number of observations ║
║ 8 = STEPSPU, steps per unit change in MCAL ║
║-1.0= QSHAPE of the shrinkage path ║
║XNAM= CYLNDS CUBINS HPOWER WEIGHT ║
╚════════════════════════════════════════════════════════════════════╝
RXridge, version 9301 . . . . . . . . . . . . . . . Page 24 of 30
Press X, Q, or the ESCape key to go on to the next file in the sequence.
Actually, rather than review all of the other "mpg-1" files now, press
the Q key to return to the main RXreview menu.
Now that we have explored most of the major features of RXridge, let us
end this on-disk tutorial...
╔══════════════════════ RXreview TRACE Files ══════════════════════╗
║ ║
║ ╔════════════════════════════════╗ ║
║ ║ Main Menu ║nu ITEM gives you access to ║
║ ║ ║════════════════════════╗g ║
║ ║ M = Model Parameters ║kage Calculations ║he ║
║ ║ ║h Shape: Q=-1.0] ║ ║
║ ║ S = Shrinkage Calculations ║ ║ ║
║ ║ ║al Max.Likelihood Shape ║ ║
║ ║ R = Review Previous Results ║ ║ ║
║ ║ ║ Shrinkage Shape, Q ║ ║
║ ║ X = eXit RXridge ║ ║ ║
║ ║ ║omputations ║ ║
║ ║ choice --> X ║ ║ ║
║ ║ ║ 3 Extent Likelihoods ║ ║
║ ╚════════════════════════════════╝ ║ ║
║ NOTE: Press║ ║ R = Return to Main Menu ║ ║
║ ║ S ║ ║ ║
║ ║ ║ choice --> R ║ ║
║ Press a Key║ T ║ ║ ║
║ ║ ╚════════════════════════════════════╝ ║
║ Press a Key║ choice --> R ║ ║
║ ║ ║ ║
║ ╚════════════════════════════════════╝ ║
╚════════════════════════════════════════════════════════════════════╝
REMINDER(S) :
RXRIDGE created an Output File named... mpg.dtl
Use the DOS invocation... TYPE mpg.dtl | MORE
to review detailed computational results from
this RXRIDGE run.
DOS Prompt> ...and we arrive back at the DOS system prompt!
┌──────────────────────────────┐
─────────────────────┤ RIDGE REGRESSION REFERENCES ├─────────────────
└──────────────────────────────┘
Dempster, A. P., Schatzoff, M. and Wermuth, N. (1976). "A simulation
study of alternatives to ordinary least squares." Journal American
Statistical Association, 72, 77-91 (with discussion, pp. 91-106.)
Efron B. and Morris, C. N. (1976). "Discussion" (of Dempster,
Schatzoff and Wermuth.) Journal American Statistical Association,
72, 91-93.
RXridge, version 9301 . . . . . . . . . . . . . . . Page 25 of 30
Golub, G.H., Heath, M., and Wahba, G. (1979). "Generalized cross-
validation as a method for choosing a good ridge parameter."
Technometrics 21, 215-223.
Hocking, R.R. (1976). "The Analysis and Selection of Variables in
Regression," Biometrics 32, 1-51.
Hoerl, A.E. (1962). "Application of Ridge Analysis to Regression
Problems." Chemical Engineering Progress 58, 54-59.
Hoerl, A.E. and Kennard, R.W. (1970a). "Ridge Regression: Biased
Estimation for Nonorthogonal Problems." Technometrics 12, 55-67.
Hoerl, A.E. and Kennard, R.W. (1970b). "Ridge Regression: Applications
to Nonorthogonal Problems." Technometrics 12, 69-82.
Mallows, C.L. (1973). "Some Comments on Cp." Technometrics, 15,
661-677.
Marquardt, D.W. (1970). "Generalized Inverses, Ridge Regression,
Biased Linear Estimation, and Nonlinear Estimation." Technometrics
12, 591-612.
Marquardt, D.W. and Snee, R.D. (1975). "Ridge Regression in Practice."
The American Statistician, 29, 3-19.
Nash, J.C. (1992). "Statistical shareware: illustrations from
regression techniques." The American Statistician 46, 312-318.
(A review article that includes coverage of softRX shareware.)
Obenchain, R.L. and Vinod, H. (1974). "Estimates of partial derivatives
from ridge regression on ill-conditioned data." NBER-NSF Seminar on
Bayesian Inference in Econometrics, Ann-Arbor, Michigan.
Obenchain, R.L. (1975). "Ridge Analysis Following a Preliminary Test
of the Shrunken Hypothesis." Technometrics, 17, 431-441. (Discussion:
McDonald, G.C., 443-445.)
Obenchain, R.L. (1976). "Methods of ridge regression." Proceedings of
the Ninth International Biometric Conference, Invited Papers, Volume
One, 37-57, Boston.
Obenchain, R.L. (1977). "Classical F-tests and Confidence Regions for
Ridge Regression." Technometrics 19, 429-439.
Obenchain, R.L. (1978). "Good and optimal ridge estimators." Annals of
Statistics 6, 1111-1121.
This highly theoretical paper derived the "ridge function," the
"excess mean squared error matrix," and the "inferior direction."
Normal distribution theory maximum likelihood estimators used in
RXridge are also discussed. Obenchain (1980) and Obenchain(1984)
describe uses of these concepts in practical applications.
RXridge, version 9301 . . . . . . . . . . . . . . . Page 26 of 30
Obenchain, R.L. (1980). Comment on "A Critique of Some Ridge Regression
Methods" by G. Smith and F. Campbell. Journal of the American
Statistical Association 75, 95-96.
Obenchain, R.L. (1984). "Maximum likelihood ridge displays."
Communications in Statistics A13, 227-240. (Proceedings of the
Fordham Ridge Symposium, ed. H.D.Vinod.)
Obenchain, R.L. (1991). "Ridge regression systems for MS-DOS personal
computers." The American Statistician 45, 245-246.
Press, W.H., Flannery, B.P., Teukolsky, S.A. and Vetterling, W.T.
(1988). "Numerical Recipes in C: The Art of Scientific Computing."
Cambridge, Massachusetts: Cambridge University Press. {Source Code:
Copyright 1985, 1987 by Numerical Recipes Software P.O.Box 243,
Cambridge, MA 02238.}
Shumway, R.H. (1982). "Maximum likelihood estimation of the ridge
parameter in linear regression." Technical Report, Department of
Statistics, University of California at Davis.
┌────────────────────────────────────────────────────────────────────┐
│ TECHNICAL APPENDIX......"Extent" and "Shape" of Shrinkage in the │
│ Two-Parameter Generalized Ridge Family. │
└────────────────────────────────────────────────────────────────────┘
MCAL = the "Multicollinearity Allowance" parameter that indexes
the "extent" of ridge shrinkage along any ridge path.
= R - trace( R x R diagonal matrix of Delta Shrinkage Factors ).
MCAL = 0 ...yields zero shrinkage. This is the "starting point" of the
ridge path, where the ridge estimator coincides with the
Ordinary Least Squares estimator at the left-hand extreme.
MCAL = R ...yields "total" shrinkage. This is the right-hand "end
point" of the path, where the ridge estimator is all ZEROS.
Again, MCAL = R - Delta(1) -...- Delta(R), where Delta(j) is the ridge
"shrinkage factor" applied to the j-th uncorrelated component, c(j), of
Bzero . The average value of Delta(1),...,Delta(R) is (R-M)/R, which is
Theil's "proportion of posterior precision in Bstar due to sample
information." More importantly, MCAL can be interpreted as the
approximate deficiency in the rank of ( I - 11'/ N ) X. For example, if
the regressor matrix has only two relatively small singular values, then
the coefficient ridge trace is expected to "stabilize" at about MCAL =
2. Perfectly stable relative magnitudes plot on the MCAL-scale as
straight lines all intersecting at MCAL = R and Bstar = 0.
Q = the ridge parameter that controls the "shape" (or "curvature") of
the ridge path through regression coefficient likelihood space.
Q = +1 ...yields uniform shrinkage (all Shrinkage Factors equal.)
Q = 0 ...yields Hoerl-Kennard "ordinary" ridge regression.
RXridge, version 9301 . . . . . . . . . . . . . . . Page 27 of 30
Q = -5 ...is usually very close, numerically, to "Principal
Components Regression," with exact agreement in the
limit as Q approaches minus infinity.
Shrinkage Factor Formulas...
P = Number of Predictor Variables (non-constant Regressors),
R = Rank of the Centered Predictor Variable X-matrix,
N = Number of Observations (or Regressor Combinations), and
generalized ridge regression "Shrinkage Factors" are of the form...
EigenValue
Delta = ----------------------------------
EigenValue + Konstant*EigenValue^Q
or, equivalently,... 1
= -----------------------------
1 + Konstant*EigenValue^(Q-1)
Empirical evidence that choice of "shape" as well as "extent" of
shrinkage can be rewarding is given in the following table...
Min.MeanSqErr Min.MeanSqErr
Data Set Number of Number of Extent of Shrinkage
Name Observations Predictors Shrinkage Shape
============= ============ ========== ============= =============
FACE data N = 21, R = 10, MCAL = 2.3, Qshape = +.77
Air Pollution N = 60, R = 15, MCAL = 5.4, Qshape = +.07
and Mortality
Acetylene N = 16, R = 9, MCAL = 5.2, Qshape = -.35
Ten-Factor N = 36, R = 10, MCAL = 3.6, Qshape = -.78
Stack Loss N = 15, R = 3, MCAL = 0.24, Qshape = -.95
Mantell, Bell N = 25, R = 3, MCAL = 0.95, Qshape = -1.1
Productivity
Wood Beam N = 10, R = 2, MCAL = 0.26, Qshape = -1.4
Longley N = 16, R = 6, MCAL = 4.0, Qshape = -1.4
Hocking MPG N = 32, R = 10, MCAL = 8.8, Qshape = -7.6
RXridge, version 9301 . . . . . . . . . . . . . . . Page 28 of 30
Diesel data N = 44, R = 9, MCAL = 4.9, Qshape = -20
Portland N = 13, R = 4, MCAL = 3.0, Qshape = -INF.
Cement, Hald.
============= ============ ========== ============ =============
Data Set Number of Number of Extent of Shrinkage
Name Observations Predictors Shrinkage Shape
Min.MeanSqErr Min.MeanSqErr
┌───────────────────────────┐
───────────────────┤ RXridge UpDate History... ├────────────────────
└───────────────────────────┘
9004 ...BETA-TEST version
9005 ...original release; LZEXE 0.91 (c) Fabrice Bellard
9008 ...bounce-bar file selection menu; change-of-address
9010 ...change screen save file to .PCX format; ANSI.SYS warning
9012 ...handle exact singularities; adopt huge memory model;
eliminate need for device=ANSI.SYS; shadows on windows
9101 ...convert to Microsoft C version 6.00AX
9103 ...documentation update; final change-of-address
9104 ...fix help-screen bug in version 9103
9108 ...fix shrinkage pattern default for special cases where
all positive regressor singular values are equal and
abandon calculations when SVD is numerically inaccurate.
9109 ...round centered and rescaled X matrix to at most six
decimal places to enhance convergence in SVD algorithm.
9110 ...make more checks on amount of RAM available for dynamic
matrix allocation; avoid "most" lack-of-memory crashes.
...add column of k = additive-eigenvalue-inflation-factor
to the .MRX (shrinkage extent selection) output file.
9205 ...fix "bug" in small-MCAL display of inferior direction.
...ask permission before overwriting an existing .ARX file.
9207 ...display "GUP" pointer to the most-likely MCAL upper limit
from the "2/R-ths Rule-of-Thumb"...this is the max extent
of shrinkage likely to be "GOOD" in multivariate/matrix
MSE risk [rather than OPTIMAL in a scalar-valued sense.]
9208 ...fix major batch-input BUG in the July 1992 version.
9301 ...add date/time stamp at beginning of .DTL output file.
9303 ...documentation review/update.
RXridge, version 9301 . . . . . . . . . . . . . . . Page 29 of 30
┌─────────┐
┌─────┴───┐ │ (R)
──│ │o │──────────────────
│ ┌─────┴╨──┐ │ Association of
│ │ │─┘ Shareware
└───│ o │ Professionals
──────│ ║ │────────────────────
└────╨────┘ MEMBER
ASP wants to make sure that the shareware principle works for you. If
you are unable to resolve a shareware-related problem with an ASP member
by contacting the member directly, ASP may be able to help. The ASP
Ombudsman can help you resolve a dispute or problem with an ASP member,
but does not provide technical support for members' products. Please
write to the ASP Ombudsman at 545 Grover Road, Muskegon, MI 49442-9427
or send a CompuServe Mail message to 70007,3536.
┌───────────────────────────────────────────────────────────────────┐
│ In an effort to make it easier for you to obtain your favorite │
│ software, the latest updates of many ASP authored programs can be │
│ found on the BBSs listed below. These BBSs are members of the... │
├───────────────────────────────────────────────────────────────────┤
│ ASP Hub Network (AHN). │
└───────────────────────────────────────────────────────────────────┘
Zone 1 - East Coast USA
North-East Coast Mid-East Coast
[Site #1] [Site #2]
The Consultant BBS The Break RBBS <East>
Jay Caplan Bruce Jackson
P.O. Box 8571 4660 Whitaker PL
New York NY 10116-4655 Dale City, VA 22193-3011
Data1) 718-837-3236 Data1) 703-680-9269
Data2) Data2) 703-551-0000
Zone 2 - North Mid-USA Zone 3 - Southern Mid-USA
[Site #3] [Site #4]
The Twilight Zone The DataExchange BBS
John Hrusovszky Don Morris
1119 E. Main St 119 Herring St.
Auburndale, WI 54412 Leesville, LA 71446
Data1) 715-652-2758 Data1) 318-239-2122
Data2) Data2)
RXridge, version 9301 . . . . . . . . . . . . . . . Page 30 of 30
Zone 4 - West Coast USA
[Site #5] [Site #6]
Attention to Details BBS Space BBS
Clint Bradford Owen Hawkins
5085 Trail Canyon Dr PO Box X
Mira Loma CA 91752 Menlo Park, CA 94026
Data1) 909-681-6221 Data1) 415-323-4398 (ASP Files)
Data2) Data2) 415-323-4193 (Other Files)
Zone 5 - Canada
[Site #7]
Knightec BBS
Phil Knight
35 Robb Blvd #6
Orangeville, ONT L9W 3L1
Data1) 519-940-0007
Data2)
┌──────────────────────────────────────────────────────────────┐
│ These BBSs are bound by special agreement with the ASP. │
│ In the case of a dispute contact the ASP Omsbudsman. │
└──────────────────────────────────────────────────────────────┘