home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Explore the World of Soft…e: Engineering & Science
/
Explore_the_World_of_Software_Engineering_and_Science_HRS_Software_1998.iso
/
programs
/
statistc
/
seqd9109.txt
< prev
next >
Wrap
Text File
|
1997-09-22
|
62KB
|
1,065 lines
TABLE of CONTENTS
Section 1 - Introduction..............................................1
Section 2 - PC System Requirements....................................8
Section 3 - SeqDir Walk-Through.......................................9
Section 4 - Effects of Changing SeqDir Parameters....................19
References...........................................................22
┌────────────────────────────┐
│ Section 1 - Introduction │
└────────────────────────────┘
SeqDir.EXE applies the Aroian(1968) "Direct Method" of Sequential
Analysis to evaluate Operating Characteristic (OC) and Average Sample
Number (ASN) curves corresponding to any specified shape of Accept /
Continue / Reject boundary. And, besides these numerical approximations
to "exact" results, SeqDir also performs Monte-Carlo simulations and
displays Sample Paths graphically! But only applications in which
sequential observations have independent, identical gamma (or
exponential) distributions with an integer valued "shape" parameter are
treated by SeqDir.
NOTE: The best known forms of sequential tests that use exponential
distributions are the ones that treat "survival/lifetime" data.
Specifically, that type of data consists of numbers and failure
times of certain units, plus "truncated" data (time so-far
on-test) for units that have not yet failed. The applications
treated by SeqDir cannot use this sort of truncated lifetime data.
SeqDir designs sequential tests to detect shifts in non-negative
measurements of "cost", "loss" or "regret"; i.e. situations where large
or increasing numerical values represent undesirable performance. These
sorts of measures can frequently be viewed as "surrogates" for the
long-term economic impact of deviations from an intended target value,
i.e. measures of the Cost-of-Poor-Quality. Almost any sort of numerical
data can be CONVERTED into cost/loss form via a so-called "regret"
function. (Four forms of regret function are illustrated below.) Once
this "conversion to regret" process is applied to historical data, the
resulting regret distribution is frequently well approximated by a gamma
or exponential. In particular, regret distributions are confined to the
range from 0 to infinity and frequently are positively skewed (i.e. they
have a [long] tail "trailing off" to the right...)
┌────────────────────────────────────────────────────────────────┐
│ │
│ │ Gamma Distribution │
│ │ ( with Shape Parameter > 1 ) │
│ │ * * │
│ │ * * │
│ │ * │
│ │ * * │
│ │ * │
│ │* * │
│ └─────────────────────────────────────────────── │
│ 0 Cost --> │
│ │
├────────────────────────────────────────────────────────────────┤
│ │
│ │ SPECIAL CASE: Exponential Distribution │
│ │* ( Shape Parameter = 1 ) │
│ │ * │
│ │ * NOTE: The Gamma MODE cost is │
│ │ * ZERO when the Shape parameter │
│ │ * is less than or equal to 1. │
│ │ * │
│ │ * │
│ │ * │
│ │ * │
│ └─────────────────────────────────────────────── │
│ 0 Cost --> │
│ │
└────────────────────────────────────────────────────────────────┘
There are two minimal requirements a function must satisfy before it can
be used to convert an arbitrary quantitative measurement, Y, into a
measure of "regret." These are...
(i) the function may assume only non-negative values, r(Y) >= 0,
and
(ii) the function must assume the value ZERO at the (or all) target
value(s) for Y, r(T) = 0.
Although not required, a third property is frequently present...
(iii) regret functions usually are monotone non-decreasing as the
deviation from target increases (above or below target.)
Three types of regret function likely to yield gamma or exponentially
distributed cost/loss results are:
┌─────────────────────────────────────────────────────────────────────┐
│ Regret Type: Absolute Value (Possibly Asymmetric) │
├─────────────────────────────────────────────────────────────────────┤
│ Percent of Label Claim --> │
│ | │
│ *** | │
│ *** | **** │
│ *** | **** │
│ +--***------|-----------**** │
│ | *** | ****| │
│ | ***| **** | │
│ Regret = 0 -----+-----------****--------+----------- │
│ 95% 100% 105% │
│ │
│ Regret = Left/Right Factor * Absolute Deviation from Target. │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Regret Type: Quadratic (Taguchi's "Loss to Society") │
├─────────────────────────────────────────────────────────────────────┤
│ Measurement --> │
│ │
│ * | * │
│ * | * │
│ ** | ** │
│ ** | ** │
│ ** | *** │
│ *** | *** │
│ **** | **** │
│ Loss = 0 ------------------*******--------------------- │
│ | │
│ Target │
│ │
│ Regret = ( Value - Target )^2 │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Regret Type: OneSided Quadratic │
├─────────────────────────────────────────────────────────────────────┤
│ * Measurement --> │
│ * │
│ * | │
│ * | NOTE: There is no │
│ ** | penalty, whatsoever, │
│ ** | for deviations on │
│ ** | ONE SIDE of the target. │
│ *** | │
│ **** | │
│ Regret = 0 ------------------**************************** │
│ | │
│ Target │
└─────────────────────────────────────────────────────────────────────┘
A fourth type of regret which can yield cost/loss measures which,
although actually bounded, are somewhat like a (truncated) gamma or
exponential is:
┌─────────────────────────────────────────────────────────────────────┐
│ Regret Type: Double Logit (Bounded Maximum) │
├─────────────────────────────────────────────────────────────────────┤
│ | Measurement--> | │
│ ****** | | | ******* │
│ ****** | | | ****** │
│ ***** | ***** <-- Half │
│ |**** | ****| Maximum │
│ | *** | *** | │
│ | **|** | │
│ Regret = 0 ----------------------*---------------------- │
│ | | | │
│ minus Target plus │
│ HalfWidth HalfWidth │
│ │
│ Regret = AbsDeviation / ( HalfWidth + AbsDeviation ) │
└─────────────────────────────────────────────────────────────────────┘
************************************************************
EXAMPLE: Suppose that historical costs derived from potency assay values
using "absolute value" regret have an exponential distribution
(gamma with shape parameter 1) with expected value of 1% of
Label Claim. Food and Drug Administration regulations require
this regret be no more than 5% of Label Claim. For each new
batch of product, the manufacturer wishes to perform as many
assays as necessary to classify the new batch is either "good"
(within established process capability) or "bad".
************************************************************
Sequential test plans detect situations where the expected value of a
cost/loss distribution has shifted (from lower-cost) toward higher-cost
values...i.e. situations where costs become markedly higher than their
historical and/or current process capability levels.
Sequential Probability Ratio Test (SPRT) procedures, Wald(1947), for
gamma distributed regret can utilize any test statistic that is a
monotonic function of the cumulative sum of regret values over
sequential steps. At step N, this regret cumulative sum is
Rsum(N) = r(1) + r(2) + ... + r(N),
where r(s) is the regret observed at sequential step "s" of the test.
The argument that Rsum(N) is the pivotal statistic for all values of the
gamma Shape parameter "p" and the gamma Scale parameter "R" goes as
follows. The general form of gamma probability density function is
f( r ) = r^(p-1) * exp( - r / R ) / [ R^p * Gamma(p) ]
over the range from r = 0 to infinity, where R^p means "R raised to the
p-th power," Gamma(p) denotes the gamma function, and the p and R
parameters are both strictly greater than zero. Although only cases
where the Shape is a positive integer ( p = 1, 2, 3, ... ) are allowed
in SeqDir, Rsum(N) is the pivotal quantity for all positive Shapes.
| ASIDE: Narrowing attention to integer valued Shapes is not |
| really very restrictive, but it greatly simplifies generation |
| of pseudo-random gamma variates. The gamma function reduces to |
| Gamma(p) = [p-1] factorial for positive, integer p. Chi-Squared |
| distributions are special cases of gamma distributions where the |
| degrees-of-freedom parameter is 2*p. Thus Chi-Squared distri- |
| butions with an odd number of degrees-of-freedom are not treated |
| in SeqDir. |
The expected value of gamma distributed regret is E( r ) = R*p, even
when p is not an integer. Similarly, the standard deviation of regret
is R*sqrt(p), and the mode density falls at the maximum of 0 and
R*(p-1).
The objective of all sequential test procedures treated here is to
distinguish between the following two hypotheses...
┌────────────────────────────────────────────────┐
│ NULL HYPOTHESIS: Mean Regret <= p * R0 │
├────────────────────────────────────────────────┤
│ ALTERNATIVE HYPOTHESIS: Mean Regret >= p * R1 │
└────────────────────────────────────────────────┘
where the gamma distribution has the SAME, known Shape (p) under BOTH
hypotheses, and R1 > R0 are known Scale parameter values.
*************************************************************
EXAMPLE: R0 = 1 and R1 = 5 are appropriate for the above example where
regret is assumed to have an exponential distribution (Shape
p=1) with expected value of 1% of Label Claim under the null
hypothesis and 5% at the regulatory specification limit.
*************************************************************
Continuing our argument that Rsum(N) is pivotal, the log-likelihood
RATIO statistic is of the form:
log(L1/L0) = - N * p * log(R1/R0) + Rsum(N) * [(1/R0)-(1/R1)].
where log => logarithm to the base e = 2.71828... In particular, note
that all terms in (p-1)*log[r(j)] cancel out when forming the RATIO
because the null and alternative gamma distributions both have the SAME
Shape parameter, p. Again, remember we are assuming that the null and
alternative gamma distributions differ only in their Scale parameters,
R0 and R1.
The Wald(1947) formula for approximate accept / reject boundaries for
Rsum(N) are then of the form...
┌──────────────────────────────────────────────────────────────────────┐
│ Reject Value: {log[(1-Beta)/Alpha]+N*p*log[R1/R0]} / [(1/R0)-(1/R1)] │
├──────────────────────────────────────────────────────────────────────┤
│ Accept Value: {log[Beta/(1-Alpha)]+N*p*log[R1/R0]} / [(1/R0)-(1/R1)] │
└──────────────────────────────────────────────────────────────────────┘
where Alpha is the objective for Producer's risk (probability of
rejecting a batch when quality is at or better than its "good" level,
R0), and Beta is the objective for Consumer's risk (probability of
accepting a batch when quality is at or worse than the "bad" level, R1.)
***************************************************************
EXAMPLE: R0=1 & R1=5 implies that [R1/R0] = 5 and [(1/R0)-(1/R1)] = 0.8.
The approximate Wald(1947) limits for p=1 & Alpha=Beta=0.05 are
then
Reject: ( log(19) + N*log(5) )/0.8 = 3.68 + N * 2.012,
and
Accept: (-log(19) + N*log(5) )/0.8 = -3.68 + N * 2.012.
Step: 1 2 3 4 5 6 7 8
Accept: -1.67 0.34 2.35 4.37 6.38 8.39 10.40 12.44
Reject: 5.69 7.70 9.72 11.73 13.74 15.75 17.76 19.77
***************************************************************
The two main troubles in practical application of Wald's approximate
method are:
...the Alpha and Beta levels actually achieved my not be very close
to their intended values, and
...accept and reject boundaries formed by a pair of parallel lines
can lead, in some situations, to large ASNs. "Truncated"
tests (in which the limits converge after a specified number of
sequential steps) frequently have much greater practical appeal.
SeqDir automates a 4-stage process that allows quality practitioners to
design sequential gamma test plans having KNOWN (exact) OC and ASN
curves that correspond to boundaries of ARBITRARY form. SeqDir
achieves this by implementing the Aroian(1968) "Direct Method" of
sequential analysis. From the definitions:
C(N) = Probability of Continuing Past Sequential Stage N,
= Probability that Rsum(N) is between the accept(N) and
reject(N) limits at sequential step N of the plan,
A(N) = Probability of Accepting the Null Hypothesis at Stage N,
= Probability that Rsum(N) is less than accept(N), and
R(N) = Probability of Rejecting the Null Hypothesis at Stage N,
= Probability that Rsum(N) is greater than reject(N),
it follows that
A(N) + C(N) + R(N) = C(N-1),
where, by convention, C(0) = 1 and A(0) = R(0) = 0.
It is straight-forward to compute C(1); this is simply a definite
integral over the Stage 1 continue region for a gamma distribution with
Shape p and any given Scale, R. The conditional distribution of Rsum(2)
[given that it is necessary to continue past stage 1] is then the
convolution of {i} the probability density of Rsum(1)=r(1) truncated to
the continue region of Stage 1 with {ii} the original gamma distribution
[of r(2)]. C(2) is then the definite integral of this convoluted
distribution over the Stage 2 continue region.
By induction, the conditional distribution of Rsum(N) [given that it is
necessary to continue past stage N-1] is the convolution of {i} the
probability density of Rsum(N-1) truncated to the continue region of
Stage N-1 with {ii} the original gamma distribution [of r(N)]. And C(N)
is the definite integral of that convoluted distribution over the Stage
N continue region. etc., etc.
Rather than use numerical methods that actually integrate and convolute
continuous distributions, SeqDir approximates all distributions (both
the original gamma or exponential and all singly/doubly truncated
"continue" distributions) by discrete distributions...with relatively
large numbers of relatively narrow cells. Definite integrals then
become sums of cell-probabilities, while convolutions become sums of
cell-probability products.
See McWilliams(1989) for a flow-chart of Aroian's method and an example
application to pass/fail (binomial distribution) testing.
SeqDir: Four Stages of "Design of Sequential Gamma Test Plans"
┌───────────────────────────────────────────────────────────────────────┐
│ STAGE ONE: Invoke SeqDir and Specify Accept/Continue/Reject Limits │
├───────────────────────────────────────────────────────────────────────┤
│ STAGE TWO: Calculate OC & ASN Values over a Range of Expected Regrets │
│ ...and possibly Confirm Calculations via Simulation │
├───────────────────────────────────────────────────────────────────────┤
│ STAGE THREE: Interactively View Plots of Boundary, OC & ASN Curves │
├───────────────────────────────────────────────────────────────────────┤
│ STAGE FOUR: Exit SeqDir to examine the Summary and/or Detailed OUTPUT │
│ Files...and Formulate Boundaries for the Next SeqDir Run. │
└───────────────────────────────────────────────────────────────────────┘
Section 3 of this documentation ("SeqDir Walk-Through") illustrates use
of SeqDir.EXE in Stages 1, 2, and 3 of the above process for Design of
Sequential Gamma Test Plans.
┌──────────────────────────────────────┐
│ Section 2 - PC System Requirements │
└──────────────────────────────────────┘
The hardware and software requirements for the personal computer system
that will use SeqDir are as follows:
...First of all, your machine should be IBM compatible with at least
300K (kilobytes) of available RAM memory. A hard disk is
optional...but greatly speeds up SeqDir execution, especially when
writing an optional (potentially very large) "details" file.
...Secondly, to use item "P" on the main menu, your monitor should be
capable of displaying medium resolution (IBM/CGA) screen graphics.
Most PC monitors used in business applications support graphics
at this resolution or better (EGA, VGA, etc.) Color is optional.
...Thirdly, a laser or dot-matrix printer can be used to produce
quick screen prints. Boundary, OC, and ASN curves displayed on
your screen can be converted into hard copy by pressing the
letter S followed by a second "special" key:
L or F10 ...Hewlett-Packard LaserJet/DeskJet
G or F9 ...IBM-compatible Graphics Dot Matrix Printer
E or F8 ...Epson FX, JX, or LQ Printers
To make efficient use of SeqDir, DOS must be informed of SeqDir's
requirements to read and write multiple files. This is accomplished by
having a CONFIG.SYS file (on your boot disk or in your C:\ root
directory) that contains lines (records) stating...
FILES=20
BUFFERS=8
*** SeqDir Input/Output FileName Conventions ***
The ASCII (text) files used by SeqDir for data input and output consist
of a filename (of at most eight characters) specified by the user
followed by a period (.) and one of three specific 3-character filename
EXTensions.
12345678.LIM ...SeqDir Batch Input of Limit Parameters
12345678.OUT ...SeqDir Summary Output File.
12345678.DTL ...SeqDir Detailed Output File.
┌─────────────────────────────────┐
│ Section 3 - SeqDir Walk-Through │
└─────────────────────────────────┘
Let us take a brisk walk through usage of SeqDir menus/prompts to
familiarize first time users. The SeqDir software system is invoked
from MS-DOS by entering its 8-character name, SEQDIR, at your DOS
prompt...
Prompt> seqdir
SeqDir does not expect or accept any "command-line" arguments. Instead,
SeqDir will prompt you to use your keyboard and/or a bounce-bar menu to
set parameters and provide all information SeqDir needs to perform
analyses. And each SeqDir prompt has an implied or [displayed] DEFAULT
VALUE that you may accept by simply pressing the ENTER key. [Displayed
default values are usually shown in square brackets.]
┌─────────────────── Direct Sequential Methods ──────────────────────┐
│ │
│ SEQDIR.EXE...Version 9109 │
│ A Quality Assurance Training Tool: │
│ Statistics Committee of the QA Section of the PMA │
│ Bob Obenchain, CompuServe User [72007,467] │
│ │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Will Accept/Reject LIMITS be Input via K = Keyboard ? ...or │ │
│ │ B = Batch File ? │ │
│ │ │ │
│ │ Press the K or B key now --> │ │
│ └──────────────────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
If you press the B key to select input from a "batch" file, the
following screen [with a pop-up, bounce-bar filename selection menu]
will appear...
┌──────────────────── Direct Sequential Methods ─────────────────────┐
│ │
│ Batch Input of A/R Limits Selected... │
│ │
│ At colon Prompts : ...simply press ENTER to get the [default]. │
│ │
│ ╔FileNames╗ Arrow Keys: Move Highlight Bar Up / Down. │
│ ║5STEP1 ║ │
│ ║5STEP5███║ Return Key: Selects the Highlighted File. >>>------+
│ ║9WALD1 ║ │ |
│ ║9WALD5 ║ Escape Key: Abandon BATCH SeqDir Input. │ |
│ ╚═════════╝ │ |
│ │ |
│ The Limits/Parms Input file is to be: 5STEP5.lim <-----------------+
│ │
│ Specify filename for Detailed Output [5STEP5.out] : │
│ │
│ The SEQDIR Output Save file is to be: 5STEP5.out <--Default Accepted
│ │
│ Press Q to QUIT now...Other Key to Continue... │
└────────────────────────────────────────────────────────────────────┘
If you initially press the K key to select KEYBOARD input, SeqDir will
prompt you for each parameter value it needs. And SeqDir will also save
your keystrokes in a file (that you may modify with any [ ASCII ] text
editor)...to rerun analyses in "batch" mode!
The Screens for entering parameter values from your Keyboard look like
this...
┌─────────────────── Direct Sequential Methods ──────────────────────┐
│ │
│ Keyboard Input of A/R Limits Selected... │
│ │
│ At colon Prompts : ...simply press ENTER to get the [default]. │
│ entered
│ Specify filename to save Keyboard Input [seqdir] : 5step5 <----------
│ The filename for saving Keyboard input is to be : 5step5.lim │
│ │
│ Specify filename for Summary Output [5step5.out] : <---Default │
│ The SeqDir Output Save file is to be: 5step5.out Accepted │
│ │
│ Save Detailed Output in filename = 5step5 ? [N|y] : Y <--This may │
│ The SeqDir Details Save file is to be: 5step5.dtl create a │
│ VERY LARGE│
│ Press Q to QUIT now...Other Key to Continue... File!!! │
└────────────────────────────────────────────────────────────────────┘
┌─────────────────────── SeqDir Parameters ──────────────────────────┐
│ │
│ What will be the Gamma SHAPE Parameter ? [ 1] : 5 <--Entered
│ SHAPE Parameter: 5. │
│ │
│ What is the Null (low) Exp. REGRET ? [ 1.00] : <--Default Accepted
│ NULL Exp.Value: 1.00. │
│ │
│ What is the Alt. (hi) Exp. REGRET ? [ 5.00] : <--Default Accepted
│ ALT. Exp.Value: 5.00. │
│ │
│ What is the Producers RISK ? [ 0.05] : <--Default Accepted
│ Producers RISK : 0.05. │
│ │
│ What is the Consumers RISK ? [ 0.05] : <--Default Accepted
│ Consumers RISK : 0.05. │
└────────────────────────────────────────────────────────────────────┘
Wald's approximate limits (lying on a pair of parallel lines) are
computed and displayed by SeqDir; these are the default SeqDir limits.
But, instead of accepting these default limits by simply pressing the
ENTER (or Carriage Return) key in response to each prompt, let us enter
custom boundaries...
┌──────────────────────── SeqDir Parameters ─────────────────────────┐
│ Respond to the following prompts for Limits... │
│ To quit data entry, enter the letter... Q │
│ │
│ Sequential Step Number: 1 │
│ Wald Lower Limit = 1.28 │
│ Enter <CR>, Lower Limit, NA or Q : na <------ Not Applicable
│ │
│ ┌────────────────────────────────────────────────────┐ │
│ │ NOTE: An NA Lower Bound is equivalent to a -1. │ │
│ │ An NA Upper Bound is equivalent to a +9,999. │ │
│ └────────────────────────────────────────────────────┘ │
│ │
│ Sequential Step Number: 1 │
│ Wald Upper Limit = 2.75 │
│ Enter <CR>, Upper Limit, NA or Q : 6 <--------------Entered
│ │
│ Sequential Step Number: 2 │
│ Wald Lower Limit = 3.29 │
│ Enter <CR>, Lower Limit, NA or Q : 0 <--------------Entered
│ │
│ Sequential Step Number: 2 │
│ Wald Upper Limit = 4.76 │
│ Enter <CR>, Upper Limit, NA or Q : 8 <--------------Entered
└────────────────────────────────────────────────────────────────────┘
etc.
┌──────────────────────── SeqDir Parameters ─────────────────────────┐
│ │
│ Sequential Step Number: 5 │
│ Wald Lower Limit = 9.32 │
│ Enter <CR>, Lower Limit, NA or Q : 12.99 <-----------Entered
│ │
│ Sequential Step Number: 5 │
│ Wald Upper Limit = 10.80 │
│ Enter <CR>, Upper Limit, NA or Q : 13 <--------------Entered
│ │
│ Sequential Step Number: 6 │
│ Wald Lower Limit = 11.33 │
│ Enter <CR>, Lower Limit, NA or Q : q <-----------Quit Signal
│ │
│ Total Number of Sequential Steps = 5 │
│ │
│ Data Entry Complete... │
│ Press Q to QUIT now...Other Key to Continue... │
└────────────────────────────────────────────────────────────────────┘
The corresponding "batch" input file created by SeqDir is formatted as
follows...
┌───────────────────────────────────────────────────────────────────────┐
│ Format for Batch Input: "FileName.LIM" │
├───────────────────────────────┬───────────────────────────────────────┤
│ assayreg ...Name of Variable │ Line 1: Name String and Comment │
│ 5 ...Shape Parameter │ Line 2: Gamma Dist. Peakedness. │
│ 1.00 ...Null Parameter │ Line 3: Exp. Regret of Null Dist. │
│ 5.00 ...Alt. Parameter │ Line 4: Exp. Regret of Alternative │
│ 0.05 ...Producers Risk │ Line 5: Wald Approx. Alpha Level │
│ 0.05 ...Consumers Risk │ Line 6: Wald Approx. Beta Level │
│ -1.00 6.00 │ Line 7: Step 1 Lower & Upper Limits │
│ 0.00 8.00 │ Line 8: Step 2 Lower & Upper Limits │
│ 4.00 10.00 │ - - - │
│ 8.00 12.00 │ - - - │
│ 12.99 13.00 │ Next to Last Line: Last Limit Pair │
│ q │ Last Line: Quit (End-of-Data) Signal │
└───────────────────────────────┴───────────────────────────────────────┘
NOTES: Limit pairs for Accept / Continue / Reject Boundaries start on
line 6; the Upper Limit (second value) must equal or exceed the
Lower Limit (first value.)
The last Lower / Upper Limit pair may, optionally, "Close Off"
the Continue Region...as illustrated above. If the Continue
Region is left "Open" at the last step specified to SeqDir, then
the value of the Operating Characteristic (Accept Probability)
plus the Producer's Risk (Rejection Probability) my sum to much
less than one for some (or all) values of True Expected Regret.
The SeqDir MAIN MENU now appears. You may proceed directly to item "D"
if you are willing to accept SeqDir's default values for Number of Cells
and Range of Integration (with implied Cell Width)...
┌────────────────────────────── SeqDir Main Menu ────────────────────┐
│ ┌──────────────────────────────┐ │
│ │ Direct Sequential MENU │ │
│ │ M = set Maximum cells │ │
│ │ D = Direct Analyses │ │
│ │ S = Simulations │ │
│ │ P = Plots: Lims/OC/ASN │ │
│ │ Q = Quit SeqDir │ │
│ │ Choice --> M │ │
│ └──────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
However, if you press the M key (as illustrated above), the following
screen will appear...
┌─────────────────────── set Maximum cells ──────────────────────────┐
│ │
│ SeqDir uses probabilities within cell intervals to approximate │
│ continuous Gamma distributions (and their convolutions) by │
│ discrete distributions. This approximation is fast but │
│ inaccurate when only a few hundred cell intervals are used. │
│ You will need to use one or two thousand cells to explore │
│ numerical sensitivities and confirm simulation results... │
│ │
│ The Maximum number of cells that can be defined is 2,500. │
│ │
│ The Maximum Rejection Upper Limit is 13.00. │
│ │
│ The Total Regret Range presently covered is [ 0.00, 25.00]. │
│ Specify Maximum Regret Limit [ 25.00] : <-----------Default Accepted
│ Maximum Regret : 25.00. │
│ │
│ Specify Total Number of Cells [ 200] : 250 <--------------Entered
│ Number of Cells : 250. │
│ │
│ The implied Regret Cell WIDTH is : 0.1000 │
│ │
│ Press any Key to Return to the Main Menu... │
└────────────────────────────────────────────────────────────────────┘
Once you return to the MAIN MENU, you should press the "D" key to
perform SeqDir Direct Calculations. Each time you change cell widths
or integration range, you should re-do calculations by pressing "D."
┌──────────────────────── SeqDir Analyses ───────────────────────────┐
│ Direct Sequential Calculations... │
│ NULL Exp.Value = 1.00 │
│ ALT. Exp.Value = 5.00 │
│ │
│ At most 50 Expected Regret Values can be evaluated. │
│ How many Expected Regret Values ? [5] : <----------Default Accepted
│ Number of Expected Regret Values = 5. │
│ │
│ Calculations for Expected Regret = 1.00. │
│ ================ Accept LowLim Contin UprLim Reject │
│ At Seq. Step 1: 0.00000 -1.00 1.00000 6.00 0.00000 │
│ At Seq. Step 2: 0.00000 0.00 1.00000 8.00 0.00000 │
│ At Seq. Step 3: 0.92822 4.00 0.07178 10.00 0.00000 │
│ At Seq. Step 4: 0.99876 8.00 0.00124 12.00 0.00000 │
│ At Seq. Step 5: 0.99998 12.99 0.00000 13.00 0.00002 │
│ Performance Statistics... │
│ Operating Characteristic = 1.00000 │
│ Producers Rejection Risk = 0.00000 │
│ Average Sample Number = 3.07 │
│ ASN Standard Error = 0.26 │
│ │
└────────────────────────────────────────────────────────────────────┘
NOTE: In SeqDir screen displays, the Accept, Continue, and Reject
probabilities are normalized so that
A(N) + C(N) + R(N) = 1 .
Internal calculations in SeqDir use this normalization to preserve
numerical precision, i.e. to prevent ALL of these probabilities from
being small. However, before results are written to the SeqDir output
file, the correct normalization is imposed:
A(N) + C(N) + R(N) = C(N-1).
For example, the analysis from the above screen for Exp.Regret=1 is
written to the output file as:
Step === Accept LowLim Contin UprLim Reject
Step 1: 0.00000 -1.00 1.00000 6.00 0.00000
Step 2: 0.00000 0.00 1.00000 8.00 0.00000
Step 3: 0.92822 4.00 0.07178 10.00 0.00000
Step 4: 0.07169 8.00 0.00009 12.00 0.00000
Step 5: 0.00009 12.99 0.00000 13.00 0.00000
1.00000 ...Operating Characteristic
0.00000 ...Producers Rejection Risk
3.07 ...Average Sample Number
0.26 ...ASN Standard Error
Similar calculations for Exp.Regret= 2, 3, 4, and 5 will be displayed as
the screen scrolls upward within the window...
┌───────────────────────── Direct Analyses ──────────────────────────┐
│ │
│ Calculations for Expected Regret = 5.00. │
│ ================ Accept LowLim Contin UprLim Reject │
│ At Seq. Step 1: 0.00000 -1.00 0.71494 6.00 0.28506 │
│ At Seq. Step 2: 0.00000 0.00 0.40090 8.00 0.59910 │
│ At Seq. Step 3: 0.00012 4.00 0.29734 10.00 0.70254 │
│ At Seq. Step 4: 0.00419 8.00 0.25264 12.00 0.74318 │
│ At Seq. Step 5: 0.09905 12.99 0.00000 13.00 0.90095 │
│ Performance Statistics... │
│ Operating Characteristic = 0.00252 │
│ Producers Rejection Risk = 0.99748 │
│ Average Sample Number = 2.11 │
│ ASN Standard Error = 0.96 │
│ │
│ Direct Sequential Calculations Complete... │
│ │
│ Press a KEY to return to the SeqDir Main MENU. │
└────────────────────────────────────────────────────────────────────┘
Once you return to the MAIN MENU, you could press the "P" key to view
medium-resolution (CGA) graphical display of the MOST RECENT results you
calculated using item "D" (or items "M" and "P"). Item "P" will NOT
OPERATE until you have selected "D" at least once.
However, let us choose option "S" next...
┌────────────────────────────── SeqDir Main Menu ────────────────────┐
│ ┌──────────────────────────────┐ │
│ │ Direct Sequential MENU │ │
│ │ M = set Maximum cells │ │
│ │ D = Direct Analyses │ │
│ │ S = Simulations │ │
│ │ P = Plots: Lims/OC/ASN │ │
│ │ Q = Quit SeqDir │ │
│ │ Choice --> S │ │
│ └──────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
SeqDir can confirm the results it calculated using Aroian's direct
method using Monte-Carlo simulation. Furthermore, the graphical
simulation display provided by SeqDir can give us a very good "visual
impression" of what Sample Paths can be expected to "look like..."
┌──────────────────────────── Simulation ────────────────────────────┐
│ │
│ Sequential Simulations... │
│ NULL Exp.Value = 1.00 │
│ ALT. Exp.Value = 5.00 │
│ │
│ Number of Expected Regret Values = 5 │
│ │
│ Maximum Number of Simulation Replications = 25,000. │
│ Desired Number of Replications ? [500] : <----------Accept Default
│ Number of Replications = 500. │
│ │
│ Input Start-up Seed ( 0 means use your Clock ): 123456 <------Enter
│ Startup Seed = -7616 │
│ │
│ Press a key to Start Simulations... │
└────────────────────────────────────────────────────────────────────┘
If you enter a non-zero seed value, as illustrated above, this exact
same "starting point" in SeqDir's pseudo-random sequence will be used
for each distinct value of Expected Regret that is simulated. The
(linear) correlation between OC and ASN estimates at different values of
Expected Regret will then be plus one; as highly correlated as possible!
On the other hand, zero is the default seed value (i.e. what you would
get by simply pressing the ENTER key in response to the above prompt)
and using that seed value has two implications.
First of all, SeqDir will get a seed value from your system clock in
this case (and write that value to the output file for simulation
documentation.)
Furthermore, SeqDir will NOT now reset its pseudo-random sequence to
the same starting point for each different Expected Regret value it
simulates. In other words, the OC and ASN estimates for different
Expected Regrets will then be statistically independent.
Usage of COLOR on SeqDir Simulation Displays:
Cyan Box => An observed Rsum(N) value in the Continue Region.
Cyan Line => Connects pairs of Cyan Boxes in the Continue Region.
Magenta Box => An observed Rsum(N) value in the Reject Region.
Magenta Line => Connects Cyan Continue Box to a Magenta Reject Box.
White Box => An observed Rsum(N) value in the Accept Region.
White Line => Connects Cyan Continue Box to a White Accept Box.
************************************************************************
WARNING: Do not press ANY KEY while Monte-Carlo Simulation is underway
unless you desire EARLY TERMINATION for the Expected Regret value
currently being simulated. If you do press key(s), SeqDir will clear
out the keyboard buffer and go on to the next Expected Regret value as
soon as at least 10 replications have been completed.
************************************************************************
Once Direct Calculations and Simulations have both been performed, it is
definitely time to "plot" results using item "P" on the Main Menu...
┌────────────────────────────── SeqDir Main Menu ────────────────────┐
│ ┌──────────────────────────────┐ │
│ │ Direct Sequential MENU │ │
│ │ M = set Maximum cells │ │
│ │ D = Direct Analyses │ │
│ │ S = Simulations │ │
│ │ P = Plots: Lims/OC/ASN │ │
│ │ Q = Quit SeqDir │ │
│ │ Choice --> P │ │
│ └──────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
Note that item "P" is separate from items "D" and "S" on the Main Menu
because you may wish to cycle through the graphical displays (the
Accept/Continue/Reject limits and the OC and ASN Curves) several times
for a single set of Calculated or Simulated Results. Simply select item
"P" several times. If no simulations have been performed, SeqDir plots
will start appearing as soon as you choose item P in the Main Menu.
If both "D"irect Calculations and "S"imulations have been performed, the
following sub-menu will appear...
┌────────────────────────────── SeqDir Plotting ─────────────────────┐
│ │
│ Critical Region, Operating Characteristic, │
│ and Average Sample Number Plots... │
│ │
│ From the NULL Exp.Value = 1.00 on the Left, │
│ To ALTERNATE Exp.Value = 5.00 on the Right. │
│ 5 = No. of Expected Regret Values per Plot. │
│ ┌──────────────────────────────┐ │
│ │Which Type of Plot ? │ │
│ │ │ │
│ │ R = Crit. Region │ │
│ │ D = Direct Calc. │ │
│ │ S = Simulation │ │
│ │ Q = Quit │ │
│ │ │ │
│ │ Choice --> │ │
│ └──────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
At this point, choose the type of plot you wish to view (R, D, or S) or
else return to the Main Menu by pressing Q.
Once a graphics screen is displayed by SeqDir, you may interactively
annotate it. For example, you may move an on-screen cursor to read
numerical values off of the plot. Simply, press the Space Bar or
Function Key F1 for on-line HELP on interacting with graphics screens!
┌─────────────────────────────────────────────┐
│ This HELP screen .... Space Bar or F1. │
│ │
│ ERASE PLOT......press ENTER or ESCape. │
│ Note: 2nd KeyPress sometimes needed. │
│ │
│ On-Screen Cursor & VALUES ... press V. │
│ Move Cursor Right with Right Arrow. │
│ Move Cursor Left with Left Arrow. │
│ │
│ Screen PRINT/SAVE Keys: First Press S │
│ and then press a Second Special Key... │
│ L or F10 dumps to HP LaserJet/DeskJet │
│ I or F9 dumps to IBM Graphics Matrix │
│ E or F8 dumps to Epson FX, JX, or LQ │
│ S saves screen to disk in PCX format. │
│ │
│ SeqDir, ver9109.........Press any key. │
└─────────────────────────────────────────────┘
Note in particular that OPTIONAL screen dumps can be obtained by
pressing first "S" then a second "special" key while the Limits, OC, or
ASN screen is being displayed. If you have annotated the screen, that
annotation will be part of the screen dump.
┌──────────────────────────────────────────────────┐
│ HP LaserJet, Plus & Series II │
│ (press either the L key or function key F10.) │
├──────────────────────────────────────────────────┤
│ IBM-compatible Graphics Dot Matrix Printer │
│ (press either the G key or function key F9) │
├──────────────────────────────────────────────────┤
│ Epson FX, JX, or LQ Printers │
│ (press either the E key or function key F8) │
└──────────────────────────────────────────────────┘
If you press any key other than one of the six "special" keys listed
above after pressing "S", the Boundary, OC or ASN screen will be erased.
After viewing the ASN curve, you will be returned to the Main Menu.
Press the "Q" key to terminate SeqDir execution and return to DOS...
┌────────────────────────────── SeqDir Main Menu ────────────────────┐
│ ┌──────────────────────────────┐ │
│ │ Direct Sequential MENU │ │
│ │ M = set Maximum cells │ │
│ │ D = Direct Analyses │ │
│ │ S = Simulations │ │
│ │ P = Plots: Lims/OC/ASN │ │
│ │ Q = Quit SeqDir │ │
│ │ Choice --> Q │ │
│ └──────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
SeqDir will write "reminders" on your screen as it terminates...
REMINDER(S) :
SeqDir created a Batch History Input file... 5step5.lim
You can invoke SeqDir and specify 5step5 [.lim]
as Batch History Input file to replay the present session.
SeqDir created an Output File named... 5step5.out
Use the DOS invocation... LIST 5step5.out
to review summary statistics from this run.
SeqDir created a Details File named... 5step5.dtl
Use the DOS invocation... LIST 5step5.dtl
to review detailed computational results.
┌────────────────────────────────────────────────────┐
│ Section 4 - Effects of Changing SeqDir Parameters │
└────────────────────────────────────────────────────┘
This section of the SeqDir documentation discusses the effects of
changing the gamma Shape parameter and the (numerical integration) cell
width.
*** gamma Shape parameter ***
You must specify a numerical value of the gamma distribution Shape
parameter, p, in response to the following SeqDir prompt...
What will be the Gamma SHAPE Parameter ? [ 1] :
Shape p=1 (an Exponential distribution) is the SeqDir default because
calculations are very simple in this case. For example, the right-hand
tail probability of regret exceeding a value X when the Scale parameter
is R would then be Pr[ r > X ] = exp( - X / R ), i.e. e = 2.71828...
raised to the power - X / R.
Consider what would happen if the gamma distribution Shape parameter, p,
were to increase as the gamma Scale parameter, R, decreases...leaving
the expected regret value, E=p*R, constant. The standard deviation of
regret, R*sqrt(p)=E/sprt(p), would then decrease and the mode regret,
max[ 0, E*(p-1)/p], will increase from 0 [for p<=1] toward its limiting
value of E.
The slope of the Wald(1947) approximate accept / reject boundaries for
Rsum(N) would be unchanged as p increases with E=p*R fixed, but the
spacing between these boundaries would decrease (assuming that Alpha and
Beta remain unchanged.) In other words, it becomes easier and easier to
distinguish between Scaling differences, R0 and R1, as the Shape
parameter increases (i.e. small regrets become relatively less likely in
the R1 scaling, and large regrets become relatively less likely at R0.)
Using those same sorts of arguments, if accept / reject boundaries are
held fixed as the Shape increases, then the Operating Characteristic
curve will increase near R0 (reduced producers risk) and decrease near
R1 (reduced consumers risk.) This point is illustrated in the following
tabulation for the 5STEP1.LIM and 5STEP5.LIM boundary examples that use
2,500 cells over regret range [0,25]...
┌─────────────────────────────────────────────────────┐
│ ExpR= 1 ExpR= 2 ExpR= 3 ExpR= 4 ExpR= 5 │
│ (NULL) (ALT.) │
┌────────────────┼─────────────────────────────────────────────────────┤
│ Shape = 1 (Exponential) │
│ Op. Char. │ 0.99227 0.75959 0.44273 0.25156 0.15122 │
│ ASN │ 3.26 3.58 3.26 2.85 2.52 │
│ ASN Std. Error │ 0.52 1.04 1.30 1.37 1.34 │
├────────────────┼─────────────────────────────────────────────────────┤
│ Shape = 5 │ │
│ Op. Char. │ 1.00000 0.92428 0.26564 0.02677 0.00210 │
│ ASN │ 3.10 4.33 3.96 2.79 2.10 │
│ ASN Std. Error │ 0.30 0.67 1.17 1.19 0.95 │
└────────────────┴─────────────────────────────────────────────────────┘
*** Range and Number of Cells ***
The following tabulation shows (again, for the 5STEP5.LIM boundary
example) that OC and ASN calculations are somewhat MORE sensitive to the
"cell width" used to represent definite integrals with finite sums of
cell probabilities than they are to the number of replications used in
Monte-Carlo simulation!
┌────────────────────┬┬────────────────────┐
│ Direct Method ││ Monte-Carlo │
│ Calculations ││ Simulations │
├────────────────────┼┼────────────────────┤
Range: 0.0 to 25.00 40.00 ││ 0 Startup
No. Intervals 250 2000 ││ -25012 -7616 Seed
Interval Width 0.10 0.02 ││ 500 10000 Replics
┌──────────────────┼────────────────────┼┼────────────────────┤
│ Expected Regret │ 1.00 1.00 ││ 1.00 1.00 │
│ Operating Char. │ 1.00000 1.00000 ││ 0.99800 0.99990 │
│ Producers Risk │ 0.00000 0.00000 ││ 0.00000 0.00000 │
│ Aver.Samp.No. │ 3.07 3.10 ││ 3.11 3.10 │
│ ASN Std. Error │ 0.26 0.30 ││ 0.31 0.31 │
├──────────────────┼────────────────────┼┼────────────────────┤
│ Expected Regret │ 2.00 2.00 ││ 2.00 2.00 │
│ Operating Char. │ 0.93415 0.92544 ││ 0.90619 0.92341 │
│ Producers Risk │ 0.06585 0.07456 ││ 0.09182 0.07609 │
│ Aver.Samp.No. │ 4.26 4.32 ││ 4.31 4.34 │
│ ASN Std. Error │ 0.68 0.67 ││ 0.69 0.67 │
├──────────────────┼────────────────────┼┼────────────────────┤
│ Expected Regret │ 3.00 3.00 ││ 3.00 3.00 │
│ Operating Char. │ 0.28768 0.26805 ││ 0.25549 0.26307 │
│ Producers Risk │ 0.71232 0.73195 ││ 0.74251 0.73603 │
│ Aver.Samp.No. │ 3.98 3.96 ││ 3.96 3.95 │
│ ASN Std. Error │ 1.16 1.17 ││ 1.20 1.18 │
├──────────────────┼────────────────────┼┼────────────────────┤
│ Expected Regret │ 4.00 4.00 ││ 4.00 4.00 │
│ Operating Char. │ 0.03081 0.02719 ││ 0.02794 0.02770 │
│ Producers Risk │ 0.96919 0.97281 ││ 0.97006 0.97210 │
│ Aver.Samp.No. │ 2.81 2.79 ││ 2.81 2.78 │
│ ASN Std. Error │ 1.21 1.19 ││ 1.23 1.20 │
├──────────────────┼────────────────────┼┼────────────────────┤
│ Expected Regret │ 5.00 5.00 ││ 5.00 5.00 │
│ Operating Char. │ 0.00252 0.00214 ││ 0.00000 0.00280 │
│ Producers Risk │ 0.99748 0.99786 ││ 0.99800 0.99710 │
│ Aver.Samp.No. │ 2.11 2.10 ││ 2.10 2.09 │
│ ASN Std. Error │ 0.96 0.95 ││ 0.92 0.95 │
└──────────────────┴────────────────────┴┴────────────────────┘
A somewhat surprising observation from the above table is the closeness
of agreement between the simulation results with 500 and 10,000
replications, respectively. And direct calculations with 2,000
intervals over the [0,40] range also agree closely with the simulation
results. In fact, direct calculations with only 250 cells over [0,25]
appear to be relatively less accurate than simulations with only 500
replications!
Of course, SeqDir direct calculations can be performed much more
quickly with fewer cells (200 to 500) than with many (say, 2000 to
2500.) But relatively "wide" cell widths (say, 0.1 to 0.25 regret
units) are recommended only for preliminary, exploratory SeqDir
runs.
As your sequential test design process approaches its final
stages, you will need to make SeqDir runs with (much) smaller cell
widths to investigate numerical sensitivity in your computed OC
and ASN curves.
McWilliams(1991) has done some independent numerical calculations
and simulations that seem to agree well not only with the SeqDir
direct calculations (at least, those using a couple thousand cells)
but also with SeqDir simulations.
REFERENCES
AROIAN, L. A. (1968). "Sequential Analysis, Direct Method."
Technometrics 10, 125-132.
AROIAN, L. A. (1976). "Applications of the Direct Method in Sequential
Analysis." Technometrics 18, 301-306.
McWILLIAMS, T. P. (1989). How To Use Sequential Statistical Methods.
Volume 13 of "The ASQC Basic References in Quality Control:
Statistical Techniques." American Society for Quality Control,
Milwaukee, WI 53203.
McWILLIAMS, T. P. (1991). Computer programs DRCONT and GAMSIM.
Personal Communication.
PRESS, et.al. (1988). "Numerical Recipes in C: The Art of Scientific
Computing." Cambridge University Press. C-source code Copr.
(C) 1985, 1987 by Numerical Recipes Software, P.O. Box 243,
Cambridge, MA 02238. (Gamma & pseudo-random number functions.)
WALD, A. (1947). Sequential Analysis. John Wiley & Sons, New York.
┌─────────────────────────────────────────────────────────────────────┐
│ SeqDir was compiled using Microsoft C, ver6.00AX, using the default │
│ mode that will automatically detect and utilize a 8087 numeric math │
│ coprocessor if your PC has this type of chip. │
└─────────────────────────────────────────────────────────────────────┘
SeqDir Software Update History:
=================================
Version 9107 ...Beta Test Version, Exponential Distributions Only
Version 9108 ...Handle Gamma & Exponential Distributions
Version 9109 ...Add graphical Simulation Module
