NetNews Usenet Archive 1992 #16/NN_1992_16.iso/spool/comp/sys/sgi/11480

Internet Message Format | 1992-07-27 | 3.7 KB

Path: sparky!uunet!dtix!darwin.sura.net!mips!mash
From: [e-mail] (John Mashey)
Newsgroups: comp.sys.sgi
Subject: Re: Specmarks for R4000's
Message-ID: <[e-mail]>
Date: 27 Jul 92 21:29:32 GMT
References: <[e-mail]> <[e-mail]>
Organization: MIPS Computer Systems, Inc.
Lines: 63
NNTP-Posting-Host: winchester.mips.com

In article <[e-mail]> [e-mail] (Philip Machanick) writes:

>SPECmark used to be a single figure (now called Spec89) that was meant to more
>accuractely characterize the performance of a complete system than MIPS. As
>with MIPS, the figure is relative to a VAX 11/780 (= 1 SPECmark).
>
>It is based on a range of benchmarks meant to be characteristic of a wide range
>of real workloads, and is based on runs of real programs (not synthetic or toy
>benchmarks). Recently, some manufacturers found a "trick" optimization that
>sped up one of the benchmarks so much that the figure became meaningless, so
>that specific program was deleted and the numbers were split into integer and
>floating point (SPECint92 and SPECfp92). Take care not to compare Spec89 and
>Spec92 numbers - some manufacturers sitll quote SPECmark meaning Spec89.

Actually, when we did SPECmarks inthe first place, we *hated* having
one number, and we *always* said you must publish all 10 numbers ...
but we did consolidate it into 1 number, even against our better thoughts,
because we knew, if we didn't, somebody else would, and that magazines
will seldom print so many numbers.

Over time, we started at least splitting the numbers into integer
and floating point (SPECint89 and SPECfp89), especially as the original
SPECmark (with 4 integer and 6 FP benchmarks) really over-emphasizes
floating point.  For SPEC92, with 6 integer and 14 floats, it would
have been even worse.

Note that there is starting to accumulate data that says, for a lot
of real fp-intensive applications in workstation environments,
the overall performance is probably better predicted by some combination
of SPECint92 and SPECfp92, than by SPECfp92 alone ...  because you
often spend a lot of time in the OS and data-pushing code that is
usually or always 100% integer.  Note that most of the SPEC CPU benchmarks,
on purpose, do very little or no I/O, and spend almost no time inside
the operating system.

Consider the following data:

>System           clock   cache   SPECint92   SPECfp92
>SGI R4000 Indigo 50/100  1M+16k     57         61
>SGI Crimson      50/100  1M+16k     61.7       63.4
>HP 750             66    0+512k     48.1       75.0
>IBM 560            50    0+72k      42.0       85.6

From this, you'd expect that both 750s and 560s would be
consistently faster than Crimsons on applications like
finite-element-analysis and computational chemistry ... and yet,
SGI has a fair amount of data that shows Crimsons often beating,
or at least equaling these. [As always, there is a lot of variability;
nobody beats the others on all benchmarks.]  This seems counter-intuitive,
but I conjecture:
    1) The real application environment has more integer work going
    on than do the specific FP benchmarks that make up SPECfp92.
    2) There is probably more context-switching going on, and that
    isn't measured at all by the SPEC CPU benchmarks.
    3) The SPEC benchmarks are mostly "small enough" that
    going from 256K -> 512K -> 1MB secondary cache doesn't improve
    the numbers very much; perhaps it does help the real applications
    more.
--
-john mashey    DISCLAIMER: <generic disclaimer, I speak for me only, etc>
UUCP:      [e-mail] [soon to be [e-mail], but not quite moved yet].
DDD:      408-524-7015,  or 524-8253
USPS:    (soon) Silicon Graphics, 2011 N. Shoreline Blvd, Mountain View, CA 94043