NetNews Usenet Archive 1993 #1/NN_1993_1.iso/spool/comp/lang/pascal/7940

Internet Message Format | 1993-01-07 | 1.8 KB

Path: sparky!uunet!spool.mu.edu!caen!zaphod.mps.ohio-state.edu!swrinde!cs.utexas.edu!ut-emx!ccwf.cc.utexas.edu!mmigdol
From: [e-mail] (michael a migdol)
Newsgroups: comp.lang.pascal
Subject: TP 6.0 Strange rounding problem (Debugger is *lying* to me!)
Message-ID: <[e-mail]>
Date: 7 Jan 93 14:32:19 GMT
Sender: [e-mail]
Reply-To: [e-mail] (michael a migdol)
Organization: The University of Texas at Austin, Austin TX
Lines: 43
Originator: [e-mail]

Here's a fragment of a program that was causing me some major grief last night.

PROGRAM FOO;

VAR
    HighFreq, ToneFreq : Integer;

Procedure Foo1(...VAR factor : Real;...)
VAR
    ...
    diff : Real;
    ...

BEGIN
    diff := HighFreq - ToneFreq;
    ...
    diff := diff / Factor;
    HighFreq := Round(ToneFreq + diff);
    ...
END; { Procedure }

    Ok, boys and girls, assuming that upon entry to Foo1, variables have the
following values, what should HighFreq be after the last statement is executed?
    HighFreq = 525
    ToneFreq = 500
    factor = 2.0

    Well, the first statement sets diff to 25.0. The second sets it to 12.5.
ToneFreq + diff is then 512.5, and round of this is...
    512???

    The plot thickens even more. If I trace variables up to the last line, and
o a watch on Round(ToneFreq + diff), THAT returns 513! But execution of
the last statement puts 512 in HighFreq! It's this discrepency that has kept
me from dismissing this as a simple rounding error somewhere. (12.5 really
equals 12.4999999999999 or something like that).

    I've got a temporary fix for the problem by replacing Round(ToneFreq + diff)
with Trunc(ToneFreq + diff + 0.5), but I'd still like to know exactly what's
going on. Does anyone have any ideas?

Thanks in advance...
     Michael