NetNews Usenet Archive 1992 #16/NN_1992_16.iso/spool/bit/listserv/csgl/616

Text File | 1992-07-23 | 6.5 KB | 135 lines

Comments: Gated by [e-mail]
Path: sparky!uunet!paladin.american.edu!auvm!DRETOR.DCIEM.DND.CA!RANDALL
Message-ID: <[e-mail]>
Newsgroups: bit.listserv.csg-l
Date:         Thu, 23 Jul 1992 23:06:18 EDT
Sender:       "Control Systems Group Network (CSGnet)" <[e-mail]>
From:         "Hugh D. Gamble" <[e-mail]>
Subject:      What is information?
X-To:         [e-mail]
Lines: 123

[Allan Randall (920723.2100)]

From Bill's ISSS talk on information:
<                 Information: a matter of perception
<                          William T. Powers
<...
<Shannon and Weaver examined a physical signal flowing from
<source to destination...information decreases the entropy of the
<receiver...Unfortunately, this is a house of cards and one of
<the bottom cards is imaginary.

Ouch! Well I wasn't going to post anything else here until I found
the time to respond to the implicit/explicit hierarchy thread, but
this I can't let pass. The entropic formulation of information
theory, along with the related algorithmic formulation, has a pretty
firm mathematical basis, and has shown itself to be quite useful, so
you will be going some to convince me it is a "house of cards."

<Information does not necessarily travel in the same direction as
<physical energy.

I think you are confusing the concept of energy and that of entropy.
They are related, but not the same. It is the latter, not the former,
that information theorists associate with information content. There
is no need in traditional information theory for the kind of "net
flow" of energy in the direction from source to destination that you
talk about in your examples, both of which I think are pretty easy to
refute.

<If you're reading these words in black print on a piece of white
<paper, you can see immediately what I mean. The energy that flows
<into your eye...comes from the reflection of light off the white
<paper. This energy flows from the page to your retina everywhere
<except where the letters are printed. If the letters are dark
<enough, a net energy flow may actually travel from diffusely
<illuminated points on your retina, through the lens, and to the
<light-absorptive ink on the paper.

Okay. Correct me if I'm wrong. Are you suggesting there is a net
flow from my retina to the words on paper, rather than vice-versa?
The "words" on the page are more than just the ink, they include
the white background. The energy from the white background is
what transmits the information to my brain. Just because it is
the ground rather than the figure that I receive as energy is
irrelevant. The energy from the white page is structured in such
a way as to correspond to the ink markings (after all, that's why
I'm able to "reconstruct" the black ink markings from the light I
receive from the white background). This is all perfectly in accord
with traditional information theory. There is no flow of energy in
the "wrong" direction.

<Or consider an old-fashioned telegraph...When the key is open, no
<information is traveling. When the key closes in long and
<short patterns, a message in Morse Code is sent to Chicago. The
<dots and dashes represent momentary drains of energy on the
<battery in the central telegraph office in Chicago; the average
<direction of energy flow is from the Chicago office to the short
<circuit in Dodge City, heating the wires along the way.

Although I disagree with this as well, it is a better example
than the first one, since here there really does seem at first
glance to be a net flow of energy in the "wrong" direction. The
mistake you are making here is using the energy output of the
battery as the transmitting energy flow. This is incorrect. You
are treating the battery as the information transmitter. But the
battery is NOT the originator of the message. It matters not a
wit whether we consider the battery to be part of the sender or
the receiver. The battery is thus more justifiably considered as
part of the medium of transmission. The message actually comes
from the human being who is putting out the dots and dashes. This
*is* a flow of energy from the human, and *does* decrease the
entropy of the receiver and increase the entropy of the source
(and the universe).

Compare what happens to the case of a transmitter that outputs
dots and dashes due to chaotic or random forces in the world
around it. These messages are less ordered, and thus higher
entropy, than the messages put out by the human. The destination
would end up with higher entropy and the source would have lower
entropy than in the case of the human operator.

<...the direction of energy flow is unrelated to the direction of
<flow of information, so that concepts like entropy (positive or
<negative) have no actual physical relationship to whatever it is
<that we mean by information.

Perhaps not what *you* mean by information, but the entropy concept
is very rigorously defined mathematically, and *is* what most
information theorists, computer scientists and physicists mean when
we speak of information. Shannon himself was very clear in his
original article that he was not giving a "semantic" definition of
information, or "meaning". He quite frankly admitted that this was
not covered by the theory he laid out. It is not entirely clear
to what degree a semantic or a perceptual theory of information
would be built on the concepts of traditional information theory
a la Shannon/Weaver/Chaitin, which does not pretend to speak to such
issues.

<When we transmit information, we hope we are transmitting more
<than words; we hope we are transmitting meanings.

I have no problem with your tying in perception/control with this
notion of information, and if this is what you want to call
"information," I'm not going to argue over definitions. But what
Shannon and Weaver showed was that there is a key aspect of
communication, which is now usually called information, that is
independant of this "meaning" or semantic content and has nothing
to do with perception. I think they succeeded. I don't think you
have, as yet, given me any reason to doubt this. Your rubber
band experiment merely shows one example of a case where control
is necessary for information to be transmitted. It says nothing
about whether such control is necessary for information
transmission in general. I don't think it is. Give me reason to
believe otherwise.

Allan Randall
NTT Systems, Inc.
Toronto, ON

PS: I'm actually a lot more favourable to PCT than I appear in
my posts. I find it to be elegant with a lot of explanatory power.
Its just that I'm, as Bill would say, controlling for the higher
error signals.