home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Monster Media 1993 #2
/
Image.iso
/
text
/
9305nni.zip
/
930513.DIS
< prev
next >
Wrap
Text File
|
1993-05-13
|
3KB
|
57 lines
Article 9010 of comp.ai.neural-nets:
Newsgroups: comp.ai.neural-nets
Path: serval!netnews.nwnet.net!usenet.coe.montana.edu!saimiri.primate.wisc.edu!sdd.hp.com!cs.utexas.edu!zaphod.mps.ohio-state.edu!howland.reston.ans.net!europa.eng.gtefsd.com!fs7.ece.cmu.edu!crabapple.srv.cs.cmu.edu!news
From: sef@sef-pmax.slisp.cs.cmu.edu
Subject: Re: Question: What constitutes "hard to learn data"
Message-ID: <C6Mz7t.DFq.1@cs.cmu.edu>
Sender: news@cs.cmu.edu (Usenet News System)
Nntp-Posting-Host: sef-pmax.slisp.cs.cmu.edu
Organization: School of Computer Science, Carnegie Mellon
Date: Fri, 7 May 1993 03:04:40 GMT
Lines: 42
From: arms@cs.UAlberta.CA (Bill Armstrong)
>Other things being equal, noisy data is harder to learn than clean data.
Suppose this is true (I have a few reservations described below), then
one corollary of this seems to be that if the signal to noise ratio
varies from low to high in different parts of the input space (e.g.
the amplitude of a sonar signal may be of high or low amplitude while
the noise is essentially constant), then you need different amounts of
training time in each part. The danger is that if you train long
enough for the low signal to noise parts to be learned properly, the
network, on the rest of the space, will be overtrained.
Well, I don't think I've run any problems that are noisy in some areas and
not in others, and I haven't thought much about this. But my guess is that
if you have enough data to separate the real signal from the noise in the
noisy part of the space, and train long enough to get this right, there
should be no problem in the clean parts. Overtraining allows the system to
model the noise as well as the signal, but in these areas the signal will
be relatively stronger and will tend to dominate.
Example: take the function x * sin(x) on [0,100], sample it at 50000
points, and add noise with a standard deviation of, say, 10. Close to
0 a lot of training might be required to learn the function to a low
level of error, but at the right end near 100, it could be easy.
Training until the part near 0 is learned could force the noise to be
learned at the end near 100.
Maybe someone should try running your problem and tell us what happens.
-- Scott
===========================================================================
Scott E. Fahlman Internet: sef+@cs.cmu.edu
Senior Research Scientist Phone: 412 268-2575
School of Computer Science Fax: 412 681-5739
Carnegie Mellon University Latitude: 40:26:33 N
5000 Forbes Avenue Longitude: 79:56:48 W
Pittsburgh, PA 15213
===========================================================================