home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Monster Media 1993 #2
/
Image.iso
/
text
/
9305nni.zip
/
930528.DIS
< prev
next >
Wrap
Text File
|
1993-05-28
|
2KB
|
47 lines
Article 9250 of comp.ai.neural-nets:
Path: serval!netnews.nwnet.net!ogicse!emory!europa.eng.gtefsd.com!fs7.ece.cmu.edu!crabapple.srv.cs.cmu.edu!news
From: sef@sef-pmax.slisp.cs.cmu.edu
Newsgroups: comp.ai.neural-nets
Subject: Re: help with simple NNs
Message-ID: <C7o0nn.JLr.1@cs.cmu.edu>
Date: 27 May 93 03:06:56 GMT
Article-I.D.: cs.C7o0nn.JLr.1
Sender: news@cs.cmu.edu (Usenet News System)
Organization: School of Computer Science, Carnegie Mellon
Lines: 31
Nntp-Posting-Host: sef-pmax.slisp.cs.cmu.edu
I'd like to know this one too, how many layers are 'enough for all problems'
with the traditional backprop NN ?
There are a number of theoretical results around showing that, under
appropriate assumptions (which cover most classification and approximation
problems you will run into in practice), a single layer of hidden units is
sufficient. However, you might need *many* more hidden units in a single
layer than you would need in a multiple-hidden-layer net.
Also whyowhyowhy are the initial weights set to random values ?????????
Surely this means that the net _could_ start in a state that causes local
minima at a later stage ?????
Any starting point, random or not, can drop you into a local minimum of the
error surface. Random initial weights are used to break symmetries. If
all the hidden units start out with the same surrounding weights (zero, for
example), and if there is no noise in the network, you will never be able
to split the hidden units apart and get them to do distinct jobs. They
will all see the same weighted inputs and the same errors.
-- Scott
===========================================================================
Scott E. Fahlman Internet: sef+@cs.cmu.edu
Senior Research Scientist Phone: 412 268-2575
School of Computer Science Fax: 412 681-5739
Carnegie Mellon University Latitude: 40:26:33 N
5000 Forbes Avenue Longitude: 79:56:48 W
Pittsburgh, PA 15213
===========================================================================