NetNews Usenet Archive 1993 #1/NN_1993_1.iso/spool/comp/ai/neuraln/4759

Text File | 1993-01-08 | 3.7 KB | 77 lines

Newsgroups: comp.ai.neural-nets
Path: sparky!uunet!haven.umd.edu!darwin.sura.net!gatech!destroyer!cs.ubc.ca!alberta!arms
From: [e-mail] (Bill Armstrong)
Subject: Re: damsel in distress over back-prop
Message-ID: <arms.726513993@spedden>
Sender: [e-mail] (News Administrator)
Nntp-Posting-Host: spedden.cs.ualberta.ca
Organization: University of Alberta, Edmonton, Canada
References: <[e-mail]>
Date: Fri, 8 Jan 1993 17:26:33 GMT
Lines: 64

[e-mail] (Nina Joshi) writes:

>I have tried to understand the back-prop algorithm from several sources, but
>find it difficult to comprehend the solution. I recognise the bp algorithm as
>being a means to determine the error derivative within a layer in networks,
>which can then be used to adjust the weights and hence output the desired
>pattern...but I get lost in the algebra...can anyone help?

Dear Damsel:

The nice thing is that you can understand the BP algorithm without
going into the algebra.  The derivatives which are computed in the
backward pass just tell you how much each weight has to be adjusted to
produce a given diminution of the error.  The weights which can reduce
the error by being changed slightly get preference over the others.
In this way, one tends to make smaller changes overall and hence tends
to disturb the responses to already learned patterns least (B. Widrow's
"Least Disturbance Principle").

If you don't want to go through all the derivatives and gradients, you
could switch to adaptive logic networks (ALNs).  There, a change in
the output of a node either changes the output or it doesn't.  This
means you can realize the "backprop" calculation faster, in
combinational logic.  You can tell whether a change in a node will
propagate to an output easily, for example a signal is enabled to pass
through an AND if the other input is a 1.  You can deal with
continuous quantities by using thresholds x > const to convert
quantized reals into booleans.

As an added advantage, you then don't have to understand Kolmogorov's
theorem to get a grip on what functions can be synthesized, since it
is obvious you can approach any continuous function y = f(x) to any
degree of precision by using a logic net realizing a predicate P(x,y)
where P is true iff y < f(x).  In fact, you are not limited to continuous
functions.

I hope ALNs will help with the distress.  Otherwise take two Aspirins
and call an algebraist in the morning.

Bill

PS The atree adaptive logic network (ALN) simulation package, atree
release 2.7, is available via anonymous ftp from menaik.cs.ualberta.ca
[129.128.4.241] in pub/atre27.exe (ftp in binary mode).  It runs on
IBM PCs and compatibles under Windows 3.x.  Included is documentation
and ON-LINE HELP explaining the basic principles of adaptive logic
networks, the atree source code and the examples.  All C and C++
source code is provided.

To set up your software on the PC under Windows 3.x, it is recommended
that you execute atre27.exe in your main directory, whereupon it will
create a subdirectory atree_27 and extract everything into it.
Running "setup" in the latter directory will create a group of icons
you can use to invoke demos and the facilities for programming
adaptive logic network applications in the lf language.  The "Open"
command gives you access to numerous instructive examples.  Clicking
on the Help button gives you access to explanations of theory and
code.  After you look at this version, you could port the Unix version
in C, atree 2.0, to whatever computer you are using.
--
***************************************************
Prof. William W. Armstrong, Computing Science Dept.
University of Alberta; Edmonton, Alberta, Canada T6G 2H1
[e-mail] Tel[phone] FAX [phone]