NetNews Usenet Archive 1992 #26/NN_1992_26.iso/spool/comp/ai/neuraln/4226

Text File | 1992-11-11 | 9.6 KB | 262 lines

Newsgroups: comp.ai.neural-nets
Path: sparky!uunet!cs.utexas.edu!qt.cs.utexas.edu!yale.edu!ira.uka.de!rz.uni-karlsruhe.de!stepsun.uni-kl.de!sun.rhrk.uni-kl.de!starke
From: [e-mail] (Gerolf J Starke)
Subject: Summary: NN and prediction
Message-ID: <[e-mail]>
Organization: University of Kaiserslautern, Germany
Date: Wed, 11 Nov 1992 12:12:51 GMT
Lines: 252


Thanks to all who answered my question about NN and prediction,
specially to:

Sanjay Patil
Shawn Day
Raj Patil
Louis Mittoni
Miachael Kuehne
Peeter M. Akerberg
H. Debarre Thesard
An Nguyen

Some asked for a summary. Here it is. I collected all the mails
I received, without any commentaries.
----------
Hello,

I have done some work using feed-forward dispersive networks with
raw time-series data as input. Below are some recent references:

Day, Shawn P. and Davenport, Michael R., "Continuous-Time Temporal
Back-Propagation with Adaptable Time Delays", To appear in: IEEE
Transactions on Neural Networks.

Day, Shawn P. and Davenport, Michael R. and Camporese, Daniel S.,
"Dispersive Networks for Nonlinear Adaptive Filtering", In Neural
Networks for Signal Processing 2 - Proceedings of the 1992 IEEE
Workshop, S. Y. Kung, F. Fallside, J. Aa. Sorenson, and C. A.
Kamm, editors, pages 540-549, IEEE Press.

Davenport, Michael R., and Day, Shawn P., "Chaotic Signal Emulation
using a Recurrent Time Delay Neural Network", In Neural Networks
for Signal Processing 2 - Proceedings of the 1992 IEEE Workshop,
S. Y. Kung, F. Fallside, J. Aa. Sorenson, and C. A. Kamm, editors,
pages 454-463, IEEE Press.

Most of this work has to do with time-series prediction, but there
are also some results on adaptive channel equalization, and chaotic
signal emulation. Postscript versions of these papers are available
for anonymous ftp from toaster.ee.ubc.ca. Go to the pub/shawnd
directory, and look for the files ieee.ps.Z, copenhagen1.ps.Z,
and copenhagen2.ps.Z.

Shawn Day           University of British Columbia
[e-mail]    Department of Electrical Engineering
[phone]      2356 Main Mall, Vancouver, BC, Canada, V6T 1Z4
----------
Well, I am not famous in the area but have done some emprocal studies for
my masters thesis couple years back. i am including the references you
should look at before going further.

Box,  G. E. P. and G. M Jenkines, "time series: forecasting and
control", holden-day, San fransisco, CA 1976.
Fishwick, P., :neural net models in simulation: a comparison with
tradiional modelling approaches, working paper contact fishwick at
univ of florida, gainsville...
Hoff J., a practicle guide to BOX-JENKINS forecasting, lifetime learning
publications, Belmont CA. ISBN 0543027199, 1983
Lapedes and Fraber, "onlinear signal processing using neural nets: prediction
and system modelling, los alomos national lab tech report, LA-UR-87-2261
july 1987
Lapedes and Farber "genetic data base analysis with neural nets" IEEE
conference on neural info. processing systems-natural and
synthetic 1987.

Makridakis S., "the accuracy of extrapolation methods: results of
forecasting competition, journal of forecasting vol 1 pp 111-153, 1982

Pack D.J, and Dowing D., "why did box-junkins win again" proceddings
third intl. symposium on forecasting, philadilphia, 1983

Sharda and Patil "Neural nets as forecasting experts: An emprical test"
IJCNN-WASH-DC, jan 1990, vol-II, pp491-494

Sutton R., S, "learning to predict by methods of temporal differences"
machine learning vol 3., 99 9-44,1988

Tang Z., Time series forecasting using neural nets, Intl workshop on neural nets feb 1990, Auburn AL.

Werbos P, "beyond regression: new tools for prediction and analysis in
behavirol sciences, ph.d thesis, harvard univ.


These are the references i used. sorry for sloppy typing i am in the middle
of some thing very urgent...let me know if you do something interesting...


best reggards

raj patil
----------
I have had luck in this area using a very simple two layered network.

Two methods I've tried is: for a chaotic signal with time constant $\tau$
give the network as input either:
    o x(t),x(t+1),...,x(t+n)   where n is say 10-15 or longer.
    o x(t),x(t+tau),x(t+2.tau)...  for the usual tau time delay.

I had more sucess with the first, I guess you can use any embedded set of
vector representation so long as you give the net enough information.

Please let me know of any other replies you recieve, maybe we can compare
performance or exchange network geometries & references.  I have a couple.

Louis
_____________________________________________________________________________
Louis Mittoni                    <Standard Disclaimer>
[e-mail]
GK Williams Co-operative Research Centre
University of Melbourne - CSIRO Division of Mineral and Process Engineering
Telephone: + 61 3 541 1289    Fax: + 61 3 562 8919
"Our country has plenty of good five-cent cigars, but the trouble is
they charge fifteen cents for them."
_____________________________________________________________________________
----------
Hallo Gerolf!
Ich beschaeftige mich zur Zeit mit der Prognose von Wirtschaftsdaten mit
Hilfe von Neuronalen Netzen. Ich verwende dafuer ein Elman-net (nach
Elman, J.L., Finding Structures in Time, Cognitive Science 1990, 14,
S 179 - 211). Das ist ein backpropagation-net mit recurrent-Strukturen.
Die Rueckkopplungen ermoeglichen eine Speicherung der Vergangenheit im
Netz selbst. Damit muessen die Eingangsgroessen zu verschiedenen
Zeitpunkten dem Netz nicht mehr parallel praesentiert werden. Ich weiss
natuerlich nicht, wie gut das Netz fuer chaotische Systeme geeignet ist.
Wenn Du interessiert bist, gebe ich Dir gerne naehere Informationen.

        Tschuess
Michael Kuehne
z.Z.
University of Wales College of Cardiff
Elsym
----------
I have some references but I would be very grateful if yoiu send me
whatever you get from others.

If you have ftp access, you can get PostScript copies of these three
papers from toaster.ee.ubc.ca. Look in pub/shawnd for
the files copenhagen1.ps.Z, copenhagen2.ps.Z, and ieee.ps.Z.

Day, Shawn P. and Davenport, Michael R., "Continuous-Time Temporal
Back-Propagation with Adaptable Time Delays", To appear in: IEEE
Transactions on Neural Networks.

Day, Shawn P. and Davenport, Michael R. and Camporese, Daniel S.,
"Dispersive Networks for Nonlinear Adaptive Filtering", IEEE
Workshop on Neural Networks for Signal Processing, Helsingor,
Denmark, 1992, pages 540-549.

Davenport, Michael R., and Day, Shawn P., "Chaotic Signal Emulation
using a Recurrent Time Delay Neural Network", IEEE Workshop on
Neural Networks for Signal Processing, Helsingor, Denmark, 1992,
pages 454-463.


1. (Just in case) see the book Nonlinear Modeling and Forecasting
   (ed. M.Casdagli and S. Eubank), SFI Studies in the Sciences of
   Complexity, Proceedings Vol. XII, Addison-Wesley NY (1992). {lotsa
   net papers}


2. D. Nychka, S.Ellner, A.R. Gallant, and D. McCaffrey. 1992.

   Finding chaos in noisy systems. J. Royal Statistical Society Ser.
B vol.54,
   399-426.

3. D. McCaffrey, S. Ellner, A.R. Gallant, D. Nychka. 1992. Estimating
   the Lyapunov exponent of a chaotic system with nonparametric
regression.

   J. Amer. Stat. Assoc. (in press; to appear this Fall).


4. Wolpert, D.M. and R.C. Miall 1990. Detecting chaos with neural
networks.

   Proc. Roy. Soc. Lond. B 242, 82-86.

 A.D. Back and A.C. Tsoi, "FIR and IIR Synapses, a New Neural
Network"
Architecture for Tine Series Modeling. Neural Computation 3, 375-385

1991.

 N.Z Hakim, J.J. Kaufman, G. Cerf and H.E. Meadows, "A Discrete-Time
Neural Network Model for Systems Identification" Proceedings
IJCNN-90.


 N.Z Hakim, J.J. Kaufman, G. Cerf and H.E. Meadows, "Nonlinear Time
Series Prediction with a Discrete-Time Neural Network Model",
Proceedings

IJCNN-91.

 J.A. Villareal and Robert O. Shelton, "A Space-Time Neural Network",
Presented at the Second Annual Joint Conference on Neural Networks
and Fuzzy Logic -90.

 B.A. Pearlmutter,
"Learning state space trajectories in recurrent neural networks"
IJCNN- 1989.

B.A. Pearlmutter,
"Dynamic recurrent neural networks"
CMU-CS-90-196.

/Peeter

Peeter Akerberg ([e-mail])
Dept. of Electrical Engineering
University of Houston
----------
I am a french PhD student and I have been working with neural nets for time
series predictions for quite a while. We have seen several approaches:
- Perceptron with a time window. I don't believe this would work very well for
    chaotic data, see Weigend and Casdagli.
- Williams and Zipser fully recurrent network. Complicated and very slow.
- Simple Recurrent Networks, like Elman or Jordan. I have choosen such an
    architecture and used it on the chaotic mapping X(n+1) = mu * X(n)
    * (1-X(n)). It works quite well in predicting the trend, but has some
    offset with the real value.
Herve
----------
A good, general collection of papers on time-series prediction w/ NN and
related subjects can be found in the volume:

   Martin Casdagli and Stephen Eubank (Eds.)  "Nonlinear modeling and
          Forecasting."  1992.

This is proceedings volume XII from the Santa Fe Institute for Studies in
the Sciences of Complexity.

Sincerely,

An Nguyen

==========================================================================
   Lex-Kon, Inc. * 3731 Shimmons Circle * Auburn Hills, MI 48328 * USA
              Tel.: [phone] * E-mail: [e-mail]
            Neural Networks, Fuzzy Systems, Genetic Algorithms
----------

Thanks again,

Gerolf.
--
Gerolf J. Starke                              phone: [phone]
Institute of Applied Mechanics                internet: [e-mail]
University of Kaiserslautern, 6750            compuserve: 100010,3451
Germany                                       fax: [phone]