NetNews Usenet Archive 1993 #1/NN_1993_1.iso/spool/bit/listserv/csgl/2112

Text File | 1993-01-11 | 11.1 KB | 205 lines

Comments: Gated by [e-mail]
Path: sparky!uunet!zaphod.mps.ohio-state.edu!howland.reston.ans.net!paladin.american.edu!auvm!BEN.DCIEM.DND.CA!MMT
Return-Path: <@VMD.CSO.UIUC.EDU:[e-mail]>
Message-ID: <[e-mail]>
Newsgroups: bit.listserv.csg-l
Date:         Mon, 11 Jan 1993 14:50:57 EST
Sender:       "Control Systems Group Network (CSGnet)" <[e-mail]>
From:         [e-mail]
Subject:      cross-post from A-LIFE via CYBSYS-L
Lines: 193

[Martin Taylor 930111 14:45]

Here is a description of the meeting "Simulation of Adaptive Behaviour 92".

CSG-L readers may be particularly interested in the comments on the paper
by Klopf, at the end of the highlights section.
---------------------

Dear ALifers,
                Here is a report on the recent SAB92 conference in Hawaii.
Hope you can use it in your ALife email network.

Cheers,
        Hugo de Garis.

                             SAB92 Report

Simulation of Adaptive Behavior SAB92 Conference, Hawaii, 7-11 Dec 1992.

                                  by

                            Hugo de Garis

                      Electrotechnical Lab (ETL)
                                Japan


Its hard to imagine a more beautiful setting for a conference than Hawaii.
As a postcard put it that I bought (showing moonlight reflecting off the water
silouetted by palm trees) "Just another ho hum day in paradise : Hawaii".
For many of us who attended this second SAB conference, it was our first visit
to Hawaii, so the 12 am to 4 pm "siesta" period was most welcome. However, the
beauty of the surroundings was offset by the distance to be travelled, hence
numbers were down (100 people, half of whom were presenters) and nearly all
were jet lagged, so the the first day or two were viewed through a mental
fog.

The first SAB conference was held in Paris in Dec 1990 where the dominant
theme seemed to be how ethologists and roboticists could help each other in
elucidating the secrets of behavioral mechanisms. Biologists were noticably
absent at this second SAB conference, dominated by computer and engineering
types. This is probably to be expected as the field matures. The conference had
a feel of "more of the same" rather than of the excitement of the
 "qualitativelynew" of the first conference. I guess you have to be a Chris
 Langton to be able
to make successive conferences exciting. Still, one came away with the
impression that some solid progress had been made since the Paris conference.
The most impressive example is the appearance of a new MIT Press journal,
"Adaptive Behavior", edited by the Frenchman, Jean-Arcady Meyer. Jean-Arcady
and future SAB conference organizers now face the difficult task of defining
a niche for the field which is narrow enough to distinguish itself from
the highly overlapping fields of Neural Networks, Genetic Algorithms,
Artificial Life, Parallel Problem Solving from Nature (PPSN), Robotics, etc
yet not be so narrow that the field becomes overdefined and stagnates.

Roughly 40 oral and 20 poster papers were presented, with the following
themes.

The Animat Approach to Adaptive Behavior.
Perception and Motor Control.
Action Selection and Behavioral Sequences.
Cognitive Maps and Internal World Models.
Learning.
Evolution.
Collective Behavior.

Highlights were (according to my own subjective view and that of one of the
organizers) :-

Pattie Maes's talk on "Behavior Based Artificial Intelligence" (the
first talk of the conference) which made the point that behavior based
thinking should be extended beyond robotics to AI in general, but was
more a statement of intent than of concrete results. Pattie's
productivity seems to have dropped since she became an assistant prof
at MIT. Its a pity that Europe and the US do not give enough "research
only" positions to its best researchers. If the West doesnt watch out,
it will lose its best to Japan, which has more sense that way by
creating many positions with no teaching load.  Quoting Pattie's
abstract - "This paper attempts to define Behavior-Based Artificial
Intelligence (AI) as a new approach to the study of intelligence.  It
distinguishes this approach from the traditional knowledge based
approach in terms of the questions studied, the solutions adopted and
the criteria used for success. It does not limit behavior based AI to
the study of robots but rather presents it as a general approach for
building autonomous systems that have to deal with multiple, changing
goals in a dynamic, unpredictable environment".

Mark Ring's (Univ. of Texas) paper "Two Methods for Hierarachy Learning in
Reinforcement Environments". His abstract read - "This paper describes two
methods for hierarchically organizing temporal behaviors. The first is more
intuitive : grouping together common sequences of events into single units
so that they may be treated as individual behaviors. This system immediately
encounters problems however, because the units are binary, meaning
the behaviors must execute completely or not at all, and this hinders the
construction of good algorithms. The system also runs into difficulty when
more than one unit is (or should be) active at the same time. The second system
is a hierarchy of transition values. This hierarchy dynamically modifies the
values that specify the degree to which one unit should follow another. These
values are continuous, allowing the use of gradient descent during learning.
Furthermore, many units are active at the same time as part of the system's
normal functioning".

Michael Littman's (Bellcore, CMU) paper "An Optimization-based Categorization
of Reinforcement Learning". Abstract - "This paper proposes a categorization
of reinforcement learning environments based on the optimization of a
reinforcement signal over time. Environments are classified by the simplest
agent that can possibly achieve optimal reinforcement. Two parameters, h and
beta, abstractly characterize the complexity of an agent: the ideal (h, beta)-
agent uses the input information provided by the environment and at most h bits
of local storage to choose an action that maximises the discounted sum of the
next beta reinforcements. In an (h, beta)-environment, an ideal (h, beta)-agent
achieves the maximum possible expected reinforcement for that environment. The
paper discusses the special cases when either h = 0 or beta = 1 in detail,
describes some theoretical bounds on h and beta and re explores a well known
reinforcement learning environment with this new notation".

Jing Peng's and Ronald William's (Northeastern Univ) paper "Efficient Learning
and Planning within the Dyna Framework". Abstract -
"Sutton's Dyna framework provides a novel and computationally appealing way to
integrate learning, planning and reacting in autonomous agents. Examined here
is a class of strategies designed to enhance the learning and planning power
of Dyna systems by increasing their computational efficiency. The benefit of
using these strategies is demonstrated on some simple abstract learning tasks".

Dave Cliff's et al (Sussex) paper "Evolving Visually Guided Robots" appealed
most strongly to me, because it used (evolved) neural networks to control
robots (my own line of work). Dave evolved a neural network which took visual
input to control the motion of a robot. His abstract was too long to quote.

Hitoshi Iba (a colleague at ETL) presented a paper called "Evolutionary Learning
of Predatory Behaviors Based on Structured Classifiers". Iba san has done some
innovative work in extending Koza's Genetic Programming Paradigm (i.e.
evolving tree structured Lisp programs) into Holland's Classifier Systems,
and applying the results to the optimization of animal foraging strategies.

Geoffrey Miller's (Stanford) and Peter Todd's (Rowland Inst) paper
 "EvolutionaryInteractions among Mate Choice, Speciation, and Runaway Sexual
 Selection".
This paper explored the effects of different mate choice mechanisms and modes
of speciation on the dynamics of runaway sexual selection. Geoffrey showed how
sexual selection could cause fitness levels to deviate from optimal values.

Long-Ji Lin's and Tom Mitchell's (CMU) paper "Reinforcement Learning with
Hidden States". Abstract - "Reinforcement learning is an unsupervised learning
technique for sequential decision making. Q-learning is a widely used
reinforcement learning method. The convergence of Q-learning relies on the
Markovian environment assumption, meaning that any information needed to
determine the optimal action is reflected in the agents state representation.
If some important state features are missing (or hidden) from the state
representation, the true world states cannot be directly identified and optimal
decisions cannot be made based on this state representation. This problem is
known as the hidden state problem. A possible solution to the problem is to use
history information to help uncover the hidden features. This paper studies
3 reinforcemnt learning architectures that learn to use history to handle
 hiddenstates: window-Q, recurrent-Q, and recurrent model. Empirical study of
 these
architectures is presented. Their relative strengths and weaknesses are also
discussed."

Harry Klopf's (Wright Lab) "Modelling Nervous System Function with a
Hierarchical Network of Control Systems that Learn". (Part of) Abstract -
"A computational model of nervous system function during classical and
instrumental conditioning is proposed. The model assumes the form of a
hierarchical network of control systems. Each control system is capable of
learning and is referred to as an associative control process (ACP). Learning
systems consisting of ACP networks, employing the drive reinforcement learning
mechanism (Klopf 1988) and engaging in real time, closed loop, goal seeking
interactions with environments, are capable of being classically and
instrumentally conditioned, as demonstrated by means of computer simulations".
This paper felt important, and that it made more than the usual incremental
contribution to the field. Klopf's neural network hierarchies actually LEARN.
If there had been a best paper award, my vote would have gone to Klopf.

                      ***   End of Highlights   ***


Now for the down side. I found it most annoying that there was no proceedings
ready at the conference. This was a point discussed in the feedback session
on the second last day. The organizers said they would try to remedy this fault
at SAB94 (which will be held very probably in the first week of August, at
the University of Sussex, Brighton, England). There seems to be a growing
tendency for conference organizers not to bother having a proceedings ready in
time for the conference. The worst offender in this regard, is Chris Langton,
who manages to come out with a book, a year after the conference, and worse,
half of the oral papers do not even appear in the book, so these poor guys
dont even get published. I would like to go on record as stating that
conference organizers who do not bother having a proceedings in time for
their conferences, should be BOILED IN OIL. (The SAB92 proceedings will be
published by MIT Press and should appear in the spring of 93).

              See you at SAB94 in Brighton, England.

                           Hugo de Garis.