[ Pobierz całość w formacie PDF ]
Neural Networks
Rolf Pfeifer
Dana Damian
Rudolf Fuchslin
Contents
Chapter 1. Introduction and motivation
1
1. Some differences between computers and brains
2
2. From biological to artificial neural networks
4
3. The history of connectionism
6
4. Directions and Applications
8
Chapter 2. Basic concepts
11
1. The four or five basics
11
2. Node characteristics
12
3. Connectivity
13
4. Propagation rule
15
5. Learning rules
16
6. The fifth basic: embedding the network
18
Chapter 3. Simple perceptrons and adalines
19
1. Historical comments and introduction
19
2. The perceptron
19
3. Adalines
24
Chapter 4. Multilayer perceptrons and backpropagation
29
1. The back-propagation algorithm
29
2. Java-code for back-propagation
32
3. A historical Example: NETTalk
35
4. Properties of back-propagation
37
5. Performance of back-propagation
38
6. Modeling procedure
45
7. Applications and case studies
46
8. Distinguishing cylinders from walls: a case study on embodiment
49
9. Other supervised networks
50
Chapter 5. Recurrent networks
57
1. Basic concepts of associative memory - Hopfield nets
57
2. Other recurrent network models
69
Chapter 6. Non-supervised networks
75
1. Competitive learning
75
2. Adaptive Resonance Theory
80
3. Feature mapping
87
4. Extended feature maps - robot control
92
5. Hebbian learning
95
i
ii
CONTENTS
Bibliography
103
CHAPTER 1
Introduction and motivation
The brain performs astonishing tasks: we can walk, talk, read, write, recognize
hundreds of faces and objects, irrespective of distance, orientation and lighting
conditions, we can drink from a cup, give a lecture, drive a car, do sports, we can
take a course in neural networks at the university, and so on. Well, it’s actually
not the brain, it’s entire humans that execute the tasks. The brain plays, of course,
an essential role in this process, but it should be noted that the body itself, the
morphology (the shape or the anatomy, the sensors, their position on the body), and
the materials from which it is constructed also do a lot of useful work in intelligent
behavior. The keyword here is embodiment, which is described in detail in [
Pfeifer
and Scheier, 1999
]and[
Pfeifer and Bongard, 2007
]. In other words, the brain
is always embedded into a physical system that interacts with the real world, and
if we want to understand the function of the brain we must take embodiment into
account.
Because the brain is so awesomly powerful, it seems natural to seek inspiration
from the brain. In the field of neural computation (or neuro-informatics), the brain
is viewed as performing computation and one tries to reproduce at least partially
some of its amazing abilities. This type of computation is also called ”brain-style”
computing.
One of the well-known interesting characteristics of brains is that the behavior
of the individual neuron is clearly not considered ”intelligent” whereas the behavior
of the brain as a whole is (again, we should also include the body in this argument).
The technical term used here is
emergence
: if we are to understand the brain, we
must understand how the global behavior of the brain-body system emerges from
the activity and especially the interaction of many individual units.
In this course, we focus on the brain and the neural systems and we try to
make proper abstractions so that we can not only improve our understanding of
how natural brains function, but exploit brain-style computing for technological
purposes. The term
neural networks
is often used as an umbrella term for all these
activities.
Before digging into how this could actually be done, let us look at some areas
where this kind of technology could be applied. In factory automation, number
crunching, abstract symbol manipulation, or logical reasoning, it would not make
sense because the standard methods of computing and control work extremely well,
whereas for more ”natural” forms of behavior, such as perception, movement, lo-
comotion, and object manipulation, we can expect interesting results. There are a
number of additional impressive characteristics of brains that we will now look at
anecdotally, i.e. it is not a systematic collection but again illustrates the power of
natural brains. For all of these capacities, traditional computing has to date not
come up with generally accepted solutions.
1
[ Pobierz całość w formacie PDF ]