TNeuralNetwork component

Properties	Methods	Example 1	Example 2

TNeuralNetwork is a component representing a general feedforward neural network of an arbitrary size. The network can be taught using the back-propagation learning method. The component is also able to normalize and de-normalize the data from the internal representation, so the original data can be used instead of numbers between 0 and 1.

Network structure and initialization

The Structure property defines the layout of the network, i.e. the number of input and output neurons and also the number of neurons in each hidden layer. The InputNum, OutputNum and LayersNum run-time only properties can be used to query the current structure of the network.

The network can be initialized by using the Initialize method.

Teaching the network

The network can be taught by putting data on the input neurons and specifying the expected output on the output neurons. Teaching consists of the following steps.

1.	Specify the minimum and maximum values for each input and output neuron. The network uses these values to normalize the input data for its internal representation and normalize back into human-readable form when querying the output. Use the SetInputMin, SetInputMax, SetOutputMin and SetOutputMax methods to specify the ranges.

2.	Specify the input for all input neurons using the SetInput method.

3.	Specify the expected output of the network for this input using the SetExpOutput method.

4.	Use the Learn method to teach this sample to the network.

5.	Continue from Step 2 with the next sample.

6.	When all samples are showed once, repeat the entire procedure from Step 2 many times. You can either show each sample for a fixed number of times, or you can repeat the procedure until the error of the network (returned by Learn) goes under a specific value. The combination of both methods is suggested, since it is possible that the error value never goes under the expected limit.
Note: showing the samples to the network in a random order can greatly increase the learning efficiency.

Querying the network

1.	Use the SetInput method to set the value of the input neurons.

2.	Call the Calc method to calculate the output for the specified input.

3.	Use the GetOutput method to query the value of the output neurons.

File Operations

To save or load the current state of the neural network (in other words, the knowledge), use the FileName property and the Load and Save methods.

See Also

A Brief Introduction to Neural Networks


Properties

InputNum	StepsNum
LayersNum	Structure
OutputNum


Methods

Calc	SetExpOutput
GetOutput	SetInput
Initialize	SetInputMin
Learn	SetInputMax
Load	SetOutputMin
Save	SetOutputMax


Structure property

Example

Applies to

TNeuralNetwork component

Declaration

property Structure: TStringList;

Description

The structure property determines the structure of the neural network. Neural networks represented by TNeuralNetwork consist of layers and each layer can have an arbitrary number of neurons. The structure property is a list of strings but every item must be an integer number between 1 and 1000. The number of items determine the number of layers in the neural network and the values of the items define the number of neurons for each layer. The first element is the input layer and the last one is the output layer.

Example

If you would like to use the network component on the picture, the Structure property should contain the following numbers:

3 [3 input neurons]

2 [2 neurons in the hidden layer]

2 [2 output neurons]

If you wish to insert another hidden layer consisting of 4 neurons right before the output layer, you should set the Structure property as follows:

3 [3 input neurons]

2 [2 neurons in the first hidden layer]

4 [4 neurons in the second hidden layer]

2 [2 output neurons]

See Also

Initialize, InputNum, OutputNum, LayersNum


FileName property

Example

Applies to

TNeuralNetwork component

Declaration

property FileName: TFileName;

Description

TNeuralNetwork component have Save and Load methods which makes possible to store neural network data to file and restore from file. The Load and Save methods uses the FileName property to determine the file name of the network file.

See Also

Load, Save


InputNum property

Applies to

TNeuralNetwork component

Declaration

property InputNum: Word;

Description

Run-time and read only. The InputNum property indicates the number of input neurons (size of input vector).

See Also

Structure, OutputNum, LayersNum


OutputNum property

Applies to

TNeuralNetwork component

Declaration

property OutputNum: Word;

Description

Run-time and read only. The OutputNum property indicates the number of output neurons (size of output vector).

See Also

Structure, InputNum, LayersNum


LayersNum property

Applies to

TNeuralNetwork component

Declaration

property LayersNum: Word;

Description

Run-time and read only. The LayersNum property shows the total number of layers in the neural network, including the input, output and hidden layers.

See Also

Structure, InputNum, OutputNum


StepsNum property

Applies to

TNeuralNetwork component

Declaration

property StepsNum: Word;

Description

Run-time and read only. The StepsNum property indicates the number of learning steps performed since the creation of the component or the execution of the Initialize method. In other words, this property shows how many times the Learn method called since the last initialization.

See Also

Initialize, Learn


SetInputMax and SetInputMin methods

Example

Applies to

TNeuralNetwork component

Declaration

procedure SetInputMax(index: integer; value: double);

procedure SetInputMin(index: integer; value: double);

Description

These methods define the possible maximum and minimum values for the index-th input neuron (the first neuron has the index of 1). The neural network uses these values to normalize the data to its internal representation.

Example

If the second output neuron will be used to teach/query the USD/DEM ratio and your historical data contains values between 0.5 and 2, then you need to use the following calls before teaching and querying the network:

   NeuralNetwork1.SetInputMin(2, 0.5);

   NeuralNetwork1.SetInputMax(2, 2.0);

See Also

SetInput


SetOutputMax and SetOutputMin methods

Example

Applies to

TNeuralNetwork component

Declaration

procedure SetOutputMax(index: integer; value: double);

procedure SetOutputMin(index: integer; value: double);

Description

These methods define the possible maximum and minimum values for the index-th output neuron (the first neuron has the index of 1). The neural network uses these values to normalize the data from and to its internal representation to.

Example

If the second output neuron will be used to teach/query the USD/DEM ratio and the expected value is between 0.5 and 2, then you need to use the following calls before teaching and querying the network:

   NeuralNetwork1.SetOutputMin(2, 0.5);

   NeuralNetwork1.SetOutputMax(2, 2.0);

See Also

SetExpOutput, GetOutput


Load and Save methods

Example

Applies to

TNeuralNetwork component

Declaration

procedure Load;

procedure Save;

Description

The Load method loads a network stored in a file specified by the FileName property. The Save method saves the current state of the network into the file specified by the FileName property.

These methods load and save the network structure, the current weights and the input and output ranges set with the SetInputMin, SetInputMax, SetOutputMin, SetOutputMax methods.

See Also

FileName


SetInput method

Example 1	Example 2

Applies to

TNeuralNetwork component

Declaration

procedure SetInput(index: integer; value: double);

Description

Sets the value of the index-th input neuron (the first neuron has the index of 1). Use this method to set all input neurons before teaching or querying the network. Before using SetInput, you have to call SetInputMin and SetInputMax once for each neuron.

See also

SetInputMin, SetInputMax, Learn, Calc


SetExpOutput method

Example

Applies to

TNeuralNetwork component

Declaration

procedure SetExpOutput(index: integer; value: double);

Description

Sets the expected value of the index-th output neuron (the first neuron has the index of 1). Use this method to set the expected output value before teaching a sample to the network. Before using this method, you need to call SetOutputMin and SetOutputMax once for each neuron.

See also

SetOutputMin, SetOutputMax, Learn


GetOutput method

Example

Applies to

TNeuralNetwork component

Declaration

function GetOutput(index: integer): double;

Description

Returns the value of the index-th output neuron (the first neuron has the index of 1). Use this method to query the output of the network after using the Calc method.

See also

Calc


Calc method

Example 1	Example 2

Applies to

TNeuralNetwork component

Declaration

procedure Calc;

Description

Calculates the output values from the input neurons. Before using Calc, use the SetInput method to set the value of the input neurons. After calculation, use the GetOutput method to query the results.

See also

SetInput, GetOutput


Learn method

Example

Applies to

TNeuralNetwork component

Declaration

function Learn(alpha, beta: double): double;

Description

Causes the network to run one learning cycle on the current sample. Before using this method, use the SetInput method to set the value of the input neurons and the SetExpOutput method to set the expected output.

Parameters

alpha:	0 < alpha < 1. This parameter is the learning rate. It influences the weight changes during the learning cycle. Larger alpha values cause larger weight changes. This might result in faster learning but lower accuracy. Smaller alpha values result in slow, higher accuracy learning, but it is also more probable that the network will not be able to break out from a local minimum. A suggested starting value is 0.35. As the error of the network decreases during the learning, it is usually a good idea to lower the value of alpha.

beta:	0 < beta < 1. This parameter is called `momentum term'. It rules the effects of the previous learning cycle. If the value of beta is zero, the previous cycle has no effect on the current one at all. Increasing the beta value increases the effects of the previous cycle (note that this is a recursive effect since the previous cycle also effected by the cycle before it). A suggested value is 0.5.

Return value

The error of the network. This number shows how well the network `knows' this sample already. The closer this number to zero, the better the network learned the sample. An average error of the network can be calculated by averaging the errors of the individual samples.

See also

SetInput, SetExpOutput


Initialize method

Example

Applies to

TNeuralNetwork component

Declaration

procedure Initialize;

Description

Use this method to initialize the network. You need to call Initialize each time you make changes to the structure of the network through the Structure property. Initialize also resets the weights of the network so you can also use it to return the network to a random starting state.

See also

Structure


A Brief Introduction to Neural Networks

From the 1960's, there is an increasing need of handling real-world problems of imprecision and uncertainty. One of the most important approaches are artificial neural networks, mainly inspired by biological neural networks. Biological neurons, believed to be the structural constituents of the brain, are much slower than silicon logic gates. However, inferencing in these networks is faster than the fastest computer. The brain compensates for the relatively slower operation by having a huge number of interconnected neurons. A biological neural network is a nonlinear, highly parallel device with the most important features of fault tolerance and robustness. It is able to

•	learn by adapting its synaptic weights to changes in the surrounding environment;

•	handle imprecise, fuzzy, noisy and probabilistic information;

•	generalized from known tasks or examples to unknown ones.

Artificial neural networks are an attempt to mimic these characteristics. This paradigm differs from an ordinary programmed instruction sequence in that information is stored in the synaptic connections. Each neuron is capable to perform primitive operations like summing the weighted inputs coming to it and then amplifying or tresholding the sum. Such an assembly can perform the same computations as a digital computer.

There are many network types, layouts and learning algorithms and a huge number of articles dealing with these. We do not go into the theoretical details here, we rather deal with practical applications. This component implements one of the most common types: a feedforward network with supervised backpropagation learning (see the picture below for its topology). If you became interested in neural networks, you can find many articles explaining these concepts.

Practical applications of the neural network implemented in this component:

•	pattern recognition: sample patterns and their categories are shown to the network and we expect the network to categorize new, unseen samples, too. Examples: object shape recognition (is it a circle or a rectangle?), on-line character and handwriting recognition (OCR), photo recognition (finding faces in a database, differentiating between female and male, recognizing license plates), speech recognition, robot-eyes, etc.

•	data generalization and prediction: we want to predict one or more values based on influencing factors. Historical data is needed: previous influencing factors and resulting values are shown to the network and we expect the network to find the relationships and be able to predict the values even for unseen influencing factor combinations. Examples: prediction of river water level based on weather data, prediction of power consumption based on weather data and society events, prediction of pools results, data error correction.

•	time series analysis and prediction: similar to data generalization. The values of for two consequent periods are taught to the network and we expect the network to be able to predict the values for a period from the next one. Influencing factors are not necessary but can greatly increase the efficiency. A simple example is to show many small parts of the sine function (e.g. 10 points as the input and the next 5 as the expected output) and if there are enough samples, the network will learn the entire sine function. Examples: prediction of stock prices or currency rates, etc.

Practical tips:

•	Use the smallest network possible, especially in generalization problems. If you use too many neurons, the network may simply learn your data 'by heart' and it will not be capable to generalize.

•	Learning is a slow process, especially in networks with multiple number of hidden layers. Be patient.

•	If possible, try a digital approach instead of a continuous one. The less possible values neurons have, the faster the network learns. For example, if the output of your network is between 0-15, it is better to have four neurons that can have 2 possible values of 0 and 1 (representing the numbers in binary format) than to have one neuron that can have 16 possible values.

•	A neural network usually does not give exact results, only approximations. If you want to find out whether an input picture is a circle or a rectangle, you differentiate these with 0 and 1 on the output during teaching the network, but you will usually get numbers such as 0.77 which means that the current input is rather a rectangle than a circle. It is your task to translate this to discrete output by printing 'Circle' for values < 0.5 and 'Rectangle' otherwise.

Prof. Dr. Jozsef Dombi

research manager

Cygron Research & Development, Ltd.


Example 1: Teaching the network

// This sample teaches the XOR rule 1000 times. The component is called NN.

procedure TMainForm.ButtonLearnClick;

var

   i: Integer;

   x,y: Byte;

begin

   Randomize;

   lasterror := 0.5;

   NN.Structure.Clear;

   NN.Structure.Add('2');

   NN.Structure.Add('2');

   NN.Structure.Add('1');

   NN.Initialize;

   NN.SetInputMin(1,0);  NN.SetInputMax(1,1);

   NN.SetInputMin(2,0);  NN.SetInputMax(2,1);

   NN.SetOutputMin(1,0); NN.SetOutputMax(1,1);

   Screen.Cursor := crHourglass;

   for i := 1 to 1000 do begin

      x := Random(2);

      y := Random(2);

      NN.SetInput(1,x);

      NN.SetInput(2,y);

      NN.SetExpOutput(1,x xor y);

      lasterror := NN.Learn( 0.3, 0.8 );

   end;

   Screen.Cursor := crDefault;

end;


Example 2: Querying the network

// This sample queries the network that learned the XOR rule shown

// in the learning example.

procedure TMainForm.ButtonQueryClick;

begin

   NN.SetInput(1,StrToInt(Edit1.Text));

   NN.SetInput(2,StrToInt(Edit2.Text));

   NN.Calc;

   Edit3.Text := Format('%f',NN.GetOutput(1));

end;


Example 3: File operations

procedure TMainForm.ButtonSaveClick(Sender: TObject);

begin

  try

     NN.FileName := Edit1.Text;

     NN.Save;

  except

     on Exception do Application.MessageBox('Cannot save network!',

      'NN Demo',MB_OK + MB_ICONEXCLAMATION);

  end;

end;

procedure TForm1.ButtonLoadClick(Sender: TObject);

begin

   try

      NN.FileName := Edit1.Text;

      NN.Load;

   except

      on Exception do Application.MessageBox('Cannot load network',

        'NN Demo',MB_OK + MB_ICONEXCLAMATION);

   end;

end;