| Map
> Data Science > Predicting the Future >
Modeling > Classification/Regression
> ANN >
Radial Basis Function |
|
|
|
|
|
|
Radial Basis Function Networks (RBF)
|
|
|
| RBF networks have three layers: input layer, hidden layer and output layer.
One neuron in the input layer corresponds to each predictor variable. With respects to categorical variables,
n-1 neurons are used where n
is the number of categories. Hidden layer has a variable number of neurons. Each neuron consists of a radial basis function centered on a point with the same dimensions as
the predictor variables. The output layer has a weighted sum of outputs from the hidden layer to form
the network outputs. |
|
|
|
|
|
|
| |
|
|
Algorithm
|
|
|
| h(x) is the Gaussian activation function with the parameters
r (the radius or standard deviation) and
c (the center or average taken from the input space) defined separately at each RBF unit.
The learning process is based on adjusting the parameters of the network to reproduce a set of input-output patterns. There are three types of
parameters; the weight w between the hidden nodes and the output nodes, the center
c of each neuron of the hidden layer and the unit width
r. |
|
|
| |
|
|
| Unit Center (c) |
|
|
| Any clustering algorithm can be used to determine the RBF unit
centers (e.g., K-means clustering). A set of clusters each with r-dimensional
centers is determined by the number of input variables or nodes of the input layer. The cluster
centers become the centers of the RBF units. The number of clusters, H, is a design parameter and determines the number of nodes in the hidden layer. The K-means clustering algorithm proceeds as follows: |
|
|
| |
|
|
- Initialize the center of each cluster to a different randomly selected training pattern.
- Assign each training pattern to the nearest cluster. This can be accomplished by calculating the Euclidean distances between the training patterns and the cluster
centers.
- When all training patterns are assigned, calculate the average position for each cluster
center. They then become new cluster centers.
- Repeat steps 2 and 3, until the cluster centers
do not change during the subsequent iterations.
|
|
|
| |
|
|
| Unit width (r) |
|
|
| When the RBF centers have been established, the width of each RBF unit can be calculated using the
K-nearest neighbors algorithm. A number K is chosen, and for each center, the
K nearest centers is found. The root-mean squared distance between the current cluster
center and its K nearest neighbors is calculated, and this is the value chosen for
the unit width (r). So, if the current
cluster center is cj, the r
value is: |
|
|
|
|
|
|
| A typical value for K is 2, in which case s is set to be the average distance from the two nearest
neighboring cluster centers. |
|
|
| |
|
|
| Weights (w) |
|
|
| Using the linear mapping,
w vector is calculated
using the output vector (y) and the
design matrix H. |
|
|
|
|
|
|
| In contrast to the
multi-layer Perceptron, there is no local minima in RBF. |
|
|
| |
|
|
| Exercise
|
|
|
|
|
|
|
|
|