Object Recognition

Participants: Greg Cohen, Luis Camunas, Ryad Benjamin Benosman, 'Jon Tapson', Michael Mathieu, Garrick Orchard, Francisco Barranco In this project a Spiking Neural Network (SNN) was implemented to visually recognise an object and provide its location and orientation in the visual field. A spiking silicon retina was used as a front end sensor, with the SpiNNaker platform used as a computational hardware platform for implementing the SNN. The model was constructed using PyNN in Python on a Linux virtual machine.

A simple Leaky Integrate-and-Fire (LIF) neuron model with refractory period was used. The neurons were implemented on SpiNNaker to create an event driven network where a neuron only updates when it receives an input. Each neuron update was initiated by an interrupt indicating that it had received a new input spike. Neuron updating consists of 4 steps. First, check if the neuron is under refraction, if not, proceed to step 2. Second, calculate and implement the neuron membrane potential decay which has occurred since the previous update. Third, factor in the effect of the input spike which has just been received. Fourth and finally, check if the spiking threshold has been exceeded and generate an output spike if necessary.

Below are example images of the input to the SNN for arrows at similar, but different orientations. Pointing at angles of 0, 15, and 30 degrees respectively. Grey regions indicate no data. White and black regions indicate areas increasing or decreasing in intensity respectively.

The first layer of the SNN consists of orientation extraction, implemented using Gabor shaped convolutional kernels. Below is the result of orientation extraction using four orientations for the same arrows as above. The horizontal and vertical axes represent spatial location in terms of pixels. Dots represent spikes and color represents orientation, with 0-165 degrees represented in steps of 15 degrees as 'blue', 'red', 'yellow', 'green', 'cyan', 'magenta', 'black', 'orange', 'purple', 'blue', 'red', 'yellow'.

The second layer of the SNN performs pooling and subsampling of the first layer. Lateral inhibition between orientations in this layer acts as a Winner-Take-All (WTA) ensuring that only the maximally responding orientation is passed through to the next layer. The second layer outputs are shown below for the same three arrows. Colors are the same as before.

The third layer of the SNN integrates responses from all orientations to create more complex features used to recognise objects. Lateral inhibition between features in this layer acts as a Winner-Take-All (WTA) ensuring that only the maximally responding feature (object) responds. 'blue', 'red', and 'yellow' represent these arrows of 0, 15, and 30 degree orientation respectively, showing that the arrows are correctly identified at the center of the region (location 4,4).

Magenta in the third image above represents misclassification as an arrow oriented at 75 degrees. Presumably because the 45 degree angles formed by the arrow head cause one of the short lengths of the arrowhead to be oriented at 75 degrees when the main axis of the arrow is in fact at 30 degrees. It is therefore correctly detecting the location and combination of oriented features consistent with a partially occluded 75 degree arrow. This is illustrated below for an arrow of 30 degrees. When we consider the region in the yellow box, we clearly see a 30 degree arrow, hence the correct classifications in the center of the region. However, if we consider the magenta box which is centered towards the lower left, we see combination of oriented features consistent with a partially occluded 75 degree arrow (occluded regions shown in grey).

Objects with their combination of features located nearer to their center would be easier to recognize, because then the magenta box would contain all the combination of features, and the presence of unexpected features (the part of the arrowhead which lies outside the bounding box in the current case) would inhibit response from magenta neurons (sensitive to 75 degree arrows).