Overview of the results

The core projects, which we pursued during the three weeks were the following three:

The cooking scenario

In this project, we set an ambitious goal of developing a neural-dynamic architecture, which enables NAO to act autonomously in a simplified "cooking" scenario. In a cooking scenario, the robot observes a human performing object-directed actions (grasping, dropping, stirring) and is able to repeat a sequence of perceived object-oriented actions, while being flexible in respect to exact location of objects during sequence generation, tolerate delays and failures of individual actions. Moreover, over time, the robot is supposed to associate shape of the object to a particular grasp type, and learn that shape of the object is the feature that defines function (the object to stir with should be longish, whereas an "ingredient" to drop into a ball is roundish). For the workshop we have developed a simplified version of this scenario:

-- the robot perceives two objects, distinguished by their colour and their shape ('vertical' vs. 'horizontal')

-- the robot is able to perform two types of grasping actions: a top-grasp and a side-grasp

-- the robot is able to distinguish the top-grasp and the side-grasp, performed by a human from its vision

-- the robot has a behaviour organisation system, which allows it to autonomously generate a following action sequence: look at an object, reach for the object, grasp object, lift object, drop object. The robot is able to 'parse' such action sequence from vision in terms of filling-in slots, such as colours and shapes of the target objects of actions and the used grasp type.

The goal of the project is to learn to use a correct grasp type for the two shapes of the objects from observing a human performing these grasp types. First, the robot detects the grasp type from the motion analysis of the scene (vertical vs. horizontal 'hand motion flow') and detect the shape of the object, at which the grasp is directed. Later, the robot learns an association between the grasp type and the object's shape.

Work on this simplified scenario required development of several individual modules, which we present one by one in the 'Individual Project' section. We have integrated some of these modules in the final (for the three weeks) architecture, whereas we plan to continue this work beyond the workshop and bring this work to the ultimate goal of autonomous behaviour and learning.

Simultaneous Planning and Action in Real-world Environments

Contributors: Daniel Neil, Mathis Richter, Erik Billing

One of the key advantages of Simultaneous Planning and Action is the ability to dynamically re-route through an environment. In this project, a webcam was attach to view a real-world environment (with blue paper for walls, a white paper plate target as the goal, and a virtual robot), and the SPA algorithm running on dynamic neural fields. A human player could interact with the environment by moving around the walls or goals dynamically, and the virtual robot will react and re-plan within the new constraints.

The demonstration successfully shows the robot navigating around the environment, even as the environment changed.

Code was also written for an e-puck robot. Unfortunately, the north-south-east-west directions need to be converted into rotation and propulsion, and the included code applies the hough transform to orient the puck along a red line. A black spot at the end of the robot shows its current heading. Surprisingly, this turned out to be quite robust.

However, the "action" map's peak lies too close to the robot's current location, so an accurate estimate of the intended heading is difficult to obtain. The quantization error of the field and the shortness of the line result in a desired angle could swing as much as 50 degrees between timesteps. So, while the robots heading could be accurately obtained, the desired heading could not be without parameter tweaking or re-architecting; this should by the next work to implement.

The code for this can be found in the attachment [erickSPA.zip Download]

Hebbian learning and self-organization with DNFs

Contributors: Thomas Trappenberg, Erik Billing, Mariel Alfaro Ponce, Yulia Sandamirskaya

Several members of the DNF group discussed and implemented Hebbian learning between DNF fields. Discussion of such learning go back at least 40 years to Willshaw and von der Marlsburg (1973). They used a model of a cortical sheet that is equivalent to the DNF equations. Indeed it is good to stress again that the DNF model is a generally accepted model of population dynamics in cortical maps that is now frequently used to explain neurophysiological findings. The exploration of the use of such biological mechanisms to engineer applications as therefore a great focus within the neuromorphic engineering community.

We have been able to both, re-implement the original Willshaw-vdMarlsburg model as well as translating this into a form with more continuous implementations. The original modeling was of course quite restricted given the state of the computer power at this time. It was therefore good to see the original model come to live in a way that it can be viewed as a movie of self organizing maps that are more commonly known in the Kohonen approximations.

The basic mechanisms described by this model is the learning of mapping functions between different maps. This connectivity matrix can be learned with basic Hebbian leaning given that there is a stable input bubble and bubble in the receiving map. While this is used to self-organize the topographic representation in a feature map, the general mechanisms of Hebbian learning is the same in a supervised scenario that we wanted to explore further in several projects.

In order to facilitate better exploration of such learning mechanisms and to integrate them in robotics environment, we implemented a Hebbian matrix module in CEDAR, which is the software framework developed at Ruhr University Bochum.

There are two specific subprojects for which we envision applications. One is the application to learning a motor plant, the other is as decision model for EEG audio recognition.

Motor plants describe the mapping between motor commands and the resulting change in pose. Such models are typically described mathematically based on the physical constraint of system, but nature shows us that we have to learn it since biological agents are also able to flexibly adapt to changing physical constrains. Using such biological-inspired adaptation of motor plants would make engineered system much more robust to wear and tear and to failure of subcomponents. We had a eye movement model with a laser pointer for which we wanted to learn this transformation, but we could not finalize this implementation due to time limitations.

A seconds subproject which we looked into is to feed EEG data in response to audio stimuli in order to recognize different speakers from these signals. Neural field models have been shown to describe such noisy decision making in the brain. To adapt the DNF model to this task we need to learn the mapping of the signals to a DNF representation. This is hence another example of supervised learning between different representations. We have implemented as basic model that demonstrate the learning of a diagonal mapping functions within the DNF framework, and we can now explore its behavior for more complicated mapping functions.

Individual Projects

DVS UDP Cedar Plugin

Contributors: Daniel Neil, Mathis Richter

The goal of this project was to quickly interface the jAER software framework with the Dynamic Neural Fields framework implemented in cedar. The code for this project can be found here:


The basic idea is to open an input stream in jAER, then pipe it over to cedar. This allows controlled reproduction of behaviours (i.e., playing logged data files), the full suite of trackers / orienters / filters, as well as live streaming data from the DVS camera. For a more encapsulated DVS approach, see the eDVS plugin below.

eDVS Cedar Plugin

Contributors: Julien Martel, Yulia Sandamirskaya

In this project, we have implemented a cedar plugin for eDVS, which uses a simple, but powerful driver, written by Julien, which allows to receive eDVS events through a usbserial port. The plugin implements a cedar step, which is a listener to the eDVS events. The events between iterations of the cedar framework (when events are "picked-up" by DNFs of the overall architecture) and zeros the events buffer when the events are passed over to the DNFs. The code for this project can be found here:


The screenshots gives you an impression of a minimal DVS2DNF architecture:

screenshot of the eDVS plugin screenshot of eDVS 2

The three screnshots demonstrate how by increasing the strength of lateral interactions in the DNF, you can force the field in making decisions about the most salient region in the scene in terms of amount of motion.

Optical Flow estimation for neural fields

Contributors: Francisco Barranco, Mathis Richter, Yulia Sandamirskaya

The main goal of this subproject is building a module for CEDAR that estimated optical flow from the input (camera). In this way, we can fed DNFs with information about the direction and the speed of objects in the scene. After this implementation we are able, for example, to track a hand and direct the attention of the NAO robot to an the closer object in the table towards the direction of the hand is pointing. Furthermore, we can also roughly estimate when the contact between the hand and the object is going to happen, and then we have more precise for the response.

About the optical flow, we have implemented two different versions:

  • A very fast version that is based on Farneback (Farneback03). It is local sparse implementation that builds a pyramid with the input to tune different sizes and speeds.
  • A second version that is slower but that estimates 100% dense results. This global variational method is based on the L-1 norml, especifically Dual TV L1 (Sanchez12).

The results of the optical flow running in real-time are shown in the next figure, using as input the integrated webcam of the laptop.

Farneback03: Gunnar Farneback, Two-frame motion estimation based on polynomial expansion, Lecture Notes in Computer Science, vol. 2749, pp. 363-370, 2003.

Sanchez12: Javier Sanchez, Enric Meinhardt-Llopis, Gabriele Facciolo. “TV-L1 Optical Flow Estimation”, 2012.

Motion estimation with eDVS and DNFs

Contributors: Julien Martel, Yulia Sandamirskaya We have used the developed cedar plugin for eDVS sensor to implement a motion-direction estimation algorithm as a (faster) alternative to the more traditional CV approach. We have calculated the (dynamic) centre of mass of the events coming from eDVS in real time and have used a DNF to track the moving object (hand). A memory trace of this DNF kept the history of the recent locations of the centre of mass of the moving blob, which aspect ratio was evaluated to estimate the prevalent direction of motion in the image. This value may serve as the detector of the grasp-type in the cooking scenario. The screenshots below show this version of the movement detection architecture: right -- detection of a horizontal movement, right -- detection of a vertical movement:

We have also tried out another strategy in motion estimation, calculating the gradient of the centre of mass of the eDVS events and using this gradient to compute the prevalent motion direction. This architecture looked like this:

Orientation estimation

Contributors: Ashley Kleinhans, Yulia Sandamirskaya, Mathis Richter

As a proof-of-concept implementation of shape-detector, we have developed a simple architecture that classifies whether a longish object is oriented horizontally or vertically in the camera image. The input color image is converted to a grayscale image, which is binarized using a threshold function. The result is convolved in two separete streams with a vertical and a horizontal filter. The responses from these convolutions are then used to scale Gaussian inputs to a dynamic neural field, which makes a selection decision based on which of the two orientations (horizontal or vertical) has more evidence in the image. Although the initial estimate is very noisy, the DNF stabilises the decision and makes the detected shape available to influence the other parts of the architecture. The CoS DNF confirms that a selection decision is stabilised.

To improve the classification, we set an attentional focus for the shape-selecting module based on the colour-based segmentation, performed by the perceptual color-space DNF of the architecture. The output of this three-dimensional neural field has a peak over the location of the attended object.

The following screenshot shows the original image taken from the robot's camera (top left), focused on a yellow wooden block that appears horizontally in the image. To the right of the image you see the activation of a three-dimensional dynamic neural field, which is defined over the two spatial dimensions of the image and a color dimension. In the plot, the color dimension was projected onto the image space (with a max operation). The activation is shown as a heat map with red colors representing suprathreshold and blue colors representing subthreshold activation. To the right of the plot, you can see the sigmoided output of the fields, which has values between 0 (blue color) and 1 (red color). The line plot to the right of that shows a one-dimensional fields spanning an abstract orientation dimension. If the attended object appears vertical in the image, the peak is on the left side, if the object appears horizontal, the peak is on the right side. Here you can see that the horizontal object has been classified correctly.

The following screenshot shows the same plots as above when presenting the robot with a vertical object. The plot of the one-dimensional dynamic neural field (rightmost) shows that the orientation of the object has been classified as vertical.

Even though the orientation estimation is somewhat noisy, the dynamic neural field stabilizes a selection for a particular orientation through its dynamics.

Behavioral organization

Contributors: Mathis Richter, Erik Billing, Yulia Sandamirskaya

As many robotic scenarios, the cooking scenario consists of a number of discrete behaviors (looking for an object of a particular color, reaching for the object while opening the hand, and grasping the object by closing the hand. All these behaviors have to be represented and organized in time. They have to be activated when necessary and possible, deactivated when finished, and in some cases, organized in a sequence.

We have built a behavioral organization system for our cooking scenario, based on previously established principles (Richter, Sandamirskaya, Schoner, 2012). We have developed a hierarchical structure of behaviors, where higher level behaviors (e.g., reaching for an object) are composed of behaviors on a lower level (e.g., moving the end-effector along a single coordinate). The activation state of each behavior is represented by a dynamic neural intention node (a zero-dimensional field with similar dynamics). Each behavior also has a condition-of-satisfaction, a signal that the behavior has been successfully executed. The condition-of-satisfaction suppresses the intention and effectively deactivates the behavior.

For this simplified architecture, the sequence of behaviors is encoded by a chaining approach (a finished behavior A activates the next behavior B). We have previously introduced a more flexible approach where the chaining connections can be activated and deactivated based on task input.

The following diagram shows the behavioral organization system that we used in the architecture.

The higher level elementary behaviors consist of an intention and condition-of-satisfaction node and activate behaviors on a lower level. The lower level does not have condition-of-satisfaction nodes because the condition-of-satisfaction is represented on the higher level already. The lower level has two behaviors for each dimension to account for a movement that sets the arm of the robot back to an initial position. All behaviors connected to this movement are suppressed as soon as other behaviors are activated that generate movement to an object.

The following screenshots show courses of activation (green lines) of all high-level behaviors over time when grasping a horizontal (first screenshot) and a vertical object (second screenshot). You can see the sequence of behaviors. You can also see that different types of reaching (top-reach and side-reach) are executed, depending on the orientation of the target object.

Learning timing of satisfaction

Contributors: Boris Duran, Yulia Sandamirskaya

The original idea with this project was to use the real Khepera or ePucks to implement the model for learning the timing of the condition of satisfaction in a given task. In this case we wanted to allow the mobile robots to reach a certain position in their environment and after a few repetitions of the same task, assuming the goals where at approximately the same distances in all trials, the system would learn how much time it would take to travel that distance. If something in the environment changes and the agent can not reach the goal at the previously experienced mean time, a new signal will fire known as the condition of dissatisfaction.

The greatest problem we had was with the communication with the real robots... nothing we did in 3 different OS worked so we decided to change to simulation. We used the camera input and extracted the position of an object using a very simple color segmentation algorithm. A one-dimensional field working with color gives input to a two-dimiensional field that associates color and orientation of the robot. The peak formed in this 2D-field was extracted in the orientation dimension to feed another one-dimensional field from which a single desired angle is extracted and used to compute the speeds of the mobile robot.

Parallel to this block, a different algorithm runs along and it's in charged of learning the amount of time the robot takes to reach the goal. One of the main components in this algorithm is a travelling peak moving inside a one-dimensional field representing time. This moving peak will stop the moment the agent reaches the goal and when doing so a memory layer will build up a bump at that location. This bump is used to modify the integration time of one of the nodes in the architecture which controls directly how fast or slow the competition between satisfaction and dissatisfaction is done.

This architecture could be easily included as part of an elementary behavior within a sequence, thus learning the time each elementary behavior and allowing the system to react when unexpected circumstances could break at any point in a sequence.