Organizers:: Yulia Sandamirskaya (RUB) Erik Billing (HIS)

Slides from seminars

Slides from Erik's seminar on planning with DNFs are found at  http://www.cognitionreversed.com/telluride/nd14.

Other slides can be found as attachments at the bottom of the page.

Invited Guests

Prof. Thomas Trappenberg. Faculty of Computer Science, Dalhousie University, Halifax, Canada (computational neuroscience, author of a text-book on theoretical neuroscience).

Prof. Estela Bicho. Engineering School, Dept. of Industrial Electronics, University of Minho, Guimar\~aes, Portugal (behavioural robotics, neural-dynamic approaches to robotic, human-robot interaction and collaboration).

Prof. Wolfram Erlhagen. Dept. of Mathematics for Science and Technology, University of Minho, Guimar\~aes, Portugal (foundation of neural fields dynamics, human-robot interaction and collaboration)

Prof. Gregor Schöner. Director of Institute of Neural Computation, Ruhr University Bochum, Germany (Dynamic Field Theory, embodied cognition, development of sensorimotor capabilities, theory of motor control, cognitive robotics)


Grasping: learn an association between visually perceived object features (its aspect ratio) and a grasp type (Project leader: Ashley).

Parsing for cooking: Temporal segmentation of actions to inform robot of a correct action sequence (Project leader: Francesco).

Learning and acoustics: Learning sensorimotor transformations with application to action segmentation based on acoustic signals (Project leader: Tom).

Dynamic SLAM and SPA: Learn a map for simultaneous navigation and planning (Project leader: Danny).

Controlling NAO with EEG: Using the pre-processed EEG signal as input to DNFs to control a robot to look in the direction of the attended source (Project leader: Vikram).

Planning with Dynamic Neural Fields: from Sensorimotor Dynamics to Large-Scale Behavioral Search

This topic area will explore different models and architectures based on our recent advances within the neural-dynamic approach to cognition. Dynamic Field Theory (DFT) is a mathematical framework based on dynamic systems, describing neural behavior at a population level. We strive to use DFT as a bride between detailed spiking models and large scale simulations working as a complete cognitive controller for a physical robot. Two Aldebaran’s humanoids Nao and three ePucks will be available during the workshop. Other hardware may also be used. The main theme of the topic area is planning, but since we make an effort to always work on complete systems other aspects of cognition is also considered.

We will offer a number of tutorials on DFT, including recent architectures for behavioural organisation, planning, and learning. We will organize a series of discussions to establish the relation between Dynamic Neural Fields (DNF) and spiking neuron networks, as well as the feasibility and need for implementing DNFs in neuromorphic hardware. We will also explore the relation between modeling learning processes from neural and behavioral view points. There will also be some robotics and dynamical systems tutorials, as well as lectures about cognition and development.

This topic area includes the following projects (you are also welcome to suggest alternatives within the broader theme of this topic):

1. Simultaneous navigation, path planning, and map learning with neural fields. This project will use a recently developed DNF architecture SPA (Simultaneous Planning and Action) and extend it with a real-world sensing capabilities and a learning module, which will allow the robot to acquire the map of the environment from experience-based exploration, memory trace formation, and reinforcement learning. Here are some snapshots from the before-the-workshop state of work, more details can be found in Billing et al. (2014 - see Attachments). The project may use our Matlab neural-dynamic library cosivina and webots simulator to get started, controlling real ePuck robots is the goal, however.

2. Planning in heterogeneous action spaces: reaching, grasping, and object manipulation. In this project we will implement the DNF SPA architecture to control the Nao robot and plan sequences of arm and hand actions to manipulate (small) objects on a (small) tabletop. We have some software prepared to control the robot and perform looking, reaching, and grasping actions with a neural-dynamic architecture, but there's still lot of work to be done to enable sequence planning in this framework. This project will use our c++ library and GUI-based neural-fields simulator cedar.

3. Learning conditions-or-satisfaction of the intended actions. Within DNF framework for generation of behavioural sequences (Richter et al. 2012 - in Attachments) -- the same framework, on which the SPA architecture is base -- we will learn conditions, which signal (predict) accomplishment of actions. This will include learning timing of actions, but also learning their sensorimotor consequences. This project may use cosivina or cedar.

4. Learning to look. Some of us are interested in working on learning sensorimotor transformations, gains, and predictive networks, involved in looking behaviour (LookingChapter?2014.pdf in Attachments). We will try to bring the recently developed and extended during the workshop models onto the Nao robot.

5. Hardware implementation of DNFs. We plan to organise several discussions and hopefully hands-on sessions on implementing dynamic neural fields on neuromorphic and other hardware, available at the workshop.


We strongly encourage those of you who are interested in this topic area to look into at least one of the following software frameworks. These frameworks will used extensively during the workshop:

1. Cedar  http://cedar.ini.rub.de/ - A rich tool for working with dynamic neural fields. Cedar provides a graphical programming interface where you can connect and adjust DFT components during runtime, without writing code. It's a powerful tool for modeling and it is interfaced to several robots, including the Nao. Cedar is implemented in C++ and can be extended with custom C++ modules if necessary. We recommend that you install and run Cedar under Ubuntu, but it can also be installed on OSX even though that requires several dependencies that has to be installed separately. Cedar comes in several versions, we recommend using  https://bitbucket.org/cedar/development. With Cedar installed, you may also want to download the Nao Plugin  https://bitbucket.org/mathisrichter/naoplugin.

2. Cosivina  https://bitbucket.org/sschneegans/cosivina - A Matlab library that facilitates implementation and plotting of DFT models. With Cosivina, you can quickly wire up DFT models using standard Matlab scripts. If you have prior experience with Matlab you will quickly get started. While allowing non-Linux users to get started more quickly than with Cedar, Cosivina is subject to several generic drawbacks with Matlab and is not interfaced to the physical robots. Cosivina is however interfaced with the Webots robot simulator (using Webots' Matlab API). In order to run Cosivina, you obviously need a Matlab licence, but in other respects there are no specific requirements.

You might also want to browse through lectures and exercises available at  http://robotics-school.org to familiarise yourself with the DNF framework.