Lessons Learned

The Telluride CogRob experiment taught us a few lessons with regard to the design of cognitive systems. They are summarized below:

  1. Infrastructure: The ROS operating system proved perfect for the job. A cognitive system integrates several processes – visual, auditory, control, and language. It needs to have the capability for several communicating processes. Although various models for this are available in the literature, the ROS operating system was adequate. Its wide availability makes it a good starting point for cognitive system design.
  1. Components: Most of contemporary computer vision or audition relies on machine learning using “learning data” provided by a number of databases. It would be nice to use images from the web to do learning and obtain software that you can apply to what your robot sees and hears. This remains wishful thinking: only if the training is done with data from the same environment, “learned” software has a chance of working. Thus, for a variety of reasons, machine learning coupled with perception for the field of robotics is still highly challenged. Current state of the art may give promising results in the field of multimedia where one works with databases, but it is far from producing useful results in robotic perception.
  1. Proof of concept – prototype: CogRob is the first cognitive system integrating vision, sound, action and language/reasoning. CogRob showed that it is possible to build a real time robotic system able to see, hear and cognize, all at the same time. In actual fact, even though many of its redundant components did not function perfectly, the system as a whole – perception, action and cognition – was almost flawless. Of course CogRob dealt with 20 objects and 15 actions, but with minor extensions in software CogRob can deal with hundreds of objects and actions, pointing the way to a new Industry of Cognitive Robotics. To deal with thousands of objects and actions, we would have to advance generalization and learning in the field of robotic perception and action, and this could be perhaps the topic of a future workshop, where neuromorphic learning is used in the CogRob architecture. One can also imagine that the various components of CogRob and their interaction could become much more sophisticated. These ideas constitute a vibrant research program. This program coincides in many ways with the program set forward by the National Robotics Initiative, a multi agency research program that was announced by the National Science Foundation while the Telluride project was happening.