Spike-Based Robotics Discussion, Saturday July 3, 2010

Discussion Leader: Jorg Conradt

Speakers: Mark Tilden, Dan Lee, Tobias Glassmachers

Conradt intros: This is about robotics, another slot in 10 days. It's about getting behaviour into machines. Different approaches: two talks with two different ways, setting the stage for some discussion. After break, Tobias.

10:30A - Mark Tilden:

Using PowerPoint?! Biomorphic humanoid marketing. Robosapien: 2004. Now full-size artificial humanoids.

-Femi-sapien: 5 motors, 6 batteries, 100 hours lifetime. Bmech + setpoint servo xtol. Femi body design. Emotive interaction. Pushing the creepy barrier. New-generation processor. Andreas A. objects to 20 mHz, 13m process technology ;). Femi nervous network: one PCB. Head very level in movement. Walking turn; step around something. Step turn. Conradt: is this coded or emergent? Tilden: set up oscillators, find different gaits. Shaking the hand of Asimo: it may fall. Femi: great behavior. 2-motor active arm complex. Application: writing on paper. 100 hours off of AA batts! Femi Pollock drawings. Femi fencing: Programmed by moving (animating) robot motors manually. Uses back-EMF. There is a remote control for femi: sapien remote can control her. She has lots of hidden secrets. Chopsaki video banned from youtube. Femi is foil for robosapien. Interactive diva. Only one burp. Femi clothing problem. Big in Japan, American sales very slow. 100K sold in Japan, controversial. Head of Bandai loved it. Femi market results.

-Successor: JoeBot?. Voice filter (1994). Three channel volume. Can recognize shape of fricatives. 5 motor structure. Basic functions from femi, turned it around backwards. Gave it an expressive face. Built-in instruction manual. Battle mode. Beatbox function. Conradt: sound or mechanical? Tilden: one mic in head. Good voice recognition (stop, back, robot)! User voice training. Hand scanning: eyes in his hands. Shoots out directed beams from hands, detector in chest. $80, 50K units sold, best to date but too late?

-Marketing tips: Nothing moves faster than a reliable prototype. Avoid complexity. Reduce your costs. Know your market. Technology is transparent (they don't care how it works, only that it does). You have to design for everyone: has to appeal to lowest common denom. Taber's law: Machines that piss people off get murdered. If you have a weird idea for a toy, changes are you're too late.

-Complexity rarely scales downwards. Scaffold frames yield significant insights to impress sponsors. You are the champion of your own technology.

-Future directions: WW Roomscooper (2010). Wall-e style compulsive cleaner. Picks things up, dumps them into bin, dumps bin in corner. Problems in Toyland. Robots too expensive for toys. Still sell 1 million robots per year. Now going after consumer electronics. Large male robots get the s**t kicked out of them. Uncanny valley: unsellable due to 17 finger traps and pinchpoints in average hot supermodel. Entering the creepy zone.

-Femi future: 14 motors, 5'7", 44 lbs, 300 hr battery life. Neuro-app biz model. US$1000. Need $200M for marketing campaign. Tele-tourism with camera in head? Exclusion zones for avoiding criminal behaviour.

-Discussion: Question: how to get robot to do a two-step? Tilden: robot obeys person with remote control. Robot is an app platform, using nervous nets: what kind of things should you put into his head? Heading to airport. Conradt: was hoping to hear about digital control. Tilden: not a Luddite. Robot is nothing more than a mouse.

11A - Dan Lee: different approach.

-Soccer robotics videos. Quadrupeds, autonomous Aibos. No offsides in robot soccer. Penn scores!! All autonomous, wireless comm between robots, or auditory comm if no WiFi?. Last week in Singapore: new team humanoid soccer. Team DARwin. Completely autonomous, no eye of God. Humanoids are slow compared to the dogs.

-For discussion: how we build these types of systems, what the limitations are, how biology can make something better, what is required to build the better system? Humanoid robots have omnidirectional walk: 100 lines of Matlab!! Tobi: robocup degenerated into a mobility/kicking speed contest. Lee: how important is team play? Special hardware for kicking is a huge factor. New competitions: hardware is the same, software is where "intelligence" is.

-At the high level, almost all robotics is a finite state machine, transitions driven by events. For a soccer striker, find the ball, see the ball, go to the ball, ball close: position near ball, dribble, kick (see photo of state diagram). Ralph: why not use a panoramic cam? Lee: not allowed to. Tobi: why not much machine learning? Lee: typically not much, very low dimensional. Question: use learning for motor control? Lee: machine learning can be used to tune params of walk, kick. Stocker: model for ball? Lee: not really, use Kalman filter with velocity and position. Goalie only has 2-3 camera frames as ball goes by. Tobi: to what extent hierarchical for team design? Lee: Have a body and head state machine somewhat independent. Head tracking ball. Can have lower states inside that control different parts. Can encapsulate into a team: defender, striker. Different strategies for different player types. (See photo) Tobi: what's the trickiness here? Lee: cooperation between players, so they don't all do same thing at same time. Bid according to how close you are to the ball. Distributed, no central authority. Question: is there a benefit to different robot physical abilities? Lee: typically homogeneous. Can bias natural problems with robots, but with truly heterogeneous robots, much weight switching between behaviors. Tobi: state machines are a giant "if" statement, do you ever use neural networks? Lee: each state is a Matlab function, strings come in as events to finite state machines. Tobi: philosophy is strings for human-readable debugging? Lee: table to "monkey with" to change behavior. Ralph: not paying attention to plant at all, all about high-level control? Lee: we built humanoid, sometimes buy them. Still lots of parameters to adjust. Nir: planning is local, is there global optimization? Lee: could do a more centralized planner, but have comm problems. For complicated planning, problem. With comm delays, do much worse than simple distributed system. Bert: how do you localize yourself on the field? Lee: at low level, vision system builds map of where you are, goal, etc, at frame rate of camera. Kick the ball means knowing where you and ball and goal are. Have to have world representation:

-Ernst: no passes? Lee: As ball passes, robot goes after ball but not really passes. Steve K.: what is defender doing when striker has the ball? Lee: can also share ball info quickly. Defender can know where striker has the ball even if can't see it. Use potential field control for that. Conradt: talking about specific implementation, but is there a more abstract view? Lee: trick is designing properly all the states, emergent behavior comes out. Most teams now don't adapt to other teams strategy. Should do. Conradt: is it just the state machine, or is there an abstraction? Lee: Even for driving, just need larger state machine. Ernst: other than robot soccer, what can we learn? Lee: this is the same technology as computer games. Alien is running state machine. To build more intelligent robot, is it just building a better state machine? Or is there something completely new? Tobi: we know there is this kind of ability in the brain, but how? Lee: state machine is a universal computer. Tobi: don't want to embed state machine in spikes. Article in Spectrum about AI in computer games. Problem: people didn't care that machine had AI, discounted it as soon as it made a mistake. Can't assign credit to state transition. Lee: state machine easier to diagnose problems. Tobi: same problem with AI in building automation. Tried to put machine learning in, 10 MS students, but people got angry when it did something they didn't expect. Mistake is 1000x more expensive.

-Steve K: looks very deterministic. If deliberately insert stochastic, is it better? Lee: It is stochastic. Can put probability of transition (Markov decision process). If do learning, typically put stochasticity, go on transition 90% of time. Learn adjustable weights for likelihood of transition. Tobi: is there a lot of room to play? Lee: main thing: designing properly the state machine. Hard to see how to learn that. Transitions easier. Question: whole system or blocks? Lee: putting skills together, all together are important. Stocker: what functions are always running? Lee: low-level controllers always running, 100 Hz balance, vision, etc. Conradt: moving towards learning. Question: if had pass option instead of kick, better? Lee: just more complicated state machine, but basic skill of passing hard with two robots, unless wheeled.

-Conradt: 10-min break.

11:50A - Tobias Glassmachers:

Conradt: a different approach.

-Tobias: machine learning: a subset of the "cognition" box. What are we talking about? having a machine tune the parameters instead of a human. What problems typically looked at, what data? Open: how we start. Much of machine learning from static data. From image sensors, engineering, etc. Assumption at least, each pattern is coming from fixed but unknown distribution. This is contrast to sequences. Learn how to play a game. Moves highly correlated. Online learning: If you want to adapt quickly on the fly, or offline. One relatively well-understood class: supervised learning. Have some input, image, sound, complicated or simple. Map it to some output (target values). (See photo) Another approach: unsupervised. Just have inputs, not given targets. Understand structure in inputs. Could do PCA or neural inspired, self-organizing maps, clustering methods. Input becomes pre-processed data rather than targets. Another approach: reinforcement learning. Linked to sequence learning. Map states or observations to actions. More detail on reinf. learning. RL: environment fuzzy, in a cloud. Agent: perhaps small robot. (see photo) Robot gets state from environment, feeds back through actions. Also gets reward.

-Can make a table from current state to current action. (State machine approach). Question: where does semi-supervised fit? Tobias: just a general diagram. Semi-s: inputs are cheap. Try to learn from distrubition of inputs. Not trying to cover everything. First important step: table representation (state, action). Going to adapt maps by learning. Can use linear architectures, but can use non-spiking ANNs, can use fancy kernel representations, graphs, Bayesian reasoning, gradient descent. Can this solve my problem at all? Second important ingredient: what is my goal? Formulate in an error or loss-function or in RL, reward over time. Closely related to representation. Decompose into structural error (if had magic learning rule to pick optimal params, how well could you do?), learning error (within my class of reps, how well can I do?). Steve K: variance important? All into structural error? Tobias: no, learning error. If insufficient data, can't expect to learn. Lee: statistics: bias error. Tobias: yes, all the same thing. Third important component: the learning rule. Q-learning, etc. Not going to focus on this.

-What do we learn? One more important thing: what statistical learning theory tells us. Don't have infinite data. Can pick a representation with lots or few params. If model very flexible, may overfit to data. Learns all the noise because model too complex. If lots of params in model, also need lots of data. Want not too many degrees of freedom. Conversely, want lots of degrees of freedom to avoid error. Key: not too complex, but pretty good solution in this class. Nir: can also parameterize the error, Gaussian? Tobias: effectively reduces complexity of class of models. This is regularization. Important to be aware of this tradeoff. Conradt: have to understand the problem first to come up with correct representation. Tobias: Yes, bad news. What machine learning literature is about. Lots of different methods. All kinds of magic in representation, very very important.

-Controversial claim: how does this relate to the workshop? We want to do cognition for robotics. Just pick right data structure, do reinforcement learning. Is this cognition? (See board photo) Nir: error has lots of information. Reaching task: endpoint error much less than joint errors. Want a high certainty on endpoint, but not high-level certainty on each joint. Tobias: is that cognitive? Tobi: Let's not get into that. Tobias: that means you have smart motor controller. How do you design such a controller? Most important is representation. Nir: human doesn't match all the state, just some. Tobias: don't try to build or mimic the brain. ANNs in machine learning don't look like brains. Tobi: they do in many ways! Tobias: just building the best machine we can. Nir: can show data on better error function. Question: from NM point of view, can we learn those representations? Tobias: there are approaches to learn also the representation. Typically evolve topology with evolutionary alglroithms, like in nature. Very slow, has many problems, works only for small sizes. Doesn't scale. Shih-chih: how do you know whether representation enough for the task? Tobias: we don't know without trying. Tobi: is there a ML engineer? Tobias: don't know. Malcolm: yes, there are, Yahoo hires them. Malcolm: new genre about computational advertising; use ML to determine what ads to show to people. Billion-dollar industry, much bigger market than robotics.

-Tobias: is this already cognitive behavior? Tobi: if you didn't read last year's 3-day discussion before the workshop, don't say the word. Conradt: out of time, game about to start. Question: need conceptual changes? Tobias: approach problems in these terms, many don't. Comment: does not inspire any thinking about how the brain works. Tobias: you may limit yourself to biologically plausible representations. Comment: can definitely use machine learning to understand what features are extracted. Comment: figure out what acutally happening, but missing how brain is doing that. Comment: Have to try a variety of techniques. Tobias: may find something better than the brain!! Brain is the best model we have.