Recognizing Complex Concepts in Robotics and Data Mining

Recognizing Complex Concepts in Robotics and Data Mining

Dougal Sutherland

A major problem in robotics and data mining is being able to recognize complex concepts in the presence of other objects which aren't important. For example, a robot trying to cook a meal in a cluttered kitchen needs to be able to tell that the frying pan on the stove is important right now, while the sink just doesn't matter at the moment. Many of the required concepts are inherently relational (you need to reason about the controls for the burner that the pan is on, not necessarily any of the others) and multidimensional (color, for example, is an inherently three-dimensional concept). It's also advantageous if it can do this without necessarily having been taught beforehand exactly what everything is; it should be able to recognize concepts as needed, dynamically.

Many of these same techniques can also be applied to completely different domains: predicting whether a given storm will produce a tornado, or playing the game of Go. In our research, we worked mostly within a block-world domain, where colored blocks are stacked on top of one another.

The Spatiotemporal Multidimensional Relational Framework (SMRF), developed by Dr. Andrew Fagg and Matt Bodenhamer at the University of Oklahoma, attempts to solve this problem. Dougal worked with them as part of an NSF-sponsored Research Experience for Undergraduates. In addition to general improvements to the framework, his core research focus was on taking advantage of cues from teachers. The person training the SMRF system probably doesn't know exactly the right way to represent the problem -- after all, then they could just code it directly -- but they probably do have some ideas for things that are important. If you're showing a robot how to make a stir-fry, for example, you know that the heat should be around a certain level, that the various ingredients you put in are important, and so on. We can use this information to bias our search towards areas that we think will be important, while still maintaining the full flexibility that the SMRF model affords us.