Autonomous Tool Use and Learning of Tool Affordances by Robots

The ability to use tools is one of the hallmarks of intelligence. Tool use is fundamental to human life and has been for at least the last two million years. We use tools to extend our reach, to amplify our physical strength, to transfer objects and liquids, and to perform many other tasks. A large number of animals have also been observed to use tools (Beck, 1980). Some birds, for example, use twigs or cactus pines to probe for larvae in crevices which they cannot reach with their beaks. Sea otters use stones to open hard-shelled mussels. Chimpanzees use stones to crack nuts open and sticks to reach food, dig holes, or attack predators. Orangutans fish for termites with twigs and grass blades. Horses and elephants use sticks to scratch their bodies. These examples suggest that the ability to use tools is an adaptation mechanism used by many organisms to overcome the limitations imposed on them by their anatomy.

Despite the widespread use of tools in the animal world, however, studies of autonomous robotic tool use are still rare. There are industrial robots that use tools for tasks such as welding, cutting, and painting, but these operations are carefully scripted by a human programmer. Robot hardware capabilities, however, continue to increase at a remarkable rate. Humanoid robots such as Honda's Asimo, Sony's Qrio, and NASA's Robonaut feature motor capabilities similar to those of humans. In the near future similar robots will be working side by side with humans in homes, offices, hospitals, and in outer space. It is difficult to imagine how these robots that will look like us, act like us, and live in the same physical environment like us, will be very useful if they are not capable of something so innate to human culture as the ability to use tools. Because of their humanoid "anatomy" these robots undoubtedly will have to use external objects in a variety of tasks, for instance, to improve their reach or to increase their physical strength. These important problems, however, have not been well addressed by the robotics community.

Another motivation for studying robot tool behaviors is the hope that robotics can play a major role in answering some of the fundamental questions about tool-using abilities of animals and humans. After ninety years of tool-using experiments with animals (Köhler, 1931) there is still no comprehensive theory that attempts to explain the origins, development, and learning of tool behaviors in living organisms.

A simple object like a stick can be used in numerous tasks that are quite different from one another. For example, a stick can be used to strike, poke, prop, scratch, pry, dig, etc. It is still a mystery how animals and humans learn the affordances (Gibson, 1979) of objects and what are the cognitive structures that they use to represent them.

Experimental Platforms


CRS+ A251 Mobile Manipulator	Dynamics Simulator

MPEG Movies from Robot Experiments

Extension of Reach

The movies show testing trials for an extension of reach task with different tools.

L-Stick (1)
T-Stick (2)
T-Hook (3)

"Broken" Tool

Demonstration of autonomous adaptation under uncertainty. Learning is performed with a T-Hook. The movies show subsequent testing trials performed with an L-Hook. The robot "believes" it is still using the orignial T-Hook tool.

L-Hook used Instead Of THook (1)
L-Hook used Instead Of THook (4)
L-Hook used Instead Of THook (4) (goal #2)

MPEG Movies from the Simulator

Useful References

Beck, B., "Animal Tool behavior: The use and manufacture of tools by animals", Garland STMP Press, NY, 1980.

Köhler, W., "The mentality of apes", Harcourt, Brace, and Co, NY, 1931.

Gibson, J. J., "The ecological approach to visual perception", Houghton Mifflin, Boston, 1979.