DARPA Seeks to Remove Communication Barrier Between Humans and Computers

Program explores ways to assemble complex ideas from elementary ones given language and context

The lifelong human imperative to communicate is so strong that people talk not only to other people but also to their pets, their plants and their computers. Unlike pets and plants, computers might one day reciprocate. DARPA's new Communicating with Computers (CwC) program aims to develop technology to turn computers into good communicators.


Straightforward as that may sound, communication involves several coordinated processes. The speaker puts ideas into words, the listener extracts ideas from words and, importantly, both rely on context to narrow down the possible meanings of ambiguous language. All of these processes are challenging for machines.

"Human communication feels so natural that we don't notice how much mental work it requires," said Paul Cohen, DARPA program manager. "But try to communicate while you're doing something else -the high accident rate among people who text while driving says it all- and you'll quickly realize how demanding it is."

Human-machine communication falls short of the human-human standard, where speakers and listeners consider such contextual aspects as what has been said already, the purposes of the communication, the best ways to express ideas, who they are speaking with, prevailing social conventions and the availability of other modes of expression such as gestures. And so computers that might otherwise contribute more significantly to solving problems in a range of areas, including national security, remain in relatively simplistic roles such as crunching large datasets and providing driving directions.

To further the goal of developing systems that communicate more like people do, the CwC program will set tasks in which humans and machines must communicate to do a job. One task will involve collaborative story-telling, in which a human and a machine will take turns contributing sentences until they have written a short story. "This is a parlor game for humans, but a tremendous challenge for computers," said Cohen. "To do it well, the machine must keep track of the ideas in the story, then generate an idea about how to extend the story and express this idea in language."

Another CwC task will be to build computer-based models of the complicated molecular processes that cause cells to become cancerous. Computers are starting to do this already in DARPA's Big Mechanism program, but they don't work collaboratively with human biologists—a shortcoming, because while machines read more quickly and widely than humans, they do not read as deeply, and while machines can generate vast numbers of molecular models, humans are better judges of the biological plausibility of those proposed models.

Of course, storytelling and cancer research are just initial challenges to help advance the technology to a point where humans and machines can take best advantage of their complementary capabilities. In the intelligence-gathering domain, for example, machines' superior ability to collect and store information and humans' superior ability to develop interpretive narratives from such information would find greater synergy if the people and the machines could communicate better.

"Because humans and machines have different abilities, collaborations between them might be very productive. But today we view computers as tools to be activated by a few clicks or keywords, in large part because we are separated by a language barrier," Cohen said. "The goal of CwC is to bridge that barrier, and in the process encourage the development of new problem-solving technologies."

Featured Product

ST Robotics - R17HS-6, the 6-axis version of the high speed R17 variant

ST Robotics - R17HS-6, the 6-axis version of the high speed R17 variant

The 6-axis version of the R17HS high speed variant of the R17 robot is now available. As is our policy the 6th axis is an optional bolt-on module. At the same time we have made it even faster and new software eliminates shake and greatly improves repeatability as this video shows. Please see https://www.youtube.com/watch?v=3wG0MeiJ-yE. The video would seem to imply a repeatability of 0.01mm but we are sticking with 0.2mm in the spec. The motor specialist was able to tune the motors from the other side of the pond. My first reaction when we put power on and entered the first commands was to jump back in amazement (and maybe an expletive). Tip speed is now 3m/s; that's 3 times as fast as its nearest competitor yet costs half the price. I love it when a plan comes together.