Perception is reality for artificial intelligence expert

Like most computer scientists, Pawan Sinha is drawn to the challenge of making computers smaller, faster and smarter, but he’s taking his lessons from the ultimate computational machine: the human brain.

Sinha, a UW–Madison psychology professor since January 1998, bridges the worlds of computer science and brain science in his research. He is trying to learn from various riddles of perception — how the brain recognizes faces, for example — to provide clues for building smarter, more perceptive machines.

His first invention almost landed in thousands of American living rooms.

As a summer intern at Princeton University in 1990, Sinha worked on a project sponsored by the A.C. Nielsen Company to improve its rating system of television shows. He helped build a machine-vision system for the top of television sets, recording not only what shows were being watched and when, but who was watching them.

“The Nielsen People Meter: Count on Us to Count You,” read the cheery slogan.

One thing they hadn’t counted on was America’s prickly sense of privacy. The idea of recording people when they were sprawled on couches and surfing channels struck a bad chord.

“There was just so much bad press about the Big Brother connotations,” Sinha says. As it turned out, not a single prototype of the People Meter ever peeped into American homes.

But the failure of this invention to sweep the nation became a blessing for Sinha, at the time a student of the artificial intelligence laboratory of the Massachusetts Institute of Technology. The object recognition technology he developed became the cornerstone of his research pursuits, and he was free to pursue applications with abandon.

Sinha recently worked on a project with Mercedes Benz to create a pedestrian recognition system for automobiles, which will warn drivers when people are dangerously close.

He is also developing a more efficient pornography filtering system for use on the World Wide Web, which will flag images rather than relying on text searches. And one of his students is working with the Madison Police Department to improve the tools used by police sketch artists, better reflecting the way people really see faces.

In 1997, a team led by Sinha won the coveted MIT Entrepreneurial Competition, and the top prize of $30,000, with an ingenious software project that will run image- based searches on the Internet. For example, one can take a picture of a cathedral and run a search; the software will find images throughout the web that closely match that cathedral.

How they won the competition, in fact, says a lot about Sinha’s style.

“We entered the contest on a whim,” he says. “The funny thing about our team is we had no business people, and we were surprised to make even the first cut. During the finals, everyone else was up there with spread sheets and analyses and revenue projections, and we were up there with cartoons.”

Yet the judges knew a bankable idea when they saw one: The idea led to a Boston-based company called Imagen, which has steadily grown over the past year.

But Sinha’s entrepreneurial interests are not what drives him. “My first passion is teaching and research,” he says. “But then, I also want to be aware of the applied aspect of what I’m discovering. It’s nice to know that what you’re doing can work in the real world.”

A native of New Delhi, India, Sinha came to the United States in 1989, surviving a bizarre and harrowing plane crash the very first time he stepped foot on an airplane. His Air France flight from New Delhi to Paris barely cleared the runway before skidding into a field swelled by recent monsoons. No one was seriously injured, although both wings were sheared off the plane.

The journey from Paris to New York was no less exciting: Perhaps to make up for the ‘inconvenience’ of the crash, Air France flew him to the U.S. in a Concorde jet.

Sinha is also an accomplished cartoonist, who penned an award-winning comic strip called “Tumbleweed Garden” for the MIT student newspaper. The strip contained random musings about MIT culture, and Sinha says there was plenty to lampoon. “One of the things that characterizes MIT is the bitter rivalry it believes it has with Harvard,” Sinha says. “But Harvard really doesn’t give a damn about MIT.”

The geeky, pocket-protector wearing stereotype about MIT students “is actually quite true,” he says. In fact, every January, MIT tries to compensate by holding a “charm school” that teaches students social etiquette and grace.

Hundreds of UW–Madison students are familiar with one of Sinha’s current projects involving a peculiar celebrity photo collection. The photos flashing across Sinha’s screen are washed-out blobs so unfocused they almost pain the eye.

The research subjects stand before the computer screen as, one by one, Sinha cues up photos of famous blurs — the likes of Sean Connery, Cameron Diaz, Brad Pitt, Meg Ryan. Astonishingly, subjects can consistently recognize event the most blurred likenesses.

How can people recognize a face without any details about eyes, ears, nose and mouth? That’s one of Sinha’s bedrock findings: Facial perception is a broad-brush, rather than fine-detail, process. Sinha is trying to discover those “holistic” features used by the brain.

Next up for Sinha is a project that ties together his fascination with face recognition and his love of cartooning. Sinha plans to enlist several of the country’s top cartoonists for a study of caricatures. Namely, how do we recognize celebrity caricatures when there is precious little information in the picture, just one or two exaggerated features?

“If we can figure out how the brain confirms these images, we will have unlocked one of the great mysteries of perception,” he says.