Desktop revolution: UW’s Condor scavenges for power

Computer scientist Miron Livny likes to show visitors the supercomputer room of the Computer Sciences and Statistics building as a barometer for how quickly technology can change.

Designed 15 years ago to hold large high-speed machines, today it’s dominated by clusters of compact personal computers and workstations packed side-by-side on metal shelves. The only conventional supercomputer in sight is a refrigerator-sized black box with a flashing display of red lights.

That machine seems almost comically past its prime. “We keep it around because we enjoy the lights,” quips Livny.

The fact is, the future of supercomputing may be less glitzy than the multi-million-dollar, lightning-fast machines people assumed would handle massive computational chores. The future may end up looking a lot more ordinary – like that computer humming away on your desktop.

That’s certainly the aim of Condor, a technology pioneered by Livny and his computer science colleagues nearly two decades ago. It is rapidly becoming a key player in high-end computing for both science and industry.

Condor – named for the bird’s scavenging nature – is a method of networking clusters of workstations and PCs together into “flocks” and harnessing all of their available power.

When Condor was first installed in the mid-1980s, Livny had fewer than 100 computers running in the computer science pool. Now UW–Madison’s Condor flock includes more than 1,300 computers scattered on desktops and shelves throughout computer science, engineering and other departments. Since June 1998, Condor has generated more than 4 million computer-hours of work, and the rate is increasing. In August alone, Condor harvested more than 338,000 computer hours – or 38 years of computing time – of work from its computer science cluster.

Some amazing results have come from this grassroots approach to computing. For example, earlier this summer scientists used Condor to become the first to solve an ostensibly “unsolvable” math problem called NUG30. It was a numerical riddle hatched by mathematicians in 1968 to test the limits of computing power, and was still considered out of reach of modern supercomputers.

By using the Internet as a means of pooling computers in Madison with many others, Condor managed to solve the problem in one week by unleashing the equivalent of 12 years of computing time.

Todd Tannenbaum, project manager for Condor, says about 150 organizations, including about 30 companies, have installed the Condor software, which can be downloaded from its web site. It’s being used in countless research efforts on and off campus, including UW–Madison’s genome mapping work and NASA’s simulation projects for space technology.

But the best applications may be yet to come. The Condor group has entered into a partnership with a Champaign-based company called Knowledge Port, which plans to market commercial products based on Condor software. A Madison-based research and development branch of the company is in the discussion stage.

Livny calls the commercial world “the ultimate test of what we do,” and he is excited about the broader possibilities of Condor. “We have a great deal of evidence now showing that enabling people with computing power acts as an intellectual fertilizer,” Livny says. “If you give them power in a form they can use, people can do wonderful things.”

What makes Condor such a functional system, Livny says, lies in the difference between high-performance and high-throughput computing. High-performance computing is the currency of the supercomputer, and the goal is speed – as in millions of computations per second. High-throughput computing, a term that describes Condor, cares less about off- the-line speed and more about stamina, essentially providing lots of power over long time spans. The Condor network is designed to be agile enough to run many different jobs over a set period.

Condor is accomplishing feats in a fraction of the time – and cost – of far bigger machines. “Our local Condor flock is a mom-and-pop operation compared to the massive federally sponsored supercomputing centers,” Livny says. “We are competing with the big boys.”

WARF and UW–Madison officials hope the approach taken by Condor could become a software technology transfer blueprint for the campus, and help other research groups that would like to have a collaborative relationship with a company.