Pencil me in: More than 1 million forms feed through this office

Quick, which tool has done the most to improve efficiency at UW–Madison during the last 30 years? Is it the Internet? E-mail? The personal computer? What about the No. 2 pencil?

If you ask Char Tortorice, associate director of the Office of Testing and Evaluation Services, she’ll choose the pencil. She can tell you all about the low-tech No. 2, a soft graphite rod jacketed in pine. When used properly, it makes a dark, even dot on a bubble sheet.

“A No. 2 works perfectly — especially if the pencil isn’t too sharp,” Tortorice says.

Bubble sheets — those fill-in-the-dot forms — have become ubiquitous as answer pages for almost all the multiple-choice exams the university administers. More than a million of them pass through UW–Madison every year, an average of nearly 25 per student, and all of them go to Tortorice and her colleagues. They possess the city of Madison’s only NCS 5000i optical scanner — the bubble-sheet reading machine.

Of course, no student takes 25 multiple-choice exams a year (though some come close). Bubble sheets have uses beyond exams. The university employs them for surveys, applications, course evaluations and a host of other functions.

“Any data that can be collected on a sheet will be,” says Tortorice, “and all those sheets come to us.”

But Testing and Evaluation’s work doesn’t end when the sheets go into the scanner. The office is a repository of expertise, not only on how to test students but also on how to examine tests. Every time the office scores an exam, it creates a detailed analysis of how every student answered every question so that faculty can evaluate their own work as well as that of their students.

“We collect all sorts of statistics on a test,” Tortorice says. “We can show instructors more than just whether their students’ overall grades fall into a bell curve, but how answers line up on each individual question.”

Successful test questions, Tortorice explains, will follow a certain statistical pattern: Students at the top of the test’s overall curve should get each question right more often than those at the bottom. If this isn’t the case — if, for instance, A students and D students are equally likely to give a wrong answer — then the instructor may conclude that the question was misleading, or that he or she covered the material ineffectively in class. Tortorice says that some professors have changed their tests based on her office’s analyses.

In Tortorice’s 24 years at UW–Madison, she’s seen the university grow increasingly dependent on its bubble sheets. “In just the last five years, there’s been a 23 percent increase in the number of sheets that pass through our office,” she says. But in spite of the spread of the bubble sheet, there are still students and faculty members who fail to remember the importance of the No. 2 pencil.

“Just last term we had a student call us up because he’d failed his final,” says Tortorice. “He wanted to know how he could’ve gotten a zero. We looked at our records and saw that the machine had read his sheet as blank, and I knew, this kid did his final in pen. The machine isn’t calibrated to read ink. It’s got to be graphite.”