Daily News Clips
Office of the Chancellor / Public Affairs
Thursday, October 14, 2004
 

Chronicle of Higher Education 10-14-04

A Better Way to Rank Colleges Could Be Crafted Out of Students' 'Revealed Preferences,' Paper Says
By DAVID GLENN

 

The selectivity measures used by U.S. News & World Report and other college-ranking services can and should be drastically restructured, four scholars have proposed in a working paper released this week.

It is theoretically feasible, the study suggests, to rank colleges and universities according to students' actual preferences -- that is, according to their decisions on where to enroll when they have been admitted to two or more colleges. Such measures of "revealed preferences" would be fair, would be transparent, and would eliminate colleges' incentives to distort and manipulate their admissions and matriculation rates, the scholars write.

Traditional measures of selectivity have long been the subject of controversy. Critics have charged that colleges have gamed their admissions and matriculation (or "yield") rates in order to improve their standing in the U.S. News charts and in similar rankings.

In a 2000 essay in The Chronicle, for example, Rachel Toor, a former admissions officer at Duke University, wrote that admissions directors at selective colleges spent much of their energy persuading large numbers of "regular, old Bright Well-Rounded Kids (BWRK's, in admissionese) to apply -- so that the college can reject them and bolster its selectivity rating" (The Chronicle, October 6, 2000).

Similarly, critics have accused selective colleges of misusing early-decision programs in order to improve their apparent yield rates.

Partly in response to such criticism, U.S. News announced in early 2003 that it would no longer employ yield rates in its rankings (The Chronicle, July 18, 2003).

The authors of the new study -- Christopher Avery, a professor of public policy at Harvard University; Mark E. Glickman, an associate professor of health services at Boston University; Caroline Minter Hoxby, a professor of economics at Harvard; and Andrew Metrick, an associate professor of finance at the University of Pennsylvania -- have proposed that the traditional selectivity measures be replaced by an analysis of students' actual matriculation choices.

Their preferred model mirrors the scoring systems used in large-scale chess tournaments and in worldwide tennis rankings. In their paper, the scholars present data about 3,240 high-achieving students who graduated from high school in 2000. After taking various steps to protect the students' anonymity, the students' high-school guidance counselors provided the scholars with detailed information about the students' college applications, acceptances, and enrollment decisions.

The scholars were then able to model thousands of "head to head" competitions between particular pairs of colleges -- for example, of students who were admitted to both Wesleyan University and Middlebury College, how many chose Middlebury? -- and to make inferences about students' overall preferences.

The authors made allowances in their equations for various factors that might have shaped students' choices, including differences in tuition levels and financial-aid offers.

The results, the authors say, offer a clear and accurate portrait of high-achieving students' preferences.

The scholars point out, however, a few odd lumps in their data: Institutions like the California Institute of Technology, which attracts students with specialized interests in science and mathematics, and institutions like Brigham Young University, which attracts students with particular religious affiliations, appear to win "head to head" competitions in unusual patterns.

But those patterns do not pose serious problems for the overall model, the authors assert. If "revealed preference" studies were ever conducted on a large scale, they write, it would be possible to calculate separate sets of rankings for students in particular subgroups.

Whether such rankings actually could be conducted on a large scale is an open question. In an interview on Wednesday, Robert J. Morse, director of data research at U.S. News, said that he doubted that high-school guidance counselors would ever provide such data en masse.

"Sometimes things are just impossible to get," he said. "You're dependent on institutions to gather it for you, and they may just say that it's impossible to gather the student-level information."

Mr. Morse also said that the paper's abstract appeared to overstate the extent to which U.S. News's overall college rankings are based on selectivity ratings. He emphasized that the magazine stopped using yield rates in 2003, and that the remaining measure of admission rates counts for only 1.5 percent of a college's total score.

David Hawkins, director of public policy at the National Association for College Admission Counseling, said on Wednesday that the paper appears, at first glance, to be an important contribution. High-school counselors, he said, have been looking for more and better data "that evaluates the college experience, or the college-desirability level, from the student's point of view, rather than from data that's provided by the colleges."

An abstract of the paper, "A Revealed Preference Ranking of U.S. Colleges and Universities," can be viewed at the Web site of the Social Science Research Network. The full text of the paper is available for download at that site; it is free to the network's subscribers, but costs $5 to nonsubscribers. The paper, which was prepared under the auspices of the National Bureau of Economic Research, has been submitted for publication, but has not yet completed the peer-review process.