US News Academic Reputation Score Changes, 2015 vs 2023, vs National Rankings

One of the most controversial aspects of the U.S. News national university rankings is the use of “undergraduate academic reputation” scores based on the “expert opinion” of the 43.6 percent of college presidents, deans, and provosts who actually return the peer assessment documents to U.S. News. Each response rates a university’s academic reputation on a scale of 1 to 5, with 5 being the highest score.

The academic reputation category itself counts for 20 percent of the total ranking, a hefty impact indeed. Since 2015, the mean scores for 99 national universities have risen by .14, from 3.67 to 3.81, or almost 4 percent. A rise or fall of .1 or .2 seems to have minimal positive impact on a university’s ranking, while a rise or fall of .3 more likely indicates a real shift in perception and ranking..

The most likely explanation for the minimal impact, in general, of reputation score increases of less than .3 is that changes in the U.S. News rankings methodology that includes measures for “social mobility” and “graduate indebtedness” have, in many cases, overridden or undermined the impact of marginally higher scores for academic reputation.

Only one university, Northeastern, rose by .4–yet the overall ranking for Northeastern fell from 42 to 44 between 2015 and 2023. The reputation score of 11 universities rose by .3: Notre Dame, NYU, Purdue, Boston University, Virginia Tech, Georgia, UMass Amherst, UConn, Utah, UC Santa Cruz, and Florida State. For this group, the ranking for 9 or the 11 universities rose, while the other 2 fell. Only Notre Dame and UConn dropped in the rankings as their reputation scores rose.

Below is a table showing the changes in academic reputation scores and overall rankings for 99 national universities, 2015-2023, sorted by academic reputation ranking in 2023.

Copy of Rsp vs rank 2023.xlsx

UniversityRep 2015Rep 2023changeRank 2015Rank 2023change
Stanford4.94.90734
MIT4.94.90725
Princeton4.84.80110
Harvard4.94.8-0.123-1
Yale4.84.80330
UC Berkeley4.74.7020200
Columbia4.64.70.1418-14
Chicago4.64.70.146-2
Johns Hopkins4.54.70.21275
Caltech4.64.601091
Cornell4.54.60.11517-2
Penn4.44.60.2871
Duke4.44.50.1810-2
Brown4.44.50.116133
Michigan4.44.4029254
UCLA4.24.40.223203
Northwestern4.34.40.113103
Dartmouth4.24.40.21112-1
Virginia4.34.302325-2
Carnegie Mellon4.24.30.125223
Georgia Tech4.24.30.13644-8
Vanderbilt4.14.30.216133
North Carolina4.14.20.130291
Rice44.20.219154
WUSTL44.20.21415-1
Notre Dame3.94.20.31418-4
Georgetown44.20.22122-1
UW Madison4.14.1047389
UT Austin44.10.1533815
NYU3.84.10.332257
Emory44.10.12022-2
Illinois4.14-0.142411
Washington 4404855-7
UC San Diego3.83.90.137343
Ohio St3.73.90.254495
UC Davis3.83.90.138380
USC3.93.901925-6
Purdue3.63.90.3625111
William & Mary3.73.90.23341-8
Maryland3.63.80.262557
UC Irvine3.63.80.242348
Florida3.63.80.2482919
Boston Univ3.53.80.342411
Boston College3.63.80.23136-5
Tufts3.63.80.22732-5
Minnesota3.63.70.169627
Penn State3.63.70.14877-29
Indiana3.63.70.176733
Colorado 3.53.70.28897-9
Texas A&M3.63.70.168671
Virginia Tech3.43.70.371629
Georgia 3.43.70.3634914
Case Western3.53.70.23844-6
Wake Forest3.53.70.22729-2
UC Santa Barbara3.53.60.140328
Michigan St3.53.60.185778
Arizona3.53.60.1120121-1
Pitt3.43.60.262620
Iowa3.53.60.17183-12
RPI3.43.60.24251-9
Brandeis3.53.60.13544-9
Northeastern3.23.60.44244-2
George Washington3.53.60.15162-11
Tulane3.43.60.253449
UMass Amherst3.23.50.376679
Oregon3.33.50.21061051
Rochester3.43.50.13336-3
Syracuse3.33.50.25862-4
Rutgers3.43.40705515
Stony Brook3.23.40.2887711
Kansas3.33.40.1106121-15
Connecticut3.13.40.35867-9
Col School of Mines3.33.40.18889-1
U of Miami3.23.40.24855-7
Clemson3.23.40.26277-15
Arizona St3.23.30.11291218
UC Santa Cruz33.30.385832
Utah33.30.312910524
NC State3.13.30.2957223
Florida St33.30.3955639
Delaware3.13.30.27689-13
Tennessee3.13.30.2106115-9
Auburn3.13.30.2103976
Baylor3.23.30.17177-6
American3.13.30.27172-1
Fordham3.23.30.15872-14
Lehigh3.33.304051-11
Iowa St3.23.20106127-21
Nebraska3.13.20.199151-52
Missouri3.33.2-0.199121-22
Brigham Young33.20.26289-27
Oklahoma33.20.2106127-21
Miami Oh3.13.20.176105-29
SMU33.20.25872-14
Alabama33.20.288137-49
UIC33.10.11499752
UC Riverside3.13.101138924
Washington St33.1138212-74
George Mason33.10.1128137-9
3.667676767683.807070707070.139795918367

 

 

 

 

 

 

The Curious Case of U.S. News and the High School Counselor Metric

Except for the nuts and bolts metrics used by U.S. News in its annual college rankings (grad and retention rates, class sizes) all of the other ranking categories receive strong criticism from education writers and the academic community. A category since 2009, the high school counselor rankings of colleges’ reputations fly a bit under the radar. But the fact is, they do appear to have a curious impact on the rankings.

A recent, excellent article about the rankings on the website Politico argues that the counselor rankings rely heavily on “guidance counselors from highly ranked high schools, while many high schools in less affluent areas have few or no counselors.”

According the the Washington Post, the rankings do include “surveys of 2,200 counselors at public high schools, each of which was a gold, silver or bronze medal winner in the 2016 edition of the U.S. News Best High Schools rankings.” U.S. News also surveys “the largest private independent schools nationwide.”

This already elite group of respondents is even more restrictive than it seems: “The counselors’ one-year response rate was 7 percent for the spring 2017 surveys,” according to U.S News.

Using the nuts and bolts categories and reputation rankings alone, as in this recent post, and separating out the peer reputation rankings from the high school counselor rankings, we can see the impact the counselor rankings have.

Using a sample of 60 national universities that are either in the top 50 nationally or have at least 7 nationally rated academic departments, we found that the high school counselor rankings of private colleges were about 11% higher than those of university peer rankings of the same colleges. (Twenty-five of the schools are public, while 35 are private.)

The fact is, high school counselor rankings on the whole run higher than those of peer reviewers. But counselor rankings of public colleges were only 6.5% higher than peer rankings.

The main question at hand is, do these (few) counselors have more useful knowledge about national universities that peer reviewers have? Peer reviewers have a response rate of more than 40%; this much broader response rate (in absolute percentages and, almost certainly, demographically) should yield a more accurate assessment from peers. (Even more accurate would be the academic departmental rankings, but those are not included.)

Related questions are, how much marketing information do counselors receive, and do they receive a disproportionate share from private colleges? Do they tour private colleges more frequently? Peer reviewers are not without biases, either, but they are not recipients of marketing information from other colleges. Finally, do counselors rely more on…U.S. News rankings?

Again using the same data set we cite above, a side by side comparison of peer and counselor assessments reveals the following:

–Of the 14 universities that rose in rankings at least two places, three were public universities (21.4%) while 11 (78.6%) were private universities. (The percentage of universities in the sample is 41.7% public and 58.3% private.)

–Of the 17 universities that fell in rankings at least two places, 14 (82.4%) were public while three (17.6%) were private.

Below is a table showing the side-by-side comparison. Please bear in mind that the rankings are our adjusted rankings, not the actual U.S. News rankings.

University Peer Only Peer + Counselors Dif +,-
Princeton 1 1 0
Harvard 1 1 0
Yale 1 1 0
Stanford 4 5 -1
Columbia 4 4 0
MIT 4 6 -2
Chicago 7 7 0
Johns Hopkins 8 8 0
Caltech 9 9 0
Penn 9 9 0
Northwestern 11 11 0
Cornell 11 14 -3
Brown 11 11 0
UC Berkeley 11 16 -5
Duke 11 11 0
Dartmouth 16 14 2
Michigan 17 17 0
Vanderbilt 18 17 1
Carnegie Mellon 18 21 -3
Notre Dame 18 17 1
Rice 18 17 1
Virginia 18 21 -3
UCLA 23 25 -2
Wash U 23 21 2
Georgetown 23 21 2
USC 26 25 1
Emory 27 27 0
Georgia Tech 28 30 -2
North Carolina 28 28 0
Tufts 30 28 2
NYU 31 32 -1
Wisconsin 31 34 -3
Boston College 33 31 2
Brandeis 34 33 1
Wake Forest 34 34 0
Illinois 36 38 -2
Florida 36 36 0
Boston Univ 38 36 2
UC Davis 38 38 0
UT Austin 38 46 -8
UCSD 41 43 -2
Washington 41 46 -5
UC Irvine 43 38 5
Case Western 43 43 0
Maryland 43 43 0
Rochester 46 38 8
Ohio State 46 50 -4
Northeastern 48 38 10
UCSB 48 46 2
Penn State 48 50 -2
Tulane 51 46 5
RPI 52 50 2
Lehigh 53 50 3
Purdue 53 55 -2
U of Miami 55 54 1
Minnesota 55 56 -1
Pitt 57 56 1
Texas A&M 58 58 0
Michigan State 58 60 -2
Indiana 58 60 -2
Rutgers New Bruns 61 58 3