The College Board has developed a new data-driven tool designed to give college admissions officers the ability to evaluate test scores in light of an applicant’s educational, social, and economic background. The effort is the Board’s latest attempt to offset criticism that its tests favor the affluent, Asian students, and white students.
The new tool could also increase Latino and African American enrollment without the specific consideration of race or ethnicity, otherwise known as affirmative action, an approach that the Supreme Court might soon disallow.
So far, 50 colleges have been using the tool; it will expand to 150 later this year and be available to all schools in 2020.
The tool utilizes 15 factors (listed below) and provides a spreadsheet for admissions officers to use in analyzing the factors in relation to scores.
The new approach is certain to draw criticism, however. Students who live in relatively affluent neighborhoods, attend strong high schools, and enroll in advanced placement courses will receive low “adversity scores” and may find themselves relatively less likely to be admitted to some colleges.
Another issue: the data is based mostly on census block and other federal data, not on individual financial information. A wealthy white student might live in a gentrified neighborhood with inaccurate data indicating that it is still a lower income area. Similarly, a disadvantaged student might live just inside a census tract with high median income stats. Students will not receive a copy of the score–another area of controversy.
Students who attend highly competitive high schools in states with automatic admission based on high school class standing, such as Texas, already find it relatively harder to graduate in the top 6th or 7th percentile of their class. They are admitted “holistically” if they are not in the top percentiles; low adversity scores might narrow their chances even more. Or help them…who knows?
On the other hand, if the new tool on its own can lead to the higher enrollment of students now benefiting from automatic admission, Texas might be able to abandon the rule altogether.
High School Information–Four Factors
Average senior class size;
Average percentage of students taking the SAT;
Average freshman SAT score at colleges attended by SAT-taking graduates of the applicant’s high school;
Percentage of students at the high school who participate in the free and reduced-price lunch program.
High School AP Opportunity–Four Factors
Number of unique AP courses taught in that high school;
Percentage of the senior class who took at least one AP exam;
Average number of AP Exams taken by graduates who sat for at least one exam;
Average AP scores across all AP Exam takers and exams.
High School Percentiles–One Factor
The 25th, 50th, and 75th old SAT percentiles on Critical Reading, Math,
and Math + Critical Reading scores for graduates.
Neighborhood and High School Context–Six Factors
Undermatch Risk–Academic undermatch occurs when a student’s academic credentials substantially exceed the credentials of students enrolled in the same postsecondary institution.
Crime Risk–The Crime Risk represents the likelihood of being a victim of a
crime–not the likelihood of committing a crime.
Family Stability–Family stability is a combined measure based on the proportion of two-parent families, single-parent families, and children living under the poverty line within each neighborhood, or across the neighborhoods of past students attending that high school.
Educational Attainment–Educational attainment is a combined measure that looks at the pattern of educational attainment demonstrated by young adults in the community. ESL participation.
Housing Stability–Housing stability is a composite measure that includes vacancy rates, rental versus home ownership, and mobility/housing turnover, again based on aggregate population statistics.
Median Family Income — Median family income is based on weighted data from the Census/ American Community Survey.
Overall context is a weighted average of the individual metrics listed above. College admissions officers receive (1) bar graphs showing the applicant’s SAT score relative to others who share the applicant’s overall percentile of neighborhood
adversity and high school adversity and (2) the average freshman SAT score of entering students at the colleges that these respective groups of students attended.
“It has become a mantra in some quarters to assert that standardized tests measure wealth more than intellectual ability or academic potential, but this is not actually the case. These tests clearly assess verbal and mathematical skills, which a century of psychological science shows are not mere reflections of upbringing. Research has consistently found that ability tests like the SAT and the ACT are strongly predictive of success in college and beyond, even after accounting for a student’s socioeconomic status.”
For years, U.S. News has used test scores and selection rates as ranking data for the annual “Best Colleges” report. The publication has slightly reduced the impact of test scores in recent editions.
Below I will explain why we do not include test scores as a metric and argue that, for honors and non-honors students, other factors are more important in predicting success. (High school GPA is certainly a major factor; but since almost all honors students have high GPAs, I do not discuss the impact of GPA in this post.)
In their published scholarly work, the authors argue that test scores by themselves correlate very strongly ( r= -.892) with the annual U.S. News Best Colleges rankings for national universities even though the test scores count for only 7.75 percent of the total ranking score. (The authors do not cite the impact of test scores on other ranking factors such as graduation and retention rates, which together account for 22 percent of the total ranking score.)
Our own work for the past eight years, however, shows that test scores do not have a similar correlation to quantitative assessments of honors programs. In our publications we list minimum and average admissions test scores for all programs we rate, but we do not count the scores alone as a rating factor.
Here’s why we do not use test scores as a measure: The factors that make for an excellent honors program are primarily structural. The major building blocks are the credits required for honors completion; the number of honors class sections offered, by type and academic discipline; the availability of priority registration and honors housing; the size of honors class sections; and the number of staff to assist students.
So, don’t the test scores drive the university graduation rates of honors program entrants, just as they do in elite colleges? The answer is not so much; the correlation is r= .50
Admittedly, it is probably difficult for a student with, say, a 1050 SAT score to succeed in an elite college or in most honors programs. But within a fairly large range of SAT scores (~1280-1510), the opportunities for success are more often present given a conducive structure. With every biannual review of honors data, I find great pleasure in discovering outstanding honors programs that are not housed in highly- ranked and extremely selective universities. The golden nuggets of excellence in higher education are scattered much farther and wider than many would have us believe.
I am strongly opposed to the numerical ranking of colleges or their honors programs, whether or not test scores are included in the methodology. I ranked honors program one time, in 2012, and regret doing so. Yes, I have data that allows me to numerically differentiate the total rating scores earned by honors programs. But anyone who wants to provide some kind of assessment of colleges or programs needs to do so with the assumption that their methodology is subjective and imperfect. Ordinal rankings based on distinctions of one point or fractions of a point give readers a veneer of certitude that a qualitative difference exists even if it (often) does not.
Although we do not rank honors programs, we do place them in one of five rating groups, a process that is similar to rating films on a five-star basis but based on quantitative rather than completely subjective data. The seven honors programs in the top group in 2018 (out of 41) had average SAT scores (enrolled students) ranging from 1280 to 1490, a sizable range.
Honors completion rates are something of an issue these days. An honors completion rate is the percentage of first year honors entrants who complete at least one honors program graduation requirement by the time of graduation from the university. About 42 percent of honors students do not complete honors requirements before graduation, although a very high percentage of honors entrants (87 percent) do graduate from the university.
The seven honors programs with honors completion rates of 75 percent or higher in our 2018 ratings had average SAT scores ranging from 1340 to 1510; the mean for this group was 1420. The mean SAT for the 31 (of 41) programs that provided completion rates was 1405, not much lower. And another seven programs with mean SAT scores of 1420 or higher had completion rates below 58 percent, the group mean.
The mean SAT score for all 41 rated programs was 1407; the mean SAT for the top seven programs was only one point higher at 1408.
It is clear, at least with respect to honors programs, that average SAT scores are not the best predictors of program effectiveness. What does this mean for the value of test scores nationwide, if anything?
I think it means that for students who are in the 1280 to 1500 SAT range, success depends as much or more on mentoring, smaller interdisciplinary sections, student engagement, course availability, community (including housing), and advising support than it does on test scores.
The good news here is that even for students who are not in honors programs, high levels of achievement are accessible to students who do not begin college with extremely high test scores, although non-honors students will probably have to assert themselves more in order to benefit from the strongest attributes of their university.
The following post is by site editor John Willingham.
For more than six years I have been fortunate enough to receive large amounts of data from public university honors colleges and programs. The data I received for the 2018-2019 edition of INSIDE HONORS pointed toward a trend in honors education: the partial substitution of experiential learning for traditional academic coursework.
First, what is experiential learning?
Students can earn credit (or “points” or “units”) for the following: doing internships, studying abroad, or conducting mentored research; publishing in journals and making presentations at conferences; applying for national awards (Truman Scholarships, Goldwater Scholarships, etc.); serving on honors committees and in other student groups, and engaging in leadership training; obtaining certification or experience in promoting diversity, social innovation, and group problem solving; and for participating in the many types of “service learning,” usually involving participation in community or university volunteer activities.
Internships, mentored research, and study abroad have long been components of many honors programs; they often carry course credit. (Some of the activities listed above do not award course credit but only points for honors completion.) Since the Great Recession, internships are increasingly important for practical reasons. The same can be said for some training or experience in collective problem solving. Sometimes the latter can take the form of group projects that have a vocational focus (entrepreneurship, engineering); other group projects take a turn toward solving social problems.
So it is clear, at least to me, that part of the focus on experiential learning is a response to changing economic conditions. And the experiential options all appear to have laudable purposes. I believe advocates of experiential learning when they say their programs are “high impact” and can teach lifetime lessons to students. The question is not whether experiential learning is worthwhile but, rather, how much of it is appropriate? Thus far, the trend toward experiential learning appears to be centered almost exclusively in public university honors programs.
I reviewed honors requirements for 40 public university honors programs, many of them in flagship institutions, along with the same number of honors programs in private universities of approximately equivalent reputation. Of the public honors programs, eight have implemented or increased the impact of experiential learning toward honors completion in the last two years. But only one private university honors program has done so.
Note: Below please see the public and private universities I reviewed for experiential learning emphasis. Those in bold have notably increased experiential learning in their honors programs during the past two years. This does mean that, in each instance, experiential learning is necessarily over-emphasized. Some of the programs have retained at least one honors completion option that requires extensive academic coursework.
As recently as two years ago, I observed only two or three public honors programs that featured the emphasis on experiential learning that I see today.
If the trend continues, it could redefine the meaning of public honors education and further differentiate that education from what is offered by private universities in ways that might not appeal to many parents and students. After all, “going to college” has meant earning academic course credits in seminars and the disciplines, with participation in campus groups or volunteer work being left up to the students.
Elite private universities, most of which do not have honors programs, continue to follow the traditional model. Internships and studying abroad are common, sometimes for academic credit; but participation in other activities is based on student choice.
Most public honors programs have promoted themselves by promising the equivalent of an elite college education within a large public research university. I call this the standard hybrid model for honors colleges and programs. Thus far, the elite college part of the hybrid has been grounded in academic coursework.
While it is fully justifiable to augment academic coursework with some experiential opportunities, providing experiential honors credit (but sometimes not course credit) for one-fourth or more of the total honors completion requirement could result in a hybrid within a hybrid: some honors programs will continue to offer mostly traditional academic courses and others will ratchet up experiential learning.
Where experiential learning is prominent, the result will be less academically focused. How will parents and students react to this? They are often trying to decide between elite private colleges, which still emphasize coursework, and honors programs, some of which are becoming more experiential.
Here are some pros and cons regarding this trend:
Pros of Experiential Learning…
A reflection of the public service mission of university (one reason for current absence of experiential learning in private honors programs?)
Personal growth for students through broader engagement outside of the classroom
A stimulating way for students to apply learning outside the classroom
An enhancement to career prospects
An antidote to the self-focused culture around us
Less costly to staff and fund; no teaching faculty involved
Cons of Experiential Learning…
Distraction from core and major requirements
Complaints from students, parents based on above, and on confusing completion options
Challenges of finding meaningful opportunities
More staff to support experiential activities
Less time for academic electives
Duplication of university-wide or other readily available experiential opportunities
Of greatest concern here is the last “Pro” listing: “Less costly to staff and fund; no teaching faculty involved.” How tempting it must be to administrators to offer honors credit without having to beg and borrow faculty and classroom space. If this becomes the primary factor in shifting from academic coursework to experiential learning, the trend could accelerate and have a profound impact on public honors education.
To remain competitive with private colleges and universities, public honors programs should continue to be enhanced academic programs at their core, seeing their central mission as providing highly talented students a top-flight education, often in-state, and almost always at lesser cost than private university alternatives. When experiential learning accounts for more than about one-fourth of honors requirements, the core mission is likely to be compromised.
“Texas CSB provides a rigorous four-year undergraduate curriculum aimed at preparing students for top technology careers. The Texas CSB offers distinct benefits for students looking toward careers in today’s tech-focused business world. University leaders anticipate that it will attract high-achieving students with strong quantitative and technical skills from across the nation. The program is a particularly attractive opportunity for aspiring entrepreneurs and for students interested in data and marketing analytics, financial engineering, and leadership roles in tech companies.”
Rest assured that the selection process is extremely rigorous. The CSB will have to approximate the standards of the Turing Scholars Program and the Business Honors Program. “Turing denies 85% of valedictorian applicants. That means it’s especially important that you demonstrate a breadth and depth of commitment in computer-related activities.” The average SAT for the BHP is north of 1500; the same is true for CSB. Only about 12% of applicants gain admission to the BHP.
Turing Scholars at UT Austin
“Our top-ranked faculty push students to think outside the box and learn the varied business and computer science disciplines,” according to the website. “The curriculum is comprised of 44 classes, taken with 30-40 students, exposing students to all facets of business and computer science.
Editor’s Note: The list appears after the introductory section. The list is current as of September 25, 2018.
In a previous post, Based on Academic Reputation Alone, Publics Would Be Higher in U.S. News Rankings, we write that many public universities have a reputation in the academic community that is much higher than their overall ranking by U.S. News. In this post, we will summarize the reasons that prospective honors students and their parents might consider paying more attention to academic reputation than to other factors in the oft-cited rankings. The list also facilitates comparisons of public and private universities.
First, these are factors to consider if the state university’s academic reputation is much stronger than its overall ranking:
1. The overall rankings penalize public universities for their typically larger class sizes, but the average honors class size in our most recent study of honors programs is 24.9 students, much smaller than the average class size for the universities as a whole. Many of these honors classes are lower-division, where the preponderance of large classes is often the norm. First-year honors seminars and classes for honors-only students average 17.5 students per section. Result: the relatively poor rating the whole university might receive for class size is offset for honors students.
2. The overall rankings hit some public universities hard for having relatively low retention and graduation percentages, but freshmen retention rates in honors programs are in the 90% range and higher; meanwhile six-year grad rates for honors entrants average 87%–much higher than the average rates for the universities as a whole. Result: the lower rates for the universities as a whole are offset for honors students.
3. All public universities suffer in the overall rankings because U.S. News assigns ranking points for both the wealth of the university as a whole and for the impact that wealth has on professors’ salaries, smaller class sizes, etc. This is a double whammy in its consideration of inputs and outputs separately; only the outputs should be rated. Result: the outputs for class size (see above) are offset for honors students, and the wealth of the university as an input should not be considered in the first place.
4. For highly-qualified students interested in graduate or professional school, academic reputation and the ability to work with outstanding research faculty are big advantages. Honors students have enhanced opportunities to work with outstanding faculty members even in large research universities, many of which are likely to have strong departmental rankings in the student’s subject area. Result: honors students are not penalized for the research focus of public research universities; instead, they benefit from it.
5. Many wealthy private elites are generous in funding all, or most, need-based aid, but increasingly offer little or no merit aid. This means that families might receive all the need-based aid they “deserve” according to a federal or institutional calculation and still face annual college costs of $16,000 to $50,000. On the other hand, national scholars and other highly-qualified students can still receive significant merit aid at most public universities. Result: if a public university has an academic reputation equal to that of a wealthy private elite, an honors student could be better off financially and not suffer academically in a public honors program.
But…what if the academic reputation of the public university is lower than that of a private school under consideration? In this case, the public honors option should offer the following offsets:
1.The net cost advantage of the public university, including merit aid, probably needs to be significant.
2. It is extremely important to evaluate the specific components of the honors program to determine if it provides a major “value-added” advantage–is it, relatively, better than the university as a whole. Often, the answer will be yes. To determine how much better, look at the academic disciplines covered by the honors program, the actual class sizes, retention and graduation rates, research opportunities, and even honors housing and perks, such as priority registration.
The rankings below are on a 5.0 scale, and there are many ties. We have included national universities with reputations rankings between 2.7 and 4.9.
This post, by editor John Willingham, is about 4,000 words in length, so not a quick read. As the title indicates, the issue of honors completion is complicated. The post makes frequent references to statistical data. Our thanks to the 31 honors deans and directors who contributed data for this report and for the Fall 2018 edition of INSIDE HONORS, due out in early October.
This post was edited on August 6, 2018. All changes were minor.
First of all, what is an honors completion rate?
It is the percentage of honors program entrants who complete the required honors curriculum by the time of graduation. Many programs have more than one honors curriculum completion option; for example, entering freshmen may be required to finish 30 honors credits and write a thesis for the main option, or they might need to complete only 18 credits without a thesis for a lower option. Honors completion rates are not the same as graduation rates. Entering honors students, because of their strong credentials, will have very high graduation rates regardless of honors completion.
Completing the requirements of an honors program is typically not directly related to graduating with Latin honors (cum laude, magna cum laude, summa cum laude) even though some universities do make the connection. Latin honors are most often tied to a student’s university grade point average or class standing. In many colleges and universities a student can graduate with Latin honors without taking any honors courses or writing a thesis. Many, if not most, honors program completers do also earn Latin honors.
At least two researchers have written that honors completion rates can be extremely low (in the 30 percent range) and that, because publicizing completion rates can be embarrassing to some programs and their parent institutions, the rates are a “Dirty little secret.” Academic studies leave the impression that having 50 percent or more of honors students complete one or more options could be, if not desirable, then acceptable. Using any set percentage as a standard is, however, questionable. (Please see below.)
University of South Carolina
The data we have received does provide evidence that some program completion rates are as low as 30 percent. On the other hand, the mean six-year completion rate in our own study of 31 honors programs, enrolling more than 64,000 honors students, was 58 percent. The years covered were 2010-2011—2015-2016.
Some honors educators believe that offering the lower, “mid-career” options increases the likelihood of at least some level of completion. The researchers cited above found evidence that mid-career completers are also inclined to add higher levels of honors completion before graduation.
Academic studies of honors retention, completion, and university graduation rates have focused on individual programs rather than on a significant range of programs. These studies have evaluated the impact that program curriculum requirements, entrance test scores, high school GPAs, housing, co-curricular activities, first-year seminars, and other factors have had on honors retention, completion, and graduation.
Our own work began with an analysis of 14 factors: program size; mean test scores of admitted students; number of honors credits required for completion; six-year graduation rate of honors entrants; six-year university graduation rates; university freshman retention rates; number of honors sections offered; average size of honors classes; number of honors sections in key academic disciplines; percentage of honors program members occupying classroom seats; comparison of main option and multiple option program completion rates; impact of a thesis requirement; the percentage of honors residence hall places available for the first two years; and the impact of requiring a separate honors application or credentials. After considering the potential value of each factor as an independent variable in our final models, we excluded some of factors based on tested co-linearity.
Again, we will publish a full statistical report in a separate post. We have also developed a formulafor estimating target completion rates, taking into account variations in rates for honors completion, freshman retention, and university-wide graduation.
Below is a detailed discussion of the honors completion issue.
Honors educators and university administrators have a keen interest in achieving high honors completion rates. Honors students bring higher test score profiles to the university as a whole, and one would anticipate that being in an honors program would make it even more likely that these students would go on to graduate and, in the process, improve the university’s retention and graduation rates.
While the academic studies make it clear that honors student entrants, whether completers or not, have high retention and graduation rates, it is not altogether clear that they have higher rates than those of non-honors students who entered the university with equivalent credentials.
Evidence does indicate, however, that after one or two years in an honors program, students do have better critical thinking skills than similarly qualified non-honors students, probably due to smaller, interdisciplinary classes in the first year and greater interaction with faculty, mentors, and fellow students. And of course these skills and a greater likelihood of obtaining strong faculty recommendations should help students to gain entrance to prestigious graduate and professional schools or find highly desirable and remunerative employment.
Arizona State University
Students who do not actually complete all honors requirements do not perform as well academically as honors completers and also take somewhat longer to complete their undergraduate work. One reason: almost all honors students enjoy some form of priority registration.
The principal goals of honors educators and administrators are to improve the metrics of the host university by enrolling high quality students and to provide those students with an enhanced education that can compare favorably with the education one might receive in an elite private college or university.
Our data and other studies show that honors programs do meet the goal of improving university metrics. Honors entrants (not necessarily program completers) on average graduate at a rate 19.7 percentage points higher than the rate for their parent universities as a whole, according to our data. For programs housed in universities with relatively low university graduation rates, the difference can be more than 35 percentage points. (Of course, honors entrants who graduate make up a part of the graduation rate of the university as a whole.)
The main goal of honors educators, however, is to provide an enhanced education.
Honors completion rates should surely be one measure of meeting this goal. Low completion rates are an especially discouraging result given the cost and effort allocated to honors. “Non-participation or minimal participation of honors students is the honors equivalent of poor overall university retention and graduation rates,” according to one paper on the subject.
The quote is probably accurate when it comes to describing the mindset of honors educators. But comparing honors completion rates to, for example, the graduation rates of elite colleges and universities is problematic. Honors programs are a hybrid; this all but universal, structural reality clearly differentiates honors programs from most elite colleges, which generally do not have honors programs. (More on the hybrid issue below.)
Comparing honors completion rates with the graduation rates of the parent university as a whole is more reasonable, provided that there is some baseline ratio of honors completion rates to university graduation and freshman retention rates. Programs in our study with completion rates above the mean of 58 percent do, on the whole, match the graduation rates for the parent universities. Programs with completion rates below the mean, on average, fail to match the university graduation rate by about 20 percentage points. (These rates and ratios will be discussed in the next post, and, again, we have developed a baseline formula for estimating target completion rates.)
Honors programs seek to combine the best qualities of an elite private college with those of a large research university. In general, this means that the “elite private college” components of this hybrid model are smaller classes, more interdisciplinary sections and class discussion, more faculty mentoring, completion of a substantial honors curriculum and sometimes an undergraduate thesis, and a high level of collegiality in the form of co-curricular activities and access to honors housing.
The advantages of the “large research university” include academic majors in abundance, relationships with a broader range of students, more undergraduate research opportunities, study under nationally recognized scholars, the enjoyment of big college football and other athletics, larger alumni networks, and life in a “college town” that is centered on the large university. Some of these advantages are, however, double-edged (see below).
The hybrid model, if realized, would be for many students an ideal college experience. But one can imagine how daunting it is to meet such expectations–to match private elites at their own game and to optimize the research university experience–all simultaneously. Honors and university administrators would like to see honors completion rates that equal or exceed parent university graduation rates, or even the graduation rates of elite colleges. But in the context of honors completion rates, some of the hybrid components are positive while others can work to lower completion rates.
THE HYBRID MODEL: WHAT MAKES IT WORK?
Six of the 31 programs in our study had six-year honors completion rates of 80 percent or higher. (But recall that honors entrants, regardless of honors completion, do graduate from the host university at a much higher rate than the rate for all students, on average about 87 percent.)
These programs are, in alphabetical order, by university: Arizona State Barrett Honors College; CUNY Macaulay Honors College; University of Illinois Campus Honors Program; Penn State Schreyer Honors College; University of South Carolina Honors College; and the UT Austin Plan II Honors Program. Programs with rates of 70 percent or higher, in alphabetical order, are Clemson Calhoun Honors College and the Colorado State Honors Program.
Colorado St University
The six programs with completion rates of 80 percent or higher have striking differences. Barrett Honors College at ASU and the Schreyer Honors College at Penn State make very extensive use of honors contracts and mixed sections. Compared with the funding required for separate honors sections, the cost of contract and mixed sections is much less. Honors contracts allow a student to take a non-honors course for honors credit if the student “contracts” with the instructor to do some form of additional work. Mixed sections are those that include honors and non-honors students; they should be more rigorous or have an honors-only discussion or lab section. Schreyer Honors College at Penn State has an extremely large percentage of mixed sections—but honors students make up a high percentage of total students in those sections.
The UIUC Campus Honors Program and the UT Plan II program, both small in size, have a far more structured curriculum that does not include contract or mixed section credit. Plan II students receive most of their honors credit through Plan II-specific courses, even in subjects such as physics. CUNY’s Macaulay Honors College students take honors-only classes offered by the honors college or by academic departments. The South Carolina Honors College offers an impressive range of honors-only classes of the same type, and only about 11 percent of sections are mixed. The program does not offer contract options.
Contract and mixed sections give students a lot of flexibility. Many of these courses are upper-division, so students can continue to receive honors credit throughout their time in the program, without having to wait for a specific honors-only course to open. It is difficult for many large honors programs to achieve four-year involvement without utilizing contract and mixed sections. Yet the South Carolina Honors College and the Macaulay Honors College at CUNY both have more than 2,000 honors students and also have strong four-year participation.
But based on statistical analyses of the data from all 31 programs, common predictive factors emerge: high university freshman retention rates; a substantial honors curriculum (30 credits or more); an emphasis on one completion option; and enhanced access to honors housing. Other positive factors include higher test scores, important to freshman retention rates; and smaller class sizes. Both of these factors have their greatest impact in programs with multiple, lower completion requirements.
As a variable in multiple regression, the impact of curriculum “flattens” because much of its effect appears in the variables for main option emphasis, honors grad rates, housing, separate honors application, and especially by participation if the latter is also a variable. The same flattening occurs with the impact of test scores, much of it expressed by the freshman retention variable.
Penn St Schreyer students
Programs looking for a “shorthand” method of assessing completion issues might find it in one overriding measure, which we call “participation”: the percentage of honors program members who occupy classroom seats in a given term. This participation rate correlates very significantly with honors completion, as one would expect. If an honors program has 1,000 members, and 1,000 honors credit classroom spaces are filled in a given term, then the participation percentage is 100. But because some honors students almost always take more than one honors class per term, it is common for participation percentages to be higher than 100. (If 1,000 members occupied 1,100 classroom spaces, then the participation rate would be 110 percent.)
The mean participation rate for all 31 programs was 129 percent. For the 16 programs with completion rates above 58 percent, the mean participation rate was 150 percent. For the 15 programs with lower completion rates, the mean participation rate was 108 percent.
For main option programs, the participate rate was 1.49; for multiple option programs, it was 1.11.
(So, a path to improvement might be >More Honors Housing >Emphasis on 30+ Credit Curriculum >Increased Freshman Retention >More Honors Class Sections and Disciplines>More Classroom Spaces Occupied>Higher Participation>Higher Completion Rate. This path, like most things in higher education, is impossible without funding. All honors deans and directors want to achieve high completion rates. The fact is that almost all honors programs do their best with the resources they have been dealt—and it is the rare honors student who does not benefit from a program even if he leaves after a year or two.)
Clemson University Honors Center
Out of 11 programs in our study with an honors curriculum requirement of 30 credits or higher, only one had a completion rate below 50 percent. The mean curriculum requirement for the six programs with a completion rate of 80 percent or higher was 37 credits. The overall mean for the 31 programs, when we averaged the multiple completion options for programs that offered them, was 27 credits.
One notable finding was that main option programs* with significantly stronger curriculum requirements (mean of 31.8 credits) had an average completion rate of 67.8 percent compared with an average rate of 48.5 percent for programs with multiple (lower) completion requirements (mean of 22.1 credits). This finding seems to contradict previous evidence and assumptions. We note that 24 of the 31 contributors to our study are either flagship or designated land-grant universities, but we have no data related to the differences between their programs and those at other types of institutions. The average U.S. News ranking of the 31 programs was 126, so the study was not limited to “public elites.” The highest ranking of any programs was 52. (In case anyone is curious: There is no significant correlation between the U.S. News rankings and honors completion rates.)
*(Included in main option programs are two programs that, while technically offering two options, have essentially the same total completion credit requirement for each option and have no mid-career option. In addition, option categories were also defined according to the curriculum requirements that honors programs offered for first-year entrants only, except we classified one program with a significant number of sophomore and junior transfers as a multiple option program.)
A precise calculation of curriculum requirements for multiple option programs would have included the proportion of honors students completing each option. In some multiple option programs more than 75 percent of students complete the highest option; in other programs fewer than 10 percent of students do so.
Although program size did not emerge as a clear predictor, the main option programs include four that have fewer than 1,000 honors students. Meanwhile, the smallest multiple option program has an enrollment of more than 1,300 students.
All 16 multiple option programs had an honors thesis option or requirement. The University of Arizona Honors College, the University of Arkansas Honors College, the Oregon State Honors College, and the Commonwealth Honors College at the University of Massachusetts, all with multiple options, do require an honors thesis for all options. Eleven of 15 main option programs had a thesis option, with nine listing a thesis as required. Regression models did not, however, establish that a thesis requirement or option had a significant impact on completion rates for the whole data set. But it appears that the combination of a significant thesis requirement along with relatively low university freshman retention rates (79-83 percent range) might contribute to low completion rates.
As for housing, the six programs with the highest completion rates offered .84 honors residence hall spaces per every first- and second-year honors student. The mean for all 31 programs was .53 residence hall spaces, or slightly more than one dorm space for first-year entrants only.
With respect to university freshman retention rates, the mean rate for the top six programs was 93.3 percent, while the overall mean freshman retention rate for 31 programs was 86.6 percent. Programs with completion rates at or above 58 percent had a mean university freshman retention rate of 88.9 percent; programs with lower completion rates had a mean university freshman retention rate of 84.2 percent. These bare statistics along with statistical models point clearly to freshman retention rates as being the most important institutional factor related to honors completion. They are co-linear with university graduation rates and almost as closely tied to the graduation rates for honors entrants, but are more significant than university graduation rates in relation to honors completion rates.
The top six programs had a mean SAT (ACT adjusted to SAT when necessary) of 1433, versus a mean score of 1406 for all 31 programs. The impact of test scores was, however, stronger for multiple option programs, though their mean test score was lower, at 1396. The mean test score for all main option programs was 1417, a difference of only 21 points. It appears, then, that the much stronger completion rates among main option programs as whole (67.1 percent versus 48.6 percent for multiple option programs) were not the result of proportionately higher test scores.
While all six of the programs with the highest completion rates were main option programs, the six programs with the lowest completion rates were all multiple option programs, with an average completion rate of 34.7 percent. For this group, the mean test score was 1358. Unlike the relationship of test scores to completion rates for the entire data set, it seems likely that, for this lowest subset, test scores would play a significant role; however, we could not confirm such a role statistically. Neither could we do so for the relationship of test scores and completion rates for the top subset.
We also calculate the ratio of enrolled individual honors students to all honors sections, and to total sections in 15 key academic disciplines. Both main and multiple option programs had about the same ratios. These ratios explain why the class size averages for main and multiple option programs are almost exactly the same (24 and 24.4, respectively). However, the absolute numbers of sections offered are much higher for main option programs because, with higher curriculum requirements, they have more students taking classes across all four years.
The six-year graduation rate for honors entrants in main option programs was 88.7 percent; for honors entrants in multiple option programs it was 85.2 percent. This difference would have an impact on university-wide graduation rates. For the six programs with the highest completion rates, the honors graduation rate averaged an impressive 91.3 percent; the honors graduation rate for the six programs with the lowest completion rates was almost eight points lower, at 83.5 percent. Freshman retention rates, test scores, and curriculum have the most impact on honors graduation rates, especially the first two factors, as one would expect. Freshman retention rates and honors graduation rates are remarkably similar.
It is somewhat unusual for honors completion rates to equal or exceed university graduation rates. Eleven of the 31 programs achieved such rates, according to our data. In alphabetical order, they are Arizona State’s Barrett Honors College; Colorado State Honors Program; CUNY Macaulay Honors College; University of Houston Honors College; University of Illinois Campus Honors Program; University of Kansas Honors Program; University of Nevada Reno Honors Program; University of South Carolina Honors College; Texas Tech Honors College; UT Austin Plan II Honors Program; and the Virginia Commonwealth University Honors College.
Texas Tech University
CHOICES, AND MORE CHOICES
The part of the hybrid structure that is related to the “large public research university” component is difficult to measure. Of course the resources allocated by the university make possible the scores of academic departments and sub-disciplines available for majors and fund the honors program, or programs, as well. Relatively generous funding allows for more honors sections, smaller classes, undergraduate research, and more housing, all of which are important to participation.
But the one characteristic of honors programs and the public universities in which they reside that receives little attention, in relation to completion rates, is the enormous range of choices that are available. An honors student at a major public university can choose to persevere through a demanding honors curriculum, or not; can choose to attend every home football game and party, or not; can choose among hundreds of degree plans and change to one that is too time-consuming to allow for honors work, or not; join eight or ten of the two hundred groups on campus, or not; and choose to live off-campus or with a non-honors friend, or not.
Their counterparts at elite private colleges do not have a hybrid structure that allows such a range of choices. Of course they can change majors, or, perhaps, change residence halls. They can also choose to spend too much time partying. But they have a smaller range of majors and college organizations from which to choose; and college sports often have limited appeal. And most do not experience large, sprawling campuses where one can feel overwhelmed, although honors programs certainly make big-campus life more collegial.
A larger range of choices, then, is an inherent piece of the “research university” component of the hybrid model, and in our opinion, it can contribute to lower honors completion rates. Some characteristics of a large public university campus (large class sizes, registration issues, social distractions) often cause parents and students to choose smaller, private colleges even at greater cost. Honors programs mitigate but do not eliminate the potential impact of these factors.
The real question is whether greater choice is ultimately negative or positive. All students make good choices and bad choices; college is often the place where they learn the first big lessons about choice. Clearly, however, students who are mature and focused enough to enjoy the large university experience without overindulgence are most likely to take full advantage of their honors opportunities.
Students should also be strongly motivated on their own if they are to undertake honors study and succeed. Their counterparts at elite private colleges must demonstrate their motivation repeatedly, not least during the application process. Our study shows that, for main option programs only, honors admission requirements that require an honors-specific application or credentials beyond those required for regular university admission do have an impact on completion rates.
Twelve of 15 main option programs require honors-specific application materials. Eleven of 16 multiple option programs do. (Some programs simply gather data from the admissions office and then issue invitations to top students already admitted to the university.)
PUBLIC HONORS VS. PRIVATE ELITES
The issue of honors completion is not only linked to the question: Do honors programs actually deliver? Another question often follows: How do honors programs really compare with private elite colleges?
Above we noted that honors participation, a major statistical (and common-sense) factor in predicting honors completion rates, is enhanced to a large extent by substantial curriculum requirements, frequently including a thesis. Assuming that a student is in a public honors program with both a strong curriculum requirement and a high completion rate, does that student graduate with an education comparable with that attained by a student at an elite private college?
The hybrid model carries with it the assumption that students at elite private colleges complete a rigorous curriculum that usually includes extensive undergraduate research and an honors thesis, and that the honors model should strive to do the same.
Like the perception that honors completion rates should approximate graduation rates at elite colleges, the perception that most or all students at elite colleges necessarily pursue an especially rigorous path is inaccurate.
Princeton is the only university in the Ivy League that requires an undergraduate thesis for graduation. Like the rest of the Ivy League, Stanford, MIT, Duke, Swarthmore, Amherst, Williams, and other elite schools do not require a thesis. Reed, Chicago, and Bates do have a high number of students who complete an undergraduate thesis. If a student wants to graduate with Latin honors at many elite colleges, or especially to graduate magna cum laude or summa cum laude, or with departmental honors, only then would the student have to write a thesis.
Anecdotal information suggests that only 20-25 percent of students at elite colleges complete a thesis or equivalent project. In addition, grades at elite institutions hover around an A-minus average, bringing into question just how much many of them are actually challenged by the courses they take.
So, yes, given appropriate effort, a student in a public honors program with a strong curriculum requirement and a thesis should receive an equivalent education or, perhaps, even a better education than most students at a private elite college. One can argue, however, that the relatively few students pursuing high Latin or departmental honors at private elite schools can receive an even better education.
Finally, another comparison: Does the education of an honors student who is in a program with, say, a 55 percent completion rate, a 24-credit completion requirement, and no requirement for a thesis compare with that of his or her counterpart at a private elite college?
The private elite college will have a graduation rate about 5-10 points higher than the graduation rate of public university’s honors students. The student body at the private elite will, on the whole, be “smarter” but less diverse, less “real-life,” economically and otherwise. The honors student may well be challenged more by honors work than most students at the private elite are in regular classes. Both students may receive some financial aid, but at the private elite most of the aid is need-based or leaves funding gaps that could leave the student with large student loans. Meanwhile, the honors student and many of his classmates enjoy a large, renewable merit scholarship.
Attacks on the humanities and social sciences have increased since the Great Recession, even at a time when the critical thinking skills associated with these disciplines are urgently needed to navigate the sometimes bizarre world of facts, alternative facts, distortions, and outright lies.
Indeed, with the decline of humanities departments, we might be nearing the time when honors colleges and programs will be the focal point of liberal arts education in many public universities. (Below is a discussion of what the nation’s largest honors college is doing to promote the humanities and “civic education.”)
The economic downturn along with rising college tuition costs forced many parents and prospective college students to zero in on courses of study that provide near-term financial results and security. The trend is so strong that, recently, the University of Wisconsin at Stevens Point announced a plan to eliminate 13 humanities and social science majors.
These include English (excluding English for teacher certification); French; geography and geosciences; German; history (excluding social science for teacher certification); philosophy; political science; sociology; and Spanish.
Studies consistently show that voters with college degrees turn out in greater numbers than those with lower levels of education, but among college-educated voters it is likely that the type of coursework taken in college is an additional contributing factor to greater and more perceptive participation in civic life.
In the higher ed world, this link between education and civic engagement is known as the “civic education hypothesis.” A recent paper by Jacob Andrew Hester of the University of Alabama and Kari Lynn Besing of Indiana University argues persuasively that honors seminars, notably in the humanities and social sciences, “can and often do impart the civic skills that, the civic education hypothesis posits, enable political participation and lead to increased involvement in politics and civic life.”
Many public universities are unable to offer small, discussion-focused classes in these disciplines. The authors contend that larger lecture sections do not develop “the classic skills associated with politics: language, rhetoric, public speaking, debate, and critical thinking.” Students can, however, develop these skills in an honors college or program that offers small seminar sections in Gen Ed courses.
Humanities and social science departments have for centuries sought to inculcate these “classic skills.” For years they have been losing faculty; now, with the elimination of majors, more faculty will be cut and course sections will be reduced, probably leading to larger classes with no opportunity for discussion. Where will these disciplines, with their manifold intersections, survive in a format conducive to civic education? Honors colleges and programs–and the mission is critical.
“Honors education and the humanities share core values, including the importance of deep, sustained reading. Students of history, literature, and philosophy confront complex and demanding texts and develop sophisticated methods of analyzing these texts….Both humanities and honors value not only high levels of reading skill but thoughtful responses to texts and an ability to integrate them into broader knowledge, reaching toward not just learning but wisdom. Such habits run counter to the mindless consumption of infobits.”
Some of the brightest students are math, science, and engineering majors, and their numbers are on the rise. Their analytical skills are seldom in question–indeed, they are often amazing. But the classes in their majors offer little discussion and, as Hester points out, “Math courses [for example] rarely involve discussion or conceptualizing social issues, and very rarely if ever do math instructors connect the development of mathematical skills to political discourse.”
On the other hand, Hester and Besing write, the “University of Alabama (UA) Honors College has an explicit goal of developing ‘agents of social change.’ At the heart of the honors experience are three-hour, interdisciplinary, honors seminars for no more than fifteen students. To graduate with honors, UA students must complete no fewer than six hours of seminar credit, but often students complete more.
“In contrast to the traditional academic lecture, the skills developed in a seminar are uniquely suited for the development and application of citizenship behaviors. In particular, UA honors seminars stress discussion, reflection, writing, and debate, providing students the opportunity to practice each behavior in a controlled environment. Through the seminar experience, honors students are expected to engage the skill sets that produce interest and competence in public affairs more frequently than non-honors students.”
To test their hypothesis that honors programs can promote civic education, Hester and Besing surveyed University of Alabama Honors College students to answer the following question: “To what extent has your experience at this institution contributed to your knowledge, skills, and personal development in the following areas: Voting in local, state, or national elections?”
The conclusion: “Students who engage in a curriculum with more opportunities to develop civic skills are more likely to respond that their institution has contributed to their interest in voting. This finding lends support for the civic engagement hypothesis within the context of an honors education. Specifically, it suggests that students in the UA Honors College are more likely to respond that their education has contributed to their interest in voting. Similarly, our findings suggest that the amount of reading and writing in their curriculum positively correlates with students’ perception that their education has had an impact on their interest in voting.”
“Our argument is that seminar courses are likely to contribute to an honors student’s interest in participating in politics, but we do not believe that honors electives have the same effect. For example, an elective honors lecture course in accounting is likely to be more enriching than a non-honors version of the course but is not likely to build political skills in the same way that a seminar does.”
“On one side of the debate, policymakers, employers, and administrators extol the benefits of a STEM education, e .g ., technological innovation, expansion of research, and the financial payoffs of a labor force with robust science and mathematics skills. On the other side, classical theories of higher education argue that a college degree is about more than the development of a professional skill set on the way to a career; it is about the development of each individual’s ability to function as a citizen in a democratic society. An honors education provides a unique opportunity for higher education institutions to satisfy both sides of the debate, proving sufficient rigor for STEM students while also grounding students in the classical purposes of higher education.”
A recent, excellent piece inInside Higher Ed, by Rick Seltzer, explores the pros and cons of public honors colleges’ charging extra fees (or differential tuition) in order to enroll and serve increasing numbers of honors students.
(Here we can pretty much confine the discussion to honors colleges because honors programs rarely charge significant fees for attendance.)
At the end of this post is a list of honors colleges that have significant honors fees, and the fee amounts.
Much of the piece involves Barrett Honors College at Arizona State, and Barrett Dean Mark Jacobs is a proselytizer for charging the extra fees and is proud that Barrett has been successful, telling Inside Higher Ed that “when you’re an educational institution, the best you can talk about in terms of the effect outside your own institution is hoping that good ideas you have might be copied and used by other people, or translated to fit their context.”
Ten years ago, Barrett enrollment cost each student $250 a semester. Now, the fee is $750 a semester, or $1,500 per academic year. With the cost of in-state attendance at ASU now at $28,491, the honors fee adds about 5% to the total cost.
One of Jacobs’ arguments mirrors those of almost all public university honors deans and directors: The “liberal arts college within a major research university” model is a bargain for students who would pay much more to attend a good liberal arts college or a strong private elite research university. So, even with the extra charge, public honors remains “a smoking deal” and “an absolute steal.”
Jacobs is in a position to know whereof he speaks; he has bachelors with high honors from Harvard, a Ph.D. from Stanford, and he had an endowed chair in biology at Swarthmore.
Another argument is that state funding cuts have put public universities in a bind, and the extra fees for honors help expand those and other programs at the universities. In addition, public honors colleges (and programs) give highly-talented students in-state options that are in great need given the increased selectivity and arbitrary admission standards of elite universities.
One thing not in doubt is whether the practice at Barrett has helped financially. “In 2017,” Seltzer writes, “the college draws 36 percent of its budget from general operations and 4 percent from endowment income. A whopping 60 percent of the budget comes from the fee.”
On the other hand, Bette Bottoms, dean emerita at the University of Illinois Honors College and a longtime leader in honors education, maintains that universities should value their honors colleges enough to put institutional money into them and not ask students to pay the costs.
“Now, if you tell me that Arizona [State] has some way of waiving the fee for lower-income students, that makes the model more palatable, but I still don’t agree with it,” she told Seltzer. “Do incoming students know this? We never charged a fee, and I found that prospective students and their families often expected it anyway — I’m sure this kept some students from even considering applying.”
“Arizona State must set aside 17 percent of its honors college fees for financial aid,” Seltzer writes, and, according to Jacobs, “Barrett students can receive need-based and non-need-based aid from the university’s central financial aid office. Students can also receive aid from the honors college in the event their financial aid packages are not enough to allow them to pay the fee for being honors students.”
The Barrett model has influenced at least a few other honors colleges. The new Lewis Honors College at the University of Kentucky will charge a $500 annual fee. The namesake of the LHC, Tom Lewis, donated $23 million to his alma mater to create the new honors college. He is also an Arizona resident and longtime supporter of Barrett, who likely believes the Barrett model is a good one to follow.
The issue of elitism at honors colleges (and programs) is also a factor. Even though Barrett goes out of its way to connect hundreds of ASU faculty, honors students, and non-honors students through the extensive use of honors contract courses, the physical separation of the honors campus can be a negative for some while it is a positive for others.
Our own view is that the extra fees can have an overall positive impact if they do not exceed, say, 5% of the in-state cost of university attendance and if the honors colleges have resources to assist students for whom the fee is a burden.
Another way to measure the impact of the extra fees is to analyze the extent to which they might discourage students from completing the full honors curriculum.
The honors college that charges the most in extra fees (actually differential tuition) is the Clark Honors College at the University of Oregon. There, students face an extra charge of $4,192 per year, which amounts to a 15.8% increase in tuition. Some scholarships to offset the very considerable charge are available after the first year.
It may be noteworthy that Barrett and Clark have similar student profile stats, though Clark students have somewhat higher test scores (new SAT 1410 to new SAT 1350). The six-year grad rate for Barrett honors entrants was 89% and for Clark entrants, 82%.
Oregon State Honors College has a differential charge of $1,353, not too much below the fee at Barrett. Oregon State honors entrants had a six-year grad rate of 87.6%, with a sizable portion of engineering students. The average (new) SAT at the OSU Honors College is about 1430.
While this is not definitive data, it only makes sense that the greater the differential cost, the more honors students will be forced to balance the value of their honors education against the cost or simply conclude that they cannot afford honors at all.
Editor’s Note: The following information is from the University of Arkansas Honors College. The college dean has designed the Honors Passport experiences, a capstone course abroad. “Honors Passport courses send honors students and top faculty scholars to historically and culturally significant sites around the globe. During these two-week intersession courses, each student much research and present on a historic site, monument or notable individual, taking an active role in teaching the course.”
Sixteen Honors College students recently spent a full semester preparing for study abroad in Peru, and landed in Lima well-versed on the Incan Empire, the Andean Hybrid Baroque and indigenismo.
Arkansas Honors Dean Lynda Coon and Prof. Kim Sexton, Fay Jones School of Architecture and Design
“The idea is to create an international capstone experience where students and professors together explore the interaction of contemporary and historical sites, texts, and artifacts,” said Honors College Dean Lynda Coon.
Honors College Dean Lynda Coon has launched a series of innovative honors courses since joining the history faculty in the J. William Fulbright College of Arts and Sciences in 1990. She helped to create the Honors Humanities Project (H2P) and as dean she has developed Signature Seminars, Forums, Retro Readings courses and this Honors Passport study abroad experience. Coon’s research focuses on the history of Christianity from circa 300-900.
Kim Sexton, an associate professor of architecture at Fay Jones School of Architecture and Design, specializes in the architecture of late medieval and Renaissance Italy. Since joining the Fay Jones School’s faculty in 1999, Sexton has taught survey courses in the history of world architecture, specialized courses on medieval and Renaissance architecture, and space and gender theory. Sexton is currently completing a book manuscript entitled Loggia Culture: Spatial Practices in Medieval Italy that positions the loggia or portico in cultural history.
Arkansas psych major Linh Luu giving a presentation at Santa Catalina, a Dominican convent in Arequipa, Peru.
Dean Coon and Professor Sexton have taught the second semester of H2P since 1999. They also developed Medieval Bodies/Medieval Spaces, an interdisciplinary honors colloquium that traces the evolution of western medieval history through text, ritual and built environments.
Editor’s Note: The following news is from the UT College of Liberal Arts.
New Plan II Director, Dr. Alexandra Wettlaufer
Plan II is pleased to announce Dr. Alexandra K. Wettlaufer as our new Director, following Dr. Michael Stoff who served as the program director for the past 11 years. Dr. Wettlaufer has been the Plan II Associate Director since 2005, overseeing the program’s senior thesis course and developing a deep love for the program and our students.
She is a Professor of French and Comparative Literature, specializing in 19th-century literature, visual arts, culture, and gender studies. A recipient of a 2014-15 Guggenheim Fellowship, Dr. Wettlaufer is currently working on a book project entitled “Reading George: Sand, Eliot and the Novel in France and Britain, 1830-1900.”
She is the author of three previous books: Pen vs Paintbrush: Girodet, Balzac and the Myth of Pygmalion in Post-Revolutionary France (2001), In the Mind’s Eye: The Visual Impulse in Diderot, Baudelaire and Ruskin (2003), and Portraits of the Artist as a Young Woman: Painting and the Novel in France and Britain, 1800-1860(2011). She has published numerous articles on Balzac, Sand, Baudelaire, Zola, Manet, Ruskin, Turner, Berlioz, Grandville, and Flora Tristan; her article “She is Me: Tristan, Gauguin, and the Dialectics of Colonial Identity” (Romanic Review,2007) was awarded the Nineteenth-Century Studies Association Essay Prize, Honorable Mention.
Dr. Wettlaufer has received fellowships from the Whiting Foundation, ACLS, Bourse Marandon, the Clark Art Institute, and the National Humanities Center. Her teaching awards include a President’s Associates’ Teaching Award, the Blunk Memorial Professorship in Teaching and Advising, a Raymond Dickson Centennial Endowed Teaching Award, a Liberal Arts Council Teaching Award, and University Coop Award for Undergraduate Thesis Advising.
She is the Co-Editor of Nineteenth-Century Contexts: An Interdisciplinary Journal and serves on the Editorial Boards of European Romantic Review, Nineteenth-Century Studies, George Sand Studies, and Dix-Neuf. Dr. Wettlaufer has also served on the Advisory Boards of the American Comparative Literature Association, Nineteenth-Century French Studies Association, Interdisciplinary Nineteenth-Century Studies Association, and on the Executive Council of the Modern Language Association. Dr. Wettlaufer is a core faculty member of Comparative Literature, Women’s and Gender Studies, and European Studies.