Beyond the Big Test: Noncognitive Assessment in Higher Education
By William E. Sedlacek
Psychology and Sociology Department
California State Polytechnic University, Pomona
The "Big Test" in the title of this book refers to the Scholastic Aptitude Test or SAT and immediately brings to mind Nicholas Lemann's book, The Big Test: The Secret History of the American Meritocracy (Farrar, Straus and Giroux, 2000), in which Lemann tells the story of how this test evolved, the ideas upon which it is based, and the politics associated with its history.
William Sedlacek takes issue with the SAT because, as he argues, "we need tests that are fair to all and that give us a good assessment of the developmental and learning needs of students, while being useful in selecting outstanding applicants to colleges and universities" (6). According to Sedlacek, as well as a variety of other researchers, the SAT is woefully inadequate to these purposes.
What is the answer? Just as Professor Harold Hill in the Music Man was enthusiastically trying to convince the residents of River City, Iowa, that a boys' band would solve many of their problems, Professor William Sedlacek, with similar enthusiasm, wishes to sell the higher education community on the value of using noncognitive variables or NCVs to augment and in some cases replace the SAT, which even today basically measures only verbal and math ability or cognitive intelligence. Professor Sedlacek's motives, however, are quite different from the selfish ones of Professor Hill, who wished to sell nonexistent band instruments to the gullible town's folk. Sedlacek wants to improve higher education: He is not trying to sell the "instruments" he and others have developed, but instead has included them in this volume and invites readers to adapt them to their purposes. Sedlacek wants institutions of higher education "To offer the best possible education for all students" (22), and he reasons that the only way to do this is to ascertain the needs of diverse students who come to the university with different experiences as well as abilities. He recognizes that universities are becoming much more diverse than they were when the SAT was first developed and implemented. Sedlacek is persuasive in his argument that the use of NCVs can help colleges and universities become more diverse while at the same time improving the experience for all those involved in the process.
Just what are NCVs? Sedlacek is principally concerned with eight noncognitive variables. They are positive self-concept, realistic self-appraisal, successful handling of the system (racism), preference for long-term goals, availability of strong support person, and leadership experience.
Sedlacek begins his book by discussing why the SAT is so popular and why there is a compelling need to go beyond it. In essence, there are two primary reasons that institutions and systems of higher education must develop alternatives to the SAT. First, the SAT does not predict the academic success of college students very well, and its failures are most evident in the case of nontraditional students. Sedlacek's use of the term "nontraditional" is rather inclusive, referring to students "with cultural experiences different from those of White, middle-class, heterosexual, males of European descent, those with less power to control their lives and those who experience discrimination in the United States" (4). UC system President Richard C. Atkinson has expressed a similar sentiment that led to a revision of the SAT, which has recently become available. The revised SAT will not, however, address Sedlacek's and perhaps not Atkinson's primary criticism.
A second important reason for developing new tests is to provide diagnostic information about new students, which can be used after they have been admitted to college in order to improve academic performance, increase retention, and improve quality of student life on campus.
Beyond the Big Test also includes a chapter on the basic concepts of test construction, with a useful refresher on reliability and validity. The subsequent chapter deals with how reliability and validity in testing are related to diversity and inequality in populations. It is interesting to note here that Sedlacek defines racism as "all the inequities that various groups of nontraditional students face" (22). Presumably inequitable treatment based on socioeconomic factors, as well as gender, national origin, and sexual orientation, physical, and mental ability would all be considered as examples of racism. Chapter 4 presents the author's noncognitive assessment model including definitions and discussion of the noncognitive variables. In Chapter 5, attention is given to the use of NCVs in the admission of student applicants and the granting of financial aid. Chapter 6 deals with the use of NCVs in teaching while Chapter 7 makes a case for the value of NCVs in the advising of students. Chapter 8 argues for employing noncognitive variables to evaluate programs and services in institutions of higher education. There are twelve appendices that contain attitude scales, noncognitive questionnaires, scoring guides, interviewer schedules and instructions, sample cases for training raters, materials for a class on racism, and a campus climate survey.
Throughout the book Sedlacek cites research to support his contention that the SAT is a poor predictor of academic success, especially after the first year of college. He also cites numerous studies to lend credibility to his argument that NCVs are better predictors of academic aptitude and success in college. It should be noted, however, that in the latter category Sedlacek is primarily citing himself, reporting on small studies, and the sample of institutions represented is rather narrow. He cites forty articles where he is the sole author and quite a number of studies in which he collaborated. Although he clearly has an enviable publication record, it would have been more persuasive to this reader if there had been more independent corroboration of the author's findings and the arguments presented. Also, in his desire to "sell" NCVs he has a tendency to stretch his interpretation of research findings to support their use, especially with non-traditional students. One example may illustrate: He argues that because "African Americans and Asian Americans were more interested in joining campus organizations than other transfer students" this suggests that "both these groups develop support systems through community involvement" (109).
In addition, with Sedlacek's concept of "nontraditional" being so broad, it would seem likely that the instruments for measuring NCVs and the methods for scoring the results might take rather different forms depending on the type of nontraditional student with which you are dealing-leading to a proliferation of test forms and templates for interpretation. There is little consideration of this implication.
One might also take issue with some of the items Sedlacek uses to measure one or another of the NCV's or the interpretation he makes of the responses. For example, in his Noncognitive Questionnaire, he uses the following item to measure self-concept and realistic self-appraisal: "About 50 percent of university students typically leave before receiving a degree. If this should happen to you, what will be the most likely cause?" The response that scores highest for the test taker is "Absolutely certain that I will obtain a degree" (169). This would seem to indicate a relatively high self-concept but should it be considered a realistic self-appraisal? The item, "It should not be very hard to get a B (3.0) average at this school" (170, emphasis in the original), is another one of the items used to measure realistic self-appraisal. The highest scoring response is "Strongly agree." Depending on the school and the student, this may or may not be a realistic appraisal.
There are several suggestions in this book for gauging students in terms of NCVs. However, if institutions were to use something as easy to administer and score as Sedlacek's Noncognitive Questionnaire for one of the criteria of admission, some problems might develop. One can well imagine the cottage industry of test preparation services that would spring up to provide, for a price, the materials, classes, and coaching designed to improve a student's showing on variables such as positive self-concept, realistic self-appraisal, successful handling of the system, and preference for long-term goals. This will be especially true when it becomes well known that students who want to score well on the NCV portion of an entrance examination need to strongly agree with items such as, "My family has always wanted me to go to college," as it indicate availability of strong support, and strongly disagree with items such as, "I expect to have a harder time than most students at this school" (170), which indicate "a realistic view of the system on the basis of personal experience of racism" (37).
Although it is clear that there needs to be a better way to give diverse students more equal opportunity for admission to colleges and universities than is currently offered by even the recently revised SAT, it seems to me that perhaps there is a stronger case in this book for employing NCVs, at least as conceptualized and measured by Sedlacek, in order to better teach and advise students once they are studying at an institution rather than using these instruments for admission. Professor Sedlacek makes a strong case for the value of NCVs in retaining nontraditional students and improving the campus climate for the entire student body. In this volume, the author provides valuable information on how to work with students to improve their self-concepts, develop more realistic self-appraisals, handle racism, develop long-term goals, and become the sort of students who will do well at the university. For these reasons, this is a valuable book not only for educational psychologists, but also for the faculty, staff, and administrators of institutions of higher education in general.
Posted March 7, 2005.
All material appearing in this journal is subject to applicable copyright laws.
Publication in this journal in no way indicates the endorsement of the content by the California State University, the Institute for Teaching and Learning, or the Exchanges Editorial Board. ©2005 by Gary A. Cretser.