CORNERSTONES
Task Force #3 -- "Institutional Integrity, Performance and Accountability"
Draft Report -- March, 1997


Table of Contents

     I.    Introduction
     II. Chapter One: Framework for Accountability
     III. Chapter Two: Report Cards: An Approach to Public Accountability
     IV. Chapter Three: Assessment of the Institutional Learning Environment
     V. Chapter Four: Employee Performance Measures
     VI. Chapter Five: Accreditation
     VII. Chapter Six: Program Review
     VIII. Chapter Seven: State and Federal Regulations
     IX. Chapter Eight: Stakeholder Information

 

 

Introduction

The California State University is committed to being fully accountable as a public institution, both internally and externally for the quality and integrity of the learning experience. Our dedication to accountability is driven internally by the desire to continue to provide an education that is both affordable and of the highest quality. We believe that as an institution, we must focus our energy on constant internal improvement. Our primary motivation for strengthening accountability is the desire to know the extent to which the CSU is doing what it professes to do.

Measuring the contributions of an institution by how much and how well its students are learning and how their educational experience is affecting their values and attitudes, speaks directly to the concerns of state government and the public at large. This philosophy is part of the CSU mission. We are clearly committed to being accountable for the value added to our students’ lives as a result of their CSU experience.

A variety of factors contribute to the increased interest in accountability and assessment. The public now commonly “perceivesÓ that diplomas and degrees from American schools and colleges no longer guarantee basic literacy or numeracy. Thoughtfully developed assessment programs demonstrate significant potential for improving communication between faculty and students, for enhancing learning, and for helping faculty and staff reach consensus on changes that will improve program quality and performance. Governing boards have the right and responsibility to receive timely, accurate and useful information about the effectiveness of institutions. Elected officials who are forced to make painful choices want clear, comprehensible evidence of institutional effectiveness upon which to base their decisions.

In this context, we examine the basis on which accountability has been defined in the past and seek both clarification of and improvement in the means by which our performance is evaluated. In particular, we focus on the linkage between internal program assessment and external communication of that performance. The goals are to discover information to help guide continuing internal improvement and to demonstrate to external constituents the results of the state’s investment. The public also needs effective education as to the important role the CSU plays in the economic and political future of both the state and nation. This report takes the disparate CSU activities designed to ensure accountability and transforms them into a comprehensive array of recommendations designed to improve performance of the institution and to affirm to the public the CSU's commitment to excellence and accountability.

Chapter One: Framework for Accountability

Accountability is the intersection of goals, policies, programs and performance. An essential element of the Cornerstones effort is the integration of an effective system of accountability into the fabric of institutional planning and resource allocation . To do that requires clearly articulated, measurable educational goals, which are the subject of much of the work of Task Forces #1 and #4, and a system of accountability itself which has been reengineered to meet the needs of the future rather than building on the systems of the past.

CSU is already "accountable" in a number of ways and to a range of constituents. But this approach to accountability has been incremental rather than comprehensive, with myriad reporting systems and controls being built successively on top of one another. There has been no systematic effort to link the internal systems of assessment and improvement with effective external communication of performance on goals. Finally there is a need for a fresh look at the whole regulatory system within which the university operates, both from the perspective of the State of California in relation to the CSU as a whole, and within the institution from the campuses to the Chancellor's Office. While CSU has come a long way from its origins as a wholly state-controlled agency, with heavy central controls from Sacramento and from the Chancellor's Office to the campuses, there is a need for an explicit re-mapping of the accountability relations as they should exist in a decentralized, deregulated environment.

For purposes of demonstrating accountability as a teaching and learning institution, we believe it useful to focus on two dimensions of accountability, which we have chosen to call performance accountability and regulatory accountability. Performance accountability as we have defined it is the alignment of system goals and purposes with overarching system performance. It is goal-oriented and assessment-centered. To achieve performance accountability requires clarity about goals and expectations, and sharp assessment of performance using a variety of measures. Performance-centered accountability speaks to institutional achievement, not control of the means by which goals are obtained.

Although performance accountability is clearly focused on educational outcomes, it also covers institutional culture and its effects on performance. For example, campus climate with respect to racial, cultural, and gender diversity and administrative style with respect to communication and shared governance are key elements of institutional culture that can impact the performance of a university in achieving its stated goals.

Regulatory accountability is quite different. Regulatory accountability is control of the means by which the institution uses resources, to ensure that they are fully accounted for and used legally. Regulatory accountability covers topics such as personnel transactions and fiscal accounting, where there is a need to show to government or other regulators that the transactions are legal and appropriate and meet the minimum standards of conformity to the law.

The CSU needs to be and is accountable in both the performance and regulatory dimensions of accountability. One goal of a realigned accountability policy is a better match between the goals of any set of activities and the means by which performance is assured. The problems we find with our current systems of accountability and strategies for improvement are detailed below.

Purpose and improvement: A fundamental driving force for our fresh approach to accountability is a dedication to providing the highest quality and highest value education. CSU is prepared to ask itself some simple but very probing questions about the extent to which we are doing what we profess to do, and to use the answers to identify where and how to do better. The purpose and mission of the CSU, and the hallmarks of excellence in our teaching and learning environments, have not been as crisply stated and precisely communicated in the past as they need to be in the future. We believe that the quality of the teaching and learning experience in the state university is as good or better than that in many institutions with whom we compete for resources and for students. An effective system of performance accountability, which links goals and performance, will define the terms of excellence for which the CSU will be accountable.

Reducing the bureaucracy: Higher education in general, and CSU in particular, has suffered from a tendency towards too much regulatory accountability and too little performance accountability. The result is an institution which too often seeks to influence not just that things get done, but how they get done. Examples of these types of regulations range from classroom space standards, to the calculation of student credit units as a proxy for student learning, to restrictions on use of tuition revenues. Regulations emanate not only from the state and federal governments, but from within the CSU, which is itself a major source of internal regulation. By its nature, regulation promotes compliance with minimum standards rather than imagination in the service of excellence. While regulatory accountability may be useful in ensuring legal use of resources, it has a profound and largely negative effect on the culture of an institution, and on means of communication, decision-making, and the core element of trust and mutual respect that is integral to shared governance. As a result, we believe that regulation -- whether by government or within the CSU -- should be an accountability tool of last, and not first, resort. There are other creative and more flexible means by which performance and accountability can be demonstrated which, in combination with selective regulation of resource use, can ensure accountability. Issues of the relative balance between central control and campus autonomy are an essential element of this within the CSU. As the institution achieves greater differentiation among the campuses, for accountability to be maintained the system will need to continually monitor each campus for how it is achieving statewide goals of access, equity and quality. Recognizing that each campus will meet these missions in more sharply differentiated ways, depending on its mix of students, programs and community needs, the public mission of the institution demands that there be systemwide net accountability among and between them on behalf of the entire state.

Achievements and not inputs: Measures of performance and achievement in higher education have generally been oriented to inputs - particularly entering student enrollment data and the quality of prior student preparation - and not to achievements. The CSU as a teaching and learning institution must find evidentiary ways of documenting its achievement in multiple dimensions, including learning outcomes among other measures of performance.

Multiple dimensions of success: It is important for CSU to take the initiative in defining the terms under which it believes it should be held accountable. A passive approach to accountability invites others - particularly government - to set out those terms instead, with a tendency to define success as a simple function of graduation rates. CSU's commitment to quality, access and success means that preparation, enrollment, retention, and graduation all need to be monitored over time. Disparities between the campuses on these key measures need to be monitored, to ensure that differences do not lead to inequalities.

Effective competition for resources: Declining public resources at both the state and federal level mean that higher education must compete for funding with other social priorities. We believe that the evidence is available to show that investment in higher education in the CSU is a very effective use of tax dollars. We must be prepared to bring this evidence together and to use it effectively in the political environment.

Distinguish between assessment and accountability: CSU has been an early proponent of collegiate assessment, and has been a national leader in developing complex measures such as the Student Needs and Priorities Survey (SNAPS). Assessment systems by their nature serve internal audiences, not external ones, and increased effective accountability to external audiences is one of our needs and priorities. In our view, assessment and accountability are complementary, and can be mutually reinforcing, but one is not a substitute for the other.

Strengthen the quality of stakeholder information: A powerful non-regulatory means of accountability is effectiveness in the competitive environment in higher education, by contributing to a better system of stakeholder information to students, families and employers. Without robust stakeholder information informing choice in the competitive environment , decisions will be made on the basis of reputation, rumor, or price without regard to quality. And without complex measures of excellence, including effectiveness of the teaching and learning environment, the single model of institutional prestige that tends to perpetuate itself is the elite research university model. We think there is a need to improve the effectiveness of the competitive environment in higher education, by contributing to a better system of stakeholder information to students, families and employers. A more mission-oriented, teaching and learning outcomes basis for measuring institutional effectiveness will improve the quality of information students have to make decisions about where to enroll.

Dimensions of accountability

What does it mean in practical terms for CSU to initiate a fresh approach to accountability? The following chapters of our report describe the essential elements of a comprehensive accountability system. The specific dimensions of accountability -- including both "performance accountability" and "regulatory accountability" -- that we suggest should be in place in a comprehensive system. These elements include the following:

Chapter Two: Report Cards: An Approach to Public Accountability

The first recommendation from the Task Force is that CSU embrace a variety of "report cards" to integrate the disparate pieces of an accountability system.

The term "report card" is generic rather than specific, as it has been used in other settings to embrace a wide variety of different types of reporting formats. What all report cards have in common is linking goals with performance on a subset of indicators. They can be data-driven but need not be data-centered, meaning that a successful "report card" for a general public audience might look more like a brochure than like a corporate report.

Improved communication about expectations and performance within the CSU is an important element of institutional accountability, and report cards can be a helpful device for facilitating that communication. They thus can be oriented to internal as well as external audiences. Regardless of format or audience, CSU will use report cards to forge the linkage between goals, performance and communication which is the essential synthesis needed for accountability to occur.

Functions of Report Cards

We recommend a variety of different report cards, appropriately formatted to reach different audiences, which can serve a variety of functions:

The Pyramid of Report Cards

We imagine a pyramid of different types of report cards that should be developed which are customized for different audiences and reflecting different purposes:

Principles for Report Cards

The crafting of detailed recommendations about what types of report cards should be developed and the benchmarks for performance that should be included in them, goes beyond the charge to this task force. A high priority for continued work that should precede any Trustee action on these policies is the elaboration of specific recommendations for concrete system policies on report cards.

To guide that work, however, we suggest some general principles that should be used in the development of CSU report cards:

Chapter Three: Assessment of the Institutional Learning Environment

The fundamental objective of the task force work on assessment is to build on current activities, practices, and policies in the CSU system for three critical yet very different reasons:

Assessment is an on-going process designed to measure the quality and efficacy of an educational experience to see if its predetermined objectives are being achieved because of the program. In other words, student achievement is measured in order to evaluate the impact, content, and pedagogy of academic and non-academic programs and activities. Assessment is an attempt by the designers and purveyors of a given educational experience to answer the question, "To what extent are we doing what we said we would do?"

To achieve the goal of institutional improvement based on the assessment of student learning outcomes, CSU first will have to agree on the components and objectives of the baccalaureate and postbaccalaureate degrees (Task Force #1 and #4). Then, using the agreed-upon objectives, campuses will be expected to measure student achievement and develop aggregate summaries. While such assessment will be undertaken first at the department and program level, measuring the institution's overall success will be an ultimate goal.

While assessment can act as a valuable tool for improving the institution’s effectiveness in delivering certain learning objectives, it cannot and should not capture the full breadth of a student’s education. The process of learning, development and maturation that a student undergoes while at the university must not be limited to that which is easily tested, or to a finite set of goals and objectives. Assessment is a tool which can assist the university in determining if its’ programs help students reach a certain level of high standards, but students must always be encouraged to reach beyond the standards we set for them.

In addition to providing data for the purpose of institutional improvement, outcomes assessment can and should be used in public accountability, that is, to document student performance in order to show external constituencies how well the university is fulfilling its educational mission. Together, the need for both institutional improvement and public accountability call for the creation of university-wide evidence about the effects of the teaching and learning process. No longer is it sufficient for a university to affirm and assert its successes; instead it will collect the data to demonstrate them.

As noted above, the third reason for assessment is as a base from which to begin examining traditional forms of learning and the accumulation of credit toward a degree. Some of the newest ideas about learning look at the form of delivery, such as the virtual classroom. Other ideas look at how units are calculated, possibly detaching seat-time from credit hours or encouraging students to learn at their own pace. Still other ideas might require faculty to evaluate individual students' prior learning, life experience and mastery as contributing to degree attainment. All of these approaches have implications for faculty roles as non-traditional teaching and learning require non-traditional ways of assigning units, potentially dictating changes in CSU policies and resource allocation.

The task force has organized its recommendations into three sets: The first two sets deal with evaluating the institution through assessing both student learning and the institutional environment. The third set of recommendations relates to the overall interaction of assessment to improved internal and external communication.

Recommendations on Program Improvement Through Student Learning Outcomes Assessment:

Recommendations on Assessment of the Institutional Learning Environment:

Recommendations Related to Improved Communication:

Chapter Four: Employee Performance Measures

The purpose of this examination of our policies and practices is to improve employee performance at every level. The expectation is that the Cornerstones recommendations will serve as the conceptual basis for this review process.

The relationships of current policy and practice concerning evaluation of employees to the broader issue of institutional improvement is the focus of this chapter. Task Force #3 examined four groups concerning the issue of employee performance measures and evaluation: campus presidents, Management Personnel Plan employees, tenured faculty, and staff. After reviewing Retention, Tenure, and Promotion policies from several campuses, the Task Force concluded that substantial change in this area is not necessary at this time.

Several issues were addressed regarding the adequacy of employee evaluations for all four constituencies:

The issue of how performance specifically related to compensation and other rewards has not been addressed in this phase of the analysis.

General Recommendations for Improvement

  1. Processes for evaluation of all groups of employees should be well understood, representative of responsibilities and more open to all involved or affected.
  2. The evaluation process should include a significant planning component, focusing the process towards the development and attainment of agreed upon outcomes.
  3. Processes should include timely and ongoing feedback.

Specific Recommendations for Improvement

Review of Presidents

The Chancellor has agreed to work with the presidents and Board of Trustees to strengthen the review process in the following areas:

Management Personnel Plan Evaluations

Two research projects have already been established in an attempt to improve the MPP evaluation process. First, the Corporate Leadership Council will provide a range of specific “best practice” evaluation instruments. The work by the Council is to be completed by January 1997. Second, a group headed by Dr. Fred Evans, Dean of the School of Business, CSU Fresno will take the Council’s suggested “best practices” and use them as the basis for an MPP performance evaluation model. This work will be completed during the summer 1997.

Task Force #3 also urges for the MPP evaluation policy:

Faculty Roles and Evaluation

The use of workload measures that truly reflect the nature and scope of faculty work has been a consistent problem in the CSU. Currently, the CSU and CFA are working on this issue. The task force recommends the following:

Staff

The Task Force encourages the current range of activities underway on the campuses to update evaluation and development processes for staff. The expectation is that the Cornerstones outputs will serve as the conceptual basis for the review process. In this regard a recent workshop sponsored by the office of the Chancellor and conducted by Mercer, Inc. emphasized performance planning, interim progress reports and a year end performance review. These concepts are congruent with the General Recommendations listed earlier.

Based on a review of some of the current plans on the campuses any future work should consider greater emphasis on linking unit and/or systems contributions to individualized performance characteristics.

Chapter Five: Accreditation

Accreditation performs a dual role of assuring minimum institutional standards and helping established institutions to improve themselves. The latter role of institutional quality improvement and change, rather than conformity to minimum standards, becomes the more important focus in the future of the CSU. There are two issues within the broader scope of accreditation review that are addressed in this chapter:

  1. The policy of the Board of Trustees concerning the accreditation of CSU academic programs by specialized accreditation groups;
  2. The role of the CSU in the regional accreditation agency, Western Association of Schools and Colleges;

Current Board policy on specialized accreditation in the CSU was approved in 1968 and encourages all campuses to seek specialized accreditation for its programs.

The task force has concluded that the current Board policy requires changes for several reasons. First, the 1968 policy is obsolete including language requiring at least three specialized professional accreditations in order to carry the "university" identifier. Other changes in the overall circumstances of higher education and in the accreditation arena necessitate a review of this policy.

Second, and most importantly, we challenge the underlying assumption of the 1968 policy that specialized accreditation is always a positive undertaking with good outcomes. While we recognize that some specialized accreditations are necessary for licensure to work in a professional area in a state, not all specialized accreditations have this established link. In addition, the current policy is silent on the link between regional (institutional) accreditation and the specialized program accreditations.

The recommendations around regional accreditation are oriented towards change in the way CSU interacts with the Western Association of Schools and Colleges (WASC).

Currently, WASC is examining the standards it employs in its accreditation procedures to stress institutional improvement for established institutions, and to encourage the use of a variety of types of evidence to evaluate quality. The changes in WASC practices provide the CSU an opportunity to use accreditation in creative ways, particularly in linking a variety of assessment tools and in relation to planning and program review.

Accreditation in its most useful form should be a non-bureaucratic, collegial endeavor. It provides an opportunity for colleges and universities to improve based on recommendations from their peers. The changing environment in California demands greater public accountability. A stronger public role for accreditation in accountability is preferable to an expanded governmental regulatory role, which by its nature is control-oriented and can lead to excessive standardization.

Recommendations for Improvement

Specialized accreditation

Regional Accreditation

Chapter Six: Program Review

The formal requirement for a qualitative review of existing academic programs has been in place in the California State University since 1971, when the Board of Trustees directed that each campus review each academic degree program on a periodic basis.

While the requirement to review all degree and general education programs is systemwide policy, criteria and procedures for conducting the review are unique to the respective campuses. The summaries of the results are presented to the Board along with the Academic Master Plan.

The task force engaged in long, comprehensive discussions about program review. The cumbersome and time consuming nature of the current process was resoundingly criticized. Also criticized was the lack of “systems focus” in the process. It does not appear to encourage departments or programs under review to plan and evaluate in the context of the overall academic program. The third major criticism was an apparent lack of outcome or consequence (positive or negative) from the process. Finally, the program review process was criticized for its focus on compliance rather than performance or planning.

Also discussed was the absence of a link between program planning and planning for individual faculty within a department. The planning and evaluation processes are entirely separate and quite disjointed.

Recommendations for Improvement

Chapter Seven: State and Federal Regulations

The work on this issue included an analysis of the regulatory environment within which the CSU operates, and recommendations about ways to strengthen the role of regulation as one element of accountability. The analysis notes that there are three primary sources of regulation for the CSU - the federal government, the state of California, and the Trustees.

For the purposes of the Cornerstones work on accountability, an initial focus on those regulations which distinctively and uniquely apply to educational institutions was agreed on. That means that regulations from OSHA and EPA are not are primary targets for this review.

The effect of this focus would be to begin with three primary targets for a fresh look at regulation and accountability: 1) Title IV of the Higher Education Act, 2) California Education Code and 3) Title 5 of the California Code of Regulations. The upcoming reauthorization of the federal higher education act provides an opportunity for headway on the federal side.

Recommendations for Improvement

Chapter Eight: Stakeholder Information

The task force challenges the assumptions of what has been designated as the resources and reputation model in higher education. In an effort to provide an alternative to prospective students and other constituents to basing decisions about the CSU on this tautology, the task force explored the issues of stakeholder and public information.

Some types of public and stakeholder information are required by law. It is the responsibility of the CSU to develop the most effective mechanisms for communication of the information required by law and other information needed by constituents.

The ability of the CSU to better articulate its mission and how well it is doing in achieving that mission is indirectly related to accountability. Stakeholders need clear, concise and honest information about the CSU.

The public relations aspect of this issue cannot be overlooked either. The only way for the CSU to overcome its “image” problem is to counteract the stereotypes with concrete, factual information.

Recommendations for Improvement