Overall Themes:

Following is a compilation of all feedback received through the discussion groups at the Academic Conference during Session E, Task Force #3. This is a summary of comments received from nearly twenty separate groups, so there are some inconsistencies and contradictions, particularly in "tone" or "voice". The comments have been grouped by topic since many groups discussed different issues than the questions posed to them.

Some overall themes/concerns:

• Concern about a "top down" system and distrust of many proposals
• Concern about workload, and many comments about the unfairness of PSSI's/ pay for performance
• Many concerns that assessment instruments would drive the curriculum, or that some would merely "teach to the test."
• Some concerns about particular words or phrases used that seemed too market-oriented or value-laden.

Topic:

Report Cards:

• The idea of report cards is useful-can have positive and negative consequences.
• Danger is in that report cards lead to simplistic way of looking at things. It should be a means to an end. We must know what the outcomes are. There must be rewards and not just punishment,
• Communication has been a problem. We need to be open and accountable. Report cards may be a starting point in addressing this problem.
• Sometimes resources are taken from one college and put into another to reward enrollment. Report cards could be misused and some departments could wither away.
• Each campus should have clearly defined outcomes and present a portfolio on what they do.
• Report cards can help explain what we are doing, and help with public relations.
• We don't have a set of rules that say how resources should be allocated. Same problem with PSSI's. We don't trust the administration and the legislature with this information.
• All report cards should be public at all levels, including for the Chancellor.
• We must know who the report cards are for and what their purpose is.
• While targeting report cards for constituents is important, sometimes they may want the "wrong" information. Example: legislators may only be interested in FTES.
• The importance of making sure the public supports the CSU was stressed.
• There should be systemwide guidelines for what to report to the legislature from the Chancellor/system, but each campus should decide what to include in the detailed report to the public.
• Information submitted to the public should not be the information upon which resource allocation is based.
• Report cards can give us the freedom to put out the kind of information we want to put out.
• Report cards could have significant resource implications.
• How do you explain to the public that a professor teaches two days a week?
• There is a fear that report cards simplify what we do.

Topic:

Assessment

• Provision must be made for workload acknowledgment of assessment.
• Campuses appropriately use a variety of assessment mechanisms (portfolios, tests, interviews)
• There are costs associated with updating self-instructional materials and assessment instruments.
• The fit with administrative and student support services may be awkward.
• If we give faculty the same workload credit for assessed learning as we do for "taught" learning, we will have a public relations problem.
• Appeal of assessment decisions could become very time-consuming and costly.
• The feasibility of assessment strategies may differ by discipline.
• Some expressed the view that a program should be so many units of value added instead of achieving a fixed standard.
• Some centralization of the mechanisms for awarding credit may be considered.
• Should there be special charges for assessing out of a course?
• We have been assessing all along. This is a significant part of our jobs as educators.

Topic:

Assessment Instruments/ measures of learning outcomes

• Student outcomes measures should have multiple indicators at various benchmarks, i.e. linking connections among the curriculum: writing requirement in each course.
• Link assessment to the reasons why we assess (for feedback to the program, for institutional accreditation, for public relations)
• Cogently tie assessment measures to resources required
• Identify workforce/employer needs to design some assessment indicators
• There is a need for specific objectives
• Not everything can be assessed
• Some outcomes measures could be: grades, standardized tests, portfolio, student self-assessment, discipline originated measures, exit interviews, state regulated competencies
• Faculty self-assessment is also important (PSSI's are unfair and divisive)
• The difference between the imposition of assessment and the creation of assessment was discussed.
• There was a concern expressed about the perceived attempts to have learning organized into discrete modules through which students could self-pace without class participation in order to save time and get more students through the system
• When to assess: unable to complete discussion but some indication that multiple points of assessing student proficiencies is needed.
• Need for benchmarks across campuses within disciplines in order to be fair to students.
• Students react positively when expectations are explicit

Topic:

Competency-Based Admission Requirements

• One group expressed strong support for a competency based admission requirement (such as the Golden State exam).
• Any competency-based admissions "exam" should include an essay, open-ended questions, demonstrated writing skills, a variety of questions, etc. It should be comprehensive
• The CSU must help K-12 to prepare students effectively to pass the exam through outreach programs and joint work with faculty from both segments. It is important that they do not just "teach to the test."
• Strong support for the idea that the CCC should be doing the majority of remedial preparation, not the CSU
• Another group had concerns about the effect of competency-based admissions on access, particularly for underserved communities. This could lead to further stratification of society, with the community colleges doing the "real" work of education.
• There appears to be a contradiction in the support for decentralization and the support for centralized, standardized testing.

Topic:

Specialized and Regional Accreditation

• Specialized accreditation is necessary for many of our academic programs, and when they are sought they should be built into the regional accreditation process
• General agreement that the evaluative process should be more mission driven where outcome assessment is an integral and significant part of the process.
• There was some concern about over-emphasis on outcomes sliding into "the ends justify the means" mentality.
• WASC does/should play a significant role in providing a check and balance to our internal evaluations, and providing public assurance about quality.
• General support for the recommendations in the task force report on academic audits and using outcomes assessment.

Topic:

Program Review

• Program review can be valuable to departments, universities, and the system.
• Reviews need to be external. They need to be carefully selected
• Can be an excellent opportunity for self-assessment, goal-setting, reaching consensus on definitions of quality or learning outcomes
• Follow-up is absolutely necessary. This may include understandings with deans or VP's, review by Academic Senate Committees, and tied to resources which reward successful ones and nurture those which need the additional assistance.
• Program reviews may be too frequent.
• The current program review process is not very valuable because there are neither rewards nor punishments for outcomes. Generally the group agreed that there needs to be incentives for being honest if we want people to really use the process to address problems.
• If programs are being assessed by outside accreditation bodies they should not also be required to do program review, or should be required to do only those aspects of program review that augment what other reviews provide.
• Ongoing assessment is much more desirable than periodic evaluation.
• Much of the data departments need or should look at in doing a self-study should be routinely collected by other support offices, but it is important for departments to have input into surveys sent to these offices.
• There are important public relations reasons not to share all weaknesses too broadly.
• Higher level administrators need to take program review seriously or not require it.
• Program review tends to be too narrow and not have the opportunity to determine if the department has enough campus support services (physical plant, technology, etc.)
• Many campuses do not know how to do program review and outcomes assessment in meaningful ways. They need training and models.
• Annual reports should be linked to unit and school. University strategic plans. These are more important on-going assessments than program review.
• When faculty sit down to look at their major curriculum we should encourage them to also look at GE and how GE goals do or don't support major goals.
• Combine program review with outcomes assessment for more and better information.

Topic:

Relationship between Program Review and Accountability

• Accountability can't be something else to do. "Supplant not supplement" is a useful idea.
• The data collected for program review may be used for the Accountability Report Cards
• There may be standardized tools (ETS program self-study) that may help both program review and accountability. However, such standardized tests must take into account socio-economic differences.
• It must be remembered that the motivation for program review, WASC and accountability are different. Program review is designed for internal audiences with the goal of helping a program develop and achieve excellence, even though a part of it is self-promotion.
• The systemwide use of program review could facilitate and promote the development of regional collaborative endeavors. However, central administration shouldn't dictate such efforts. Meetings of the chairs of departments across the system could be helpful.

Topic:

Assessment and Program Evaluation

• How do you determine who makes a difference as a faculty members?
• There are many intangibles that are qualitative rather than quantitative.
• We must assess the very long term effects of teaching on our students
• The development of goals should occur on the local level.
• All stakeholders (faculty, students, alumni, employers) should have a role in developing goals.
• All courses may not have external goals that are relevant.

Topic:

Presidents/ MPP Evaluations

• All evaluation processes need to be better explained to faculty.
• The criteria for evaluation processes need to be objective, confidential, and fully explained to faculty.
• The president needs to respond to the faculty in regard to the evaluation assessment.
• Campus constituents need to know what has happened to their input.
• Increased "outcomes" assessment with focus on "small things" as well as "big things."
• Evaluate the administration as well as the individual.
• Presidents and senior management team encouraged to have annual evaluation outside the normal timeline of system evaluation or what they have done for the year.
• COLA's should be equivalent for management and faculty.
• Presidents and senior management team members should develop annual and multi-year activity/development plans.
• Should be more of an emphasis on performance criteria rather than on a perceived "popularity contest"
• One group was divided on the level of confidentiality that should be granted to presidential/MPP evaluations. The need to balance confidentiality with the constituency's "right to know" was debated, but not resolved.
• Needs to be more of a focus on the whole management team, not just the individual.
• It was suggested that PSSI logic for compensation be applied to presidents and MPP

Topic:

Post-Tenure Review

• Post-tenure review needs to be strengthened. PTR should be based on a professional development plan/ work plan created by the faculty members with consent of colleagues. Evaluation by colleagues with possible "outsiders" to keep the process honest should result in a reward from PSSI moneys for successful accomplishment of plan.
• Majority in one group supports step increases bonus approach. Consensus opposes current "PSSI by presidential fiat" model.

Topic:

Demonstrating CSU's contributions to the State

• Need for more effort put into fundraising
• We should communicate the community service done by faculty and students.
• Advertise the number of government officials who have CSU degrees
• Forgivable loads to undergraduates who become teachers
• Report the number of professionals educated by CSU campuses
• Hire professionals to do public relations, report cards, or fundraising. Don't require faculty to do it.
• Legislative days on campuses
• Campuses are reluctant to provide information to the chancellor's office. A bottom-up approach would be best.
• Cultural contributions to the community need to be emphasized.
• Use the Internet
• Advertise positions in academic societies held by CSU faculty.
• Encourage the task force to show the relationship between prison spending and educational spending. This will help to show that spending is a choice.
• There is a belief that the Chancellor's PSSI initiative will destroy our efforts to be prepared for Tidal Wave II. PSSI's are divisive, and make more demands on faculty time.