The Reinvention Center

at the

University

of Miami

 
 

Our Goals
Who We Are
Events
Resources
Spotlight
Reinventions
Boyer Report
Brochure
Networks
Studies

Home

 
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  Contact us Join our network Level up  
 
  SPOTLIGHT: Creating a Culture of Assessment
 

Thought:

From Assessment to Accountability: Reframing the Conversation
Robert Thompson, Jr.

Duke University

 

bookshelf


Invitation for Future Spotlights

Models:

University of Nebraska, Lincoln:

Student Learning Outcomes: From Assessment to Accountability

 

North Carolina State University:

Assessment of Undergraduate Student Learning at NC State: A Decentralized Approach

 

University of Wyoming:

The University of Wyoming's Curricular Alignment Initiatives
 

National Council of Teachers of English,  Conference on College Composition and Communication and Council of Writing Program Administrators:

The White Paper on Writing Assessment

 

Previous Spotlights:

Diverse Approaches and Venues for Undergraduate Research

 

Creative Uses of Instructional Technology

Engaging Humanities Students in Research

Application of Quantitative Concepts and Techniques in Undergraduate Biology

 

The Minor as a Vehicle for Interdisciplinary Education

Undergraduate Research, Scholarship, and Creative Activity

First-Year Initiatives

Achieving an Interdisciplinary General Education

  Thought
 

Every few months The Center spotlights a topic of significance to research university faculty and administrators. Its approach is through Thoughts and Models. The Thought consists of a short essay on the particular topic being highlighted. The Models represent different campus approaches to the topic.

In recent years, empirical assessment methods increasingly have been applied to higher education.  Under the broad rubric of assessment, faculty and administrators at research universities are thinking about ways to design studies and gather evidence to measure the effectiveness of their programs and the extent to which their curriculum and pedagogy are meeting their goals and achieving desired learning outcomes.  Further, they are experimenting with a variety of approaches and tools ranging from on-line surveys of targeted populations, to small-scale, informal assessments of individual class sessions, to multi-year e-portfolios to measure students’ progress, to standardized testing of all students.  The goal is to build a culture of assessment that informs their undergraduate education.

This Spotlight consists of a Thought and four Models of assessment presented at Reinvention Center meetings this past year.  The Thought, written by  Robert Thompson, Jr., Professor of Psychology and Dean of Trinity College at Duke University, calls upon research universities to become proactive in shaping the national conversation on assessment; instead of allowing external forces to control the conversation and demand “accountability” based on criteria that may be inappropriate or inapplicable to higher education, universities should reposition the discussion of assessment in terms of their own academic values. The Models that follow illustrate the range of assessment activities being implemented and the ways in which the findings can be used to improve teaching and learning.  The University of Nebraska-Lincoln has undertaken a centralized, systematic assessment of student learning across schools and colleges.  Its centralized approach has enabled the University to gather data that are being used to inform major curricular changes.  In contrast, North Carolina State University has decentralized its assessment activities and given responsibility for creating an assessment plan to departments which devise their plans based on faculty expertise and disciplinary requirements.  In an altogether different vein, at the University of Wyoming, university faculty and high school teachers are engaged in a joint effort to  gather data that will help the University to understand the educational experiences of Wyoming high school students and  design appropriate transitions from high school to university education.   The final Model represents a collaboration among the National Council of Teachers of English, the Conference on College Composition and Communication and the Council of Writing Program Administrators.  These organizations are working together to create models for assessing student writing, the teaching of writing and writing programs.


From Assessment to Accountability: Reframing the Conversation

Robert Thompson, Jr., Dean of Trinity College, Duke University

“And if assessment becomes synonymous with standardized testing, what will happen to assessment undertaken for the purpose of guiding improvement in instruction, curricula and student services?” (Banta, 12)

 

Many in higher education are uneasy about the relationship between assessment and accountability. We worry about the impact of increasing expectations for accountability, about whether these expectations will alter the educational process and about how assessment data will be used. Some have tried to differentiate assessment for accountability from assessment for improvement. Carol Schneider, for example, captures this difference in her article for the Peer Review when she states: “Assessment can and should be designed to deepen and strengthen student learning, not just to document it” (3). If we accept this difference in terms as a signal of the different purposes of assessment, it would seem that assessment and accountability are not mutually exclusive and could, in fact, be complimentary. But a part of our uneasiness stems from the more fundamental problem of how accountability is being framed, and it is this framing that bothers and threatens us. An alternative framing of accountability would serve to enhance the quality of undergraduate teaching and learning and help us to identify where change in our practices is warranted. Let me first describe the way accountability is currently framed, and then demonstrate how a reframing would allow a more productive conversation.

Currently, conversations about assessment and accountability are framed by the following recommendations of the Commission on Higher Education (Eaton, 2007):

·       Provide more evidence of student achievement and institutional performance and make this evidence primary when judging academic quality

  • Make information easily understandable and readily accessible to the public
  • Develop various means to compare institutions regarding their success in student achievement and institutional performance
  • Establish threshold standards for collegiate learning

The first two recommendations evoke little disagreement; we have no trouble agreeing that providing evidence of student achievement and institutional performance to the public is a good idea and that using such evidence to judge academic quality is appropriate. Similarly, we have no trouble with the idea of making information easily understandable and readily accessible. But the last two recommendations evoke concern and, it is agued, constitute a misplaced emphasis on an external locus of control and a market place perspective. When accountability is considered from the market place perspective and coupled with the threat of an external locus of control that would define standards and judge academic quality:

·         Education is seen as a product that is provided

·         The focus is on student achievement and institutional performance outcomes, and

·         The approach is comparative and competitive among institutions

In contrast, when we take an academic view of accountability, with an internal locus of control with regard to defining standards and judging academic quality:

    • Education is seen as a process of enabling growth
    • The focus is on instituting a culture of experimentation and evidence with regard to teaching and learning, and
    • The approach is evaluative within institutions and collaborative among institutions in pursuit of best practices.

Once the marketplace perspective of accountability is adopted, even the idea that colleges and universities should make information more readily available becomes distorted to that market place perspective. For example, Kevin Carey demonstrates such a shift when he states that “students choosing colleges currently have little or no information about which institutions actually provide the best education” (Change, 26). The difficulty is with the expectation that there is “the best education” as opposed to the perspective that educational experiences vary along a number of dimensions and that what should be sought is the optimal match between the education an institution can provide and a particular student’s interests and talents.

By shifting to the academic perspective of accountability, the values of scholarship rather than those of the marketplace will be at work. In reframing accountability in this way, institutions of higher education will hold themselves accountable to their multiple constituencies by continuously engaging in systematic, iterative processes to improve the quality of teaching and learning. But to attain an internal locus of control with regard to accountability rather than succumb to the threat of external controls, we must be willing to work through institutional and faculty resistance and establish ”a willingness to reexamine familiar practices and search for new methods that could serve the purpose better” (Bok, 2006, 313). A research university must, in effect, become a learning organization that engages in a continuous and systematic process of experimentation with innovative teaching and learning approaches, evaluation of attainments, and revision of approaches. This can be accomplished by committing to the self-evaluative and self-correcting process with regard to teaching and learning that characterize universities’ approach to scholarship.

References

Banta, T. (2007). Can Assessment for Accountability Complement Assessment for Improvement? Peer Review. 9 (2), 9-12.

Bok, D. (2006) Our Underachieving Colleges. Princeton: Princeton University Press.

Carey, K. (2007) Truth without action: The myth of higher-education accountability. Change, 39 (5), 24-29.

Eaton, J. (2007). Institutions, accreditation, and the Federal government: Redefining their appropriate relationship. Change, 39 (5), 16-23.

Schneider, C.G. (2007) From the President. Peer Review, 9 (2), 3


  Models
 

The following four models provide examples of innovative programs with a broad range of assessment experiences. Each model is highly flexible, applicable to range of disciplines, and reproducible on many scales. For additional examples and information, please visit the Undergraduate Research Opportunities section of our Resources page.

University of Nebraska-Lincoln

Rita Kean, Dean of Undergraduate Studies

Student Learning Outcomes: From Assessment to Accountability

In today’s milieu of educational accountability, the research university must clearly demonstrate to both its internal and external constituencies that the discovery, application and efficacy of knowledge are the foundation for the undergraduate experience. If we hold true to the belief that students should be active participants with faculty in the discovery and application of knowledge, then we should be able to assess that participation, discovery and application in a way that makes clear to our constituencies the value of undergraduate experience at a research university. Institutions of higher education are responsible for insuring the merits of the undergraduate experience. The University of Nebraska–Lincoln (UNL) instituted a systematic assessment of student learning outcomes at the program and institutional levels in 1996.

This attention to assessment at both the program and institutional levels has been essential to creating a culture of assessment as a process for improvement as well as for accountability. As faculty at UNL have become actively involved in formulating and evaluating their program assessment plans, assessment has become embedded throughout the undergraduate curriculum in all eight of the University’s undergraduate colleges. The numbers of units participating in assessment of student learning outcomes at the program level and the use of assessment as a means of implementing program improvements have increased steadily over the past twelve years. Results of assessment measures have highlighted a broad array of insights into student learning, including the continuous improvement of learning outcomes, coherence of the curriculum, advising, co-curricular experiences, student-faculty interaction and recruitment/ retention. For more information on these aspects of assessment at UNL, please see http://www.unl.edu/ous/faculty_resources/assessment.shtml

The key objectives underlying the UNL assessment process are threefold:

  1. Determine the desired student learning outcome(s): What do we want students to know?

2.      Identify best measures for determining whether these outcomes have been realized: What are the intellectual frameworks/underpinnings of the knowledge base? How we can best deliver them to meet the needs of the learners (pedagogy)?

  1. Use results to either confirm or improve instructional and curricular practice: What did we discover about the learner’s understanding of the content and delivery (reflection, critical analysis, evaluation) and in the same context, what did we learn that changed our own understanding of the content and/or our delivery?

Constructing a culture of effective assessment is a developmental process. If the goal of assessment is continuous improvement in the learning environment, then faculty members need to understand the benefits of using assessment to improve student learning. Disciplines approach assessment differently; therefore, respect needs to be shown for each discipline’s scholarship and traditions. The School of Biological Sciences, for example, employs a standard measurement for assessment of their graduating seniors. The Major Field Test for Biology consists of 150 multiple-choice questions, a number of which are grouped into sets based on the description of laboratory and field situations, diagrams or experimental results. Questions within each of the major areas are designed to test students’ analytical skills. The Department of Art uses senior exhibits for assessment purposes. Peer review for assessment techniques and student learning outcomes is beneficial. For many UNL faculty members, the peer review process has been invigorating and confirming.

To facilitate the peer review process, UNL has invested in tools to assist faculty with their assessment efforts. In 2004, UNL’s College of Education and Human Sciences and the College of Agricultural Sciences and Natural Resources established a partnership with Colorado State University to pilot an on-line assessment management system designed for program improvement. As of January 2008, four of the eight UNL undergraduate colleges and the Division of Student Affairs were employing this system, referred to as PEARL http://www.unl.edu/ous/pearl/indepth.shtml. PEARL allows faculty and staff to take the lead in the assessment process through participation in a continuous on-line dialogue between a trained peer reviewer and the faculty/ staff member responsible for the program’s assessment, increasing interaction and engagement among faculty/ staff. Information obtained from the PEARL system has already been used in academic program reviews as well as in reporting to professional groups/agencies. For example, the Department of Food Science and Technology, in their self study for their academic program review in fall 2007, described their use of the PEARL system to assist with their assessment of student learning outcomes, as well as the mechanisms used to assess those outcomes. The Department’s learning outcomes, opportunities for student learning, questions of interest, assessment methods, assessment results and use of the results for improvement were included in the academic program review.

UNL expects systems such as PEARL to be key elements of the assessment of its new general education program, Achievement Centered Excellence, which is based upon student learning outcomes http://ace.unl.edu. After attending the 2005 AAC&U Summer Institute on General Education, UNL’s General Education Planning Team (GEPT) concluded that a student’s general education experience should not be viewed as a separate function of his/her educational program, but, rather should be integrated throughout the entire educational experience. 

The GEPT and the larger General Education Advisory Committee (GEAC), approached reform efforts by defining four overall institutional objectives, each accompanied by a set of measurable student learning outcomes. Table 1 presents each of these objectives, followed by an example of a student learning outcome for that objective. A major strength of a general education program based on student learning outcomes is that assessment of student work is accomplished through direct measures. One measure, for example, might be a sample of student products from ‘certified’ general education courses, along with reflections from the unit as to how the student work met the stated student learning outcomes for the course or program.

 

Table 1 ACE Institutional Objectives with Examples of Student Learning Outcomes

 

Objective

Example of Student Learning Outcome

Develop intellectual and practical skills, including proficiency in written, oral and visual communication; inquiry techniques, critical and creative thinking; quantitative applications; information assessment; teamwork; and problem-solving.

Write texts, in various forms, with an identified purpose, that respond to specific audience needs, incorporate research or existing knowledge, and use applicable documentation and appropriate conventions of format and structure.

 

Build knowledge of diverse peoples and cultures and of the natural and physical world through the study of mathematics, sciences and technologies, histories, humanities, arts, social sciences and human diversity.

 

Use scientific knowledge of the natural and physical world to address problems through inquiry, interpretation, analysis, and the making of inferences from data, to determine whether conclusions or solutions are reasonable.

 

Exercise individual and social responsibilities through the study of ethical principles and reasoning, application of civic knowledge, interaction with diverse cultures, and engagement with global issues.

 

Explain ethical principles, civics, and stewardship, and their importance to society.

 

Integrate these abilities and capacities, adapting them to new settings, questions, and responsibilities.

 

Generate a creative or scholarly product that requires broad knowledge, appropriate technical proficiency, information collection, synthesis, interpretation, presentation, and reflection.

 

UNL has been involved in several efforts to assess the undergraduate experience at the institutional level. A key tool has been the National Survey of Student Engagement (NSSSE), administered at UNL on a triennial basis. UNL participated in the 2005-06 Council of American Education pilot of the Collegiate Learning Assessment (CLA).

During AY 2006-07, UNL administered the CAAP measurement to a sample of students through their participation in Parsing the First Year of College Study, led by Patrick Terezini and Robert Reason of the Pennsylvania State University Center for Higher Education Research. Terezini and Reason offer a comprehensive model and measure of the students’ educational experience as well as providing data for institutional advancement and improvement. This study considers the relationships between inputs (entering student characteristics and experiences), processes (organizational context and educational environments) and outputs (learning, development and persistence), thus offering a more holistic view of our students’ undergraduate experience.

Presently, the Office of Institutional Research and Planning is coordinating UNL’s contribution to the Voluntary System of Accountability (VSA). UNL believes the assessment of the student learning outcomes associated with their general education program will provide them with institutional data as to the efficacy of the program and its effect on student learning; however, they also realize the choice of instrument may be determined by the larger University system rather than the individual institution.


North Carolina State University

Alan Dupont, Director of Assessment

 

Assessment of Undergraduate Student Learning at NC State: A Decentralized Approach

Like most large, public, research-extensive universities, North Carolina State is decentralized. It has nine colleges offering undergraduate degrees (a tenth college, the College of Veterinary Medicine, does not offer undergraduate degrees); they range in size from the College of Engineering with approximately 5,400 undergraduates (18% of total undergraduate enrollment) to the College of Design with roughly 470 undergraduates (1.5% of total undergraduate enrollment). Each college has its own culture that shapes how things are organized and how things get done. The roughly 85 departments into which these nine colleges are organized, in turn, have their own distinctive cultures and differing degrees of autonomy. Alan Wolfe, in a 1996 article in The Wilson Quarterly, wrote that “so organized, the university is not an entity with a common purpose or at least organized around a self-defining core. It is a set of linked fiefdoms that find temporary advantage in belonging to a larger organization.”

It has been noted that this decentralization and concomitant faculty autonomy have been integral to the research success of the modern university (see, for example, the Kellogg Commission report Returning to Our Roots: Toward a Coherent Campus Culture, 2000). However, this structure also means that processes or activities mandated “from above” are often resisted strenuously by the faculty. This resistance does not apply to basic business functions, of course, which are generally carried out by non-faculty employees. A good example of a “mandate from above” is assessment of student learning outcomes, which is (or should be) carried out by the faculty. Until 2006, NC State utilized a centralized approach led by the Committee on Undergraduate Program Review (CUPR). Programs (typically but not always synonymous with departments) were given lengthy assessment guidelines that went into some detail about the format of the required reports. The communications went from CUPR to the departments, bypassing the college administration in most cases. This approach caused many faculty members to see assessment as an added burden that was disconnected from their day-to-day work.

In 2004, the most common question faculty asked about assessment was “What does CUPR want?” The question was indicative of a “compliance mentality” that sought to complete the reports with the least amount of time and effort expended. While some programs and faculty were authentically engaged in assessment, most program faculty saw assessment as a fad, an unjustified additional burden, and/or not worth doing even if they had the time and energy to spare. To put it bluntly, the centralized approach provided some extrinsic motivation (though not enough in most cases to motivate the faculty to authentically engage in the process), but it did not provide intrinsic motivation. Given that the purpose of assessment is to improve student learning, it is critical that any assessment process be primarily developed and implemented by the faculty as it is the faculty who make the changes to courses and curricula that result from assessment.

Recognizing this, and spurred on by increasing calls by the faculty and the department heads to re-think its top-down approach, NC State switched to a decentralized process in which every college is responsible for developing an assessment plan and carrying it out. This approach gives the colleges considerably more latitude in determining how to organize and implement assessment than they had under the centralized system. Specifically, the Associate Dean of the college who oversees its undergraduate programs now has, as part of the job, responsibility for ensuring that every program within the college submits an annual program assessment report. This decentralized approach has been a success, at least in part because the faculty members in the colleges know their Associate Dean and have in many cases known them for years. Perhaps just as important, the assessment process can be tailored to fit the unique culture and needs of the individual college. For instance, the College of Engineering has a specific approach to student learning outcomes assessment that is driven by its national accrediting body, ABET. Under the decentralized approach, the reports that the College prepares for ABET purposes can be used with only minimal modification to meet the University’s assessment requirements. .Similarly, the College of Management can take advantage of its AACSB accreditation process and the College of Education can align its assessment with its NCATE accreditation process, as can other colleges and departments that have special assessment situations. It should be noted that assessment support is itself decentralized; in addition to the Office of Assessment in the Division of Undergraduate Academic Programs, the College of Engineering and the College of Education have their own assessment professionals who focus on the specific assessment and accreditation needs of their respective colleges. The Graduate School, the Division of Student Affairs, and Distance Education also have assessment professionals on their staffs. Finally, University Planning and Analysis carries out some university wide assessment projects under the direction of an assessment professional on the UPA staff.

While the decentralized approach increases the level of control by the faculty over the process, it does not in and of itself provide any intrinsic motivation. In an effort to engage faculty, and in conjunction with the decentralization, the staff of the Office of Assessment in the Division of Undergraduate Academic Programs has successfully switched re-cast its role so that instead of being perceived as “assessment enforcer” its members have become “assessment consultants.” This change makes talking to program faculty about the purpose of assessment much easier and staff members are very pleased with the new approach. There appears to be an attitude among many more faculty members that assessment is a useful tool for improving undergraduate student achievement of specified learning outcomes. The annual program assessment reports reflect the improved attitude and increased faculty engagement as they now focus more on questions that program faculty find compelling than they did in the past, and they include complex analysis and judgment. At the same time, there appears to be a corresponding decrease in annual reports that reflect a “compliance mode.”

There remains a level of University oversight within this decentralized structure in that the program reports and the Associate Deans’ summaries of those reports are made available to the Office of Assessment, whose staff members review them and provide feedback to the Associate Deans and to the departments’ curriculum or assessment committees. The staff members also look for themes or issues that cross college boundaries. These are brought to the Associate Deans for their consideration. Finally, the Associate Deans themselves share the assessment results from their departments at an annual Council of Associate Deans meeting that is devoted to assessment. The Office of Assessment staff also prepares an annual report for the Provost and other interested parties (primarily the liaison to the University’s regional accreditation agency) that includes pertinent information needed by those individuals and offices.

The decentralized approach to assessment at NC State meshes well with the decentralized institutional culture, allowing colleges and departments to fashion assessment plans that work for the program faculty. The program faculty can address the questions that are of interest to them and gather information that is meaningful to them. This approach puts the staff of the Office of Assessment in a position to be consultants rather than enforcers, which leads to more productive interactions.


University of Wyoming

Audrey Kleinsasser, Professor and Director, Wyoming School-University Partnership

Rollin Abernathy, Professor and Associate VP for Academic Affairs

 

The University of Wyoming's Curricular Alignment Initiatives

A land grant institution, the University of Wyoming was charged by the Wyoming Constitution to provide instruction “as nearly free as possible” (Wyoming Constitution, Section 16, 1889). The Wyoming Legislature made the ideal of nearly free postsecondary education a reality in 2006 by enacting provisions for a $400 million Hathaway Scholarship Endowment Program. Interest from the endowment is providing merit and need-based scholarships to graduates of Wyoming high schools who enroll at the state’s community colleges or the four-year university, utilizing ACT scores and high school grade point averages as determinants of the level of scholarship support. As with many scholarship programs in other states, Wyoming’s Legislature stipulated that recipients of the Hathaway Scholarships complete a Success Curriculum beginning in ninth grade: four years of English, mathematics, and science; three years of social studies; and two years of a sequenced foreign language. The Success Curriculum will be fully phased in by 2011. Concurrent with the implementation of the Wyoming Hathaway Scholarship Program, the University partnered with the Community College Commission, the Wyoming Department of Education, and business to craft the Wyoming P-16 Education Council. The goals of the Council include effective implementation of the Success Curriculum and data systems that allow statewide assessment of effectiveness for improvement.

Along with this extraordinary opportunity come enormous challenges. During the first year of full implementation, 2006-2007, approximately 25% of the Hathaway recipients failed to maintain a high enough GPA to keep their scholarships. The 25% failure rate indicates significant alignment issues between secondary and postsecondary schooling in Wyoming, especially Grades 11-14.  

To address curricular alignment across grades 11-14, reduce remedial education, and close gaps apparent in Wyoming’s P-16 education system, UW is leading an innovative professional development model piloted by the Wyoming School-University Partnership. The Partnership model, which builds on a successful UW 2003-2006 FIPSE-funded initiative, brings together a multi-level cohort of high school, community college, and university instructors who teach students in Grades 11-14. This four-year grade span represents a series of necessary academic, social, and emotional transitions that are critical to limiting remedial education, completing a postsecondary degree, and thriving as a lifelong learner. The model is applicable to virtually all secondary through postsecondary alignment initiatives in the nation

Wyoming Life Sciences Summits

In February 2006, over fifty life sciences teachers representing the University of Wyoming and statewide high schools and community colleges convened in a day-long science summit organized by the Wyoming School-University Partnership. Summit planners asked each participant to bring multiple copies of student work representing high, medium, and low quality. During the meeting, this work was examined in multi-level, small groups using an assessment protocol designed and guided by a skilled facilitator. For all of the participants, including the planners, the event was unique. None had ever met in a multi-level cohort to examine student work and the learning it revealed. Summit evaluations confirmed that participants valued the opportunity for candid peer discussion about assignments and student learning outcomes. They returned to their classrooms with immediate plans for changing their practices. Secondary teachers reported holding their students to higher expectations. University teachers created course stewardship committees to align curriculum and evaluate assessment practices. In effect, this first summit made the barrage of national reports concerning the secondary-to-postsecondary transitions real, local, and high priority. It revealed that such multi-level discussions are foundational to major changes that must occur in Grades 11-14.

By February 2008, six follow-up life sciences summits had occurred in Wyoming, involving over 200 participants who came to some deeper insights of student learning across levels. They identified several valuable features they wanted to replicate. First, student work revealed student understandings and misconceptions as well as a range of performances. Second, by focusing on genuine student work—tests, quizzes, lab reports, homework assignments—the teachers came to better understand particular grade level challenges. The vocabulary of science, for example, is especially challenging for eighth and ninth grade students, with more new words introduced in a year than students would learn in a modern language course over a similar time period. Third, and most consequential, teachers came to see that even across levels there were more similarities than differences in teachers’ concerns about student achievement. Teachers at all levels became alert to students’ failure to critically read assignments of increasing length and complexity, especially as students in high school bridged college-level work. Uniformly, teachers saw problems with writing and the necessity of multiple drafts to produce final texts. For the life sciences faculty, a significant issue became student difficulty at all levels with interpreting and analyzing quantitative data represented in graphs and charts.

An early result of the first few summits was the introduction of a mathematics achievement gate for UW’s Life Sciences Program. The program director and faculty have documented that a mathematics ACT score of 21 predicts success in the University of Wyoming’s General Biology course as measured by a grade of C or better. The course is pivotal since students in many majors across all of the university’s six undergraduate colleges must successfully complete it. By establishing several ways for students to meet the equivalent of an ACT score of 21 before taking the course, the program director has significantly decreased the number of students who must retake the General Biology course.

The requirement became the source of much discussion at four 2007-2008 life sciences summits, in particular the conclusion that the increasingly synthetic nature of the life sciences makes the subject harder for students, especially in its quantitative reasoning demands. The teachers also discussed the reality that most students took their first and often only high school life sciences course in eighth or ninth grade. Then, four or five years later, students entered a college-level course with heavy reading and lab requirements, a faster pace, and work that required students to make significant connections between math and science. For the postsecondary teachers, that four- to five-year gap explained many of the difficulties first-year students exhibit.

Writing Colloquium

Building on the success and momentum of the life sciences summits, Wyoming School-University Partnership planners adapted the model for a statewide Teaching Writing in Wyoming Colloquium in April 2008. Key elements of the two-day meeting for more than fifty participants included multi-level work groups involving junior high, senior high, community college, and university instructors, the examination of student writing samples and scoring rubrics, a discussion of the first year university-level writing course, and a critical reading workshop.

Similar to teachers attending the life sciences summits, writing colloquium participants found the meeting unique and valued the multi-level cohort experience, especially peer discussion. They identified significant gaps in expectations for homework, and they began to understand that outside-of-class reading and writing assignments are key features of successful postsecondary student achievement. Participants also drafted a document that compared eleven instructional features of secondary and postsecondary writing courses. The document is remarkable in that it is created and co-owned by a broad-based, multi-level group of teachers.

The Power of this Model

The two statewide projects described above, one in life sciences and one in writing, illustrate the power of a multi-level cohort professional development model in a core content area. The model breaks down the barrier of dedicated secondary and research university teachers working together to strengthen the quality of secondary instruction and help students successfully negotiate transitions to the next level of study. At the same time, the model is non-threatening, featuring low risk strategies that are also frequent and interactive. The model honors the hard and necessary intellectual work of effective teaching—deliberations around student learning and academic achievement that are longitudinal.

In the Wyoming work to date, the participants have been genuinely curious and respectful about each other’s successes and challenges. As a result, the meetings have not devolved into mutual blaming or debilitating cynicism. Trust grew, in fact, with participants eager to meet again in a similar configuration to examine student work. Most important, we have growing evidence in the UW Life Sciences Program that the model supports a more sophisticated way of solving secondary to postsecondary transition problems and reflects deeper student learning as documented by higher achievement and reduced remedial education.

For more information

Audrey Kleinsasser, Director, Wyoming School-University Partnership, dakota@uwyo.edu

Mark Lyford, Director, University of Wyoming Life Sciences Program, mahler@uwyo.edu

April Heaney, Director, University of Wyoming LeaRN Program, aprilh@uwyo.edu


National Council of Teachers of English, Conference on College Composition and Communication and Council of Writing Program Administrators

Linda Adler-Kassner and Howard Tinberg for the NCTE/WPA/CCC Task Force

 

The White Paper on Writing Assessment

Valid and reliable assessment is consistent at the level of principle and conceptualization: it is discipline-based, locally determined, and used to inform teaching and learning at the local level.

Assessment. The word often strikes fear into the heart of college and university faculty, who sometimes view it as an imposition into their academic freedom and teaching practices. This perception was particularly acute after the publication of A Test of Leadership, the final report from the Spellings Commission on the Future of Higher Education.  But assessment is really about investigating the questions at the heart of our professional lives: What do we teach, how are we teaching, and why are we teaching as we are? What is the effect of our teaching on students’ learning – that is, how do we know students are learning what we say they are? The responses to these questions also form the cores of our disciplines. After all, if faculty are not in a position to define what students should learn, how and why they should learn it, and how that learning should be assessed, then we run the risk of losing control over what happens in our classrooms. At the same time, assessment also provides opportunities for us to engage in dialogue with interested others outside of our disciplines – and perhaps even our departments or institutions – who are also interested in what students learn, how, and why. In our own discipline of composition and rhetoric, for instance, we are regularly reminded that the questions at the center of our work – how to help students become flexible writers, readers, and thinkers – are often of interest to other faculty, administrators, and employers.

The challenge that those of us working inside the academy now face is how to can embrace the opportunities provided by increased interest in assessment to investigate our teaching practices and build alliances with interested others with an eye toward improving teaching and learning within our disciplines and across our campuses. This challenge includes, for instance, reframing assessment – away from terms like “accountability” and “transparency” toward ones like “responsibility” and “visibility.”

As a part of this work (and stemming, in part, from regional hearings on the Spellings Commission Report), a joint National Council of Teachers of English (NCTE) and Council of Writing Program Administrators (WPA) Task Force was convened to create a White Paper that summarizes existing position statements on assessment in writing drafted by National Council of Teachers of English (NCTE), the Conference on College Composition and Communication (CCCC), and the Council of Writing Program Administrators (WPA). The White Paper articulates the common thread running through these research and best-practice informed documents: Valid and reliable assessment is consistent at the level of principle and conceptualization and is discipline-based, locally determined, and used to inform teaching and learning at the local level. 

Among the principles articulated in the White Paper

·         Writing assessment should place priority on improvement of teaching and learning.

·         Writing assessment should provide the foundation for data-driven, or evidence-based, decision making.

·         Writing assessment should use multiple measures and engage multiple perspectives to make decisions that improve teaching and learning.

·         Writing assessment should be based on continuous conversations with as many stakeholders as possible and engage multiple perspectives to make decisions that improve teaching and learning.

To illustrate these principles, readers of the White Paper will be directed to descriptions of specific assessments undertaken by nine institutions of varying types across the U.S. While the assessments address questions developed at the local level and use methods that are locally determined and appropriate for improving teaching and learning at the local level, these assessments also demonstrate how these varied institutions have drawn on common principles and conceptualizations of writing and writing assessment to develop assessment practices that are appropriate for them, and used to improve teaching and learning at the local level. Assessment models run the gamut, from sophomore writing portfolios (Carleton College) to external assessments of a community writing center (Salt Lake Community College) to evaluating students’ “expert insider prose” in a writing in the disciplines program (Seattle University) to assessment of a first year program (University of Kentucky). 

Readers also will be directed to a set of communication strategies that addresses the question: how can we communicate with external stakeholders about writing assessment? These strategies provide a snapshot of practical, hands-on activities that writing instructors and program administrators can use to communicate with others to help them understand that valid and reliable assessment is consistent at the level of principle and conceptualization: it is discipline-based, locally determined, and used to inform teaching and learning at the local level. Among the strategies discussed:

Act locally

Develop alliances

Identify shared values with all stakeholders.

Together, these documents will provide concepts, models, and strategies for faculty to engage with assessment practices that can improve teaching and learning. Such assessment need not be considered extrinsic to the work of the classroom. Rather these documents demonstrate that assessment remains at the core of our professional work. The resources will be available July 1 through the NCTE and WPA web sites (www.ncte.org and www.wpacouncil.org ).          


If you have a creative program in instructional technology you would like listed on the Resources page, please send us a brief description (250 words maximum). Be sure to include the name of the program as well as a link to a Web site or the name and email address of a contact person.

AN INVITATION
We invite you to take the lead in framing future Thoughts and Models. If you're interested and have a "Thought" in mind, please send us an e-mail: reinventioncenter. We will identify "models" that relate to it.

THOUGHT: The Thought will consist of a short essay focusing on an issue central to undergraduate education at research universities. The specific topic to be addressed may vary. It may for example relate to an institutional challenge, an aspect of student learning, a societal need, or a recent research finding that may influence the way undergraduate education generally or in a specific discipline is conceived and delivered at research universities.

MODELS: Each Thought will be accompanied by reports on programs and experiences that exemplify or expand upon the Thought. The models will be drawn from different research universities, utilize different strategies, and, to the extent possible, focus on different disciplines. Collectively, they will become part of a database that will yield insights into what works or does not work and why.

Together, the Thoughts and Models will be incorporated into reports to be distributed through this web site, professional society newsletters and our own mailings.

We welcome your comments and look forward to hearing from you.

 

RC Home | Contact us | Join our network | University of Miami
Back to top
 

PHOTO CREDITS:
Ballet Rehearsal courtesy of University of Indiana Jacobs School of Music, Ballet Department
University of Missouri researchers in boat photo by Jim Curley
Library bookshelf photo by artlung
Univ of South Florida students at NOAA Hurricane center by Naomi Yavneh
musicians photo © Michael Jastremski for openphoto.net CC:Attribution-ShareAlike
work and why.

Together, the Thoughts and Models will be incorporated into reports to be distributed through this web site, professional society newsletters and our own mailings.

We welcome your comments and look forward to hearing from you.

 

RC Home | Contact us | Join our network | University of Miami
Back to top  

PHOTO CREDITS:
Ballet Rehearsal courtesy of University of Indiana Jacobs School of Music, Ballet Department
University of Missouri researchers in boat photo by Jim Curley
Library bookshelf photo by artlung
Univ of South Florida students at NOAA Hurricane center by Naomi Yavneh
musicians photo © Michael Jastremski for openphoto.net CC:Attribution-ShareAlike