New school of thought
UND develops trend-setting approach to assess student-learning outcomes regarding performance tasks such as problem solving and communication
So you’ve signed up for a Shakespeare class. You bet, the Bard is cool.
But is old Bill a career builder?
That’s the irksome question many grads ask about courses like this one as they dive into the shark tank we call the “job market.”
Anne Kelsch and Joan Hawthorne have an answer. They’ve been working on the Collegiate Learning Assessment (CLA) for UND, including performance tasks that help students learn lots more about the utility of the knowledge and skills they build in college—including courses in William Shakespeare!
“The CLA is a test designed to be a novel measure of learning in higher education,” says Hawthorne, director of assessment and a recipient of the national Jerry G. Gaff Faculty Award from the Association for General and Liberal Studies. So for sure, there are many life and work-related lessons to be learned—and assessed—in a Shakespeare class.
The CLA includes performance tasks that aim to assess a student’s capabilities to apply “book learning” in dynamic real-world situations that demand problem-solving skills.
“Part of the rationale behind this system of assessment is an effort that started about 10 years ago to avoid the creation of new federal regulations regarding standardizing learning outcomes assessment in higher education,” said Kelsch, program director of instructional development. On developmental leave this academic year, Kelsch and Hawthorne have worked with faculty on UND’s performance tasks.
“To avoid an external entity imposing something on us, we said ‘we will assess ourselves,’” Kelsch said.
In 2012, Hawthorne and Kelsch connected with their counterparts at North Dakota State University, agreeing that the CLA-style performance tasks are a great way for faculty to generate assignments for their classes.
The national CLA organization has a traveling workshop called ”CLA in the Classroom” where they take faculty, in a couple of days, through all the stages of development of performance tasks for use in their classes and in assessments.
“We took a group of about 40 UND faculty down to NDSU and did this big two-day workshop,” Kelsch said. “After Joan and I met with faculty who developed performance tasks, we’d have them review each other’s performance tasks. It turned out to be a great format for an assignment. This is a really smart way of getting at what student do and do not know. So eventually we came to the decision to use this to assess the Essential Studies program here.”
“Before that we had actually started using the CLA test itself, which includes performance tasks, as part of the national Voluntary System of Accountability program to assess our first year students and our seniors in 2010,” Hawthorne said. “That was posted on our website for the national accountability process. It helped to increase the familiarity of people across campus with performance tasks.”
Hawthorne noted that UND had been doing various kinds of outcomes assessments for Essential Studies for a number of years.
“We knew that we had to collect work products from seniors that would allow us to measure their quantitative literacy and oral communication,” Hawthorne said. “We knew that this was not going to be easy. The only students who are doing quantitative literacy kinds of things as seniors are students in fields such as chemistry and engineering. We were never going to get English or Art or Nursing majors in that pool. So we couldn’t get a representative sample if we just collected from senior classes. And similarly with oral communication, how were we to collect and score all this information from senior classes?”
So the assessment team here at UND knew they had a challenge on their hands.
“We decided to do performance tasks as outcomes assessments for quantitative reasoning and for oral communication as a first step,” Hawthorne said. “At the same time, some UND students would take the CLA, which is a pre-developed performance test, standardized, and developed by somebody else.”
Students are assigned to these assessment tests randomly. They show up at the test site, where there are faculty proctors.
“It seems like we can get lost in the jargon around assessment and accountability,” Kelsch said. “I think that it’s important that we appreciate that in higher education, we’re all struggling to come up with good ways to understand student learning across the board. It’s a place where higher education has come under a lot of fire lately.
“What we ended up doing here at UND is coming up with a way to do that that’s homegrown, developed by our faculty. They use it with their students from their capstone classes, and then we have a team of faculty come together to score those tests. So we’ve figured out a way to do this that’s effective and that gets faculty engaged in conversations about students’ learning, not just in their class but institutionally.”
In other words, UND has developed a unique way to benchmark and think about how to improve student learning.
“It’s all been done collaboratively, this is faculty donating their time, for us to try to come up with good ways to do institutional assessment,” Kelsch said. “At most institutions people just buy a standardized test; we really believe that this is something that’s far more effective and it works far better in terms of faculty really caring about the results.”
Hawthorne and Kelsch are enthusiastic proponents of the do-it-yourself—do it locally—school of thinking on this issue.
“At UND we’ve found that the best assessments are ones that faculty here develop, that faculty believe in,” Hawthorne said. “Faculty can hardly wait to see the results when it’s their own students, doing assessments they’ve helped develop; then faculty are willing to spend time scoring and talking about the tests. What we hear is that faculty are so glad they did. That’s the kind of finding from their assessment work that causes them to say, ‘you know, we should do such-and-such in my class,’ because they are seeing student-learning demonstrated differently.”
“Something we know from the CLA—the standardized version—is that UND students do better, compared to students nationally, on closed-answer questions than they do on ‘muddy’ questions,” Hawthorne said. “Seriously, when you get out of college, do you ever get multiple choice questions? You get open-ended muddy real-world questions.”
Thus the ultimate intent of the CLA and performance tasks is to assess—and learn from—students how to improve action-oriented thinking in future classes.
“One of the great things about the performance task format is it’s readily adaptable to almost any level of student learning,” Kelsch said. “I think what we want to say is that these are ways in which faculty at the University of North Dakota are addressing issues of national concern in higher education, ways that are being recognized and acknowledged. I think our performance tasks and assessment approach demonstrate that UND has very strong leadership in terms of assessing student-learning outcomes in ways that are innovative and effective. And they reflect a strong faculty commitment to student learning and a strong commitment to collaborating at an institutional level.”
Hawthorne, Kelsch and other colleagues have presented on UND’s work with performance tasks at national meetings and have also published on the process. One of the performance tasks, developed by a team of UND faculty, to assess quantitative reasoning, is now featured on the National Institute for Learning and Outcome Assessment website.
Ultimately this is all about ratcheting up student learning with a commitment to get beyond knowledge-based degrees—the aim is to help students get to be effective problem solvers and communicators.
“We also want to help our graduates learn about working on diverse teams. These are all things that every college graduate needs to have, no matter what they majored in,” Hawthorne said. “We’re doing that in innovative ways, both in terms of the assessment and in terms of the pedagogy behind that. This is a tremendous faculty collaboration and engagement kind of thing. This is a campus-wide commitment to student learning. We all know it matters. Faculty care deeply about student learning.
“Our primary interest in this isn’t measuring student learning, although we try to help faculty do that. We want faculty to be able to have informed conversations about students’ learning because the whole purpose of doing this is to make a difference in what faculty do next year and the year after that.”
And it’s really not about having a number to put in a report.
“It’s really about helping faculty see in a way that gives them a bit of distance from the teacher-student relationship and helps them make good decisions about what happens next,” Hawthorne said. “It’s about constant improvement. It’s about being able recognize when we want to make a change.”
Juan Miguel Pedraza
University & Public Affairs writer