National Survey of Student Engagement: A truer measure of quality

Student-engagement surveys evaluate best educational practices and provide valuable feedback for university administrators

Aaron Hutchins

<p>Prof. Akindele Odeshi from the department of mechanical engineering lectures to his first year GE124 Engineering Mechanics class in the “airplane” room at the University of Saskatchewan. Photograph by Derek Mortensen</p>

Prof. Akindele Odeshi from the department of mechanical engineering lectures to his first year GE124 Engineering Mechanics class in the “airplane” room at the University of Saskatchewan. Photograph by Derek Mortensen

February 12, 2015

When measuring the quality of education, Alexander McCormick is hardly swayed by a university’s reputation, nor its entrance exam scores.

“Neither of those really tells you anything about teaching and learning on the campus,” says the professor of educational leadership and policy studies at Indiana University Bloomington. Reputation is highly correlated with the wealth of an institution, while exam scores offer little indication of what happens after students are admitted.

“What ends up standing in for quality are not terribly good measures of what actually goes on in a university.”

As director of the National Survey of Student Engagement (NSSE), a North American survey offered annually since 2000, McCormick is most interested in improving higher education by asking current undergraduates about specific measures for levels of engagement on and off campus.

With NSSE, approximately 100 questions are designed to evaluate best educational practices such as student-faculty interaction, collaborative learning and supportive campus environments, and give feedback to schools. How often have students attended an arts performance in the past year? How much of their course work emphasizes memorization? How often have they prepared two or more drafts of a paper before turning it in? How often do they ask another student for help with course material? How often have they had discussions with people of a different race? A different economic background? A different religious belief?

If it all sounds extremely specific, that’s the goal. “We ask about activities and experiences that previous research has shown to be positively related to learning outcome,” McCormick says. The higher the score, the better the odds that undergrads are getting the most from university.

When Quest University Canada was founded back in 2002 as a small, private, but non-profit liberal-arts school in Squamish, B.C., it was designed to provide many of the best practices found in this kind of survey. A sign of its success is that, in the five years it has participated in NSSE, Quest has come out at the top four times, and second once, when students rated their entire educational experience. Similarly, when students were asked if they would attend the same school all over again, Quest was in the upper echelon there, too.

While receiving high praise from students is always good, Quest president David Helfand believes the two overall-satisfaction questions Maclean’s publishes are the least important from NSSE, which had 114,511 surveys returned from first- and fourth-year students across all participating Canadian campuses last year, a response rate of 36 per cent. “These are the only two questions in the survey that represent student opinion,” he says. “The most important are those that measure what the students actually do.”

This year, 73 Canadian institutions took part in the survey, the largest number in its 15-year history. “We want to know that we’re actually having an impact on these students,” Helfand says, “that, when they come in, they’re in one state, and when they go out, they’re in a different state, presumably, better prepared for life.” The questions and answers on student engagement that Helfand refers to (six of which are published on Macleans.ca) are a measure of what Quest is accomplishing.

NSSE itself aggregrates the results and does not identify institutions. “We don’t give anyone the information that allows them to say: ‘We’re the best,’ ” says McCormick. Schools get their own scores, which they can compare to averages in different groups, including the top 50 per cent and the top 10 per cent. (Maclean’s asks all participating schools for their data on two student-satisfaction questions and six of 10 student-engagement measures; this year, 66 of 73 consented. Because most schools do not do the survey every year, we use data from the previous year’s results to fill in the blanks.)

Overall, compared to our American counterparts—622 U.S. institutions took part in last year’s NSSE—Canadian schools didn’t appear to perform as well, says McCormick. Another overarching trend was: The bigger the university, the lower it tended to perform on student-faculty interaction.

Last, Maclean’s includes the results from two questions posed to middle-year students by the Canadian University Survey Consortium. Of 28 universities that took part last year, 22,537 students filled out the survey—a response rate of 30 per cent—answering questions about participation in activities and the perception of the quality of the educational experience.

Robert Lapp, president of Canada’s Society for Teaching and Learning in Higher Education, says, “The differences within universities can be greater than between universities.” The point is to make use of “formative feedback.”

In Lapp’s ideal scenario, university courses would be constructed collaboratively between a professor—the expert in content—and an educational developer, an expert in best teaching practices. “That would be utopia.”

Engagement indicator results from the 2014 National Survey of Student Engagement (NSSE)

NSSE created 10 engagement indicators organized across 4 major themes: academic challenge; learning with peers; experience with faculty; and campus environment. Each indicator reflects students’ responses to a set of questions relating to these themes. Results are organized to compare performance across all participating schools—American and Canadian.

The following charts show results from the 2014 survey for six of the 10 NSSE engagement indicators. We have listed the universities in descending order of achievement, according to their senior-year scores.

Scores are on a scale of 60.

Scroll down to view the charts sequentially, or view them individually via the links below:
Higher-Order Learning
Quantitative Reasoning
Collaborative Learning
Student-Faculty Interaction
Effective Teaching Practices
Supportive Environment

Student questions:
How would you evaluate your entire educational experience at this institution?
If you could start over, would you go to the institution you are now attending?

Higher-Order Learning

This engagement indicator assesses how often coursework emphasizes such skills as applying facts, analyzing ideas in depth, evaluating points of view, and forming new ideas from various pieces of information.

[rdm-footable id=’277′]