National Survey of Student Engagement: A truer measure of quality

When measuring the quality of education, Alexander McCormick is hardly swayed by a university’s reputation, nor its entrance exam scores. “Neither of those really tells you anything about teaching and learning on the campus,” says the professor of educational leadership and policy studies at Indiana University Bloomington. Reputation is highly correlated with the wealth of an institution, while exam scores offer little indication of what happens after students are admitted. “What ends up standing in for quality are not terribly good measures of what actually goes on in a university.” As director of the National Survey of Student Engagement (NSSE), a North American survey offered annually since 2000, McCormick is most interested in improving higher education by asking current undergraduates about specific measures for levels of engagement on and off campus.

RELATED:  Canadian University Survey Consortium: Surveying student satisfaction

With NSSE, approximately 100 questions are designed to evaluate best educational practices such as student-faculty interaction, collaborative learning and supportive campus environments, and give feedback to schools. How often have students attended an arts performance in the past year? How much of their course work emphasizes memorization? How often have they prepared two or more drafts of a paper before turning it in? How often do they ask another student for help with course material? How often have they had discussions with people of a different race? A different economic background? A different religious belief? If it all sounds extremely specific, that’s the goal. “We ask about activities and experiences that previous research has shown to be positively related to learning outcome,” McCormick says. The higher the score, the better the odds that undergrads are getting the most from university. When Quest University Canada was founded back in 2002 as a small, private, but non-profit liberal-arts school in Squamish, B.C., it was designed to provide many of the best practices found in this kind of survey. A sign of its success is that, in the five years it has participated in NSSE, Quest has come out at the top four times, and second once, when students rated their entire educational experience. Similarly, when students were asked if they would attend the same school all over again, Quest was in the upper echelon there, too. While receiving high praise from students is always good, Quest president David Helfand believes the two overall-satisfaction questions Maclean’s publishes are the least important from NSSE, which had 114,511 surveys returned from first- and fourth-year students across all participating Canadian campuses last year, a response rate of 36 per cent. “These are the only two questions in the survey that represent student opinion,” he says. “The most important are those that measure what the students actually do.” This year, 73 Canadian institutions took part in the survey, the largest number in its 15-year history. “We want to know that we’re actually having an impact on these students,” Helfand says, “that, when they come in, they’re in one state, and when they go out, they’re in a different state, presumably, better prepared for life.” The questions and answers on student engagement that Helfand refers to (six of which are published on Macleans.ca) are a measure of what Quest is accomplishing. NSSE itself aggregrates the results and does not identify institutions. “We don’t give anyone the information that allows them to say: ‘We’re the best,’ ” says McCormick. Schools get their own scores, which they can compare to averages in different groups, including the top 50 per cent and the top 10 per cent. (Maclean’s asks all participating schools for their data on two student-satisfaction questions and six of 10 student-engagement measures; this year, 66 of 73 consented. Because most schools do not do the survey every year, we use data from the previous year’s results to fill in the blanks.) Overall, compared to our American counterparts—622 U.S. institutions took part in last year’s NSSE—Canadian schools didn’t appear to perform as well, says McCormick. Another overarching trend was: The bigger the university, the lower it tended to perform on student-faculty interaction. Last, Maclean’s includes the results from two questions posed to middle-year students by the Canadian University Survey Consortium. Of 28 universities that took part last year, 22,537 students filled out the survey—a response rate of 30 per cent—answering questions about participation in activities and the perception of the quality of the educational experience. Robert Lapp, president of Canada’s Society for Teaching and Learning in Higher Education, says, “The differences within universities can be greater than between universities.” The point is to make use of “formative feedback.” In Lapp’s ideal scenario, university courses would be constructed collaboratively between a professor—the expert in content—and an educational developer, an expert in best teaching practices. “That would be utopia.”

Engagement indicator results from the 2014 National Survey of Student Engagement (NSSE)

NSSE created 10 engagement indicators organized across 4 major themes: academic challenge; learning with peers; experience with faculty; and campus environment. Each indicator reflects students’ responses to a set of questions relating to these themes. Results are organized to compare performance across all participating schools—American and Canadian. The following charts show results from the 2014 survey for six of the 10 NSSE engagement indicators. We have listed the universities in descending order of achievement, according to their senior-year scores. Scores are on a scale of 60.

Scroll down to view the charts sequentially, or view them individually via the links below:Higher-Order LearningQuantitative ReasoningCollaborative LearningStudent-Faculty InteractionEffective Teaching PracticesSupportive EnvironmentStudent questions:How would you evaluate your entire educational experience at this institution?If you could start over, would you go to the institution you are now attending?

Higher-Order Learning This engagement indicator assesses how often coursework emphasizes such skills as applying facts, analyzing ideas in depth, evaluating points of view, and forming new ideas from various pieces of information.

* NSSE average is the average score for all Canadian and U.S. universities and colleges that participated in the 2013 and 2014 surveys.[Back to top]Quantitative Reasoning This engagement indicator assesses how often students use numerical information, such as numbers, graphs and statistics, to reach conclusions, examine real-word issues, and evaluate what others have concluded from numerical information.

* NSSE average is the average score for all Canadian and U.S. universities and colleges that participated in the 2013 and 2014 surveys.[Back to top]Collaborative Learning This engagement indicator gauges how often students work with other students on course projects or to prepare for exams, as well as how often students explain course material to other students or ask other students for help.

* NSSE average is the average score for all Canadian and U.S. universities and colleges that participated in the 2013 and 2014 surveys.[Back to top]Student-Faculty Interaction This engagement indicator assesses such things as how often students discuss topics with a faculty member outside class, work with faculty on activities other than coursework, or discuss their academic performance and career plans with faculty.

* NSSE average is the average score for all Canadian and U.S. universities and colleges that participated in the 2013 and 2014 surveys.[Back to top]Effective Teaching Practices This engagement indicator assesses how well instructors teach in an organized way, clearly explain course requirements, use examples or illustrations to explain difficult points, and provide prompt and detailed feedback on tests and assignments.

* NSSE average is the average score for all Canadian and U.S. universities and colleges that participated in the 2013 and 2014 surveys.[Back to top]Supportive Environment This engagement indicator assesses how well universities provide support for academic and non-academic endeavours, ranging from tutoring services and writing centres to recreation, health care and counselling. This measure also examines social interaction, including contact with students from different background.

* NSSE average is the average score for all Canadian and U.S. universities and colleges that participated in the 2013 and 2014 surveys.[Back to top]

How satisfied are students?

NSSE asks students dozens of specific questions about how they spend their time in and out of the classroom. Below are two questions that are the broadest and most representative of the student experience. Responses are ordered according to the percentage who chose the highest level of satisfaction, e.g., "Excellent." How would you evaluate your entire educational experience at this institution?

[Back to top]If you could start over, would you go to the institution you are now attending?

[Back to top]