Assessment: SENCER Results
Does the SENCER approach work? Do students learn better in SENCER courses? What about the SENCER approach helps students gain more in learning, develop positive dispositions towards science, and increase their inclination and capacity to be more civically engaged? What can faculty members do to make continuous improvements in the quality of their courses? How do members of the SENCER community appraise the quality of the SENCER project, itself? How do we know? How can you find out? What assessment and evaluation tools are available from SENCER?
Answering these questions - and developing tools and strategies for obtaining the answers - has been a key feature of the SENCER project from its inception. This section of the SENCER Web site is designed to present evidence of program effectiveness and to provide access to assessment tools and resources.
SALG Results Show SENCER Faculty Achieve in Raising Higher-Order Learning Gains
By Dr. Stephen Carroll, Principal Investigator, SALG
Analysis of the SALG (Student Assessment of their Learning Gains) data from September 2007 to September 2011 reveals that the SENCER project is succeeding on a number of levels. First, the data show that an ever-increasing number of SENCER faculty are using the SALG to assess and improve their teaching. Over the past four years, SENCER faculty conducted 1,314 SALG surveys, slightly more than 27% of the total number of SALG surveys delivered. The number of SENCER surveys has increased every year. The first year, 2008-2009, saw the largest increase, from 67 to 1019. In the next year, 2009, the number of users swelled to 1594 and expanded to 2026 in 2010 and 2174 in 2011. This substantial and consistent growth shows that SENCER faculty are getting useful data from the SALG. Learn more.
Reshaping How Educators View STEM Learning: Assessment of the SENCER Experience
By Janice Ballou
Transforming undergraduate STEM education is the core objective of the SENCER program. The SENCER Impact Assessment Survey was conducted to find out whether or not SENCER program attendance influences the various objectives related to this goal. As the robust descriptive results show, overall respondents perceive their pedagogical practice, perception of students as science learners, and ability to help students achieve 21st Century learning goals was influenced by attending SENCER programs. The description of the components of these objectives and the types of participants where there is more or less consensus on perceived SENCER influence can inform SENCER planning and contribute to addressing the challenges related to improving STEM education. Read full article.
Summary Results of 2010 Survey
SENCER Impact Assessment Survey results show that, in general, across all of the questionnaire items, majorities or more have been positively influenced by SENCER experiences. To illustrate key program benefits, examples of results notable for their overwhelming consensus are highlighted. The highlights are followed by a summary by Janice Ballou of other outcomes that respondents report were influenced by SENCER program participation.
Top Ranking SENCER Influences
These highlights were selected to exemplify areas of notable consensus among the survey respondents. Reports of about eight-in-ten or more survey participants credit SENCER with:
- Developing courses or programs: 84.9 percent (49.2% a great deal; 35.7% some)
- Setting learning goals for their course(s): 79.8 percent (40.6% a great deal; 39.2% some)
- Refining (SENCERizing) their courses: 79.6 percent (39.4% a great deal; 40.2% some)
- New course or new program development: 79.4 percent (42.8% very successful; 36.6% somewhat successful)
Influencing pedagogical practices that increase student opportunities to:
- Make interdisciplinary connections: 85.6 percent (strongly agree 49.8%; agree 35.8%)
- Make connections between science and civic problems/topics: 85.0 percent (strongly agree 51.6%; agree 33.4%)
- Identify scientific problems and questions: 78.1% percent (strongly agree 33.2%; agree 44.9%)
Helping their students achieving 21st Century learning goals, such as:
- Ability to engage in critical thinking: 79.9 percent (37.3% a great deal; 42.6% some)
- Capacity to collaborate or engage in group work: 78.6 percent (42.9% a great deal; 35.7% some)
- Problem solving: 77.7 percent (29.5% a great deal; 48.2% some)
Influencing their image of students as science learners able to:
- Take knowledge gained and apply it in a civic/community setting: 78.9 percent (37.5% strongly agree; 41.4% agree)
Independent (External) Evaluation of the SENCER Project
With supplemental support from the National Science Foundation, SENCER contracted with the University of Colorado, Boulder for a multi-year, independent program evaluation that included the development and validation of the Student Assessment of Learning Gains (SALG) evaluation instrument. Dr. Elaine Seymour has served as the evaluation consultant since SENCER's inception. She and Dr. Timothy Weston have served as the principal investigators of the multi-year evaluation project. The results of their evaluation are contained in a final report issued in December 2006. (You can access a copy of the project report below.)
The results of the evaluation study are most encouraging. Seymour and Weston note that "SENCER's goal of encouraging faculty to teach courses with civic content and innovative pedagogy is a reality." The researchers also confirm that the SENCER reforms are durable, noting that they found that "92% of instructors believed that their courses would be continued in the future, and 80% considered their course part of the permanent curriculum at their institution."
The data provided by >10,000 students in 345 SENCER courses who completed the SENCER-SALG have been analyzed. In addition to the important outcome of helping faculty make "meaningful changes to their instruction," the data tell who gains and what they gain from their study in SENCER courses. The researchers found it significant that:
- Students gained most in the areas of science literacy, followed by general course skills.
- Women gained more than men and non-science majors gained more than science majors on many of the items and composite variables (a fact that the evaluators note "is encouraging evidence given that females and non-science majors have traditionally been underserved or overlooked in many university science programs").
- The patterns of gains were in line with efforts by SENCER to encourage awareness of the link between civic issues and scientific content.
- Roughly a fifth of students who had never engaged in civic activities said they were more likely to participate in these activities after a SENCER course completion.
- Ten percent of students who on the pre-survey were not interested in taking additional science or mathematics courses reported on the post-survey that there were very or extremely interested in doing so.
- Similarly, slightly more than 6% percent now say they would like to consider exploring career opportunities in science and nearly 5% are now "interested in teaching science."
Internal Evaluation Reports
Evaluation is broad and deep within the SENCER project. All sub-awardees are required to file progress and final reports; all Institutes and major program activities have included process evaluations. In addition, from time to time, SENCER commissions "synthesis" reports that feature information on the effects of SENCER participation measured in terms of pursuit/receipt of intramural and extramural support, special recognition, records of tenure and promotion, and other outcomes.
SENCER Annual Report
Review SENCER activities, community participation, resources developed, and project evaluation covering February - October 2012