Educators and researchers are both interested in what students “get” from a curriculum. We want to know both if and how a curriculum changes students’ understandings about the world. We also want to know if students apply what they have learned and change their behaviors. Yet, to truly understand the effectiveness of a curriculum, one needs to know what students experience. Process evaluation does just that by answering questions such as, Were the lessons taught as designed? What parts were left out? What parts were augmented? What did the students talk about? What made them excited or angry? At the Center for Food & Environment, our process evaluation research is based on a comprehensive and systematic conceptual framework and uses both quantitative and qualitative methodologies. We also take our process evaluation to the next step by exploring if there are linkages between classroom implementation and how much — and in what ways — students changed their food and activities behaviors.
Process Evaluation of C3 Formative Evaluation Study
The process evaluation research during our formative evaluation of the curriculum indicated that participating teachers found the professional development workshops and support we provided useful in helping them to closely follow the lesson plans in their classroom. Additionally, we had an implementation coordinator visit the classroom to provide on-going support. Teachers taught the curriculum daily while the coordinator visited 1-2 times per week. The teachers greatly valued being able to have their questions — and their students’ questions — answered. Student satisfaction with the curriculum was rated a 3 on a 4-point scale, with 4 being very satisfied with the curriculum.
Outcomes of the C3 Summative Evaluation Study
In our summative, or final, evaluation study, we had 20 C3 intervention classrooms. Our implementation coordinators, who were previously science teachers, observed about one-third of all C3 sessions for each class. From these observations, we rated how completely the C3 lessons were taught by the teachers. We called this “dose delivered.” The dose delivered levels were high at 73%. We also rated student engagement and participation in the lesson activities. We called this, “dose received,” that is the dose of the curriculum that was received by the students. The dose of the curriculum received was 69%.
The students in classrooms with higher dose delivered and dose received reported greater satisfaction with the curriculum. They also made greater changes on out targeted behaviors (eat more fruits and vegetables; drink more water; drink fewer sweetened beverages; eat fewer processed, packaged snacks such as chips, candy and baked goods; and eat less often and make healthier choices at fast food places) and they increased activity (more purposeful walking and taking the stairs and less leisure screen-time) behaviors. More importantly, on an individual student level, higher satisfaction with the curriculum was the strongest predictor for behavioral changes.
Our goal is to identify process-related factors that predict student outcomes, and to use this to make future interventions more effective. We are working to discover consistent process components across studies in order to develop conceptual maps that describe what is involved to achieve high rates of fidelity to curriculum implementation. The more we link process evaluation data to student outcomes data, the more we will build an understanding about what makes curriculum interventions effective.