Field Trip Evaluation and Creative Problem Solving
A recent article about bus delays hampering school field trips in New York City jolted me back to my days as a freelance museum educator. Although the last field trip I led was years ago, I remember all too clearly sitting in the lobby, waiting for the bus to arrive, and mentally whittling down my lesson plan as the minutes went on.
Anyone who manages museum field trips will tell you that even with the best laid plans, some unpredictable logistical challenge will come up. Now, as someone who evaluates students’ and teachers’ experiences on museum field trips, these challenges bring up a whole new crop of questions and potential solutions. I wanted to share some solutions to a few common field trip evaluation challenges in the hopes that whether you’re a field trip program manager, educator, internal evaluator, or external evaluator you can find the best ways to continuously evaluate school field trips—even when the bus doesn’t show!
Evaluation challenge: The group arrived 30 minutes late!
This is the most common issue, and often the most concerning as an evaluator. If the field trip starts late and the tour guide has to cut important parts of the lesson plan, how can you accurately evaluate the program’s intended impact? If the main data collection method you had planned to use was observations, how can you assess those elements of the trip that you never see?
Some solutions
Prioritize outcomes with the program staff. Before the field trip, discuss which outcomes are higher and lower priority. This will help both the tour guide and evaluator know ahead of time what might need to be cut if a group is late. If it’s not possible to discuss this ahead of time, you could check in with the tour guide and ask them a couple questions while you wait: What are you considering cutting? What do you consider to be a nonnegotiable part of the field trip? Why is that? Their answers will not only help you anticipate what the modified program will look like, but also sharpen your focus on the parts that you will see.
Have one or two backup data collection methods ready to go. If you are evaluating multiple field trips on an ongoing basis, but are finding that delays are impacting your ability to answer your questions, consider shifting your approach. While observations are great for providing real time, unfiltered data about the student experience, you could supplement your observational data by adding follow up surveys or interviews. Offering teachers and/or students the opportunity to reflect on their experience will provide additional information about the field trip that you might not have been able to observe and allow you to triangulate what you observed on the tour (even if it’s limited) against what teachers and/or students thought about their experience. That said, if you’re working with a limited budget, (especially if you are an external evaluator) it may impact your ability to change the design of the evaluation on the fly.
Start a list of questions for next time. No evaluation is perfect, and there will always be questions left unanswered. And even the most complete evaluations often lead to new questions! Start a running list of questions you have about the program to address in future evaluations.
Evaluation challenge: Participants aren’t signing up for the study!
We all know teachers are busy, and there’s somehow never a good time of year to ask them to do extra work. Best practice is to offer an incentive, typically a cash payment or gift card to thank them for their time. But even with an attractive incentive, convincing teachers to sign up for something on top of everything else on their plate can be a tall order.
Some solutions
Think about incentives broadly. Museum budgets don’t always allow for cash incentives, especially for larger scale evaluations. Is there anything else participants might appreciate? The museum might offer teachers a free trip (or two), to share the evaluation results with them, provide posters or other resources for their classrooms, or even professional development.
Get personal. In an ideal world museums would have deep, personal relationships with all the schools and teachers who visit, making participation in an evaluation a natural outcome of a mutually beneficial relationship. In real life, however, program managers don’t always get to cultivate those relationships. When recruiting participants you don’t know well, make your invitation as personal as possible: begin with a customized greeting and let them know they have been specially recruited because of their particular experiences (which is true!).
Plan with teacher convenience in mind. Sometimes asking for even 20 minutes more of teachers’ time can be a big ask. For example, at Kera, we were having trouble recruiting teachers to participate in a virtual interview following their museum tour. So we shifted from a follow-up interview to an onsite survey that teachers filled out at the end of their tour. While we weren’t able to get all the nuance that an open-ended interview would have provided, we still got to hear from teachers in a way that was more convenient for them.
Evaluation challenge: Hearing from classroom teachers is great, but I want to know what students think!
Field trip evaluations tend to focus on the adult participants for lots of reasons—speaking to minors introduces important ethical concerns, as well as logistical questions. When and where would I speak to students? During the tour, right after, or back at school? What would that look like? Do I try to speak to each student or pull a sample? If I choose a sample, how do I select which students to include? As an evaluator, it’s enough to make you want to skip hearing from students all together, but often, they have the most compelling feedback.
Some solutions
Try an embedded method: One solution is to work closely with the classroom teacher or museum educator and integrate a data collection method into the field trip’s regular teaching and learning activities. For example, students might fill out a worksheet in the last 10 minutes of the program, which would serve both as a way for them to share what they have learned and an activity to reinforce program outcomes . Just be sure to make sure there is enough time at the end for students to show their work without needing to rush. Here is an example of a project where we used an embedded assessment to learn what students were taking away from a middle school civics program.
Organize a diary study: Another way to hear from students directly is something called a diary study, where participants respond to a pre-determined number of prompts over a few weeks. Students could respond through writing or sketching answers to different questions after the tour back in their classrooms. Collaborating with the classroom teacher will ensure that the questions elicit responses that not only provide feedback about the program but also complement their curriculum (e.g., What connections do you remember seeing between what you saw at the museum and something you learned at school?).
Field trips present lots of different, hard to predict evaluation challenges: shortened programs, slow recruitment, relationship management. Hopefully these ideas can help you anticipate possible issues in the future and think of creative ways to address them!
