Aligning the Philadelphia Museum of Art’s Sherlock Program to its Intended Impact
Client: Philadelphia Museum of Art | Location: Philadelphia, PA
As a thought partner, we supported the Philadelphia Museum of Art over three years in aligning a multi-visit school program to its intended impact.
OVERVIEW
Over the course of three years, we served as a thought partner and evaluator for the Philadelphia Museum of Art’s pilot program called Sherlock. Sherlock derived from a program for medical students that the PMA hypothesized would be fitting for fifth and sixth graders.
APPROACH
We first worked with the Philadelphia Museum of Art to clarify the impact they intended to have on fifth and sixth graders. Given that the PMA was translating an existing program for another audience, the general outcomes existed, but they needed to be clarified for a new audience. We then began evaluation that was scaled up and refined over each academic year:
Year 1: A remedial evaluation with program observations and teacher interviews
Year 2: A summative evaluation with student assessments, student case studies, teacher interviews, and museum educator interviews
Year 3: A COVID-modified summative evaluation with museum educator observations, student feedback, and teacher interviews
CLIENT TAKEAWAYS
From the beginning, the PMA had clear, intended student outcomes. Over the first two years of the program, they continued to experiment with the curriculum to select lessons that they perceived to support these outcomes and made adjustments both years based on results from observations, student assessments, and teacher interviews. By the end of year two, we had measured effects on students in the areas of observation, critical thinking, and creative thinking.
In the third year of the program, the PMA was interested to see if those results would be replicated and furthered by continued program refinement. From our COVID-modified evaluation, it seemed the program was replicating these results.
We also focused evaluation efforts in the third year on supporting co-teaching in the program. As the program scaled up to serve more students by the third year, the museum was experiencing some growing pains. Therefore, observations in this third year were focused on museum educator dynamics and clarifying what co-teaching looks like in the program.
PMA educators will use the student outcomes and evaluation to help them maintain focus and stay grounded in impact as they make decisions about Sherlock, with the intent of continual program improvement.
