Prototyping, Summative, and Process Evaluations for the Change Your Game / Cambia tu juego Project

Experience Design Research
Written By Cathy Sigmond

Client: Lemelson Center for the Study of Invention and Innovation, Smithsonian National Museum of American History | Location: Washington, DC | Funding: National Science Foundation

 
 

We served as the external evaluator for the NSF-funded project Change Your Game / Cambia tu juego—providing formative, summative and process evaluation services throughout the grant.

OVERVIEW

With funding from a National Science Foundation (NSF) Advancing Informal STEM Learning (AISL) grant #2005404, the Lemelson Center for the Study of Invention and Innovation at the Smithsonian National Museum of American History (NMAH) launched the project Change Your Game / Cambia tu juego. The project aimed to develop a STEM exhibition for the Smithsonian’s National Museum of American History with embedded research into visitors’ exploration of inventive identity.

APPROACH

From 2020-2024, we conducted multiple evaluations to support the project team in developing the exhibition and ultimately measuring the impact of the project.

  • Alpha and beta prototyping with priority audiences including females 10-17 years, African American males 10-17 years, and people with disabilities

  • Title testing with English- and Spanish-speaking walk-in visitors at NMAH

  • Summative evaluation exploring the impact of the exhibition on walk-in visitors and the aforementioned priority audiences

  • Process evaluation exploring the efficacies and challenges of the exhibition development process to contribute to understanding new approaches and understanding of the design and development of STEM learning in informal environments

CLIENT TAKEAWAYS

The summative evaluation of the Change Your Game / Cambia tu juego exhibition indicated that project team members did a good job at identifying stories that represent many different people and sports. Visitors found it novel to consider invention in relation to sports and games, thus serving as an entry point to inventive identity exploration in terms of ontological-epistemological beliefs within audiences’ role as a museum visitor. Beyond ontological-epistemological beliefs, the evaluation results are indeterminate about how the exhibition affected audiences in other areas of identity exploration. The evaluation raises questions about inventive identity to be considered along with the research findings for further exploration.

In the process evaluation, the project team complimented the collaborative project plan, particularly valuing how the project was designed to include many voices into the exhibition development process.  NMAH staff perceived the process as particularly collaborative and innovative for them in its focus on promoting inventive identity, which fostered a relationship with educational psychology researchers around a dynamic systems model of identity. However, the project team identified challenges that prevented the project from fulfilling the collaborative nature they had desired. The COVID-19 pandemic added a layer of uncertainty and required adaptability not previously encountered. The project was also challenged by turnover within the project team. 

Looking to the future, we helped NMAH staff identify three areas for improvement in their process: build in ample time and resources on collaborative projects to address challenges that may emerge, identify fewer priority audiences for an exhibition, and engage priority audiences even earlier in the exhibition development process.

Cathy Sigmond

Cathy brings many years of experience in education and experience design to her role as Head of Strategy at Kera Collective. 

Having previously worked in a variety of educational settings, Cathy is driven by her constant fascination and delight at how people make discoveries about the familiar and the unfamiliar. 

Cathy loves helping to shape experiences that spark curiosity and make a difference in people’s lives. She particularly enjoys the rapid, iterative nature of design-based research and the deep insights that come from qualitative research, especially on projects exploring interactions with the digital and built environments. 

Cathy shares her passion for experience design research widely and regularly guest lectures for graduate programs, including the Fashion Institute of Technology’s Exhibition and Experience Design Program and the Pratt Institute’s School of Information. Cathy served as the co-chair of the Museum Computer Network’s Human-Centered Design special interest group from 2018-2021.

Outside of work, you can usually find Cathy playing soccer, thrifting, or making her way through her large cookbook collection. 

Cathy’s favorite museum experiences are immersive; she will always vividly remember walking through the giant heart at the Franklin Institute, being surrounded by birds at the Peabody Essex Museum, and hearing centuries-old instruments come to life at the Museum of Musical Instruments. 

Previous

Keeping Audiences’ Needs Central When Creating a New Exhibition for a Science Center
Next

Evaluating Visitors’ Experiences in Vesterheim’s Folk Art Galleries