In this talk, Cathy Sigmond explores how formative evaluation supports the process of creating audience-centered exhibitions. Cathy covers the ins and outs of formative evaluation for exhibition development, from core principles to strategies for doing formative evaluation in all types of museums. 

Museum educators, does any of this sound familiar?

You care deeply about making a positive difference in museum audiences’ lives, yet...

• You are pulled in many directions by competing needs and agendas

• You have limited resources and capacity

• You need to raise money to fund your programs

• You feel you are having a positive impact but you have no evidence, and 

• You have been told to consider expanding or scaling up your program

If any of these are true of your museum program or department, you probably need a logic model.

Interviewing visitors is a valuable way to gain insight into what they think of your museum, either during or as they finish their visit. Their feedback can also help shape the development of future exhibitions and programs. I’ve had the opportunity to interview many museum visitors onsite at museums, and I personally really love chatting with them. I get to meet a lot of great people and hear so many unique and diverse perspectives. To help out anyone interested in trying out interviews, I would like to share a few tips as a seasoned interviewer!

At the Visitor Studies Association (VSA) conference in July 2024, I co-presented with two museum exhibition practitioners about questioning the status quo of exhibition evaluation practices. We each approached the session with questions and skepticism about traditional summative evaluation for exhibitions. The big question I explored was: Are summative evaluations worth the cost (money and time)?

We have seen a surge in requests for audience research projects over the last couple of years. All types of museums, from science to history to botanic gardens, are asking for studies that can help them understand more about their visitors. And with this flurry of audience research projects, we have been doing a lot of thinking about museums and their audiences.

I’ve been a fan of collaborative research ever since I learned about it when working on my dissertation on the role of Whiteness in the gallery teaching practices of White art museum educators (like myself). Initially, I was trying to study Whiteness through a traditional approach, where I developed study questions, recruited participants (other White museum educators), designed instruments, and analyzed data on my own. But several months into data collection, my findings just weren’t addressing my questions with the richness that I expected. I finally realized: I need to examine Whiteness by critically examining it in myself.

Have you ever found yourself staring at a mountain of qualitative data, feeling like you're lost without a map? Whether it’s transcripts from interviews or focus groups, written responses from a survey or assessment, or entries in a diary study, figuring out where to start can feel daunting.

Over the years, I’ve had the pleasure of working with many design firms and museums to prototype early versions of new exhibits with visitors. Prototyping helps us understand what works well about an exhibit and what doesn’t, and the results guide us in refining exhibits so they are more engaging and effective. Some of my favorite examples include testing out a tactile map of a large National Recreation Area; a hands-on airplane seat-building challenge; interactives about climate change; a Sims-style urban-planning experience; and, most recently, a 10-foot tall Plinko-like game about natural resources.

Observations are an important tool for evaluating museum programs from the perspective of a third party. Observations can reveal important dynamics and surprising ways a program may be addressing its intended outcomes. Observing programs in museums brings their own set of considerations, considering their logistical variations and museums’ history of exclusion.

As evaluators, we use many different methods to collect data, and one of the most-frequently used methods we use to collect qualitative (descriptive) data are interviews. Interviews take many forms depending on the context and project but what does not change is the essential element of curiosity that is built into the interview process.

At Kera Collective, we average around 20-30 active projects at a given time, of varying size and scope. As a small team, people often ask us how we handle so much work at once. The short answer? We prioritize staying organized.

We recently conducted an exploratory study of the efficacy of various interpretative strategies at the Denver Art Museum (DAM). Given the goals of the study, we proposed an infrequently-used method: walk-and-talk interviews.

Every gathering has the potential to be great. And anyone can learn how to “gather well.” This is the premise of Priya Parker’s 2018 book, The Art of Gathering: How We Meet and Why It Matters. As someone who plans a lot of gatherings in my personal life, I recently picked up a copy hoping to gain some insight into how I could be a better host for my friends and family.

“Why don’t we just do a survey?” If I’ve heard it once, I’ve heard it a thousand times. Surveys are the first instinct for many museums seeking to learn about their audiences (and I’m sure the same is true across many industries).

In our work at Kera, we often ask museum practitioners to articulate audience outcomes as a starting point for planning and evaluation. Audience outcomes are the difference a museum intends to make in its audiences’ lives through an experience (like an exhibition or a program).

When analyzing qualitative data, our end goal is always to provide a sense of how much or how little an idea or trend came up within the whole sample. Sometimes, it’s very clear—people either say “yes, I liked this” or “no, I didn’t like this,” and there were no overlapping reasons why. But most of the time, it’s less straightforward and more nuanced. We have to interpret what people say based on many factors.

In African American literature, my field of study, citation is pivotal practice—one of call and response. Throughout college and graduate school, my African American literature professors would turn toward us, their students, and pause. After a breath, they would all say something to this effect: “Citation is important. Why? Because you are calling on the ancestors, Black thinkers and leaders, before you. Their voices are often left out of the historical record. In your research, you must not forget the voices you have learned from.” Today, as I approach museum work, I take this lesson with me.

Evaluating audiences’ experiences in a museum program, exhibition, or initiative, like most things, requires thoughtfulness and planning to be useful. But, when things get busy and deadlines loom, sometimes the last thing you want to do is slow down and consider why you are doing something.

Observations have always been one of my favorite ways to collect data. Watching how people move about and behave in a space is inherently addictive—there is so much you can discover if you pay close attention to what is happening around you, and it feels a bit like going undercover as a spy.

While there is often a bias toward quantitative data, numbers aren’t everything. I love the complexity and nuance revealed through qualitative data. Because qualitative data is open-ended, it helps you understand peoples’ thoughts and experiences in their own words–this can reveal interesting, profound, funny, and unexpected insights that would be lost in quantitative methods.