Over the years, I’ve had the pleasure of working with many design firms and museums to prototype early versions of new exhibits with visitors. Prototyping helps us understand what works well about an exhibit and what doesn’t, and the results guide us in refining exhibits so they are more engaging and effective. Some of my favorite examples include testing out a tactile map of a large National Recreation Area; a hands-on airplane seat-building challenge; interactives about climate change; a Sims-style urban-planning experience; and, most recently, a 10-foot tall Plinko-like game about natural resources.

Observations are an important tool for evaluating museum programs from the perspective of a third party. Observations can reveal important dynamics and surprising ways a program may be addressing its intended outcomes. Observing programs in museums brings their own set of considerations, considering their logistical variations and museums’ history of exclusion.

As evaluators, we use many different methods to collect data, and one of the most-frequently used methods we use to collect qualitative (descriptive) data are interviews. Interviews take many forms depending on the context and project but what does not change is the essential element of curiosity that is built into the interview process.

At Kera Collective, we average around 20-30 active projects at a given time, of varying size and scope. As a small team, people often ask us how we handle so much work at once. The short answer? We prioritize staying organized.

We recently conducted an exploratory study of the efficacy of various interpretative strategies at the Denver Art Museum (DAM). Given the goals of the study, we proposed an infrequently-used method: walk-and-talk interviews.

“Why don’t we just do a survey?” If I’ve heard it once, I’ve heard it a thousand times. Surveys are the first instinct for many museums seeking to learn about their audiences (and I’m sure the same is true across many industries).

In our work at Kera, we often ask museum practitioners to articulate audience outcomes as a starting point for planning and evaluation. Audience outcomes are the difference a museum intends to make in its audiences’ lives through an experience (like an exhibition or a program).

When analyzing qualitative data, our end goal is always to provide a sense of how much or how little an idea or trend came up within the whole sample. Sometimes, it’s very clear—people either say “yes, I liked this” or “no, I didn’t like this,” and there were no overlapping reasons why. But most of the time, it’s less straightforward and more nuanced. We have to interpret what people say based on many factors.

In African American literature, my field of study, citation is pivotal practice—one of call and response. Throughout college and graduate school, my African American literature professors would turn toward us, their students, and pause. After a breath, they would all say something to this effect: “Citation is important. Why? Because you are calling on the ancestors, Black thinkers and leaders, before you. Their voices are often left out of the historical record. In your research, you must not forget the voices you have learned from.” Today, as I approach museum work, I take this lesson with me.

Evaluating audiences’ experiences in a museum program, exhibition, or initiative, like most things, requires thoughtfulness and planning to be useful. But, when things get busy and deadlines loom, sometimes the last thing you want to do is slow down and consider why you are doing something.

Observations have always been one of my favorite ways to collect data. Watching how people move about and behave in a space is inherently addictive—there is so much you can discover if you pay close attention to what is happening around you, and it feels a bit like going undercover as a spy.

While there is often a bias toward quantitative data, numbers aren’t everything. I love the complexity and nuance revealed through qualitative data. Because qualitative data is open-ended, it helps you understand peoples’ thoughts and experiences in their own words–this can reveal interesting, profound, funny, and unexpected insights that would be lost in quantitative methods.

This post will help you determine—if indeed you are conducting human subjects research—whether it is exempt from IRB review.

This third post defines criteria for human subjects research by breaking down the term “human subjects research.”

In my first post in this IRB 101 series, I described what IRBs are and why they exist. IRBs exist to protect research participants. In this second post, I focus on risks to research participants.

My hope in leading the IRB 101 workshop and writing these posts is to help others overcome their nervousness about IRBs. Let’s start with a shared, foundational understanding of IRB.

Since the reality of coronavirus set in back in March, our team has been having a lot of conversations about study design.

Based on my experience presenting quantitative results to museums, I came up with a list of 10 things to know about quantitative results so you can better assess their meaningfulness.

This article describes critical appraisal, a method applied by us to help a museum’s education department make data-driven decisions about programming.

Figuring out sample size for interviews can sometimes feel trickier than for quantitative methods, like questionnaires because there aren’t tools like sample size calculators to use. However, there are several important questions to consider that can help guide your decision-making.