Using Transcript Review to Acknowledge Power Dynamics in Evaluation
As a researcher for Kera Collective, I have been fascinated with the overlap between museum evaluation and my background in English literature, folklore studies, and storytelling. For me, research is storytelling. With a report, there is an author; there is a narrative. The reader follows the narrative, walking along the path the author laid out. In all the paragraphs the author has crafted, there is a message and an interpretation.
While evaluation reports are different from fiction, I think it is important to remember the aspects of storytelling in the work we do—stories are embedded in research. Moreover, by recognizing storytelling in evaluative research, we can contemplate important questions about how we engage with and present data. Here are a few essential queries that I try to carry with me in every research step: where does power rest (who is able to tell this story?) and how do we tell this story in a genuine, respectful way?
Additionally, how might we, as evaluators, acknowledge power dynamics so overly present in evaluation? What can we do to honor interviewees’ feelings about and interpretations of their own stories?
Transcript review may be one possible method. Generally, during the transcript review process, a researcher sends an interviewee their transcript. Similar to storytelling evaluation methodologies, interviewees are able to see their own words in writing and sit with the words on the page. Usually within one or two weeks, through email, they can respond to the interviewer by electing to omit statements and/or add more information or context.
In our research, we have also provided interviewees with an additional option: instead of conversing over email, interviewees can have a follow-up conversation with the researcher to review their transcript together. In this conversation, the interviewee has the floor; in a more informal, dialogic setting, they can discuss any clarifications or raise any concerns. Often, these conversations range in topic, from addressing typos to clarifying an important statement. In some cases, depending on the interviewees’ concerns and desire for involvement, the researcher may also follow-up with the interviewee during the report writing stage, where the interviewee reviews and responds to a paraphrased summary of their words.
Ultimately, the entire transcript review stage serves as an important reminder: research is a learning process and an exchange of information and ideas.
Even so, we must proceed with caution. We must ensure the evaluation project design clearly describes the transcript review process to participants (to allow participants to opt in or out). We also must make sure potential follow-up conversations align with participants’ incentive amounts. Moreover, we should understand that transcript review is a component of a project but is markedly different from co-research and participatory evaluation (which involve research participants in the development of evaluation design). Certainly, transcript review is a learning process that can be improved upon, especially with input of those involved in the research.
Overall, I invite you to consider how we can acknowledge the many perspectives and stories that exist in evaluation. How can we work with research participants to genuinely and accurately tell their stories?
