13 Jan How to measure the impact of interactive documentaries? | David Dufresne
New York, October 30, 2014, in a large glass room of the Center for Social Innovation. Throughout the day, about 70 people will reflect on the simple question, the “killer” question: “how can the impact of a particular interactive documentary be measured?”
The Interactive Department of Tribeca Film Institute and the MIT Open Documentary Lab have initiated this gathering. Already, for several months, the two organizations have formed a working group to explore how non-fiction interactive projects can and should be measured in terms of impact and engagement.
Goal of the day: how to clarify the real needs of interactive creators? How to give them the tools to measure impact and provide them with evaluation frameworks? And how to make this emerging creative sector sustainable?
Because it is obviously the new battlefront that is significant to all directors, producers as well as broadcasters of interactive documentary, we ask what metric can be used? Considering that interactive productions can offer in one space the data from the problem (history) AS WELL AS the discussion of tools and / or the weapons to resolve it, we understand how the question of measuring the engagement takes all its meaning. It also becomes evident why NGOs and foundations are more and more interested in the genre.
Therefore, what measure of “success” (audience, impact or repercussions) should we invent? Purely quantitative references, dear to the television industry, have a very limited sense in the interactive area.
Other measures are to be taken into account. Other measures are to be invented.
And this in fact is also part of the “interactive bet”: a complete overhaul of production habits.
Hence the organization of this day of October 30, following several months of collaboration between different actors in the field.
In the room were producers, broadcasters (POV, Video for Change …), foundations (Mozilla Foundation, Ford Foundation, Fledgling Fund, MacArthur Foundation, Bertha Foundation …), broadcasters, researchers ( Center For Media And Social Impact ,Impact Media Project , American University, MIT Center for Civic Media , i-Docs , Center for Investigative Reporting ) and six filmmakers who came to present their work, and their repercussions.
Among the questions of the day ( Storify here ):
- How to speak the same language? Indeed, common definitions are lacking for many of the players in the industry, and even more to the public. How can the impact of works be measured if their definition remains unclear?
- What is meant by “participation”? How to define “impact”?
- Can we make a list of the various expected results / interactive documentary impacts before they go online?
- How to differentiate between a “real” or “natural” impact (from the public) from an impact “expected” or “orchestrated” by the production team?
- Should we create a precise “contribution agreement” between the participants? What would be their exact role? What do we ask from them exactly and at what stage of the project? Do they contribute content, financing or promotion?
- Is the audience of a traditional documentary and that of an interactive documentary about the same? And should we expect the same participation from the audience?
- Is a large audience a guarantee of participation?
- How to focus less on the possibilities offered by our technologies and more on human interaction? How can we attract sociologists and anthropologists to these questions?
- Who measures? Directors? Financial backers? Broadcasters? The press?
- What are the duties and the responsibilities of those who create the metrics?
- How can you share and inform the metrics and other impacts of this and that work?
In the afternoon, we filmmakers were there to share our experience, including the the teams of Question Bridge or Priya’s Shakti (under construction). Elaine McMillion of Hollow highlighted the many impacts in the community which provided the subject of her film, noting that “the public remains very active in the platforms that it already uses like Facebook, etc.).”
The team of Sandy Storyline detailed its strategy: a call for evidence for Hurricane Sandy, both online as well as in the heart of the New York library system.
As for Karim Ben Khelifa, whose “The Enemy project” for France TV promises to be one of the strongest ever made about the war, he expressed his hope: to reach the 12-18 year olds (the next generation) and change the perception that they might have of war (religion, violence, etc.).
For my part, I focused on the large number and quality of the comments in debates and especially the incredible creativity of some players from Fort McMoney (re-appropriation of the logo, mapping experience, sharing all kinds of spontaneous creations of groups of players on FaceBook, Twitter accounts to hack the game, etc.).
At the end of the conference, everyone left with a list of tentative conclusions:
- Financial support should be provided for researchers to work alongside creative teams.
- The need to make honest statements (after or during each interactive experience) remains relevant.
- “Everything has an impact” is not enough motivation in the world of documentary.
- Sharing failures is good for the whole community (producers, broadcasters, writers, etc.).
- The creators call for more sharing: sharing of learning, sharing of resources, sharing of knowledge from each other.
More to follow.
Some Resources :
- Media Impact Funders
- Measuring Media Impact: An Overview of the Field ( PDF ):
- The Impact Field Guide & Toolkit
A big thank you to Lindsay N. Green-Barber, the Center for Investigative Reporting for his notes.
For the original French version, click here.
David Dufresne
Fellow, MIT Open Documentary Lab
Comparative Media Studies/Writing