How to quantify our progress
Should we count how many aquatic animals are positively affected?
Since our project aims to contribute to a large-scale cultural change, so that in the long run, thousands of billions of aquatic animals will be spared intense suffering, one might want to quantify how many aquatic animals are positively affected by this campaign each year (at least this is a metric that we have already been asked for by a funder). It seems interesting to us to make public the thoughts we have had on this question. Even if we think that WoDEF actions play a worthwhile and supportive role in the movement, we understand and share the desire to quantify their impact.
We have asked for help from organizations and individuals who seemed relevant to the topic, including Faunalytics, Animal Ethics, Effective Altruism France and the data scientist Harish Sethu. The consensus was that it is very difficult to produce such an estimate. For instance, the latter said that “Something as complex as a rising social movement relies on thousands of varied actions over multiple decades, and it can be extremely difficult to quantify the impact of any one or any set of them accurately, especially over the long term.”
We have come to the conclusion that it is beyond our means to evaluate our impact in terms of numbers of individuals affected positively by our project, or even that it could result in misleading numbers. Indeed, we take a long-term approach, so the impact of our actions is not designed to be measured by short-term metrics. The actions organized for the World Day are of cultural nature, whose aim is to influence people's attitudes in order to provoke social and political changes in the next decades. Political science teaches us that major reforms in our societies rarely happen without popular support, that this support depends, among other things, on what people think, and that their ideas are influenced by collective mobilizations such as campaigns (among other sources of influence), over several decades. Moreover, as Animal Ethics recommended us to point out, most animals will live not in the very near future, but in the long-term, and our actions now can impact how their situation will be. This concerns a very large number of animals that it is not possible to count without ending up with an estimate “likely to be highly imprecise, with confidence intervals large enough to render them meaningless” (Harish Sethu). However, it seems to us that we are not alone in considering it crucial to work on cultural change. Andrea Polanco, Research Scientist at Faunalytics, tells us for example: “I agree with your theory of change that changing individual attitudes can increase public support of an issue, which can then positively impact institutional outcomes and long-term change.”
We could have tried to make an estimate by calculating this way: If X number of people attended the actions organized during the World Day and Y percent of these people reduced or stopped eating aquatic animals, then we could assert that Z aquatic animals were positively influenced by this project. However, we believe that even an estimate like this one would not be able to highlight the impact that our project has and would be extremely imprecise, so much so that there would be few benefits in doing it. Furthermore, our focus is not much on individual dietary changes but mainly on cultural and political change. A person who is more aware of the suffering of aquatic animals and would be willing to sign petitions or get involved to help improve or ban certain practices – such as banning the sale of live crustaceans, banning live scalding, improving water quality in aquaculture, etc. – is already having a positive impact on aquatic animals without necessarily reducing their consumption of aquatic animals.
How to measure our progress, then?
We are still thinking about the relevant metrics to assess to what extent we are contributing positively to cultural change, and to what extent this cultural change is actually helping to achieve our long-term goal.
In the conclusion of Animal Charity Evaluators’ Protest Intervention Report from March 2018, it states that “In short, we think that the most effective protests are those that are conducted professionally (i.e., with a strategy for making a visual impact and a designated individual on site to speak with the press), with a specific target and a specific ‘ask’ ”. Several actions carried out as part of WoDEF fall under this definition of the most effective protests, or at least come close to it in terms of defining a specific demand for a well identified target, which leads us to believe that we are already making progress in the right direction.
For instance, in 2022:
- Manitoba Animal Save organized an action against the sale of live lobsters in Winnipeg grocery stores.
- PETA Deutschland carried out an action against the aquaculture farm that was being built in Malchow.
- We ourselves are encouraging our partners to target, among others, aquaculture farms. A member of our team was invited to speak during a podcast made specifically for WoDEF. During this interview, she spoke about the salmon farms that were supposed to be built in France. The other speaker, Amandine Sanvisens, talked a lot about the octopus farm project planned in Spain.
- Aquaculture farms were the theme of WoDEF 2021 and demonstrations were held in Boulogne to demand the banning of the huge salmon farm that Pure Salmon wanted to build (about 2 million salmons were expected to be raised and killed there each year).
The metric that could be defined here is the number of actions organized each year that meet the definition mentioned above. For the time being, we do not claim to carry out such an exhaustive count, as this would imply to objectify precisely according to which criteria an action is considered professional (for example, when is the target considered enough specific for the action to match the definition mentioned?). Yet we do not have the resources, at the moment, to carry out such a meticulous evaluation work.
We are working to collect more and more data on the impact our communication is having on partner organizations and their audiences, which is another aspect on which we can build metrics:
- number of activists involved in each organization who would not have addressed the topic of aquatic animals without us,
- number of people reached by their actions,
- number of press articles generated,
- number of people exposed to press articles reporting on the actions,
- number of existing campaigns redesigned to better address aquatic animal issues as a result of our intervention,
- number of new actions organized to address aquatic animal issues as a result of our intervention beyond just WoDEF…
These are admittedly not direct indicators of the number of individuals affected or years of suffering avoided, but they should at least help to compare our work with that of other organizations dealing with cultural change.