A rescued rooster at Farm Sanctuary's Acton shelter
As a research organization dedicated to rigor and collaboration, we feel it’s important to share not only our accomplishments but also our mistakes. Our decision to have a dedicated page for those mistakes was inspired by the mistakes pages of GiveWell and Animal Charity Evaluators. Some mistakes are omitted to protect privacy.
While we have received positive feedback on the SI Podcast, it may not have been worth as much effort as we put in, given it has not reached a large audience. Relatedly, it may have been a mistake to not put much effort into spreading it to particular audiences (e.g., skeptics and humanists) where it could have gained momentum.
We incorrectly entered payroll information for an employee, which was not as easily fixable as we expected because of issues in the payroll software codebase. This created a time sink of contacting various customer support teams to fix the payroll records.
We had errors in the initial publication of our global and US factory farming estimates: using an incorrectly low estimate of global egg-laying hens, erroneously swapping two variables within the fish estimates, and using a US sales figures for meat chickens instead of inventory.
Our worker’s compensation policy was cancelled, and at the advisement of the broker we work with, we paid for a new policy ourselves, but that did not constitute purchasing the policy, which resulted in a fine. Our mistake was in not seeking a second opinion on how to acquire a new policy after the initial policy was cancelled.
We significantly underestimated the time a large research project would take, leading to a backlog and less time spent on other important projects.
Jacy should have titled his TEDx talk “The Future Is Vegan” because, while the TEDx talk racked up tens of thousands of views, “The End of Animal Farming” didn’t get the talk to show up in the top search results for common relevant queries, most of which include the word “vegan,” e.g. “vegan TED talks” (and presumably has also limited it in showing up in autoplay and/or the sidebar).
From July to November, Jacy probably spent a little too much time speaking for local effective altruism and animal advocacy groups, given this was less cost-effective than expected.
The book might have done better with a title of “The End of Meat” to focus more on virality than intellectual rigor. (“The Future is Vegan” was also a possibility, but probably would have narrowed the audience too much.)
A major media outlet expressed interested in an op-ed centered on the book, and Jacy thinks he would have been more likely to get it published if he had pushed more aggressively for its publication.
Some of Jacy’s book interviews could have been improved if Jacy had rehearsed and outlined responses to common questions. In other words, Jacy thinks he should have focused more on ‘sticking to talking points.’
Jacy should not have started the large-scale messaging strategies RCT in 2018 (now a collaborative study with two academics) as we have deprioritized direct experimental research and it would have been better tackled by the quantitative researcher we expect to hire in 2019.
Kelly failed to notice one moderately sized donation and as such to thank the donor and reach out to them again in a timely fashion.
After our 501(c)(3) application was returned due to a form error, Kelly should have conducted the rest of the application process on her own or with hired legal service instead of the pro bono service we were using, in order to avoid several months of waiting.
Our series of three blog posts on the tractability of changing the course of history arguably should have been one report, but by the time we realized this, it would have required too much editing to be worthwhile.
In our first fundraising pitch shared in May 2017, we did insufficient review to make sure it was clear, free of typos, and detailed in specifying which research projects were our top priority.
We failed to include one full-time EAA researcher on our “Effective Animal Advocacy Researcher Survey June 2017” who should have been included. Fortunately we were able to send that person the survey before we published the results.
When we initially transferred the Research Network from Sentience Politics, we sent out emails from multiple addresses and should have instead used a single point of contact.
Two generous lawyers handled our incorporation and 501(c)(3) application pro bono. However, that process took a long time, and in retrospect, it might have been better to do it ourselves, even though the likelihood of making an error would have increased. The delay in incorporation postponed when we could make our first hire, and while our fiscal sponsor, the Centre for Effective Altruism, has been very generous in continuing to accept earmarked donations for us for longer than anticipated, receiving our 501(c)(3) status sooner would spare them some time.
In our first researcher job interviews, we took too few notes on each applicant, overestimating how many details from the interviews we’d remember without writing them down. We didn’t want to make candidates nervous and we have personally had plenty of interviews where people didn’t take (or appear to be taking) notes, which made it easy for us to overestimate how easily we’d remember everything. However, our applicant pool was small, we used an evaluation chart with predetermined criteria, our application form included questions that significantly informed our evaluations, and our process involved an editing project which heavily informed our evaluations, so the lack of more detailed notes from our interviews probably did not significantly affect the process.
We underestimated the initial administrative time costs of establishing the organization, which led us to rush to complete one of our initial research projects, our social movement case study of the British antislavery movement, by the end of November. The project could also have been somewhat smaller (i.e. the research phase could have stopped somewhat earlier in light of diminishing returns on significant new information).
In our “Survey of US Attitudes Towards Animal Farming and Animal-Free Food October 2017,” we should have included two additional questions despite increasing costs: political affiliation and self-identification as vegetarian. We also should have phrased the last two questions about humane farming more similarly (either both “treated well” or both “treated humanely”). This might have been fixed if we had sought additional peer review, but we’re not sure about that.
We overestimated how much interest in the “Survey of US Attitudes Towards Animal Farming and Animal-Free Food October 2017” we’d get from mainstream media outlets. In retrospect, we should not have sent it to those outlets because that might have reduced our chances of getting coverage from them for future projects.
In our social media posts and press release about the “Survey of US Attitudes Towards Animal Farming and Animal-Free Food October 2017,” we should have put more emphasis on the strategic implication that we should focus more on changing institutions. We partly made this mistake because we expected our coverage to be more from mainstream outlets, who wouldn’t be as interested in the strategic takeaways. We also should have included the animated video Mercy For Animals made about the survey results in the press releases we sent following its publication, but we hadn’t looked carefully enough at the MFA blog post to see it at the time.
Subscribe to our newsletter to receive updates on our research and activities. We average two to five emails per year.