Image Credit: Pharexia, Ratherous, AKS471883, Source Data from  Johns Hopkins University CSSE, The Centers for Disease Control and Prevention, New York Times, CNBC.

As it quickly became clear in late February and early March that COVID-19 was not going away anytime soon, attention turned to trying to figure out when and where the virus would spread. Epidemiologists and virologists have had their work cut out for them, trying to simultaneously reassure and warn people the world over about the dangers, the nature and the potential timeline of the virus.

So it came as somewhat of a surprise to see ecologists try and tip their hat into the ring. Early on in the pandemic, teams of ecologists sprang up, trying to use Species Distribution Models to predict the spread of the virus. And whilst this might sound helpful, many of these studies lacked collaboration with epidemiologists, and their predictions very quickly fell flat. Some studies suggested that areas like Brazil and Central Africa would be largely spared by the virus, which quickly turned out not to be the case. Flaws in the studies were spotted quite quickly by concerned members of both the ecological and epidemiological communities alike, and a few teams got started on responses.

One such team consisted of statistical ecologist Joseph Chipperfield, alongside epidemiologist Colin Carson, climate ecologist Richard Telford, and statistical ecologists Blas Benito and Bob OHara. Joseph and his colleagues have published several responses to these initial models (linked below), cautioning the ecological community about the use of Species Distribution Models in the current crisis. I spoke to Joseph about the problems with applying ecological modelling techniques to the coronavirus, some of the fallout from the models that were published, and what ecologists can learn from this affair going forward.

Sam Perrin (SP): Why do you think that ecologists felt the need to weigh in on this in the first place? Surely we have enough other global crises to be focusing on?

Joseph Chipperfield, Ecologist at the Norwegian institute for Nature Research (JC): There’s a couple of things here that’s been enticing for someone looking to use a Species Distribution Model (SDM) for this. There were initially some basic patterns that presented themselves, and if you don’t think about it too much, then you can almost convince yourself that it’s a worthwhile endeavor. There was a nice dashboard that had been made by the John Hopkins University, where they had reasonably up-to-date data from across the world. They got this up and running quite early on in the pandemic and so if you’re a biogeographer who is trained to see spatial patterns, and you have lots of training in SDMs to predict the range of species, you think to yourself, “I could use these tools to predict the range of the virus”. You think, “well we use these models for invasive species, why not a virus?”.

As we explain in our paper, the particular peculiarities of the ecology of this virus and other directly contacted viruses, means that the distributional data associated with them is not appropriate for analysis using SDMs. However there’s also a little bit of naivete in that you assume the epidemiologists don’t have access to these spatial models. That was a little bit frustrating. I’m an ecologist, my particular background is SDMs, and I know for a fact that most of the models I use are taken from epidemiology. So the epidemiologists are quite far ahead of us in this respect.

Related: Bob O’Hara: The Rise of the Ecological Modeller

SP: The first thing that sprung out when they tried to apply SDMs to the coronavirus, is that this relies enormously on human and social factors, which are study areas that ecologists have a poor history of incorporating into studies (though we’re starting to improve). Could this be a wake-up call that we need to do more to normalise interdisciplinary work going forward?

JC: Yes. I think ecology, as it deals with the interactions between aspects of ecosystems, is nicely positioned for some forms of interdisciplinarity. But I think a very major failing in this case is that actually in the preprints that we’ve seen, there are almost no public health or epidemiology authors on them. And I think that was the major problem. If some of these papers had one epidemiological author on them, the epidemiologist would have had the relevant training and said ‘this has been done before, we have better models’. I wouldn’t have even envisaged trying to write a response if we had not had Colin on board. Before we met Colin we were making so many stupid mistakes. I didn’t even know the difference between SARS-Cov-2 and COVID-19.

Related: Species distribution models are inappropriate for COVID-19

SP: So tell me about the initial responses you wrote.

JC: We first had a pre-print response where I was the lead author. It was largely a response to one particular study, led by Miguel Araújo. The Araújo study was the first preprint that got noticed, mainly because they had made quite a big press release before they had published it. It was tweeted by the journal Ecography, even before it was available as a preprint, just linking to a blog post with a map showing predictions of ‘spreading risk’. Widely circulated press outlets picked up on it, including the BBC. We saw this and were concerned. We decided initially that this particular paper needed a response. We needed to explicitly say why this paper was wrong.

But we decided very quickly that we had to be more general with this, because using this particular class of models in the way that they have is not useful, and we realised there were going to be more biogeographers who would probably do this. So Colin took the lead on a more general response, because it required a lot more background epidemiological knowledge. But I can’t imagine writing either paper without having an epidemiologist involved. We would have just made simple terminology mistakes, we just simply aren’t trained in that discipline, so the response would have been lacking.

Related: On the inadequacy of species distribution models for modelling the spread of SARS-CoV-2: response to Araújo and Naimi

It seems very odd to me that we have a large community of ecologists who were very happy to publish on epidemiology. I don’t know if they had reached out and just not been able to get an epidemiologist involved (I mean epidemiologists are pretty busy right now). So I can understand why they might not want to jump on board papers with ecologists. But I think that this has been a major failing.

SP: How much responsibility do you think people need to take for these models they’ve put out?

JC: In a case where the models have turned out to be of such poor quality, I think it would be fair for the people behind them to at the very least make a public statement admitting that the models hadn’t worked in the way they’d hoped, and that they were sorry. At least own the failure publicly enough that it can get picked up by same outlets that disseminated it in the first place. It doesn’t necessarily undo the damage, but at least it helps to lessen it somewhat.

I think in this case, one of the crucial issues is that ecologists were not really used to working in a domain that has such instant and obvious political and social ramifications. One of the crucial things is that people working in public health and epidemiology are used to working within risk frameworks that have that immediate impact. When they put work out, they have to ask themselves if a study can cause more damage than it can prevent, so they therefore have a protocol to go through before they put papers out.

It’s very hard to ascribe intentions on the part of the authors, but I’m always going to be generous and assume that they were the best intentions and they were trying to help. But this has shown a certain amount of naivete from ecologists here. We’re not used to having our work picked up and maybe used inappropriately or applied in a way we weren’t expecting. That’s not to say it doesn’t happen in ecology, but here the ramifications were obvious and impending, and I don’t think enough forethought was given to the way the research would be distributed widely, and then acted upon in some political spheres. I think then the responsibility is on the authors to hold their hands up and admit that the models haven’t performed very well, that they advise interested parties not to act on these, certainly not base policy decisions on these models. I think that’s very important.

Our coauthor Colin Carson has a lot more experience than we do on working on the interface with public health and epidemiology. So he sees ecologists riding roughshod over the risk framework that he has to adhere to, and that has been particularly frustrating for him. It’s something that has been a bit disappointing to see: ecologists not being very helpful in that respect. I’m an ecologist, and I’d like to think that my discipline is helpful. But in this particular debate, I feel like, if anything, we’ve muddied the waters and have created a lot of unnecessary noise.

SP: The concept of risk seemed to be quite oversimplified by initial models. “Risk” is obviously very different for different countries based on their population densities and infrastructure.

JC: I think it all goes back to the fact that epidemiologists have a very strict risk framework, and rightfully so. People working in public health know that when they make public statements they have to be careful about what they say. Of course ecologists can be misconstrued (most ecologists working in the field of climate change will attest to that), but the consequences normally aren’t as severe. I’m sure we’ve all been burnt by having something we’ve said twisted a little bit, but it hasn’t resulted in human lives being on the line in the same way.

SP: What is the preprint process, and why has it been so important here?

JC: If you have a paper that you’ve written, and it’s only in manuscript form and it hasn’t been published, maybe you want people to know about it. You can place it on a preprint server, and people can see it, and review it, but in a casual sense. There isn’t any formal peer review necessarily. Preprints are quite often looked at by people in the field, and there are many preprints that are very high quality. The issue is that there’s not really necessarily the same rigorous checking that you’d have to have if you went through a traditional peer review system.

But my opinion of the preprint server before was that it was quite a nice way to get work out, because the traditional peer review system can be slow and laborious. Sometimes you might do something that can be interesting to your field but not necessarily be publishable, it might not have an obvious outlet. But you don’t want to lose that work. I think in a world where things don’t have definitive consequences, that is fine. It’s just getting information out there. And people in our community know that preprint servers have not been peer-reviewed. So people know not to put too much scientific weight on the results contained within them.

SP: Has this affair changed your opinion of preprint servers?

JC: Up until the COVID outbreak I was pretty relaxed about them. I think this has been a real test of the system though, and my opinions have shifted somewhat. As a community, we have to show some restraint regarding what we put on preprint servers. I think it’s very naive to put a paper on a preprint server with global predictions of coronavirus risk and assume that this won’t be picked up or used negatively. In the Araújo study this was compounded by the fact that this submission was done to much fanfare. This paper was disseminated far and wide.

By uploading such preprints, we as a community, misunderstand the incentives that journalists and politicians have. Journalists have a different incentive structure, they want a scoop over their competitors. If they can pick something up at the preprint level, so they get their head above the competitors, who might wait until it’s published, then they’ll publish it. And sometimes the context of that study and the fact that it hasn’t been peer-reviewed will get lost. Sometimes it won’t, but also the general public don’t generally know the difference between a preprint server and traditional published article either.

Now the incentive structure for politicians is that they generally don’t want to do major interventions. They’re costly, they’re not politically welcome a lot of the time and people don’t like being put under lockdown. People don’t want to be socially distant, no-one enjoys it. Politicians want to be reelected. They will use things that agree with them, is what it boils down to. If there’s a paper that looks authoritative and scientific, you might not know yourself about the difference between preprints and published literature, but it seems to support your case that maybe you don’t need a lockdown. You go with it. It’s not just ecologists, there have been a wide range of scientists putting their pet model out there, and what has happened is it has allowed politicians and news outlets to cherry-pick from a wide array of unpublished science.

SP: There have been claims by authors of some of the initial papers that the models were not predictive, they were instead ‘what-if scenarios’. The assumption that people can pick apart the nuances between the two strikes me as somewhat irresponsible. I’ve always been one for science communication, but I’ve also believed that not everyone should have to do it. Do you think an issue like this, where scientists have so poorly understood how our terminology is interpreted, should renew calls for better scientific communication training?

JC: I think it has to. I think for us to maintain any credibility as a discipline, we really can’t carry on making big missteps like this. Today it’s the coronavirus, but tomorrow there might be another big event that ecologists might feel the need to comment on. If you make a model which shows predictions like Brazil being fine, then Brazil quite quickly becomes the second most infected country in the world, you can’t just hold up your hands and go “oh that was a what-if scenario”. A couple of other scientists have spoken up and said that this what-if scenario has had very real consequences for them. Bottom line, if it’s just a what-if scenario, it shouldn’t be disseminated by high-impact journals and media outlets.

It undermines credibility in your entire field. Particularly when we’re trying to be taken seriously in other areas where we do have competence like climate change mitigation and biodiversity conservation. So this is something we need to learn as a discipline and fast actually.

SP: Is there a use for ecologists in this current crisis?

JC: There are plenty of cases where it is appropriate, for instance where you have a biological vector that we need to map. There are currently projects underway to map the potential reservoir species for this and other beta coronaviruses, so we might know better where future outbreaks might come from. If in the future we get human-to-human spread under control, we still need to be vigilant against occasional zoonotic outbreaks where there are high reservoir concentrations. In those situations it might be useful to know where those areas are so we can put more effort into monitoring those areas and so we can have an early response should we get an outbreak again.

If you are an ecologist who wants to find out more about the use of ecology to combat future outbreaks, check out the VERENA Consortium, headed up by Colin Carson. The consortium includes ecologists, virologists, data scientists, and policy experts. They are currently focused on recruiting junior scientists with an interdisciplinary skillset, including labwork, bioinformatics, and statistical modeling.

Sam Perrin is a freshwater ecologist currently completing his PhD at the Norwegian University of Science and Technology. You can read more about Sam’s research on his Ecology for the Masses profile here, and follow him on Twitter @samperrinNTNU.