Calls for clear new pandemic modelling rules

2 July 2020

Experts are calling for transparent and clear computer modelling of COVID-19, as the world looks for a way to understand and make decisions about the coronavirus pandemic. 

They have pulled together a manifesto, published in Nature, pushing for models to be used appropriately - without political bias or overestimating with "magic numbers". 

"COVID-19 has really put modelling into the spotlight and there have been very effective simple models such as flattening the curve, that everyone can understand," Professor Gabriele Bammer, from The Australian National University (ANU), said. 

"In Australia we are mostly doing well, but the world needs a uniform set of standards for computer modelling.  

"Models shouldn't predict more certainty than they allow for. They can't replace complex decision making and they should be upfront about unknowns."  

The experts say political rivals often brandish models to support predetermined agendas. 

"The reality is at the moment decision makers have to make hard choices and if models are not done well, you can find a model that says whatever you like," Professor Bammer said. 

"What people generally want is the magic number or a simple solution, but that is often not possible for complex societal or environmental problems. In those cases, good modelling can help politicians and others make better decisions.  

"For COVID-19, a vaccine might be a magic bullet that can really help us deal with it but until we've got the vaccine, we've got this complex problem and we have to figure out the best ways to deal with it."  

 The authors have penned a five-point manifesto to call for "full and frank disclosure" in an effort to streamline computer modelling so that predictions are transparent, humble and responsible. 

The experts say modellers need to acknowledge their own hubris, as well as unknowns, assumptions, framing and consequences when modelling. 

"To make sure their predictions do not become mere adjuncts to a political cause, modellers, decision makers and citizens need to establish new social norms such that modellers are not permitted to project more certainty than their models deserve, and politicians are not allowed to offload accountability to models of their choosing," the authors write. 

The authors warn that models can be a dangerous way to assert answers and model predictions can contain "unstated interest and values".  

"We are cautioning people to apply the pub test, if it doesn't sound right - be sceptical. The models that seem very certain require reasonable scepticism," Professor Bammer said. 

"The problems we highlight with models are relevant for a wide range of topics from economics to flood prediction, fisheries management and more. 

"We now have an opportunity to think about how we use modelling and get a strategy in place so these processes are crystal clear."