epidemiological modelling

The constructionist hypothesis and expertise during the pandemic

Now that COVID-19 cases are rising again in the country, the trash talk against journalists has been rising in tandem. The Indian government was unprepared and hapless last year, and it is this year as well, if only in different ways. In this environment, journalists have come under criticism along two equally unreasonable lines. First, many people, typically supporters of the establishment, either don’t or can’t see the difference between good journalism and contrarianism, and don’t or can’t acknowledge the need for expertise in the practise of journalism.

Second, the recognition of expertise itself has been sorely lacking across the board. Just like last year, when lots of scientists dropped what they were doing and started churning out disease transmission models each one more ridiculous than the last, this time — in response to a more complex ‘playing field’ involving new and more variants, intricate immunity-related mechanisms and labyrinthine clinical trial protocols — too many people have been shouting their mouths off, and getting most of it wrong. All of these misfires have reminded us of two things: again and again that expertise matters, and that unless you’re an expert on something, you’re unlikely to know how deep it runs. The latter isn’t trivial.

There’s what you know you don’t know, and what you don’t know you don’t know. The former is the birthplace of learning. It’s the perfect place from which to ask questions and fill gaps in your knowledge. The latter is the verge of presumptuousness — a very good place from which to make a fool of yourself. Of course, this depends on your attitude: you can always be mindful of the Great Unknown, such as it is, and keep quiet.

As these tropes have played out in the last few months, I have been reminded of an article written by the physicist Philip Warren Anderson, called ‘More is Different’, and published in 1972. His idea here is simple: that the statement “if everything obeys the same fundamental laws, then the only scientists who are studying anything really fundamental are those who are working on those laws” is false. He goes on to explain:

“The main fallacy in this kind of thinking is that the reductionist hypothesis does not by any means imply a ‘constructionist’ one: The ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct the universe. … The constructionist hypothesis breaks down when confronted with the twin difficulties of scale and complexity. The behaviour of large and complex aggregates of elementary particles, it turns out, is not to be understood in terms of a simple extrapolation of the properties of a few particles. Instead, at each level of complexity entirely new properties appear, and the understanding of the new behaviours requires research which I think is as fundamental in its nature as any other.”

The seemingly endless intricacies that beset the interaction of a virus, a human body and a vaccine are proof enough that the “twin difficulties of scale and complexity” are present in epidemiology, immunology and biochemistry as well – and testament to the foolishness of any claims that the laws of conservation, thermodynamics or motion can help us say, for example, whether a particular variant infects people ‘better’ because it escapes the immune system better or because the immune system’s protection is fading.

But closer to my point: not even all epidemiologists, immunologists and/or biochemists can meaningfully comment on every form or type of these interactions at all times. I’m not 100% certain, but at least from what I’ve learnt reporting topics in physics (and conceding happily that covering biology seems more complex), scale and complexity work not just across but within fields as well. A cardiologist may be able to comment meaningfully on COVID-19’s effects on the heart in some patients, or a neurologist on the brain, but they may not know how the infection got there even if all these organs are part of the same body. A structural biologist may have deciphered why different mutations change the virus’s spike protein the way they do, but she can’t be expected to comment meaningfully on how epidemiological models will have to be modified for each variant.

To people who don’t know better, a doctor is a doctor and a scientist is a scientist, but as journalists plumb the deeper, more involved depths of a new yet specific disease, we bear from time to time a secret responsibility to be constructive and not reductive, and this is difficult. It becomes crucial for us to draw on the wisdom of the right experts, who wield the right expertise, so that we’re moving as much and as often as possible away from the position of what we don’t know we don’t know even as we ensure we’re not caught in the traps of what experts don’t know they don’t know. The march away from complete uncertainty and towards the names of uncertainty is precarious.

Equally importantly, at this time, to make our own jobs that much easier, or at least less acerbic, it’s important for everyone else to know this as well – that more is vastly different.

Pandemic: A world-building exercise

First, there was ~~light~~ news of a vaccine against COVID-19 nearing the end of its phase 3 clinical trials with very promising results, accompanied with breezy speculations (often tied to the stock prices of a certain drug-maker) about how it’s going to end the pandemic in six months.

An Indian disease-transmission modeller – of the sort who often purport to be value-free ‘quants’ interested in solving mathematical puzzles that don’t impinge on the real world – reads about the vaccine and begins to tweak his models accordingly. Soon, he has a projection that shines bright in the dense gloom of bad news.

One day, as the world is surely hurtling towards a functional vaccine, it becomes known that some of the world’s richest countries – representing an eighth of the planet’s human population – have secreted more than half of the world’s supply of the vaccine.

Then, a poll finds that over half of all Americans wouldn’t trust a COVID-19 vaccine when it becomes available. The poll hasn’t been conducted in other countries.

A glut of companies around the world have invested heavily in various COVID-19 vaccine candidates, even as the latter are yet to complete phase 3 clinical trials. Should a candidate not clear its trial, a corresponding company could lose its investment without insurance or some form of underwriting by the corresponding government.

Taken together, these scenarios portend a significant delay between a vaccine successfully completing its clinical trials and becoming available to the population, and another delay between general availability and adoption.

The press glosses over these offsets, developing among its readers a distorted impression of the pandemic’s progression – an awkward blend of two images, really: one in which the richer countries are rapidly approaching herd immunity while, in the other, there is a shortage of vaccines.

Sooner or later, a right-wing commentator notices there is a commensurately increasing risk of these poorer countries ‘re-exporting’ the virus around the world. Politicians hear him and further stigmatise these countries, and build support for xenophobic and/or supremacist policies.

Meanwhile, the modeller notices the delays as well. When he revises his model, he finds that as governments relax lockdowns and reopen airports for international travel, differences in screening procedures in different countries could allow the case load to rise and fall around the world in waves – in effect ensuring the pandemic will take longer to end.

His new paper isn’t taken very seriously. It’s near the end of the pandemic, everyone has been told, and he’s being a buzzkill. (It’s also a preprint, and that, a senior scientist in government nearing his retirement remarks, “is all you need to know”.) Distrust of his results morphs slowly into a distrust towards scientists’ predictions, and becomes ground to dismiss most discomfiting findings.

The vaccine is finally available in middle- and low-income countries. But in India, this bigger picture plays out at smaller scales, like a fractal. Neither the modeller nor the head of state included the social realities of Indian society in their plans – but no one noticed because both had conducted science by press release.

As they scratch their heads, they also swat away at people at the outer limits of the country’s caste and class groups clutching at them in desperation. A migrant worker walks past unnoticed. One of them wonders if he needs to privatise healthcare more. The other is examining his paper for arithmetic mistakes.

The number of deaths averted

What are epidemiological models for? You can use models to inform policy and other decision-making. But you can’t use them to manufacture a number that you can advertise in order to draw praise. That’s what the government’s excuse appears to be vis-à-vis the number of deaths averted by India’s nationwide lockdown.

When the government says 37,000 deaths were averted, how can we know if this figure was right or wrong? A bunch of scientists complained that the model wasn’t transparent, so its output had to be taken with a cupful of salt. But as an article published in The Wire yesterday noted, these scientists were asking the wrong questions – that the number of deaths averted is only a decoy.

So say the model had been completely transparent. I don’t see why we should still care about the number of deaths averted. First, such a model is trying to determine the consequences of an action that was not performed, i.e. the number of people who might have died had the lockdown not been imposed.

This scenario is reminiscent of a trope in many time-travel stories. If you went back in time and caused someone to do Y instead of X, would your reality change or stay the same considering it’s in the consequent future of Y instead of X? Or as Ronald Bailey wrote in Reason, “If people change their behaviour in response to new information unrelated to … anti-contagion policies, this could reduce infection growth rates as well, thus causing the researchers to overstate the effectiveness of anti-contagion policies.”

Second, a model to estimate the number of deaths averted by the lockdown will in effect attempt to isolate a vanishingly narrow strip of the lockdown’s consequences to cheer about. This would be nothing but extreme cherry-picking.

A lockdown has many effects, including movement restrictions, stay-at-home orders, disrupted supply of essential goods, closing of businesses, etc. Most, if not all, of them are bound to exact a toll on one’s health. So the number of deaths the lockdown averted should be ‘adjusted’ against, say, the number of people who couldn’t get life-saving surgeries, the number of migrant labourers who died of heat exhaustion, the number of TB patients who developed MDR-TB because they couldn’t get their medicines on time, even the number of daily-wage earners’ children who died of hunger because their parents had no income.

So the only models that can hope to estimate a meaningful number of deaths averted by the lockdown will also have simplified the context so much that the mathematical form of the lockdown will be shorn of all practical applicability or relevance – a quantitative catch-22.

Third, the virtue of the number of deaths averted is a foregone conclusion. That is, whatever its value is, it can only be a good thing. So as an indisputable – and therefore unfalsifiable – entity, there is nothing to be gained or lost by interrogating it, except perhaps to elicit a clearer view of the model’s innards (if possible, and only relative to the outputs of other models).

Finally, the lockdown will by design avert some deaths – i.e. D > 0 – but D being greater than zero wouldn’t mean the lockdown was a success as much D‘s value, whatever it is, being a self-fulfilling prophecy. And since no one knows what the value of D is or what it ought to be, even less what it could have been, a model can at best come up with a way to estimate D – but not claim a victory of any kind.

So it would seem the ‘number of deaths averted’ metric is a ploy disguised as a legitimate mathematical problem whose real purpose is to lure the ‘quants’ towards something they think challenges their abilities without realising they’re also being lured away from the more important question they should be asking: why solve this problem at all?

The Wolfram singularity

I got to this article about Stephen Wolfram’s most recent attempt to “revolutionise” fundamental physics quite late, and sorry for it because I had no idea Wolfram was the kind of guy who could be a windbag. I haven’t ever had cause to interact with his work or his software company (which produced Wolfram Mathematica and Wolfram Alpha), so I didn’t know really know much about him to begin with. But I expected him, for reasons I can’t explain, to be more modest than he comes across as in the article.

The article was prompted in the first place by a preprint paper Wolfram and a colleague published earlier this year in which they claimed they had plotted a route to a fundamental theory of everything. Physics currently explains the universe with a combination of multiple theories that don’t really fit together. A ‘theory of everything’ is the colloquial name of a universal theory that many physicists argue exist and which could explain everything about the universe in a self-consistent manner.

Wolfram’s preprint paper was startling as things go not because of its substance but because a) he made no attempts to engage with the wider community of physicists that has been working on the same problem for decades, and b) for Wolfram’s insistence that those dismissing its conclusions are simply out to dismiss him. Consider the following portions:

“I do fault myself for not having done this 20 years ago,” the physicist turned software entrepreneur says. “To be fair, I also fault some people in the physics community for trying to prevent it happening 20 years ago. They were successful” [emphasis added].
“The experimental predictions of [quantum physics and general relativity] have been confirmed to many decimal places—in some cases, to a precision of one part in [10 billion],” says Daniel Harlow, a physicist at the Massachusetts Institute of Technology. “So far I see no indication that this could be done using the simple kinds of [computational rules] advocated by Wolfram. The successes he claims are, at best, qualitative.” …
“Certainly there’s no reason that Wolfram and his colleagues should be able to bypass formal peer review,” Katie Mack says. “And they definitely have a much better chance of getting useful feedback from the physics community if they publish their results in a format we actually have the tools to deal with.”

Reading of this attitude brought to mind an episode from six or seven weeks ago, after a pair of physicists had published a preprint paper modelling the evolution of the COVID-19 epidemic in India and predicting that multiple lockdowns instead of just one would work better. The paper was one of many that began to show up around that time, each set of authors fiddling with different parameters according to their sense of the world to reach markedly different conclusions (a bit of ambulance-chasing if you ask me).

The one by the two physicists was singled out for bristling criticism by other physicists because – quite like the complaints against Wolfram – their paper allegedly described a model that seemed to be able to reach any conclusion if you tweaked its parameters enough, and because the duo hadn’t clarified this and other important caveats in their interviews to journalists.

Aside 1 – In physics at least, it’s important for theories to be provable in some domains and falsifiable in others; if a theory of the world is non-falsifiable, it’s not considered legitimate. In Wolfgang Pauli’s famous words, it becomes ‘not even wrong’.

Aside 2 – Incidentally, Harlow – quoted above from the article – was one of the physicists defending physicists’ freedom to model what they will but agreed with the objection that they also need to be honest with journalists about their assumptions and caveats.

In a lengthy Facebook discussion that followed this brouhaha, someone referred to a Reddit post created three days earlier in which a physicist appealed to his peers to stop trying to model the pandemic – in his words, to “cut that shit out” – because a) no physicist could hope to do a better job than any other trained epidemiologist, and b) every model a physicist attempted could actually harm lives if it wasn’t done right (and there was a good chance it was at least incomplete).

Wolfram is guilty of the same thing: his preprint paper won’t harm lives, but the mortal threat is the only thing missing from his story; it’s otherwise rife with the same problems. His hubristic remark in the article’s denouement – that he deserves “better” questions than the ones other physicists were asking him in response to his “revolutionary” paper – indicates Wolfram thinks he’s done a great job but it’s impossible to see people like him as anything more than windbags convinced of their intellectual superiority and ability to singlehandedly wrestle hideously intractable problems to the ground. I, and likely other editors as well, have glimpsed this attitude on the part of some authors who dismiss criticism of their pieces as criticism of anything but their unclear writing, and some others who refuse to be disabused of a conviction that their conclusion is particularly fascinating.

I’d like to ask Wolfram what I’d like to ask these people as well: What have you hit on that you think others haven’t in all this time, and why do you think all of them missed it? Granted, everyone is allowed their ‘eureka’ moment, but anyone who claims it on the condition that he not be criticised is not likely to be taken seriously. More importantly, he may not even deserve to be taken seriously if only because, to adapt Mack’s line of reasoning, he undermines the very thing on which modern science is founded, the science he claims to be improving: processes, not outcomes; involving communities, not individuals.

Featured image credit: Anna Shvets/Pexels.