signal-to-noise ratio

COVID-19, due process and an SNR problem

At a press conference streamed live on March 18, the head of the European Medicines Agency (EMA) announced that the body – which serves as the European Union’s drug and vaccine regulator – had concluded that the AstraZeneca COVID-19 vaccine was not associated with unusual blood clots that some vaccine recipients had reported in multiple countries. The pronouncement marked yet another twist in the roller-coaster ride the embattled shot has experienced over the past few months. But it has also left bioethicists debating how it is that governments should respond to a perceived crisis over vaccines during a pandemic.

Over the last two weeks or so, a fierce debate raged after a relatively small subset of people who had received doses complained of developing blood clots related to potentially life-threatening conditions. AstraZeneca, a British-Swedish company, didn’t respond to the concerns at first even though the EMA and the WHO continued to hold their ground: that the vaccine’s benefits outweighed its risks, so people should continue to take it. However, a string of national governments, including those of Germany, France and Spain, responded by pausing its rollout while scientists assessed the risks of receiving the vaccine.

Aside from allegations that AstraZeneca tried to dress up a significant mistake during its clinical trials of the vaccine as a ‘discovery’ and cherry-picked data from the trials to have the shot approved in different countries, the company has also been grappling with the fact that the shot was less efficacious than is ideal against infections by new, more contagious variants of the novel coronavirus.

But at the same time, the AstraZeneca vaccine is also one of the more affordable ones that scientists around the world have developed to quell the COVID-19 pandemic – more so than the Pfizer and Moderna mRNA vaccines. AstraZeneca’s candidate is also easier to store and transport, and is therefore in high demand in developing and under-developed nations around the world. Its doses are being manufactured by two companies, in India and South Korea, although geographically asymmetric demand has forced an accelerating vaccination drive in one country to come at the cost of deceleration in another.

Shot in the arm

Now that the EMA has reached its verdict, most of the 20 countries who had hit the pause button have announced that they will resume use of the vaccine. However, the incident has spotlighted a not-unlikely problem with the global vaccination campaign, and which could recur if scientists, ethicists, medical workers and government officials don’t get together to decide where they can draw the line between abundant precaution and harm.

In fact, there are two versions of this problem: one in countries that have a functional surveillance system that responds to adverse events following immunisation (AEFIs) and one in countries that don’t. An example of the former is Germany, which, according to the New York Times, decided to pause the rollout based on seven reports of rare blood clots from a pool of 1.6 million recipients – a naïve incidence rate of 0.0004375%. But as rare disorders go, this isn’t a negligible figure.

One component of the post-AEFI response protocol is causality assessment, and one part of this is for experts to check if certain purported side-effects are clustered in time and then to compare those to the illness’s time distribution for a long time before the pandemic. It’s possible that such clustering could have prompted health officials in Germany and other countries to suspend the rollout.

The Times quoted a German health ministry statement saying, “The state provides the vaccine and therefore has special duties of care”. These care considerations include what the ministry understands to be the purpose of the rollout (to reduce deaths? To keep as many people healthy as possible?) read together with the fact that vaccines are like drugs except in one important way: they’re given to healthy – and not to sick – people. To quote Stephan Lewandowsky, an expert of risk communication at the University of Bristol, from Science:

“You’ve got to keep the public on board. And if the public is risk-averse, as it is in Europe … it may have been the right decision to stop, examine this carefully and then say, ‘The evidence, when considered transnationally, clearly indicates it is safe to go forward.’”

On the other hand is the simpler and opposing calculus of how many people didn’t develop blood clots after taking the vaccine, how many more people the virus is likely to have infected in the time the state withheld the vaccine, how many of them were at greater risk of developing complications due to COVID-19 – topped off by the fact of the vaccines being voluntary. On this side of the argument, the state’s carefulness is smothering, considering it’s using a top-down policy without accounting for local realities or the state’s citizens’ freedom to access or refuse the vaccine during a pandemic.

Ultimately there appears to be no one right answer, at least in a country where there’s a baseline level of trust that the decision-making process included a post-vaccination surveillance system that’s doing its job. Experts have also said governments should consider ‘mixed responses’ – like continuing rollouts while also continuing to examine the vaccines, given the possibility that a short-term review may have missed something a longer term exercise could find. One group of exerts in India has even offered a potential explanation.

The background rate

In countries where such a system doesn’t exist, or does but is broken, like India, there is actually one clear answer: to be transparent and accountable instead of opaque and intractable. For example, N.K. Arora, a member of India’s National COVID-19 Task Force, told The Hindu recently that while the body would consider post-vaccination data of AstraZeneca’s vaccine, it also believed the fraction of worrying cases to be “very, very low”. Herein lies the rub: how does it know?

As of early March, according to Arora, the Union health ministry had recorded “50-60” cases of AEFIs that may or may not be related to receiving either of the two vaccines in India’s drive, Covaxin and Covishield. (The latter is the name of AstraZeneca’s shot in India.) Reading this with Arora’s statements and some other facts of the case, four issues become pertinent.

First is the deceptively simple problem of the background rate. Journalist Priyanka Pulla’s tweets prompt multiple immediate concerns on this front. If India had reported 10 cases of disease X in 20 years, but 10 more cases show up within two weeks after receiving one dose of a vaccine, should we assume the vaccine caused them? No – but it’s a signal that we should check for the existence of a causal link.

Experts will need to answer a variety of questions here: How many people have disease X in India? How many people of a certain age-group and gender have disease X? How many people of different religious and/or ethnic groups have disease X? How many cases of disease X are we likely to have missed (considering disease-underreporting is a hallmark of Indian healthcare)? How many cases of disease X should we expect to find in the population being vaccinated in the absence of a vaccine? Do the 10 new cases, or any subset of them, have a common but invisible cause unrelated to the vaccine? Do we have the data for all these considerations?

Cornelia Betsch, a psychologist at the University of Erfurt, told Science that “most of the cases of rare blood disorders were among young women, the group where vaccine hesitancy already runs highest”. Can India confirm or deny that this trend is reflected in its domestic data as well? This seems doubtful. Sarah Iqbal reported for The Wire Science in September 2020 that “unequal access to health”, unequal exposure to potentially disease-causing situations, unequal representation in healthcare data and unequal understanding of diseases in non-cis-male bodies together already render statements like ‘women have better resistance to COVID-19’ ignorant at best. Being able to reliably determine and tackle sex-wise vaccine hesitancy seems like a tall order.

The second issue is easy to capture in one question, which also makes it harder to ignore: why hasn’t the government released reports or data about AEFIs in India’s COVID-19 vaccination drive after February 26, 2021?

On March 16, a group of 29 experts from around the country – including virologist T. Jacob John, who has worked with the Indian Council of Medical Research on seroprevalence surveys and has said skeptics of the Indian drug regulator’s Covaxin approval were “prejudiced against Indian science/product” – wrote to government officials asking for AEFI data. They said in their letter:

We note with concern that critical updates to the fact sheets recommended by the CDSCO’s Subject Expert Committee have not been issued, even though they are meant to provide additional guidance and clarify use of the vaccines in persons such as those with allergies, who are immunocompromised or using immunosuppressants, or using blood thinners/anticoagulants. There are gaps in AEFI investigations at the local level, affecting the quality of evidence submitted to State and National AEFI Committees who depend on these findings for making causality assessments. The National AEFI Committee also has a critical role in assessing cases that present as a cluster and to explore potential common pathways. In our letter dated January 31, 2021, we asked for details of all investigations into deaths and other serious AEFIs, as well as the minutes of AEFI monitoring committees, and details of all AEFI committee members and other experts overseeing the vaccine rollout. We have not received any response.

City of Omelas

The third issue is India’s compliance with AEFI protocols – which, when read together with Pulla’s investigation of Bharat Biotech’s response to a severe adverse event in its phase 3 trials for Covaxin, doesn’t inspire much confidence. For example, media reports suggest that medical workers around the country aren’t treating all post-vaccination complaints of ill-health, but especially deaths, on equal footing. “Currently, we are observing gaps in how serious adverse events are being investigated at the district level,” New Delhi-based health activist Malini Aisola told IndiaSpend on March 9. “In many instances local authorities have been quick to make public statements that there is no link to the vaccine, even before investigations and post mortem have taken place. In some cases there is a post mortem, in some cases there isn’t.”

Some news reports of people having died of heart-related issues at a point of time after taking Covishield also include quotes from doctors saying the victims were known to have heart ailments – as if to say their deaths were not related to the vaccine.

But in the early days of India’s COVID-19 epidemic, experts told The Wire that even when people with comorbidities, like impaired kidney function, died due to renal failure and tested positive for COVID-19 at the time of death, their passing could be excluded from the official deaths tally only if experts had made sure the two conditions were unrelated – and this is difficult. Having a life-threatening illness doesn’t automatically make it the cause of death, especially since COVID-19 is also known to affect or exacerbate some existing ailments, and vice versa.

Similarly, today, is the National AEFI Committee for the COVID-19 vaccination drive writing off deaths as being unrelated to the vaccine or are they being considered to be potential AEFIs? And is the committee deliberating on these possibilities before making a decision? The body needs to be transparent on this front a.s.a.p. – especially since the government has been gifting AstraZeneca’s shots to other countries and there’s a real possibility of it suppressing information about potential problems with the vaccine to secure its “can do no wrong” position.

Finally, there’s the ‘trolley problem’, as the Times also reported – an ethical dilemma that applies in India as well as other countries: if you do nothing, three people will get hit by a train and die; if you pull a lever, the train will switch tracks and kill one person. What do you do?

But in India specifically, this dilemma is modified by the fact that due process is missing; this changes the problem to one that finds better, more evocative expression in Ursula K. Le Guin’s short story The Ones Who Walk Away from Omelas (1973). Omelas is a fictitious place, like paradise on Earth, where everyone is happy and content. But by some magic, this is only possible if the city can keep a child absolutely miserable, wretched, with no hope of a better life whatsoever. The story ends by contemplating the fate of those who discover the city’s gory secret and decide to leave.

The child in distress is someone – even just one person – who has reported an AEFI that could be related to the vaccine they took. When due process plays truant, when a twisted magic that promises bliss in return for ignorance takes shape, would you walk away from Omelas? And can you freely blame those who hesitate about staying back? Because this is how vaccine hesitancy takes root.

The Wire
March 20, 2021

Powerful microscopy technique brings proteins into focus

Cryo-electron microscopy (cryo-EM) as a technology has become more important because the field that it revolutionised – structural biology – has become more important. The international scientific community had this rise in fortunes, so to speak, acknowledged when the Nobel Prize for chemistry was awarded to three people in 2017 for perfecting its use to study important biomolecules and molecular processes.

(Who received the prize is immaterial, considering more than just three people are likely to have contributed to the development of cryo-EM; however, the prize-giving committee’s choice of field to spotlight is a direction worth following.)

In 2015, two separate groups of scientists used cryo-EM to image objects 2.8 Å and 2.2 Å (1 nm is one-billionth of a metre; 1 Å is one-tenth of this) wide. These distances are considered to be atomic because they represent the ability to image features about as big as a smallish atom, comparable to that of, say, sodium. Before cryo-EM, scientists could image such distances only with X-ray crystallography, which requires the samples to be studied to be crystallised first. This isn’t always possible.

But though cryo-EM didn’t require specimens to be crystallised, they had to be placed in a vacuum first. In vacuum, water evaporates, and when water evaporates from biological objects like tissue, the specimen could lose its structural integrity and collapse or deform. The trio that won the chemistry prize in 2017 developed multiple workarounds for this and other problems. Taken together, their innovations allowed scientists to find cryo-EM to be more and more valuable for research.

One of the laureates, Joachim Frank, developed computational techniques in the 1970s and 1980s to enhance, correct and in other ways modify images obtained with cryo-EM. And one of these techniques in turn was particularly important.

An object will reflect a wave if the object’s size is comparable to the wave’s wavelength. Humans see a chair or table because the chair or table reflects visible light, and our eyes detect the reflected electromagnetic waves. A cryo-EM ‘sees’ its samples using electrons, which have a smaller wavelength than photons and can thus reveal even smaller objects.

However, there’s a catch. The more energetic an electron is, the lower its wavelength is, and the smaller the feature it can resolve – but a high-energy electron can also damage the specimen altogether. Frank’s contributions allowed scientists to reduce the number of electrons or their energy to obtain equally good images of their specimens, leading to resolutions of 2.2 Å.

Today, structural biology continues to be important, but its demands have become more exacting. To elucidate the structures of smaller and smaller molecules, scientists need cryo-EM and other tools to be able to resolve smaller and smaller features, but come up against significant physical barriers.

For example, while Frank’s techniques allowed scientists to reduce the number of electrons required to obtain the image of a sample, using fewer probe particles also meant a lower signal-to-noise ratio (SNR). So the need for new techniques, new solutions, to these old problems has become apparent.

In a paper published online on October 21, a group of scientists from Belgium, the Netherlands and the UK describe “three technological developments that further increase the SNR of cryo-EM images”. These are a new kind of electron source, a new energy filter and a new electron camera.

The electron source is something the authors call a cold field emission electron gun (CFEG). Some electron microscopes use field emission guns (FEGs) to shoot sharply focused, coherent beams of electrons optimised to have energies that will produce a bright image. A CFEG is a FEG that reduces the brightness in favour of reducing the average difference in energies between electrons. The higher this difference – or the energy spread – is, the more blur there will be in the image.

The authors’ pitch is that FEGs help produce brighter but more blurred images than CFEGs, and that CFEGs help produce significantly better images when the goal is to image features smaller than 2 Å. Specifically, they write, the SNR increases 2.5x at a resolution of 1.5 Å and 9.5x at 1.2 Å.

The second improvement has to do with the choice of electrons used to compose the final image. The electrons fired by the gun (CFEG or otherwise) go on to have one of two types of collisions with the specimen. In an elastic collision, the electron’s kinetic energy doesn’t change – i.e. it doesn’t impart its kinetic energy to the specimen. In an inelastic collision, the electron’s kinetic energy changes because the electron has passed on some of it to the specimen itself. This energy transfer can produce noise, lower the SNR and distort the final image.

The authors propose using a filter that removes electrons that have undergone inelastic collisions from the final assessment. In simple terms, the filter comprises a slit through which only electrons of a certain energy can pass and a prism that bends their path towards a detector. This said, they do acknowledge that it will be interesting to explore in future whether inelastically scattered electrons can be be better accounted for instead of being eliminated altogether – akin to silencing a classroom by expelling unruly children versus retaining them and teaching them to keep quiet.

The final improvement is to use the “next-generation” Falcon 4 direct-electron detector. This is the latest iteration in a line of products developed by Thermo Fisher Scientific, to count the number of electrons impinging on a surface as accurately as possible, their relative location and at a desirable exposure. The Falcon 4 has a square detection area 14 µm to a side, a sampling frequency of 248 Hz and a “sub-pixel accuracy” (according to the authors) that allows the device to not lose track of electrons even if they impinge close to each other on the detector.

A schematic overview of the experimental setup. Credit: https://doi.org/10.1038/s41586-020-2829-0

Combining all three improvements, the authors write that they were able to image a human membrane protein called ß3 GABA_A R with a resolution of 1.7 Å and mouse apoferritin at 1.22 Å. (The protein called ferritin binds to iron and stores/releases it; apoferritin is ferritin sans iron.)

A reconstructed image of GABA_A R. The red blobs are water molecules. NAG is N-acetyl glucosamine. Credit: https://doi.org/10.1038/s41586-020-2829-0

“The increased SNR of cryo-EM images enabled by the technology described here,” the authors conclude, “will expand [the technique] to more difficult samples, including membrane proteins in lipid bilayers, small proteins and structurally heterogeneous macromolecular complexes.”

At these resolutions, scientists are closing in on images not just of macromolecules of biological importance but of parts of these molecules – and can in effect elucidate the structures that correspond to specific functions or processes. This is somewhat like going from knowing that viruses infect cells to determining the specific parts of a virus and a cell implicated in the infiltration process.

A very germane example is that of the novel coronavirus. In April this year, a group of researchers from France and the US reported the cryo-EM structure of the virus’s spike glycoprotein, which binds to the ACE2 protein on the surface of some cells to gain entry. By knowing this structure, other researchers can design the more perfect inhibitors to disrupt the glycoprotein’s function, as well as vaccines that mimic its presence to provoke the desired immune response.

In this regard, a resolution of 1-2 Å corresponds to the dimensions of individual covalent bonds. So by extending the cryo-EM’s ability to decipher smaller and smaller features, researchers can strike at smaller, more precise molecular mechanisms to produce more efficient, perhaps more closely controlled and finely targeted, effects.

Featured image: Scientists using a 300-kV cryo-EM at the Max Planck Institute of Molecular Physiology, Dortmund. Credit: MPI Dortmund.

Crowdsourcing earthquakes is not a big deal – keeping it reliable is

"Symbols show the few regions of the world where public citizens and organizations currently receive earthquake warnings and the types of data used to generate those warnings (7). Background color is peak ground acceleration with 10% probability of exceedance in 50 years from the Global Seismic Hazard Assessment Program." DOI: 10.1126/sciadv.1500036 — “Symbols show the few regions of the world where public citizens and organizations currently receive earthquake warnings and the types of data used to generate those warnings (7). Background color is peak ground acceleration with 10% probability of exceedance in 50 years from the Global Seismic Hazard Assessment Program.” DOI: 10.1126/sciadv.1500036

This map stakes well the need for a decentralized earthquake warning system. The dark and light blue rings show where early earthquake warnings are available, while the reddish and yellow patches describe areas prone to earthquakes. There’s a readily visible disparity, which a team of scientists from the University of California, Berkeley, leverages to outline how early earthquake warnings can be crowdsourced. In a paper in Scientific Advances on April 10, the team proposes using the accelerometer in our smartphones to log and transmit tiny movements in the ground beneath us to a server that analyzes them for signs of a quake and returns the results (insert cute quote about crowdsourced information being used by the crowd).

This idea isn’t entirely new. In 2013, two seismologists from the Instituto Nazionale di Geosifica e Vulcanologia in Italy used cheap MEMS (micro-electromechanical system) accelerometers to determine that they’re good for anticipating quakes that are rated higher than five on the Richter scale if located close to the epicenter. Otherwise, the accelerometers weren’t reliable when logging seismic signals that weren’t sharp or unique enough – such as is the case with weaker earthquakes or the strong ground-motion associated with moving faults – because the instruments produced sufficient noise to drown their own readings out.

In fact, this issue might’ve been evident in 2010 itself. Then, a team out of Stanford University proposed using “all the computers” on the Internet to “catch” quakes. To be part of this so-called Quake Catcher Network, users would have to install a piece of QCN software along with a ‘low-maintenance’ motion sensor on their desktops/laptops to empower them with the same capabilities as a smartphone-borne accelerometer, but more sensitive. The software would log motion data due to mild tremors or stronger and strong ground-motion and relay it over the web in near-real-time. The QCN has been live for over a year now, although most of its users are situated in Europe and North America.

Perhaps the earliest instance of crowdsourcing in the Age of the Smartphone was with Twitter. In 2008, a 7.9-magnitude earthquake in China killed over 10,000 in a rain-hit region of the country. The CNN wrote, “Rainy weather and poor logistics thwarted efforts by relief troops who walked for hours over rock, debris and mud on Tuesday in hopes of reaching the worst-hit area”. Twitter, however, was swarming with updates from the region, often revealing gaps in the global media’s coverage of the disaster. The Online Journalism Blog summed it up:

Robert Scoble was following proceedings on his much-followed Twitter, and feeding back information from his followers, including, for instance (after he tweeted the fact that Tweetscan was struggling) that people were saying Summize was the best tool to use.

If you followed the conversation through Scoble using Quotably, you could then find Gregg Scott, who in turn was talking to RedChina, Karoli, mmsullivan, and inwalkedbud who was in Chengdu, China (also there was Casperodj and Lyrrael).

If you wanted to check out inwalkedbud you could do so using Tweetstats and find he has been twittering since December. Sadly the Internet Archive doesn’t bring any results, though.

The mainstream media had differing reports: RTE (Ireland) said “No major damage after China earthquake” – but UK’s Sky News reported four children killed and over 100 injured; Chinaview (China) said no buildings had collapsed – but an Australian newspaper said they had.

Filtering the noise

In all these cases – the Italian MEMS experiment, the QCN desktop/laptop-based tracker and with updates on Twitter – the problem has not been to leverage the crowd effectively. In 2015, we’re already there. The real problem has been reliability. Quakes stronger than five on the Richter scale signal danger everywhere, and there are enough smartphone-bearing users around the world to be on alert for them. But quakes less strong are bad news particularly in developing economies, where bad infrastructure and crowding are often to blame for collapsing buildings that claim hundreds of lives.

Let’s take another look at the disparity map:

The redder belts are more prevalent in South America, Central and East Asia and in a patch running between Central Europe and the Middle East. Not being able to detect weaker quakes if not for centralized detection agencies in these regions keeps hundreds of millions of people under threat. So, the real achievement when scientists confidently crowdsource early earthquake warnings is the use of specialized filtering techniques and algorithms to increase the sensitivity of smartphones to subtle movements in the ground and so the reliability of their measurements. Where concepts like phase smoothing, Kalman filters and GNSS receivers thrumming in a smartphone’s chassis spell the difference between news and help.

Tech 1, Coarseness 0.

These are only some of the techniques in use – and whose use the Berkeley group thinks particularly significant in their early warning system’s designs. Phase smoothing is a technique where errors associated with data transmission between smartphones and satellites – such as measurement noise or reflection by metallic objects in the transmission’s path – are mitigated by keeping track of the rate of change of the distance between the phone and the satellite. A Kalman filter is an algorithm that specializes in picking out data patterns from a chaos of signals and using that pattern to fish for even more signals like it, thus steadily filtering out the noise. Together, they help scientists adjust for drift – which is when an object moves by a greater distance than an earthquake would have it move.

Finally, the scientists further refine the data by comparing it to legacy GNSS (Global Navigation Satellite System) data, which is the most accurate but also the most costly system with which to anticipate and track earthquakes. In their Science Advances Paper, the Berkeley group writes that the data obtained through thousands of smartphones “can be substantially improved by using differential corrections via satellite-based augmentation systems, tracking the more precise GNSS carrier phase and using it to filter the [crowdsourced] data (“phase smoothing”), or by combination with independent INS data in a Kalman filter.”

A warning system all India’s

But the best part: “Today’s smartphones have some or all of these capabilities”, negating the otherwise typical coarseness and unreliability associated with crowdsourced data. Here’s more evidence of this:

(B) Drift of position obtained from various devices (GNSS, double-integrated accelerometers, and Kalman filtering thereof) compared to observed earthquake displacements. DOI: 10.1126/sciadv.1500036

Chart (B), which is the one of interest to us, shows the amount of drift present in data acquired by various methods over time. The black lines show the observed displacements due to earthquakes of different magnitudes. So, a colored line represents reliable data as long as it is below the corresponding black line. For example, the red line for “C/A code + p-s + SBAS” shows a largely reliable reading of an M6 earthquake until about 50 seconds, after which it starts to drift. Similarly, most colored lines are below the black lines for M8-9 earthquakes, so all those methods can be used to reliably track the stronger earthquakes. The line described by the Berkeley group is the red line – the crowdsourced line.

The ideal thing would be to develop more sophisticated filtering mechanisms that’d bring the red line close to the blue GNSS line at the bottom, which of course exhibits zero drift. Fortunately, self-reliance on this front might be possible soon in the Indian Subcontinent region. Since 2013, the Indian Space Research Organization has launched four of its planned seven Regional Navigation Satellite System (IRNSS) that could augment regional efforts to crowdsource earthquake-warnings. The autonomous system is expected to live in 2016.