Tag: objectivity

Scientists’ conduct affects science

Nature News has published an excellent feature by Edwin Cartlidge on the “wall of scepticism” that arose in response to the latest superconductivity claim from Ranga Dias et al., purportedly in a compound called nitrogen-doped lutetium hydride. It seems the new paper has earned a note of concern as well, after various independent research groups failed to replicate the results. Dias & co. had had another paper, claiming superconductivity in a different material, retracted in October 2022, two years after its publication. All these facts together raise a few implications about the popular imagination of science.

First, the new paper was published by Nature, a peer-reviewed journal. And Jorge Hirsch of the University of California, San Diego, told Nature News “that editors should have first resolved the question about the provenance of the raw data in the retracted 2020 Nature article before even considering the 2023 paper”. So the note reaffirms the role of peer-review being limited to checking whether the information presented in a paper is consistent with the paper’s conclusions, and not checking whether it is well-founded and has integrity in and of itself.

Second, from Nature News:

“Researchers from four other groups, meanwhile, told Nature’s news team that they had abandoned their own attempts to replicate the work or hadn’t even tried. Eremets said that he wasted time on the CSH work, so didn’t bother with LuNH. ‘I just ignored it,’ he says.”

An amusing illustration, I think, that speaks against science’s claims to being impartial, etc. In a perfectly objective world, Dias et al.’s previous work shouldn’t have mattered to other scientists, who should have endeavoured to verify the claims in the new paper anew, given that it’s a fairly sensational claim and because it was published in a ‘prestigious’ peer-reviewed journal. But, as Eremets said, “the synthesis protocol wasn’t clear in the paper and Dias didn’t help to clarify it”.

The reciprocal is also true: Dias chose to share samples of nitrogen-doped lutetium hydride that his team had prepared only with Russell Hemley, who studies material chemistry at the University of Illinois, Chicago, (and some other groups that he refused to name) – and that Hemley is one of the researchers who hasn’t criticised Dias’s findings. Hemley is also not an independent researcher; he and Dias worked together on the work in the 2020 paper that was later retracted. Dias should ideally have shared the samples with everyone. But scientists’ social conduct does matter, influencing decisions about how other scientists believe they should respond.

Speaking of which: Nature (the journal) on the other hand doesn’t look at past work and attendant misgivings when judging each paper. From Nature News (emphasis added):

The editor’s note added to the 2023 paper on 1 September, saying that the reliability of data are in question, adds that “appropriate editorial action will be taken once this matter is resolved.” Karl Ziemelis, Nature’s chief applied- and physical-sciences editor, based in London, says that he and his colleagues are “assessing concerns” about the paper, and adds: “Owing to the confidentiality of the peer-review process we cannot discuss specific details of what transpired.” As for the 2020 paper, Ziemelis explains that they decided not to look into the origin of the data once they had established problems with the data processing and then retracted the research. “Our broader investigation of that work ceased at that point,” he says. Ziemelis adds that “all submitted manuscripts are considered independently on the basis of the quality and timeliness of their science”.

The refusal to share samples echoes an unusual decision by the journal Physical Review B to publish a paper authored by researchers at Microsoft, in which they reported discovery a Majorana zero mode – an elusive particle (in a manner of speaking) that could lead the way to building quantum ‘supercomputers’. However, it seems the team withheld some information that independent researchers could have used to validate the findings, presumably because it’s intellectual property. Rice University physics professor Douglas Natelson wrote on his blog:

The rationale is that the community is better served by getting this result into the peer-reviewed literature now even if all of the details aren’t going to be made available publicly until the end of 2024. I don’t get why the researchers didn’t just wait to publish, if they are so worried about those details being available.

Take all of these facts and opinions together and ask yourself: what then is the scientific literature? It probably contains many papers that have cleared peer-review but whose results won’t replicate. Some papers may or may not replicate but we’ll never know for a couple years. It also doesn’t contain replication studies that might have been there if the replicators and the original research group were on amicable terms. What also do these facts and view imply for the popular conception of science?

Every day, I encounter two broad kinds of critical imaginations of science. One has emerged from the practitioners of science, and those studying its philosophy, history, sociology, etc. These individuals have debated the notions presented above to varying degrees. But there is also a class of people in India that wields science as an antidote to what it claims is the state’s collusion with pseudoscience, and such collusion as displacing what is apparently science’s rightful place in the Indian society-state: as the best and sole arbiter of facts and knowledge. This science is apparently a unified whole, objective, self-correcting, evidence-based, and anti-faith. I imagine this science needs to have these characteristics in order to effectively challenge, in the courts of public opinion, the government’s oft-mistaken claims.

At the same time, the ongoing Dias et al. saga reminds us that any ‘science’ imprisoned by these assumptions would dismiss the events and forces that would actually help it grow – such as incentivising good-faith actions, acknowledging the labour required to keep science honest and reflexive, discussing issues resulting from the cultural preferences of its exponents, paying attention to social relationships, heeding concerns about the effects of one’s work and conduct on the field, etc. In the words of Paul Feyerabend (Against Method, third ed., 1993): “Science is neither a single tradition, nor the best tradition there is, except for people who have become accustomed to its presence, its benefits and its disadvantages.”

On the NBDSA opinion against Zee News

On April 5, JNU PhD student Shehla Rashid tweeted that the National Broadcasting and Digital Standards Agency (NBDSA) had ordered Zee News to remove links to a show it had broadcast in November 2020, alleging that Rashid was indulging in “anti-national activities” and that she was “funding terror”.

The program was hosted by Zee News editor-in-chief Sudhir Chaudhary, who, the NBDSA statement found, hadn’t bothered to present Rashid’s defence of herself on the same show nor stopped to check if the claims being made on the show – by Rashid’s father, from whom she her mother and her sister are estranged – were true.

Even if they haven’t watched the show in question, readers of this blog must know by now what its tone, style and decibel-level must have been: shows like this have been regular programming at a clump of pseudo-news-sites trumpeting Hindutva ideals and hate. These outlets deserve without exception to be brought to book by the relevant national institutions, including the NBDSA – even as Rashid added, agreeably, that simply asking hateful content to be taken down from their pages doesn’t suffice. They need to issue public apologies and pay steep fines, say as a fixed percentage of their revenues, for every transgression identified by the NBDSA.

But even as we rejoice – just a little – in the knowledge that the NBDSA had some integrity where many other national institutions haven’t, I wish it had used different language in its statement upbraiding Zee News. Rashid had shared a screenshot of a part of the NBDSA’s statement showing the bottom fifth of one page and the top four-fifths of the next one. In this portion, the NBDSA is concerned repeatedly with the “impartiality and objectivity” and their absence in Zee News’s show. To quote:

With regard to the broadcast, NBDSA was of the view that the issue under consideration is whether the programme lacked objectivity, impartiality, neutrality and whether it violated the complainant’s privacy. NBDSA noted that by allowing the interviewee … the channel had presented only one side of the story. Further, not only had the broadcaster failed to approach the complainant for her version prior to telecasting the impugned programme but bymaking only a fleeting reference to her denial of the allegations, the broadcaster had also failed to adequately present her version. In any case, the Authority noted that to broadcast the version of the complainant available in her social posts was not sufficient compliance of the [Code of Ethics and Broadcasting Standards and] Guidelines.

That Zee News failed to accord any meaningful screen time to Rashid in a programme about slinging mud on her work and views is a cardinal sin. Even if you have a great investigative story on your hands, with documentary evidence for every claim, not giving the implicated parties a chance to defend themselves will only undermine the story. It’s a small step and often an easy one: either the parties will decline comment, as many in India have chosen to do of late, or seize the chance to air their side of the story. This opportunity reflects, among other things, the media’s refusal to serve as an arbiter but also its implicit acknowledgment of the possibility that it its narrative isn’t the ‘ultimate truth’, however that’s defined.

However, the NBDSA has tied this rule to “objectivity”, which doesn’t make sense. This is because Zee News, from its point of view, surely believes that it has been objective: Rashid’s father does hold the views that he does, and Zee News has broadcast them without alterations. There is no violation of objectivity here per se.

Even when unscrupulous outlets like OpIndia quote ministers and government officials spouting hateful rhetoric or articulating policies grounded in dubious assumptions, they are being objective in the sense that these statements do exist – i.e. their existence is a fact – leaving the outlets to simply report these facts to a larger audience. And even if OpIndia, and Zee News for that matter, aren’t objective in other ways (esp. when they publish false news first-hand), the definition of objectivity encompasses the reporting of facts and thus allows them to claim to the consumers of their bilge that they’re being “objective” where others aren’t.

This is how the pursuit of objectivity can be, and often is, anti-democracy. Many feckless (pseudo) news publishers in India have hijacked the false virtue of objectivity to project themselves as the purveyors of ‘real journalism’, while many of the rest of us have allowed them to do so by vying for the same objectivity. We don’t need to be objective; we need to be pro-democracy: the latter compels a greater fidelity to the truth, in substance as well as spirit, that proscribes technicalities of the sort that pro-Hindutva ‘reporters’ have been known to employ.

If the NBDSA had instead pulled up Zee News for not being on the side of democracy, instead of not being on the side of objectivity, (that part of) its statement would have been perfect. It would also have set an important precedent for other news outlets, at least those that are interested and willing, to follow.

Defending philosophy of science

From Carl Bergstrom’s Twitter thread about a new book called How Irrationality Created Modern Science, by Michael Strevens:

https://twitter.com/CT_Bergstrom/status/1372811516391526400

The Iron Rule from the book is, in Bergstrom’s retelling, “no use of philosophical reasoning in the mode of Aristotle; no leveraging theological or scriptural understanding in the mode of Descartes. Formal scientific arguments must be sterilised, to use Strevens’s word, of subjectivity and non-empirical content.” I was particularly taken by the use of the term ‘individual’ in the tweet I’ve quoted above. The point about philosophical argumentation being an “individual” technique is important, often understated.

There are some personal techniques we use to discern some truths but which we don’t publicise. But the more we read and converse with others doing the same things, the more we may find that everyone has many of the same stand-ins – tools or methods that we haven’t empirically verified to be true and/or legitimate but which we have discerned, based on our experiences, to be suitably good guiding lights.

I discovered this issue first when I read Paul Feyerabend’s Against Method many years ago, and then in practice when I found during reporting some stories that scientists in different situations often developed similar proxies for processes that couldn’t be performed in their fullest due to resource constraints. But they seldom spoke to each other (especially across institutes), thus allowing an ideal view of how to do something to crenellate even as almost every one did that something in a similarly different way.

A very common example of this is scientists evaluating papers based on the ‘prestigiousness’ and/or impact factors of the journals the papers are published in, instead of based on their contents – often simply for lack of time and proper incentives. As a result, ideas like “science is self-correcting” and “science is objective” persist as ideals because they’re products of applying the Iron Rule to the process of disseminating the products of one’s research.

But “by turning a lens on the practice of science itself,” to borrow Bergstrom’s words, philosophies of science allow us to spot deviations from the prescribed normal – originating from “Iron Rule Ecclesiastics” like Richard Dawkins – and, to me particularly, revealing how we really, actually do it and how we can become better at it. Or as Bergstrom put it: “By understanding how norms and institutions create incentives to which scientists respond …, we can find ways to nudge the current system toward greater efficiency.”

(It is also gratifying a bit to see the book as well as Bergstrom pick on Lawrence Krauss. The book goes straight into my reading list.)

A trumpet for Ramdev

The Print published an article entitled ‘Ramdev’s Patanjali does a ‘first’, its Sanskrit paper makes it to international journal’ on February 5, 2020. Excerpt:

In a first, international science journal MDPI has published a research paper in the Sanskrit language. Yoga guru Baba Ramdev’s FMCG firm Patanjali Ayurveda had submitted the paper. Switzerland’s Basel-based MDPI … published a paper in Sanskrit for the first time. Biomolecules, one of the peer-reviewed journals under MDPI, has carried video abstracts of the paper on a medicinal herb, but with English subtitles. … The Patanjali research paper, published on 25 January in a special issue of the journal titled ‘Pharmacology of Medicinal Plants’, is on medicinal herb ‘Withania somnifera’, commonly known as ‘ashwagandha’.

This article is painfully flawed.

1. MDPI is a publisher, not a journal. It featured on Beall’s list (with the customary caveats) and has published some obviously problematic papers. I’ve heard good things about some of its titles and bad things about others. The journalist needed to have delineated this aspect instead of taking the simpler fact of publication in a journal at face value. Even then, qualifying a journal as “peer-reviewed” doesn’t cut it anymore. In a time when peer-review can be hacked (thanks to its relative opacity) and the whole publishing process subverted for profit, all journalists writing on matters of science – as opposed to just science journalists – need to perform their own checks to certify the genealogy of a published paper, especially if the name of the journal(s) and its exercise of peer-review are being employed in the narrative as markers of authority.

2. People want to publish research in English so others can discover and build on it. A paper written in Sanskrit is a gimmick. The journalist should have clarified this point instead of letting Ramdev’s minions (among the authors of the paper) claim brownie points for their feat. It’s a waste of effort, time and resources. More importantly The Print has conjured a virtue out of thin air and broadcast asinine claims like “This is the first step towards the acceptance of ‘Sanskrit language’ in the field of research among the international community.”

3. The article has zero critique of the paper’s findings, no independent comments and no information about the study’s experimental design. This is the sort of nonsense that an unquestioning commitment to objectivity in news allows: reporters can’t just write someone said something if what they said is wrong, misleading, harmful or all three. Magnifying potentially indefensible claims relating to scientific knowledge – or knowledge that desires the authority of science’s approval – without contextualising them and fact-checking them if necessary may be objective but it is also a public bad. It pays to work with the assumption (even when it doesn’t apply) that at least 50% of your readers don’t know better. That way, even if 1% (an extremely conservative estimate for audiences in India) doesn’t know better, which can easily run into the thousands, you avoid misinforming them by not communicating enough.

4. A worryingly tendentious statement appears in the middle of the piece: “The study proves that WS seeds help reduce psoriasis,” the journalist writes, without presenting any evidence that she checked. It seems possible that the journalist believes she is simply reporting the occurrence of a localised event – in the form of the context-limited proof published in a paper – without acknowledging that the act of proving a hypothesis is a process, not an event, in that it is ongoing. This character is somewhat agnostic of the certainty of the experiment’s conclusions as well: even if one scientist has established with 100% confidence that the experiment they designed has sustained their hypothesis and published their results in a legitimate preprint repository and/or a journal, other scientists will need to replicate the test and even others are likely to have questions they’ll need answered.

5. The experiment was conducted in mice, not humans. Cf. @justsaysinmice

6. “‘We will definitely monetise the findings. We will be using the findings to launch our own products under the cosmetics and medicine category,’ Acharya [the lead author] told ThePrint.” It’s worrying to discover that the authors of the paper, and Baba Ramdev, who funded them, plan to market a product based on just one study, in mice, in a possibly questionable paper, without any independent comments about the findings’ robustness or tenability, to many humans who may not know better. But the journalist hasn’t pressed Acharya or any of the other authors on questions about the experiment or their attempt to grab eyeballs by writing and speaking in Sanskrit, or on how they plan to convince the FSSAI to certify a product for humans based on a study in mice.

Two sides of the road and the gutter next to it

I have a mid-October deadline for an essay so obviously when I started reading up on the topic this morning, I ended up on a different part of the web – where I found this: a piece by a journalist talking about the problems with displaying one’s biases. Its headline:

It’s a straightforward statement until you start thinking about what bias is, and according to whom. On 99% of occasions when a speaker uses the word, she means it as a deviation from the view from nowhere. But the view from nowhere seldom exists. It’s almost always a view from somewhere even if many of us don’t care to acknowledge that, especially in stories where people are involved.

It’s very easy to say Richard Feynman and Kary Mullis deserved to win their Nobel Prizes in 1965 and 1993, resp., and stake your claim to being objective, but the natural universe is little like the anthropological one. For example, it’s nearly impossible to separate your opinion of Feynman’s or Mullis’s greatness from your opinions about how they treated women, which leads to the question whether the prizes Feynman and Mullis won might have been awarded to others – perhaps to women who would’ve stayed in science if not for these men and made the discoveries they did.

One way or another, we are all biased. Those of us who are journalists writing articles involving people and their peopleness are required to be aware of these biases and eliminate them according to the requirements of each story. Only those of us who are monks are getting rid of biases entirely (if at all).

It’s important to note here that the Poynter article makes a simpler mistake. It narrates the story of two reporters: one, Omar Kelly, doubted an alleged rape victim’s story because the woman in question had reported the incident many months after it happened; the other, the author herself, didn’t express such biases publicly, allowing her to be approached by another victim (from a different incident) to have her allegations brought to a wider audience.

Do you see the problem here? Doubting the victim or blaming the victim for what happened to her in the event of a sexual crime is not bias. It’s stupid and insensitive. Poynter’s headline should’ve been “Reporters who are stupid and insensitive fail their sources – and their profession”. The author of the piece further writes about Kelly:

He took sides. He acted like a fan, not a journalist. He attacked the victim instead of seeking out the facts as a journalist should do.

Doubting the victim is not a side; if it is, then seeking the facts would be a form of bias. It’s like saying a road has two sides: the road itself and the gutter next to it. Elevating unreason and treating it at par with reasonable positions on a common issue is what has brought large chunks of our entire industry to its current moment – when, for example, the New York Times looks at Trump and sees just another American president or when Swarajya looks at Surjit Bhalla and sees just another economist.

Indeed, many people have demonised the idea of a bias by synonymising it with untenable positions better described (courteously) as ignorant. So when the moment comes for us to admit our biases, we become wary, maybe even feel ashamed, when in fact they are simply preferences that we engender as we go about our lives.

Ultimately, if the expectation is that bias – as in its opposition to objectivity, a.k.a. the view from nowhere – shouldn’t exist, then the optimal course of action is to eliminate our specious preference for objectivity (different from factuality) itself, and replace it with honesty and a commitment to reason. I, for example, don’t blame people for their victimisation; I also subject an article exhorting agricultural workers to switch to organic farming to more scrutiny than I would an article about programmes to sensitise farmers about issues with pesticide overuse.