Dreams of a Final Theory

Engels, Weinberg

Dialectics of Nature, Friedrich Engels, 1883 (ed. 1976):

… an acquaintance with the historical course of evolution of human thought, with the views on the general inter-connections in the external world expressed at various times, is required by theoretical natural science for the additional reason that it furnishes a criterion of the theories propounded by this science itself. Here, however, lack acquaintance with the history of philosophy is fairly frequently and glaringly displayed. Propositions which were advanced in philosophy centuries ago, which often enough have long been disposed of philosophically, are frequently put forward by theorising natural scientists as brand-new wisdom and even become fashionable for a while. It is certainly a great achievement of the mechanical theory of heat that it strengthened the principle of the theory of heat that it strengthened the principle of the conservation of energy by means of fresh proofs and put it once more in the forefront; but could this principle have appeared on the scene as something so absolutely new if the worthy physicists had remembered that it had already been formulated by Descartes? Science physics and chemistry once more operate almost exclusively with molecules and atoms, the atomic philosophy of ancient Greece has of necessity come to the fore again. But how superficially it is treated even by the best of natural scientists! Thus Kekulé tells us … that Democritus, instead of Leucippus, originated it, and he maintains that Dalton was the first to assume the existence of qualitatively different elementary atoms and was the first to ascribe to them different weights characteristic of the different elements. Yet anyone can read in Diogenes Laertius that already Epicurus had ascribed to atoms differences not only of magnitude and form but also of weight, that is, he was already acquanited in his own way with atomic weight and atomic volume.
The year 1848, which otherwise brought nothing to a conclusion in Germany, accomplished a complete revolution only in the sphere of philosophy. By throwing itself into the field of the practical, here setting up the beginnings of modern industry and swindling, there initiating the mighty advance which natural science has since experienced in Germany and which was inaugurated by the caricature-like itinerant preachers Vogt, Büchner, etc., the nation resolutely turned its back on classical German philosophy that had lost itself in the sands of Berlin Old-Hegelianism. Berlin Old-Hegelianism had richly deserved that. But a national that wants to climb the pinnacles of science cannot possibly manage without theoretical thought. Not only Hegelianism but dialectics too was thrown overboard—and that just at the moment when the dialectical character of natural processes irresistibly forced itself upon the mind, when therefore only dialectics could be of assistance to natural science in negotiating the mountain of theory—and so there was a helpless relapse into the old metaphysics. What prevailed among the public since then were, on the one hand, the vapid reflections of Schopenhauer, which were fashioned to fit the philistines, and later even those of Hartmann, and, on the other hand, the vulgar itinerant-preacher materialism of a Vogt and a Büchner. At the universities the most diverse varieties of eclecticism competed with one another and had only one thing in common, namely, that they were concocted from nothing but remnants of old philosophies and were all equally metaphysics. All that was saved from the remnants of classical philosophy was a certain neo-Kantianism, whose last word was the eternally unknowable thing-in-itself, that is, the bit of Kant that least merited preservation. The final result was the incoherence and confusion of theoretical thought now prevalent.
One can scarcely pick up a theoretical book on natural science without getting the impression that natural scientists themselves feel how much they are dominated by this incoherence and confusion, and that the so-called philosophy now current offers them absolutely no way out. And here there really is no other way out, no possibility of achieving clarity, than by a return, in one form of another, from metaphysical to dialectical thinking.

Dreams of a Final Theory, Steven Weinberg, 1992:

Even where philosophical doctrines have in the past been useful to scientists, they have generally lingered on too long, becoming of more harm than ever they were of use. Take, for example, the venerable doctrine of “mechanism,” the idea that nature operates through pushes and pulls of material particles or fluids. In the ancient world no doctrine could have been more progressive. Ever since the pre-Socratic philosophers Democritus and Leucippus began to speculate about atoms, the idea that natural phenomena have mechanical causes has stood in opposition to popular beliefs in gods and demons. The Hellenistic cult leader Epicurus brought a mechanical worldview into his creed specifically as an antidote to belief in the Olympian gods. When Rene Descartes set out in the 1630s on his great attempt to understand the world in rational terms, it was natural that he should describe physical forces like gravitation in a mechanical way, in terms of vortices in a material fluid filling all space. The “mechanical philosophy” of Descartes had a powerful influence on Newton, not because it was right (Descartes did not seem to have the modern idea of testing theories quantitatively) but because it provided an example of the sort of mechanical theory that could make sense out of nature. Mechanism reached its zenith in the nineteenth century, with the brilliant explanation of chemistry and heat in terms of atoms. And even today mechanism seems to many to be simply the logical opposite to superstition. In the history of human thought the mechanical worldview has played a heroic role.
That is just the trouble. In science as in politics or economics we are in great danger from heroic ideas that have outlived their usefulness. The heroic past of mechanism gave it such prestige that the followers of Descartes had trouble accepting Newton’s theory of the solar system. How could a good Cartesian, believing that all natural phenomena could be reduced to the impact of material bodies or fluids on one another, accept Newton’s view that the sun exerts a force on the earth across 93 million miles of empty space? It was not until well into the eighteenth century that Continental philosophers began to feel comfortable with the idea of action at a distance. In the end Newton’s ideas did prevail on the Continent as well as in Britain, in Holland, Italy, France, and Germany (in that order) from 1720 on. To be sure, this was partly due to the influence of philosophers like Voltaire and Kant. But here again the service of philosophy was a negative one; it helped only to free science from the constraints of philosophy itself.

‘Nothing in the history of science is ever simple’

Once I finished Steven Weinberg’s book Dreams of a Final Theory, I figured I’d write a long-winding review about what I think the book is really about, and its merits and demerits. But there is a sentence in the seventh chapter – titled ‘Against Philosophy’ – which I think sums up all that the book essentially attempts to explain.

Nothing in the history of science is ever simple.

And Dreams of a Final Theory wants to make you understand why that is so. To Weinberg’s credit, he has done a good job – not a great one – with complexity as his subject. I say ‘not a great one’ because it has none of the elegance that Brian Greene’s The Elegant Universe did, and it laid out string theory from beginning to end. At the same time, it is still Weinberg, one of the towering figures of particle physics, at work, and he means to say, first, that there is no place for simplicity in his line of work and, second, even in all the terrible complexity, there is beauty.

The book, first published in 1992, is a discourse on the path to a ‘final theory’ – one theory to rule them all, so to speak – and the various theoretical, experimental, mathematical and philosophical challenges it presents. Weinberg is an erudite scientist and you can trust him to lay out almost all facets of all problems that he chooses to introduce in the book – and there are many of them. Also, I wouldn’t call the book technical, but at the same time it demands its fair share of intellectual engagement because the language tends to get (necessarily) intricate. And if you’re wondering: There are no equations.

In fact, I would be able to describe the experience of reading Dreams of a Final Theory using a paragraph from the book, and such internal symmetry is unmistakable throughout the book:

But why should the final theory describe anything like our world? The explanation might be found in what [Robert] Nozick has called the principle of fecundity. It states that the different logically acceptable universes all in some sense exist, each wit its own set of fundamental laws. The principle of fecundity is not itself explained by anything, but at least it has a certain pleasing self-consistency; as Nozick says, the principle of fecundity states ‘that all possibilities are realized, while it itself is one of those possibilities’.

Buy the book.

Debating the business of beauty in ‘Dreams of a Final Theory’

In his book Dreams of a Final Theory, Nobel-Prize-winning physicist Steven Weinberg discusses the various aspects of the journey toward a unifying theory in fundamental physics. One crucial aspect is the aesthetic of such a theory, and Weinberg’s principal contention is that a unifying theory must be beautiful because if it weren’t beautiful, it wouldn’t be final in every sense. However, thinking so presupposes all scientific pursuits are motivated by a quest for beauty – this may not be the case. More importantly, beauty in being a human construction can be fickle and arbitrary, and interfere with the pursuit of science.

We are trained to expect nature to be a certain way and we call that beauty. As a result, we strive for solutions that are beautiful, i.e. commensurate with the way we see nature to be. But if the physicist confesses to you that the problems he chooses to solve are so beautiful, then that implies he thinks the problem is beautiful in its own right and independently of its solution’s beauty. Does this mean problem-solving in fundamental physics is dominated by a selection bias: whereby scientists choose to solve some problems over others because of the way they appeal to their aesthetic sense? Weinberg thinks so, and presents an example of scientists going after an ‘ugly’ problem – the thermal demagnetization of iron and critical exponent associated with it (0.37) – in the hope that it will have a beautiful solution. He writes,

Why should leaders of condensed matter theory give the problem of the critical exponents so much greater priority? I think the problem of critical exponents attracted so much attention because physicists judged that it would be likely to have a beautiful solution.

The result of their selection bias is the emergence of a dividing line between what needs to be studied and what doesn’t, between what knowledge is codified in the form of principles and what knowledge remains as individual facts. There is an obvious conflict with objective rationality here, which guides the fundamental investigations of nature and excludes unreasonable judgments like those backed by one’s sense of beauty. It seems, according to Weinberg, we are all motivated only to discover a beautiful universe – one that appeals to our preexisting convictions of what the universe ought to be – as if we are defining the beauty we feel we are bound to abide by. What else are we doing when we reject ‘ugly’ solutions but rejecting a form of the truth that doesn’t appeal to our sense of beauty²? By Weinberg’s own admission, what constitutes beauty¹ has been changing with the discovery of more truths: just as beauty was a universality among the dynamics of forces in the early 20th century, beauty in the 21st century seems to be the presence of symmetry principles.

Therefore, by making such decisions, we are actively precluding the ‘existence’ of certain kinds of beauty because we are also forestalling the discovery of certain truths. Weinberg defends this by saying that if aesthetic judgments are working increasingly well, it could be because they are applicable – but the contention he does not address at all is that it is an arbitrary mechanism with which to arrive at the truth. We are simply consigning ourselves to understand beauty in different eras as new deviations from previous definitions of beauty, and removing opportunities to understand other³ (i.e. seemingly unrelated) kinds altogether. For example, the physicist who decides that the ‘ugly’ critical exponent of 0.37 must belong to a more beautiful, overarching theory is immediately pigeonholing other seemingly random exponents to the same fate. What if such exponents are indeed ones of a kind – perhaps even part of a much larger renormalization framework that researchers are desperately seeking to make sense of the many ‘fine-tuned’ constants in high-energy physics, rather than buoys of apparently hidden symmetries themselves that lead nowhere?

There are three additions to this discussion (referenced in the paragraph above):

1. Has beauty always been the pursuit of science? Elegance is definitely a part of the pursuit – if not more – because the elegance of natural phenomena is sure to reflect in the natural sciences, to paraphrase Werner Heisenberg. At the same time, Weinberg goes to some length to mark a distinction between beauty and elegance: “An elegant proof or calculation is one that achieves a powerful result with a minimum of irrelevant complication. It is not important for the beauty of a theory that its equations should have elegant solutions.” That said, the answer to this question is unlikely to be short or general for it questions the motivations of scientists over many centuries. At the same time, some of the greatest scientists – typically Nobel Prize winners – have said the quest for beauty has constituted a significant part of their work simply as an abrogation of randomness. Here is Subrahmanyan Chandrasekhar writing about the work of Lord Rayleigh in his book, Truth and Beauty: Aesthetics and Motivations in Science:

… after a scientist has reached maturity, what are the reasons for his continued pursuit of science? To what extent are they personal? To what extent are aesthetic criteria, like the perception of order and pattern, form and substance, relevant? Are such aesthetic and personal criteria exclusive? Has a sense of obligation a role? I do not mean obligation with the common meaning of obligation to one’s students, one’s colleagues, and one’s community. I mean, rather, obligation to science itself. And what, indeed, is the content of obligation in the pursuit of science for science?

2. We started with the assumption that beauty is what we have learnt nature to be. Therefore, by saying a problem or a solution doesn’t appeal to our sense of beauty, it only means it doesn’t appeal to what we already know. This attitude is best characterized by the tendency of well-entrenched paradigms to not give way to new ones, to not surrender in the face of new knowledge that they can’t account for. An example I am particularly fond of in this regard is the story of Dan Shechtman‘s discovery of quasicrystals, which went against the grain of Linus Pauling’s theory of crystals at the time.

Before introducing the third point (which is optional): While it is clear that Weinberg is enamored by the prospect of beauty legitimizing the study of fundamental physics, all of science cannot afford to be guided by as fickle a metric because beauty is what we expect nature to be – according to him – and that signifies a persistence with ‘old knowledge’ while discovering ‘new knowledge’. That deprives the scientific method of its objectivity. Also, the classification of knowledge impedes what scientists choose to study and how they choose to study it as well, and judging the legitimacy of knowledge based on its beauty lends itself to a mode of classification that is not entirely rational. Finally, that scientists also wouldn’t reject new knowledge if it was ugly but that beautiful knowledge would find acceptance faster and scrutiny slower is not… proper.

3. Orson Scott Card’s Speaker for the Dead provides an interesting way to understand this ‘otherness’. It describes a so-called hierarchy of foreignness to understand how alien a person or object is relative to another, in four stages (quoted from the book): Utlänning, “the stranger that we recognize as being a human of our world, but of another city or country”; framling, “the stranger that we recognize as human, but of another world”; raman, “the stranger that we recognize as human, but of another species”; and varelse, “the true alien … which includes all the animals, for with them no conversation is possible. They live, but we cannot guess what purposes or causes make them act. They might be intelligent, they might be self-aware, but we cannot know it.” Similarly, the ‘other’ kinds of beauty we stand to lose, according to Weinberg, are varelse, while we stick to the more fathomable (utlänning, framling and raman) kinds.