Karl Popper’s Theory of Evolution

Natural selection as creative and purposeful problem-solving ☁️

The Pink Cloud, by Henri-Edmond Cross (c. 1896)

Darwin’s theory of evolution through natural selection is one of the most influential ideas ever generated by humanity, and one of the biggest contributors to the way I think. But there is an important kind of criticism that can be levelled against it: it is rather tautological. The theory says that evolution happens due to the selection of those organisms that survive and reproduce. But how do they manage to survive and reproduce? By being “the fittest”¹… which means that they are good at surviving and reproducing.

It’s not really tautological, of course. There are plenty of cracks that can be filled with interesting detail and testable predictions, which is why natural selection is one of the most influential ideas ever generated by humanity and not just some minor philosophical curiosity. But it can be worth restating it to avoid the appearances of vagueness and tautology.

On the recommendation of a thoughtful friend,² I have been reading the book Objective Knowledge: An Evolutionary Approach by the philosopher Karl Popper, known for his epistemology and for inspiring David Deutsch and the whole “critical rationality” movement. In the chapter “Of Clouds and Clocks,” written in 1965 as a lecture, Popper lays out his restatement of evolutionary theory. It comes in the form of 12 theses, which I shall in turn restate in my own words below.

1. All living organisms are constantly engaged in problem-solving.

2. The problems they’re solving are objective problems, not necessarily consciously chosen; an example is the problem of surviving within a particular environment.

3. The problem solving works by trial-and-error: organisms try new behaviors and reactions, as well as new organs and physiological changes, and new hypotheses; and then do error-elimination to get rid of the bad attempts.

4. Error-elimination can take several forms, including killing the organism, which is natural selection at its most pure; but it can also take the form of evolving control mechanisms to control organs and get rid of bad behaviors of hypotheses

5. The evolution of these controls occurs over time, during the evolution of an organism’s “phylum” (I think this refers to e.g. species or larger groups), but are all recapitulated in the single organism body.

6. This recapitulation into a single body is also, itself, a tentative solution to problems. In other words, each individual organism is a trial.

7. We can schematize this process like this:

   \(P_1 \rightarrow TS \rightarrow EE \rightarrow P_2\)

   where TS means “tentative solution,” EE is “error elimination,” and the two P’s are problems: the initial problem to be solved, and the updated problem that arises as a result of trying out solutions. (This schema comes up a lot throughout Popper’s book.)

8. But in real problem-solving there are many tentative solutions:

   \(TS_1, TS_2, TS_3, ..., TS_n\)

   which you should imagine these laid out vertically between P1 and EE. Organisms propose lots of solutions, which are all submitted to a mechanism of error elimination.

9. In classical Neo-Darwinism, i.e. Darwinian evolution combined with knowledge of genetics, all problems are simply survival; all the TS’s are genetic mutations; and EE is always the killing of the organism. So P1 and P2 don’t look different: once the organism is dead, there’s no update; the next organism also just has to survive.

10. In Popper’s theory, there are also non-survival problems, which means that P2 can differ from P1. Popper gives this example:

    1. P1 is the problem of reproduction.

    2. Once solved, P2 is the problem of spreading the offspring so that they don’t suffocate the parent.

    3. (A solution to P2, he says, may have been the evolution of multicellular organisms: the parent and offspring cells work together instead of being of being in competition.)

11. Thus organisms often have to deal with new problems that arise due to solutions to previous problems.

12. Finally, organisms can evolve error-eliminating mechanisms that are distinct from simply dying. Eyes can evolve to fix the error of being near visible danger; physiological feedback mechanisms like hunger or temperature control can evolve to fix the error of not having enough nutrients or being too cold. So there can be tentative solutions TS’s that aren’t the entire life of the individual organism, e.g. specific behaviors (fleeing from a predator you saw with your eyes, looking for food to satisfy hunger) or physiological adjustments (cells accumulating fat to protect from cold). Ultimately, complex and conscious organisms like humans can formulate hypotheses and have those die in our stead.

From here we can readily reach various extensions of evolutionary theory, beginning with Popper’s general view of what knowledge is.

1. Epistemology and science

Popper’s theory of knowledge is that we don’t gain it simply by observing the world through our senses and noticing patterns. Instead we notice problems, and then make guesses (theories, conjectures) to explain them. The guesses are usually wrong, and are eliminated through criticism and the collection of evidence (this is the one step where we use our senses). The best theory is simply the one that has best resisted to criticism so far, although it could always be eliminated later.

Having done this conjecture-and-criticism work, we usually end up with somewhat different problems than before — and this is how we grow knowledge. In Popper’s philosophy, objective knowledge, which is distinct from the subjective beliefs held in someone’s mind, refers to the current state of scientific discourse, mostly contained in books and papers. It consists of known problems, as well as proposed theories and explanations, and (valid or invalid) critical arguments against them. In other words, knowledge is the sum of every part of the schema we saw above:

\(P_1 \rightarrow TS \rightarrow EE \rightarrow P_2\)

Thus knowledge works similarly to biological natural selection: at any given point, many explanations are proposed (in books, papers, lectures, etc.), most are selected against (in books, papers, lectures, etc.), and the surviving ones form the basis for another iteration of the cycle.

Or, equivalently, we might say that living organisms are made of knowledge: each strand of DNA, each organ, each body is an encoding of a “theory” or error-elimination mechanism, as well as the source of new problems.

2. Tools and technology

If body parts evolve through natural selection in order to solve problems, it seems pretty likely that artificial body parts — tools and instruments — would work the same way. Rather obviously, technologies are always attempts at coming up with solutions. Slightly less obvious is the fact that technologies are also eliminated through criticism. I say slightly because anyone who has ever invented something knows it is a constant iterative process, involving things like brainstorming (the explicit generation of ideas to be eliminated later) and tinkering (making small adjustments to an existing solution, thus rejecting a version in favor of a new one).

I think Popper’s evolutionary view is a good argument against tech trees as prospective planning (as opposed to tech trees as a representation of past history): these future-facing trees can only suggest a small number of solutions, most of which will be eliminated.

Popper only briefly mentions this point, but it’s a nice segue into—

3. Culture and memes

I’ve written a lot about cultural evolution and memes on this blog (most recently here). In my post on the watchmaker analogy, I wrote that this view of the world, though introduced by Richard Dawkins in 1976, has permeated public discourse only recently, in the early 2000s (though one extra data point that I hadn’t considered before is Susan Blackmore’s The Meme Machine in 1999). All of this is too late for Popper to have engaged with it — he died in 1994.

But it’s interesting to note that he anticipates cultural evolution in some important ways. Many memes, after all, are a form of knowledge. They’re proposed solutions to problems, most transparently in the case of the ideas surrounding tools and technology, but also in many other cases, like, say, explanations for economical phenomena like the housing crisis.

More importantly, memes are subject to intense selective pressure. Much of this pressure ends up favoring the memes that are easily transmitted (say, a simplistic but attractive explanation that the housing crisis is due to Airbnb). As far as I know, Popper didn’t write directly about the transmission properties of explanations. But the other half of the selective pressure, the one that makes memetics have deep evolutionary implications, takes the form of criticism and elimination (say, an economist pointing out that Airbnb has only a tiny effect of home prices). This is very Popperian. Cultural evolution proceeds by trying out a lot of memes/ideas (whether in language, religion, technology, economics etc.), and then rejecting the bad ones. The result is the culture we live in.

One passage in Objective Knowledge that stood out to me is a 1971 addendum to the chapter “Evolution and the Tree of Knowledge,” called “The Hopeful Behavioural Monster.” The “hopeful monsters” idea is an argument by Richard Goldschmidt, who wrote around 1940 that Darwinian evolution couldn’t fully explain why complex living structures arise based on the accumulation of many small mutations, even given immense periods of time. There would have to be large “macromutations” that occur from time to time; he called the organisms that would result “hopeful monsters.”

The theory is difficult to defend, because large mutations in an organism are all but certain to be lethal. But Popper thinks there is merit to the idea: while hopeful anatomical monsters are exceedingly unlikely, behavioral ones, in which the genetic macromutations change not the body parts but the way the organism uses them, are totally possible. A behavioral monster would behave radically differently from its parents, but that isn’t necessarily lethal, and may speed up biological innovation.

This is another way of saying that the P1 → TS → EE → P2 schema works better and faster when the TS’s don’t kill the whole organism. And it seems pretty obvious to me, though apparently not to Popper, that hopeful cultural monsters are even more readily possible than the behavioral version. Large cultural change can occur without being eliminated immediately. I think this is one reason why cultural evolution is much faster than biological evolution, in addition to the fact that it relies more on memetic (horizontal) transmission.

4. Art

Art also gets brief mentions. I like this part, from “Of Clouds and Clocks”:

It is perhaps not uninteresting to see that artists, like scientists, use this trial-and-error method. A painter may put down, tentatively, a speck of colour and step back for a critical assessment of its effect in order to alter it if it does not solve the problem he wants to solve. And it may happen that an unexpected or accidental effect of his tentative trial—a colour speck or brush stroke—may change his problem, or create a new subproblem, or a new aim: the evolution of artistic aims and of artistic standards . . . proceeds also by the trial-and-error method.

This seems trivially true, and I’ll note that there has been a lot of trial-and-error in the writing of this essay. (There’ll be some more after it’s published, I’m sure.)

And now we get to the fun part: the teleological implications of all this.

Teleology refers to having purposes or goals. It is generally frowned upon by evolutionary biologists to say that evolution has anything of the sort. No, evolution didn’t “want” to give birds their wings, nor did birds “want” to evolve wings: it was just a bunch of random mutations combined with the fact that those mutations helped their bearers survive and reproduce. At no point in the process was there anyone “wanting” anything.

Of course, such a teleological view of evolution comes naturally to us — despite my education in biology, I say that evolution “wants” behaviors or “gave us” organs all the time. But I say it while being perfectly aware that it’s just a convenient shorthand, nothing more.

Or is it? Popper writes that “Darwin’s theory of natural selection showed that it is in principle possible to reduce teleology to causation by explaining, in purely physical terms, the existence of design and knowledge in the world” (italics his). I’m not sure I fully understand his reasoning, but it seems to be something like:

• teleological explanations can always, in principle, be reduced to a causal, mechanistic explanation;

  • e.g. the fact that I purposefully eat a cookie can be explained by facts about my biology and biochemistry;

• so causal, mechanistic explanations, like natural selection, can be said to simulate purpose;

• therefore it’s totally fine to say that evolution acts with purpose.

This is the place where Popper differs most from traditional Darwinism. He even says that Darwinists should respect the (earlier, discredited) theory of Lamarckism as an approximation! But there’s something compelling about it. Notice that purpose plays a role in most of the topics — science, technology, art — that we saw above. In all these cases, the generation of solutions, followed by criticism and elimination of the bad ones, is absolutely compatible with having goals.

When scientists, technologists, or artists enact the evolutionary process of advancing knowledge and culture, they do it with purpose and creativity. It’s not just memes spreading mindlessly. It’s problems being solved.

Darwinian evolution itself is not actually teleological, but it’s just because there’s nobody to do the teleology: the organisms generally don’t know they’re solving problems, and their only way to fail is to die. Nevertheless it is, like all problem-solving, creative. And seen under this angle, definitely not tautological. The species that survives — or the scientific theory, or technological tweak, or meme, or speck of paint — is the one that best solves the problems it faces, thereby creating new problems that go far beyond meme survival. And in doing so, moving biology / scientific knowledge / technology / culture / art forward.

It’s a nice way to reconcile what often seems like the vast incomprehensible forces of evolution, whether biological or cultural, with the idea that we have agency. Popper was very much into showing that we humans are not little deterministic clockwork machines,³ and have a lot of freedom afforded to us. His restatement of evolutionary theory as generalized problem solving makes that pretty convincing.

Sunset on the Lagoon, Venice, by Henri-Edmond Cross (c. 1898–1893)

1. Not a term used by Darwin, but rather by Herbert Spencer. But Darwin liked it, and moreover “fitness” has become an important concept in modern evolutionary biology, referring to a quantitative measure of an organism’s ability to pass on its genes, which is pretty close to “ability to survive and reproduce.”

2. Who will be glad to know I can now return his copy

3. These deterministic machines are the “clocks” in the title “Of Clouds and Clocks.” The clouds represent unpredictable, nondeterministic systems. Popper argues that everything, including living organisms, including humans, is somewhere in between.