Sabine
patreon
Complexity: The biggest gap in modern science
🕑 Added 2023-12-16 13:00:07 +0000 UTC[This is a transcript with links to references.]
Complexity. It sounds very sciency, doesn’t it. Everyone loves to talk about complex problems and complex systems, but no one has any idea what it means. I think that understanding complexity is THE biggest gap in science today. What do we even mean by complexity? What do we know about it? And what’s the problem with trying to explain it? That’s what we’ll talk about today.
Before we talk about complexity, let me tell you about something very simple, a special offer from our sponsor galaxy lamps.
If the days are short and cold and the nights are long and getting longer, why not throw some space colours on the wall with the galaxy projector. It’ll instantly turn your room into a wonderland. And it’s not just a lamp, it’s a smart lamp. It comes with a phone app that lets you adjust the colour and brightness and lets you pick different themes as well. It even works with voice control through Alexa or the Google assistant.
I have one of those projectors at home and I find it both relaxing and inspiring, especially around this time of the year. They’ll also make a great Christmas gift for yourself and all your space loving nerd friends, so don’t miss this opportunity because they’re on sale right now. Just click on the link in the info below and check it out. Now let’s talk about complexity.
How complex do you think these objects are: a rock, a clockwork, a baby. I am guessing most of you would rate the baby the most complex, the clockwork somewhere in the middle, and the rock the least complex. Unless possibly you’re a geologist working on a grant proposal, in which case you probably shouldn’t be watching YouTube.
But why? I mean, babies aren’t all that difficult, are they? Stuff goes in one end and comes out the other, and for the first couple of months that’s pretty much it.Ok, you might say, but they grow, they learn to speak, they go on and win Nobel prizes, sometimes. Clockworks don’t do that. The baby’s more complex not because of what it does, but what it could go on to do. How can we possibly scientifically capture a property like that?
Maybe let’s start with a simpler example. Coffee. Or, if you’re British, imagine it’s tea. And we pour milk into it. Initially you have the coffee and milk separated. That’s very orderly, very non-complex. Then you mix them together and well some complex things happen. Then it’s completely mixed and that’s maximum entropy and non-orderly again. So complexity is somewhere in the middle between the two, between strict order and maximum disorder, between very low and very high entropy.
This isn’t just the case for latte, it’s also the case for the entire universe. The universe started out in a very orderly state, with energy and matter smoothly distributed in space. Smoothly, but not perfectly smoothly, with some tiny quantum fluctuations in it. Then universe began to expand, and gravity did its thing. The places where matter and energy were a little more dense, collapsed, and heated up. They went on to form stars and then galaxies and galaxy clusters. And some of those stars got planets and on at least one of those planets, life evolved.
Yet as time goes on, entropy continues to increase. The stars will burn out, the universe will cool, and life will no longer be possible. Complexity, life, society, technology, is only possible in this intermediate phase, between order and disorder, between low and high entropy. The difficulty is making this intuition scientifically precise. And so far, no one’s come up with a convincing idea.
It’s not that no one’s tried. The issue is that scientists have too many different ways of quantifying complexity, and they all fall short of capturing it.
There most used one is probably algorithmic complexity also known as Kolmogorov complexity. That’s the length of the shortest possible computer program that can generate the data that describes your system. This seems to make sense at first sight doesn’t it. Certainly, you need a more difficult algorithm to calculate what a baby does than what a rock does, right? Yeah, well, think again.
Or better, think physics. Both the rock and the baby are made of atoms. And those atoms are made of elementary particles. And the behaviour of all those particles is always described by the same mathematics, that’s the standard model of particle physics, plus Einstein’s general relativity. Baby, rock, clockwork, they all can be described by the same maths. In fact, everything you’ve ever seen can be described by that, at least in principle. So, if you went by algorithmic complexity, everything would be equally complex. That makes no sense.
Ok, you could say, but leaving aside that the algorithm is fundamentally the same because everything is made of the same elementary particles, it’s arguably true that what a baby does is more difficult to calculate than what a rock does. Can’t we figure out a way to measure that?
Well, the reason the baby is more difficult to calculate than a rock is not to do with algorithms or their complexity, but with the the arrangement of the particles. For a rock, they’re simple. All those particles collect to certain mineral configurations that are more or less regular, and then they just sit there. For a clockwork, you have metals but in very specific shapes so that different parts move against each other. And for the baby you have lots of different arrangements of molecules in cells and organs and the thing moves and wiggles and squeaks.
That, in some sense, is what we mean by complex, it’s about how the thing is made up, not the algorithm that describes it. What we seem to be missing is a type of structural complexity.
Measures for structural complexity also exist but they, too, have their problems. They are most commonly used for graphs or network. Graphs and networks are the same thing, they’re lines connected by dots basically. It’s just that in some areas of science they’re called graph, elsewhere networks. Oh and the lines and dots aren’t called lines and dots, because that sounds very kindergarten. If you want to sound educated, you call them links and nodes, or edges and vertices.
Now about that structural complexity. Let me give you a simple example. Suppose you have a completely disconnected network, that’s just a bunch of points. And then you have a completely connected network, that connects each point with each other point. Which one is more complex?
I’d say they’re both equally simple and not complex. For me, complexity resides somewhere in the middle. And indeed, you could use some measure, that would tell you exactly that: that they’re both equally non-complex. For example, the Shannon entropy for the degree distribution, just so you’ve heard of it. It’s a measure of how dissimilar different parts of the network are. And for both the completely disconnected and the completely connected network, all parts are similar to each other. So according to this measure, they have small complexity.
Then again, some people might say the graph with the many connections is more complex. And there are measures for that too. You could for example do something like count the number of closed loops, or do more complicated things like taking into account the size of the loop or how many edges go out of each nodes, or combinations thereof and so on. And for all of those measures, the connected network would come out being more complex.
But just how much more complex, well, that depends on the measure. You could use global complexity, average complexity or topological complexity or some other one. All of them measure something, and we can debate their pros and cons, but which one’s the right one? It’s an entire multiverse of complexities.
In the end, the structural complexity of networks also doesn’t help us much. And just in case you haven’t noticed, a baby isn’t a network.
Ok, so just trying to take some mathematical description, an algorithm, a graph, and trying to quantify something about it didn’t really get us very far, did it. Maybe it’d be better to first discuss what we expect of a complex system.
One typical feature we expect of a complex system is that they’re surprising in some sense. They do things that are difficult to predict and often difficult to understand. We say that they have a lot of emergent properties and behaviours.
What does emergent mean? Everything is of course, still made up of those elementary particles and they always obey the same laws. But if you combine those particles to larger things, it often makes sense to describe them differently, not as a lot of elementary particles, but as bigger things with properties and natural laws of their own right. This is what we mean by emergent. The properties and laws of the bigger things that we use in their own right.
A very simple, not a complex, example of an emergent property is the conductivity of a metal. The conductivity of a metal tells you how easily the material transports electrons, so how well it conducts electricity. But that isn’t something you find in the standard model of particle physics. It doesn’t make any sense to speak of the conductivity of a particle. Still, it’s *useful to talk about the conductivity of a metal. There are many other such properties, like viscosity, rigidity, magnetism, and so on. They’re all emergent.
Similar story for chemistry. Yes, chemistry is ultimately all physics, but that’d be cumbersome to use because it’s just the wrong language. Much of physics is unnecessarily complicated on the level of chemical reactions. Chemists therefore describe the behaviour of different compounds and their interactions by using emergent properties. Like solubility, acidity, hydrophilicity, and so on.
Those were all quite simple examples. But the thing is that number of those emergent properties and their behaviours seem to increase with what we call complexity. In biology we might talk about different types of cells and boy are there a lot of them. And they combine to organs and do all kinds of things, like leaving comments on YouTube videos. Lots of emergent behaviour.
So emergent properties, are one of the characteristics we associate with complex system but it’s not the only one. We also expect their behaviour to be difficult but not quite chaotic. As Stuart Kauffman put it so poetically, the most complex systems seem to be those that live on the “edge of chaos”. Complexity increases until chaos sets in.
The defining feature of chaos is that very small changes in the configuration of a system can have huge consequences later on. And since you never know the configuration of a system exactly, chaos makes it for all practical purposes impossible to make predictions beyond a certain time. The edge of chaos therefore is also the edge of unpredictability. It’s on this edge that complexity thrives, because it gives you as much variety as you can possibly have without screwing everything up.
So we have emergence, the edge of chaos, and a third feature that we often see in complex systems is evolution: Complex systems can learn, they adapt, they self-organize, and they improve over time. This seems to be the major way how complexity grows.
This third feature is very much about feedback and the resulting change, and it seems to be missing from the measures of complexity that we previously looked at. It’s not a property of the structure of a system per se, it’s a property of how that structure changes.
That makes three characteristics of complex systems. Emergent properties and behaviour, the more, the more complex, inching towards the edge of chaos, the closer, the more complex, and the ability to learn and adapt, the better the more complex.
If looking for emergent behaviour is the thing to do, you might ask why it is taking so long. It’s not all that hard to tell a brain from a piece of wood, in most cases, so why haven’t scientists figured it out? Well, yes, *you can tell a brain from wood, but if you had to write it down in maths, could you?
When it comes to complexity, we quickly arrive at a point where we’re better off classifying systems by what we see and recognize rather than by maths. We’re using one complex system -- our brain – to recognize others. This is ultimately the same problem we have with consciousness. We’re still operating on the level of: we know it when we see it, but we can’t quite pin down just what we mean by it. It’s a pre-scientific stage.
The scientific challenge is then to quantify complexity in a way that we can use it to formulate new laws of nature. I think it’s the biggest gap in science at the moment because our lacking understanding of complexity is the reason why science is confined to simple systems, or simple questions about complex systems. Think about it, in physics we deal with particles, or materials, or stars. These are all systems with very low complexity. If you look at areas where the complexity is high, like biology or sociology, we don’t have the maths.
By the way this video comes with a quiz on quiz with it, that’ll allow you to check later how much you remember. You can even collect points from all our videos.
There have been a few recent approaches to this question, which I have found very interesting because it’s such an important problem that receives so little attention.
The first one is an idea they call assembly theory. They propose to measure the complexity of objects by how difficult they are to assemble. Basically, they take all the possible ways you can assemble an object and then ask what’s the shortest one.
They also take into account that to assemble big things you need to make the smaller things first, so that gives you a history-dependence that induces a kind of selection. Some things that assemble fall apart and disappear. Other things that assemble stick and make more of those things. So this is how this assemble idea goes together with the notion of selection and adaptation.
And they’ve even calculated this assembly-based complexity for some molecules, and found that organic molecules, those that are being used by living things, are indeed more complex according to this measure. So that’s very promising.
I find this a nice idea, but I think it’s not quite there yet. It’s because some complex objects like, say, humans, often assemble other objects that are far less complex. Say, a pencil. Pencils only exists because humans exist, so I’d argue that when it comes to the entire assembly, a pencil can’t be less difficult to make than a human. Yet it isn’t itself complex.
Another idea is to use a notion of “functional information”. This idea dates back to 2003, when a biologist pointed out that there are many molecules that fulfil the same biological function, and really fulfilling those functions should be the marker for how complex the molecules are. In a recent paper now a group of philosophers pushed this further, arguing that the functions that a system can fulfil are a measure for its complexity. This tries to capture the idea that what makes a system complex is what it can do.
Again I am think this does capture some aspect of complexity, but it isn’t quite there yet, because it’s unclear what a “function” is. You could argue that the function of a rock is sitting on the ground, and it’s pretty good at it. Yet sitting on the ground isn’t exactly a measure of complexity, is it?
Nevertheless, I find this development extremely exciting because we might be on the edge of discovering new laws of nature. And I believe that understanding complexity is the stepping stone to understanding consciousness. Forget theories of everything, that’s so yesterday. The next big thing in science are theories of complexity.
The quiz for this video is here.
Comments
But is it really "that we don't have the maths"? Mathematics provides the means to model the world, but it may not be "all that" as neural networks are able to find patterns, using algorithms that are not explicit equations, in what appears to be similar to how brains function, at least at the subconscious level. The visual center and neural network starts by identifying straight lines and curves at a lower layer and then at a higher layer puts those together to form faces. So, maybe it's time for a change of paradigm that recognizes that mathematics can go only so far.
Seems that any shot at defining complexity, can start anywhere. And end anywhere. Or not end anywhere. Even a closed system with perfectly understandable existence is connected to other systems and hence its fullness is the complexity of all the circumscribing that's needed to grasp it using additive, subtractive, and distortable horizons. And we know its baked in that we will always miss something. If we did not, then it wouldn't be complex. Rad: Thanks...golf is analogous.
I feel your words, Oscar. When I was a wee lass, physics-wise, my Master's advisor had me calculate the fractal dimension of his pulsar emission model (based on soliton collapse in the magnetosphere) and then do the same for actual pulsar data to see if some connection could be made. Sure enough, the model showed chaotic behavior and a well-determined dimension because it came from a pretty straightforward set of nonlinearly coupled differential equations. The actual pulsar timing data didn't show any hint of fractal behavior -- was this because of noise, because time between pulses was not a state variable, because it was not in a chaotic regime, ... The study of chaos always _seems_ like something big is on the horizon, but in actuality there is so little to show in practice.
One possible approach to a mathematical formulation (probably limited to the macro classical level) of what is a complex system and how much so, could be based on the size of the state (or phase) space (i.e. the number of states/dimensions) needed to describe its dynamics as well as the properties of the differential (or difference) equations that prescribe the individual and interactive behavior of those states as the system evolves in time, perhaps along a way that is forced to follow by its interaction with other things outside the system. Or with everything else, if physical things are coupled in some way (e.g. gravitation) throughout the Universe, so the whole of it is one enormous system in the sense described. Or not, if, for example, the attempt to define and deal mathematically with complexity is a scientific wild-goose chase.
