Scaling Laws in Living Things & Human Society
My notes after reading Goeffrey B. West's book "Scale: The Universal Laws of Growth, Innovation, Sustainability, and the Pace of Life in Organisms, Cities, Economies, and Companies".
Scaling Laws that Govern Living Things
People who are familiar with large language models should be pretty familiar with neural scaling law - the empirical (usually following power law) curve that is fitted from smaller scale models to inform the optimal configuration for training a large model. Scaling law was applied as early as the 19th century by William Froude to design ships with stability, and is widely used today in industrial design, so it is not surprising it is used for design of machine learning models.
One thing I didn’t realize though, until I read Geoffrey West’s book Scale, is that scaling law is also used by nature to design species (retrospectively, it does make lots of sense). One specific example is Kleiber's law, which states that for a vast majority of animal species, the basal metabolic rate scales to the roughly 3/4 power of the animal's mass. In other words, a species A with 4 times the mass of species B only consumes 3 times the energy consumed by species B. Beyond metabolic rate, there are a lot of other biometrics that follow similar power law scaling, from heart rate, life span to aorta (the largest artery in the body) radius and more. From that sense, a horse is like an upscaled version of a mouse, and a downscaled version of an elephant.
Most of the scaling laws about life have been known since a long time ago. Coming from a theoretical physicist background, West’s biggest contribution to the field was applying first principle thinking to quantitatively model the origin of these scaling laws. West and his collaborators observed that in order to sustain life, materials and energy must be delivered to all the subunits (e.g. cells) of the body in an efficient and “democratic” way, and natural selection has evolved “branching networks” as a solution. The most obvious example of such branching networks is the cardiovascular system, in which blood comes out from the aorta, into smaller and smaller veins, and finally into capillaries where energy and materials are diffused into cells. If we hypothesize that the branching network that sustains life following the following 3 simple principles:
The network is space filling, in that the branches of the network have to extend throughout the whole system that it is serving (i.e., the whole body).
The terminal units of the network are similar in size and characteristics. For example, the capillaries of all mammals, be it adults or children, are roughly the same. This also makes sense given how hard it is to come up with new fundamental building blocks of living things.
The system minimizes the energy needed to pump fluids through the system, driven by natural selection.
Then we can derive Kleiber's law and other other scaling laws mathematically [1]. That’s a really elegant theory!
Another interesting insight West derived from his work is that because the scaling of metabolism is sublinear (meaning that the exponent is smaller than 1), it leads to bounded growth & life span of lives, in a predictable, power law scaling way. Sublinear scaling and its implications contrast significantly with the scaling of socioeconomic aspect of human society, which is the focus of the second part of the book.
Scaling Laws that Govern Human Society
Human society resembles lots of the patterns in organisms. Take cities as an example, they consist of infrastructure networks like roads and utility lines that deliver food, water and energy. And like the networks in organisms, these infrastructure networks are “space filling” in the sense that they aim to serve every household; they also have invariant terminal units such as houses and faucets; and they are also constructed to optimize for least energy and material usage. In fact, sublinear power law scaling is also found in cities’ physical infrastructure, such as the number of gas stations. The only difference is that they scale by an exponent of 0.85 instead of ¾.
When it comes to the social economic aspect of cities, however, superlinear scaling is widely observed, in strong contrast to organisms. Wages, GDP, patent count and crime counts, etc, all increase faster than the increase of city population, scaling roughly to 1.15 power of population. Interestingly, the walking pace of pedestrians demonstrates similar scaling as well. In other words, living in bigger cities gives you more material success, but at the same time, you have to accept more social unrest and a faster pace of life.
The cause of the superlinear scaling of social economic activities is far from being conclusive, but in the book, it is hypothesized that it comes from the superlinear scaling of social interactions in cities’ social economic network, which can further be explained by the different nature of physical networks and social networks [2].
Superlinear scaling has far reaching implications on the growth dynamics of cities. As is mathematically shown in [3], in contrast to sublinear scaling which leads to bounded growth and linear scaling which leads to exponential growth, superscaling leads to superexponential growth and finite time singularity, i.e., growing to unlimited in a finite time. This is of course unsustainable because unlimited population implies unlimited resource consumption. In reality, the outcome of superexponential growth is stagnation and eventual collapse.
There is one approach to avoid finite time singularity, though, which is to reset the system parameters periodically, through innovations and major paradigm shifts. This resets the clock and buys us time to come up with new innovations before the next singularity. Unfortunately, there is a big catch - in order to keep pushing the time of finite time singularity to the future, the pace of innovation has to come faster and faster as well.
As you might already know, the acceleration of the innovation cycle appears to be what’s happening. The question is, can we keep accelerating innovations? West didn’t think so, and suggested that a paradigm shift towards a more ecological, sublinear scaling lifestyle might be needed.
At the end of the book, he called for devotion to a “grand unified theory of sustainability” through multidisciplinary efforts to answer the challenge.
Thoughts After Reading the Book
Like any other scientific work, West’s theory and conclusions are not without challenges and controversy (example). More recent data seems to suggest that Kleiber's law only holds for birds and mammals, but across all forms of non-plant life, the scaling of metabolic rate is actually close to 1 instead of ¾ [4]. If that’s true, then West’s model would only apply to a small subset of living things instead of being universal. That said, even if that’s the case, I still think it is a pretty insightful and elegant model.
It might be the case that, for complex systems like organisms and human society, there is just no way to characterize them with a simple model and West’s model about scaling law in biology, even if it is correct, might just be a rare exception. We might just have to resort to models and simulations to tackle most of the complexity system problems, but are we going to be satisfied with just our knowledge being stored as network weights instead of human digestible logic? How can we share and expand our knowledge in that world?
Moving onto the superscaling of cities. Urbanization is still ongoing but since West’s team published their paper, lots of big cities have slowed or stopped their rapid population growth. What happened to those cities and how are they doing so far? Are they heading into stagnation and eventual collapse, or was it because of innovations and paradigm shift? Has the digital economy enabled more capacities for social and economic connections beyond city boundaries, and thus sustain further growth? Although I am skeptical about technology singularity, if we are indeed cursed with shorter and shorter innovation cycles to avoid the population singularity, could technology singularity be the solution to that?
The book left me with more questions to answer than what has been answered, but I think that’s exactly what makes it a good read for me.
References
[1] West, Geoffrey B., James H. Brown, and Brian J. Enquist. "A general model for the origin of allometric scaling laws in biology." Science 276.5309 (1997): 122-126.
[2] Bettencourt, Luís MA. "The origins of scaling in cities." science 340.6139 (2013): 1438-1441.
[3] Bettencourt, Luís MA, et al. "Growth, innovation, scaling, and the pace of life in cities." Proceedings of the national academy of sciences 104.17 (2007): 7301-7306.
[4] Hatton, Ian A., et al. "Linking scaling laws across eukaryotes." Proceedings of the National Academy of Sciences 116.43 (2019): 21616-21622.