An Immense World of Intelligence
A little tale about the scale of nature and how we are part of it.
A Remarkable Story of Scale
The race to the crown of artificial intelligence has been a race of scale in the past couple of years - bigger models, more training data, but most importantly, more computation. More computation requires more energy; to win the AI race, AI companies have made ambitious goals to build gigawatts (GW) scale computer clusters. OpenAI’s StarGate project and xAI’s planned expansion of “Colossus” cluster are some of the examples.
How much is one gigawatt? According to U.S. Energy Information Administration, in 2022, an average US household consumes 10,791 kWh per year, which translates to an average power consumption of 1,232W. One gigawatt of electricity would then be sufficient to power over 800,000 households. Orion Solar Belt, the largest solar farm in the US equipped with 1.3 million solar panels, generates merely 875 MW of electricity at peak, short of gigawatt scale.
The gigawatt clusters carry the dream of cultivating superintelligence. Imagine a superhuman model was born after 6 months of flipping and kicking inside the chips of the mother computer cluster, who feeds him/her with energy that equals to what takes to grow 72 million humans from an embryo to an infant (a recent study shows the birth of a baby takes up 50,000 kCal throughout the 9-month pregnancy). Isn’t that a remarkable story of scale if it came true?
But the remarkable scale of human engineering is dwarfed when compared to the scale of nature. If you think of the Earth as a computer cluster, the sun would be its primary energy source, which supplies the Earth with 44 petawatts of power, equivalent to 44 million gigawatt-scale clusters. From the first time DNA appeared on the Earth, life and intelligence has been “trained” for 4 billion years, 8 billion times longer than training an LLM. Unlike ML models which are trained on classical computers, evolution of life runs on truly quantum computers, where molecules move, split and synthesize according to the fundamental laws of physics.
When you have the scale of the Earth, you are lucky to have the perfect initial conditions of the Earth and you can afford to wait for billions of years of evolution, you can make wonders with your models. You don’t need to collect the data and preprocess the data. You don’t need to design the architecture and decide the hyperparameters. You don't even need a loss function. You just let them replicate and mutate, compete or collaborate. In the end, what you get are intelligent agents of the real world, who can sense and deal with the environment with extreme energy efficiency, and you don't just get one model or one type of model - you have created an immense world of intelligent agents. They are all different, but effective and intelligent in their own way.
An Immense World
An Immense World is the title of a book that I recently read. The book is about the senses of animals - the unique sensing capabilities different animals have and how their senses shape their perception and behavior. Though I have known for a long time that nature's creations are wonderful, it was still an eye-opening experience reading the book.
When animals evolve their senses, tradeoffs have to be made to fit their needs given the energy budget. Humans have two eyes facing forward that form a single acute zone. This is actually a quite unique feature of humans and other primates. It gives us a very good perception of depth but limits the area that we can see without turning our head. Cows and some other animals living in flat habitats for example, have their visual fields wrap all around their heads. They give up the perception of depth for a view of the entire horizon at once, which is crucial for their survival in the savannah.
Humans have the sharpest vision among mammals, and are only second to some prey birds like eagles across the whole animal kingdom. But sharp visions require more photoreceptors (light sensors) to be packed into a given area, which means each receptor will receive fewer photons, reducing the eye’s sensitivity to light. That’s why human’s night vision is much poorer compared to a lot of other mammals.
The process of adapting to the environment has pushed some animals’ vision to the extreme. Killer flies possess one of the fastest visions, which is optimized for hunting flying objects. It takes them less than 10 ms to send signals to the brain, process the signal and send commands to muscles. Their brains process more than 350 frames in a second while humans can do only 60, which makes the movies that we watch look like a slideshow to them.
The same story of evolution and adaptation happens to other senses, creating countless examples of amazing, sometimes weird sensing capabilities. The forked tongue is a snake’s smell organ. With flicks of the tongue, a snake can track down the direction of a prey or mating partner by comparing the differences of odor molecules coming from the left and right tip of the tongue. Underground star-nosed moles have turned its nose into 22 tentacles for finding tiny prey and building a mental image of their surroundings. Treehoppers communicate different messages by creating vibrations that travel through plant stems. Weakly electric fish generates a weak electric field around its body and detects small variations in the field with its electroreceptors to locate nearby objects.
Evolution has given animals diverse sensing capabilities that meet their unique needs, but what is even more remarkable is how well these senses coordinate with animals’ actions as a wholesome system. How can we tell how much an object is moving when we are moving around while tracking a moving object? How can fish use lateral lines to detect water movement and pressure gradient to find prey and avoid predators, when their own movements disturb the water and change the pressure? The answer to these questions is the same. When we take actions, our brains have predicted the effect on receptors from our own actions. Having primed with anticipated effect from our own movements, our brain can then isolate the effect from external objects. All these complicated interactions happen continuously in extreme low latency without our awareness.
Despite all these remarkable findings about creatures, much more mysteries are waiting for further discovery. Each species is like a different genre of model that is way more complicated and interesting than a large language model. The immense world will always be a source of wonder and inspiration for human beings.
Who Are We?
In this immense world of intelligent beings, humans are the superstars. Although we are not the best for any single sense modality, our sharp vision, combined with the touch sensitivity and versatility of our hands, make us unrivaled in precise manipulation of objectives. Of course, what makes us particularly special is that 3 pound mush resting inside the skull that enables us to reason, plan, abstract and think outside of the box - capabilities that we might never be able to fully understand.
Humans are unique and special, but at the same time limited. We are limited because we are the product of the tremendous scale of nature, something that we simply can’t replicate. We are limited because nature has defined who we are by giving us our value, our passion and our struggles, and we spend our whole life sorting them out.
But being limited is not a bad thing. As a social animal, our limitation is the source of gratitude, curiosity, love and belonging. Some people accept their limitations and live happily with them. Other people fight to overcome their certain limitations, enjoying the process of fighting while accepting their limitations elsewhere. Only a few people among us long for becoming unlimited and unconstrained. Historically the story of such pursuit never realized nor had a happy ending. I consider technological singularity the newest version of such stories.
In a time filled with hypes and arrogance, I can't find a better way to end this little piece of writing by quoting Newton’s famous words:
I don't know what I may seem to the world, but as to myself, I seem to have been only like a boy playing on the sea-shore and diverting myself in now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered.
If one of the greatest scientists and mathematicians of all time was so humble, maybe we all should be.