The AGI Rush
For a long time, I have been working in big tech companies, building & improving machine learning models for products with billions of users. ML engineer teams spend lots of effort improving the machine learning models, and then do large scale A/B testing to see if the change expectedly improves certain “metrics” - some predefined quantity that reflects the business goal. If everything goes well, you will see a small amount of metric improvement, with the cost of multiple months of engineer time to build and hundreds/thousands of CPUs to fully deploy.
This is where the magic of scale comes in. Because of the enormous user base, the aggregated gain (e.g, money they can make) from that small increase actually outweighs the cost, not to mention the benefit of leaving less room for competitors since user’s time and attention are bounded.
Making some changes to billions of users and seeing people’s behavior changing in the way that you expected gives one a god-like feeling. And taking a step further, what if one can collect all the data about the world and the user, and build a giant model? Would it become so good at predicting what users need or want that users will follow whatever it suggests to do? Surely, building such a giant model is very experience, but because it only needs to be built centrally once and can be deployed cheaply to everyone, the scalability of the solution will still make it a very profitable business.
In some sense, this is what tech companies are trying to do around the world today. They collect all the text, images & videos that they can find on the internet, and from simulations or synthesis, with the goal of eventually building the so-called AGI that outsmarts human beings, or at least be as smart as human beings. They are extremely expensive to build, but scalability will eventually make it profitable. Some worry that AGI will kill all human jobs, eliminating the value of humans. Some celebrate that they will liberate humans from labor and create a cycle of exponential technology growth, resulting in extreme prosperity.
Will such centrally built AGIs that can take any human jobs ever exist?
If we believe that human society and the world humans are experiencing is complex enough that it cannot be simulated or approximated by a much simpler system, then such “AGI”s will always be a poor compression of the combined knowledge & experience of human beings. And without constant inputs from humans, the system will get even worse over time as the world & society evolve, because even if the system has a way to update itself, the way that it updates will become less efficient & relevant overtime because the update mechanism itself is built based on poor compression of the past and can’t capture how things will evolve in the future.
For a centrally built system to replace human labor, it has to reside in a constrained environment where the future can be efficiently simulated. Moreover, unscalable solutions need to be added to interface between the constrained environment & the wild world, for the interaction with the wild world has to deal with unique local environments and unpredictable future events.
We have seen lots of such systems in operation. For example, airplanes are centrally built to excel at rapid mass transportation in the air but they need to be operated by localized teams to take care of taking off & landing, where the system interfaces with the wild. When accidents happen, humans need to be involved to investigate and find solutions to improve the system. Sure, more and more AIs can be added to give the system higher & higher levels of automation but at some level, the system has to interface with the environment which is not anticipated by the creator. Finally, some time in the future, a new technology will unpredictably emerge from somewhere by someone and revolutionize the established system.
Human intelligence is unscalable, because there are so many of us and each of us is unique. Our uniqueness comes from our unique genes, from the unique environment we grow and live in, the unique people that we love and interact with. The uniqueness has made us matter to the people and environment around us, and the uniqueness of us has enabled us to discover & invent things that were unknown to established, scalable solutions. This is what happened in the past, and this is what will happen in the future.
Let’s embrace the exciting advances in AI. Let’s expect continuing to be awed by more advances. But let’s not overly trust or be overly obsessed with “AGIs”, or any scalable solutions, because either they are incomplete, or there exists better scalable solutions. If we do that, we are ignoring the unique problems right around us, and we are giving up our unique capabilities to make the world better.