LLM Helped Me Build My First Mobile App within a Few Hours, but…
The experience gave me clearer ideas on how GenAI will be part of future software development workflow - its benefits and challenges, and how humans and LLMs can work together.
Although I have been writing code for more than 20 years, have built web applications in the past and have lots of random ideas of building good apps, I never seriously considered building a mobile app by myself. That was not only because of how busy my life is, but also because of the opportunity cost of learning a new technology, which I don’t know is worth it or not.
But, LLM has clearly changed the equation. Without any prior knowledge about Android Studio and Kotlin, I was able to build my first mobile app within a couple of hours. LLM generated code & configurations for me, showed me how to navigate the IDE, and helped me debug problems that came along the way. Most importantly, the experience of being able to start tackling your problem without needing to go through hours and hours of tutorials beforehand was just so satisfying.
I guess it is time for me to show my end product - an app called “Pleasantly” for maintaining your reading list:
Yes it has only pretty simple features and the UI is pretty rudimentary. But you know, I am not a UI person and this took me just a few hours, so it is still very satisfactory. The code is here in case anybody is interested.
Strength & Weakness of LLMs
During the creation of “Pleasantly”, the most impressive thing about LMM was that, no matter what my question was - setting up the skeleton, adding new functions, or debugging errors - most of the time LLM answered my questions & requests accurately and comprehensively, with step by step guide, code snippets and explanations. It was an amazing copilot & tutor throughout the journey.
However, while LLM did a great job at the individual Q/A level (which is also what the LLM was mostly optimized for I believe), its quality was not so great when you look at the conversation as a whole. It has lots of repetitions. It lacks consistency - when I asked it to change something, it changed some irrelevant parts as well. In some other occasions, it should have asked clarifying questions instead of making assumptions and generating code with baked in assumptions. So at this stage, LLM helps expand the knowledge of a software engineer, making them much more efficient in using unfamiliar tools, but it needs someone who is familiar with coding to steer through the process and get things done.
Why LLM Degrades in Long Interactions
While repetition & inconsistency in conversations seems easy to solve, optimizing for a long, interactive process will generally be a very challenging problem for LLMs, and this is not due to the limit of context length.
Today’s LLM learns by copying human languages and adjusting to human preferences. When the question doesn’t need multiple rounds of interactions to answer, it is much easier/cheaper to judge whether the answer is good or not and thus easier to provide labels for LLMs to copy. When there is a long interactive process, the number of different ways the interaction can play out grows exponentially. When the conversation gets long, it is also much harder to judge how well the interaction goes, and where it can be improved, thus it is very hard to create good labels for LLMs to learn from.
However, software development is an interactive process where the developer doesn’t know everything that they want before they see the code running, and they need to interact with product, business, users, existing systems and hardware environment to know the next steps.
Why Humans Excel
Humans learn from directly interacting with the world, and to us, other people are part of the world. In those interactions, we build our internal model - we become opinionated and know what is good or bad, right or wrong. A good software engineer doesn’t copy instructions; they know what is the right thing to do and how to do it right and will go beyond what is expected.
In short, the reason that humans excel is that humans learn from the world and LLMs learn from humans, but the world is the ultimate judge!
This is also a good reminder for parents not to give too many instructions to their kids to follow, otherwise, the kids will become only as good as LLMs :)
The Future of Software Development
By now, we can vision a future of software development that harnesses the best of LLM & humans. Except for highly optimized/specialized algorithms & libraries, it looks like LLMs could take care of well contained, clearly specified components, such as implementation of functions, classes. It could be a great tool and brainstorming partner for other parts of software development, such as system design & optimization, data analysis, new product features, but since they involve lots of interactions with the “wild world”, humans will play the major role here.
While there are lots of speculations in what I said above, one thing for sure is that the velocity & capabilities of an individual developer can be greatly amplified. It is going to be a great time for being a builder.