We are living through an impressive state of AI advancement. The progress over the last five years represents a genuine paradigm shift, and there’s no denying its power. But alongside the excitement, many are rightfully concerned about the hype. Can AI deliver on the transformative future we keep talking about?
Table of contents
Open Table of contents
The Linear Growth Assumption
My expertise is in software, and what I see there gives me pause. We often have a linear understanding of progress, assuming that the rapid advances we’ve made will continue at the same rate, if not faster. Some even predict exponential growth, similar to what we saw with search technology or processing speeds. But history shows us that neither linear nor exponential growth is guaranteed.
Consider the promise of fully autonomous, self-driving cars. Five years ago, I would have said that by now, we’d have high-capacity self-driving capabilities. Instead, the industry has stagnated. After a period of incredible progress, we’ve hit a kind of writer’s block, struggling to solve the most pressing questions. I believe we’re in a similar situation with AI today.
AI’s Current Strengths: Rule-Based Systems
It’s not surprising that AI has done so well producing new code. Software is a rule-based system, and AI has been trained on decades of high-quality examples, bug fixes, and best practices. In structured domains like this, it excels. Similarly, it can generate a detailed travel itinerary for London because there is a finite number of places to go and a massive amount of well-structured content about them online.
Where AI Thrives
Current AI systems excel in domains with:
- Clear rules and patterns: Programming languages, mathematical problems
- Abundant training data: Popular topics with extensive documentation
- Structured information: Well-organized databases and knowledge bases
- Finite solution spaces: Tasks with limited, well-defined outcomes
The Challenge of Unstructured Reality
The problem arises when we ask AI to step outside of these sandboxes. What about things that don’t have a lot of content, or where the available information is complex and unstructured? You can’t easily draw a simple conclusion from messy, real-world data. Asking an AI who you should vote for is infinitely more complicated than planning a trip. It involves long-term consequences, the actions of others, and a web of interconnected issues.
This reveals the core challenge: the current AI paradigm is limited by the quality of its training data. We have chosen to scale this technology at a time when misinformation is rampant. If the content available on the internet is often incorrect, the AI’s output will be too.
The Misinformation Problem
The internet, AI’s primary training source, is increasingly filled with:
- Contradictory information
- Biased perspectives
- Deliberate misinformation
- Low-quality content generated by AI itself
This creates a feedback loop where AI systems trained on internet data may perpetuate and amplify existing problems.
The Scaling Question
This brings us back to the question of future growth. People are making bold predictions about AI replacing white-collar jobs, but we don’t know if the current paradigm can even scale to that point. There are indicators that we have already extracted much of the value from the available data.
AI is not yet close to replacing the full cognitive capacity of a human, even in more menial roles like entry-level accounting or analysis. The prevalence of errors, hallucinations, and unreliable information means we still need a human in the loop.
Current Limitations
Despite impressive capabilities, AI systems still struggle with:
- Reliability: Inconsistent outputs and hallucinations
- Context understanding: Missing nuanced implications
- Real-world complexity: Handling edge cases and unexpected scenarios
- Verification: Distinguishing between accurate and inaccurate information
The Need for Another Paradigm Shift
To make its next big leap, AI will most likely need another paradigm shift. Until then, we should be careful. The future of AI may not be a straight line of progress, but a series of plateaus, and we may have just reached our first one.
Historical Parallels
Technology advancement often follows this pattern:
- Breakthrough period: Rapid initial progress
- Plateau phase: Diminishing returns on current approach
- Paradigm shift: New methodology enables next breakthrough
We’ve seen this with:
- Search engines: From simple keyword matching to PageRank to AI-powered search
- Computing power: From increasing clock speeds to multi-core processors
- Transportation: From horse-drawn carriages to automobiles to aviation
Managing Expectations
The key is not to abandon AI development, but to set realistic expectations about what current technology can achieve. We should:
- Recognize current limitations while appreciating genuine achievements
- Invest in research for the next paradigm shift
- Implement AI thoughtfully in areas where it genuinely adds value
- Maintain human oversight in critical applications
Conclusion
The AI revolution is real, but it may not unfold as smoothly as many predict. Like the autonomous vehicle industry, we may be entering a period where the most obvious improvements have been captured, and the next breakthrough requires fundamentally new approaches.
Rather than expecting continuous exponential growth, we should prepare for a more nuanced future where AI progress comes in waves, with periods of rapid advancement followed by plateaus. This doesn’t diminish AI’s importance—it simply means we need to be more strategic about how we develop and deploy these powerful tools.
The future of AI will likely be determined not by scaling current methods, but by discovering new paradigms that can handle the complexity and uncertainty of the real world. Until then, tempering our expectations while continuing to push the boundaries of what’s possible is the wisest path forward.