AI is wild, but hasn't changed my model of the world
An attempt to give a semi-sensible opinion amongst the hype
I could be updating insufficiently. I could be discounting new knowledge too severely.11 Conservatism bias.
But my model of how the world really works — and what I expect to happen — hasn’t changed much with the rise in AI prominence.
Of course the surface is different. There’s AI in my emails and task management. It now helps write, and review, nearly everything, for everyone — be it copy or code.
But here are a few things I think are true, and not necessarily novel, which were true beforehand.
The best performers without AI are the best performers with it
Over the past few years, I have not seen someone who was an under-performer rise to over-achiever (due to AI). Equally, I have not seen anyone who was genuinely useful or productive beforehand stop being useful or productive — irrespective of AI-adoption level.
The productive people are productive still, regardless of how little or how much they use it. The unproductive people are still unproductive, regardless of how much they use it.
If you were checked out already, AI has just become a way to get what you would have done anyway a little faster and easier. Busywork and slop.
If you cared deeply about what you produce and the outcomes you get, AI has become a way to create more value and build better things.
Deep domain knowledge or broad contextual awareness are the valuable skills
Everyone now has extensive cross-domain knowledge at their fingertips. Dentists can build their own websites. Carpenters can do their taxes.
But there are limitations, quite possibly through no fault of the model. To extract the full capacity of its software engineering or tax-law knowledge, you need to have sufficient understanding of those fields in order to prompt it correctly. Though, I do believe there are limits to what the model can reach due to the dearth of expert data it can be trained on. For these reasons, deep domain expertise is still going to be highly valuable.
AI is an amazing way to scale expertise, but we still need something like 100 experts before we have enough data for AI to be trained sufficiently so that any one model can reach expert level.
Another place where knowledge is going to be increasingly useful is at the intersections of fields, disciplines, and domains. Really anywhere that knowledge has had an arbitrary boundary placed upon it. This is the principle of consilience. Although at a university the physicists may work and study in a different building to the geologists, that is a division imposed by the institution and not the shared reality they are both trying to understand. Categories exist in the mind not in nature.
It is for this reason that evolutionary biology and psychology had to give rise to evolutionary psychology. Neuroscience and chemistry birthed neurochemistry. The truth of these “new” fields was always there. But it was inaccessible to us due to how we had organised our collective knowledge.
The same continues to occur in the age of AI. By understanding the problems that exist across teams, domains, and industries, that is another place where ingenuity and creativity are needed, and value to be found.
ML is insufficient for AGI
I’ve been referring to AI (artificial intelligence) throughout this post, but you should know it has pained me a little to do so. I’m using that term as it gestures broadly at what I’m talking about — and it is one people recognise — but saying ML (machine learning) would point more directly at it.
All ML is AI but not all AI is ML.
What we’ve been having shoved down our throats seeing over the last few years is almost exclusively ML: a type of intelligence based on statistical models, averages, expectations and updating parameters using backpropagation. None of this is to diminish the incredible impressiveness of these modelling techniques and how useful they can be. I’m no Luddite; this stuff is cool.
My point is that I did not believe ML would get us to AGI beforehand and I still do not believe it will. These models transfer across tasks in ways that genuinely surprised me — but a system that needs the whole internet to get there is interpolating, not understanding. The data appetite is the tell. (The 100 experts from above, and finding a single expert is tricky work.)
These limitations aren’t a hot take. Inside ML they’re close to common knowledge — there are established terms for them, like overfitting, and known engineering trade-offs, like that between precision and recall. It’s in the media (traditional and social) that the hype decouples from reality.22 Not that this is anything new.
The idea that these (again) impressive improvements in ML techniques and architectures is going to give rise to an intelligence explosion seems suspect to me. Are models improving within domains? Yes. Are we applying them to new domains and finding success? Also yes. Are we finding recursive and self-improving feedback cycles? Mmmm, I’m not sure about that.
This is not to suggest we are out of danger with it. My claim is narrower. I’m saying that your YouTube recommendation algorithm could not drive your car. Claude or Chat couldn’t land a spacecraft (even though that was first achieved by software decades ago). We still seem far away from AGI when we actually stop and think about it.
This technology is powerful and disruptive. We are building the intellectual equivalent of atomic bombs — and like the bomb, the danger is not that the thing wakes up, but the force it puts in our hands. What we have currently are narrow optimizers with no model of what we actually want, aimed at a measurable proxy, with near-endless compute33 Which plays a massive role. and given civilisational reach. This is Goodhart’s law, holding the largest shotgun we’ve ever seen. We might be lucky it is only pointing at our foot.
Hence, whether we should build them is still a very open question. But without fundamental breakthroughs in our understanding of intelligence and how to architect it,44 On the order of the transformer architecture by Google in 2017 that gave rise to the current generation of AI. I don’t see AGI, specifically, happening. Maybe it’s inevitable, and advanced ML is what will take us to the precipice of the next era.
AGI (and other terms) are losing their meaning
Until then, though, ML will be playing catch up. And we are the ones determining, consciously or not, what they optimise for. The target doesn’t drift on its own. We are restless, satisficing creatures — we drag it along behind us. ML will learn from the experts — the few that we have — and that knowledge will be codified in models and scaled via products. We can then collectively use AI tools as if we have a team of experts on speed dial.
Then — maybe slowly, maybe quickly — how humans spend their time, and the things they want, will change.55 Proximally, not distally, is my guess. Desire for health, wealth, status etc. will remain fundamental.
What AI needs to do will then change. It will learn from us and we will use it. It will use us and we will learn from it. It will be dynamic and we will continue to co-evolve on a sociocultural-level, if not biologically,66 Remains to be seen. I think we will master gene-editing before the time required for genuine evolutionary changes to occur passes. What that does to our models of biological evolution, I have no idea. with these tools. And the more symbiotic the relationship becomes, the more meaningless terms human-level intelligence and AGI become.
This is all based on my limited knowledge and experience. If you’re a researcher, engineer, investor or anyone else who is seeing things that my view isn’t accounting for I would love to hear about it. This isn’t an opinion piece. If I’m wrong, I want to know it.