What does it take to get to the next level of artificial intelligence?
We’ve heard a lot about that in the past year – we saw a perceived crisis in scaling laws, where movers and shakers in the industry were worried that we would run out of power to keep ramping up systems.
But we also had quite a lot of debate about what constitutes artificial general intelligence – when we can say we’ve gotten to that point, and what it means.
One excellent example of this debate comes in the form of an essay written by Thomas Wolf, apparently partially in reaction to Dario Amodei’s ‘Machines of Loving Grace’ essay.
This opinion was covered on the AI Daily Brief podcast, where Nathaniel Whittemore talked about the premise and its ramifications for the future.
Einstein and the Good Student
One thing we know from history is that Einstein wasn’t great in school.
The eminent theorist and mathematician had trouble fitting into the classical form of education in his day. He didn’t appear to be someone who had it all together, and Wolf suggests that’s really the norm.
“History is filled with geniuses struggling during their studies,” Wolf writes. “Edison was called ‘addled’ by his teacher. Barbara McClintock got criticized for ‘weird thinking’ before winning a Nobel Prize. Einstein failed his first attempt at the ETH Zurich entrance exam. And the list goes on.”
It’s a mistake, he contends, to think that you can just scale up good students and get a genius intellect.
I liked Wolf’s analogy saying that Copernicus “went against his training data set” in suggesting that the earth orbits the sun – yes, we know that now, but at the time, it was a revolutionary idea.
“To create an Einstein in a data center, we don’t just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask,” Wolf writes. “One that writes ‘What if everyone is wrong about this?’ when all textbooks, experts, and common knowledge suggest otherwise. Just consider the crazy paradigm shift of special relativity and the guts it took to formulate a first axiom like ‘let’s assume the speed of light is constant in all frames of reference’ defying the common sense of (those) days.”
Citing Jennifer Doudna and Emmanuelle Charpentier and their work on CRISPR, he noted how out-of-the-box thinking can mean using a designed product for radically different use cases.
These rare paradigm shifts, he said, often receive Nobel prizes.
“Real scientific breakthroughs will come not from answering known questions, but from asking challenging new questions and questioning common conceptions and previous ideas,” Wolf adds, invoking the premise of Douglas Adams’ famous “42” anecdote, where we know the answer, but not the question.
“In my opinion this is one of the reasons LLMs, while they already have all of humanity’s knowledge in memory, haven’t generated any new knowledge by connecting previously unrelated facts,” Wolf writes, calling knowledge an “intangible fabric of reality.” “They’re mostly doing ‘manifold filling’ at the moment – filling in the interpolation gaps between what humans already know.”
On the other hand, he suggests, once they become able to transcend that in-the-box thinking, the sky’s the limit.
“We’re currently building very obedient students, not revolutionaries,” Wolf writes. “This is perfect for today’s main goal in the field of creating great assistants and overly compliant helpers. But until we find a way to incentivize them to question their knowledge and propose ideas that potentially go against past training data, they won’t give us scientific revolutions yet.”
Competing in Academics
After the break, Whittemore went into his own personal experience, talking about studying for an academic decathlon where students would study all year.
Whittemore revealed that after competing in the top five rankings nationally for two years, he tracked the top five students in the competition later in life to see what kinds of trajectories their careers took.
“The one thing they all shared was an insane willingness to work hard,” he said. “But most of them, as I would later find out, tracking their time through college and in their careers, were very inside-the-box thinkers. They came from schools that had good programs that knew what to do to turn out champions, and so they put in the work and got out the result … but none of them were disruptors, none of them were entrepreneurs, none of them were builders.”
By contrast, he talked about the kinds of people you read about in tech media, the prime movers who become household names.
“They had a restlessness, a curiosity, a set of qualities that drove them to yearn for more and to be willing to play outside the rules of the system to get it,” he said.
In using LLMs, he noted, we might want to challenge this.
“We assume this straight line between the LLMs of today, which are basically like the best academic decathlon students we’ve ever possibly imagined, having read all the things, studied all the things, and who now can remember all the things and tell you all the things, but who aren’t creating anything for themselves,” he said, wondering aloud if we can get the neural nets to think in different ways. “Given how much we point to scientific achievement and scientific advancement as the universally agreed-upon upside of AI, I actually think these questions are worth pondering, and worth really digging into.”
Multi-Agent Processes
With that said, Whittemore also conceded that we’re in an age of thinking about multi-agent progress with AI. He talked about how the cooperation of agents supercharges network capabilities, and why that will matter in the future.
I personally always cite Marvin Minsky‘s Society of the Mind here, partly because of his MIT connection, but also because it’s a fundamental premise that keeps coming back – that the human brain, as powerful as it is, is not one computer, but hundreds of connected computers, and that to really compete with humans, artificial intelligence entities will need that same interplay and interconnection between multiple agents. You can call it ensemble learning, network knowledge, or group cognition. Or you can just use the overall term “collaboration.”
The upshot is that true AGI might mean systems that can not only get better at answering questions, but also become able to ask them.
