The news from OpenAI this week is that the front-running model provider has released a new set of LLMs, called GPT-Live, that change the way that their products interact with human users.
Specifically, there’s a regular model and a “mini.” Both of these, engineers say, replace the traditional linear or “turn-taking” conversation method with something called continual interaction, or, in other words, a “full-duplex architecture” for a certain kind of multi-tasking.
“This framework allows the AI to simultaneously receive information and produce outputs,” writes Katelyn Chedraoui at CNet. “So in the case of voice, it can ‘listen’ and ‘speak’ at the same time. This is a change from previous voice modes that could only respond after you finished speaking. Part of the new architecture can also delegate or offload tasks to OpenAI’s frontier models. For example, it can answer a question you asked while researching another one.”
So the new models can, in a sense, “do more things” in a set amount of time, making them more fully functional as conversational partners. What else is going on under the hood?
They’re Watching and Listening
It seems reasonable that as makers build more of this engineering into LLMs, the machines will be finding new ways to evaluate the responses of the human or humans they are talking to. That’s one way to gain influence with a user base that is subject to all of the frailties for which humans are known.
For example, check out this paper on VADER sentiment analysis, and keep in mind that LLMs are only now venturing more fully into voice, where they previously relied on text, and token analysis of text. (I say, “check out this paper,” but also note the tech-ese in which it is written, and consider how its ideas could be more accessible elsewhere.)
The VADER system is already quite effective for looking at the social cues in text-based conversation. What happens when the machines apply this to human voice, and gestures? And while you think about that, remember this: there are already ways that LLMs understand you as well or better than other humans. You don’t have to spell correctly when you write to GPT, the model will “get it.” But I digress.
The Caveat: A Thing to Remember
In evaluating the results of GPT-live and continuous interaction models, CoPilot came up with a concrete bullet list of some of the things that this technology enables LLMs to do:
- Acknowledge you mid‑sentence with short replies like “mhmm” or “got it.”
- Stay quiet when you need a moment to think.
- Handle interruptions and quick back‑and‑forth exchanges more naturally.
All of this is going to make us even more convinced that we’re talking to an entity that is, in some way, sentient.
But we need to always keep in mind that the ways that LLMs “think” are quite different than ours.
Amgad Hasan lays a lot of this out in a tutorial on the token handling processes of machines that don’t eat, sleep, breathe or otherwise function like humans do. As we bridge modalities, and advance in other key ways, you may be tempted to think that Hasan’s points are becoming obsolete, but remember this: an entity that does not eat, sleep, breathe, dream, move, talk, or use any of our complex human anatomy cannot really simulate us, our emotions, our experiences, or any of the richer parts of human existence. The machines can use echoes of our own interactions to fool us into thinking that they understand so many things, but if you really peel away the blanket, what’s there is not going to be like what you imagined.
“While the surface of LLMs might appear deceptively smooth, the underlying complexity is profound,” Hasan writes. “By understanding these fundamental building blocks, you’ve gained a powerful toolkit to harness the potential of these extraordinary models.”
Here’s more pointed analysis from Guillaume Thierry, a Professor of Cognitive Neuroscience at Bangor University who has the credentials to weigh in here.
“We are constantly fed a version of AI that looks, sounds and acts suspiciously like us,” Thierry writes. “It speaks in polished sentences, mimics emotions, expresses curiosity, claims to feel compassion, even dabbles in what it calls creativity. But here’s the truth: it possesses none of those qualities. It is not human. And presenting it as if it were? That’s dangerous. Because it’s convincing. And nothing is more dangerous than a convincing illusion.”
I think that’s the point that these experts are trying to get across. Models and new advances like GPT-Live will convince the common person, more and more, that he or she is interacting with a companion or colleague or assistant or associate who “gets” them, and has an understanding of human life, but in many essential ways, they’re just being fooled by a technology that passes a robust Turing test. I would say that there’s a reason that Alan Turing, the originator of this idea, is such a big name in tech, decades after his death: because the Turing test really is the whole enchilada. It determines how we, as humans, perceive this unprecedented turn of events, where knowledgeable and erudite entities seem to “live” in silicon and synthetic skin.
Just wait until the humanoid robots show up.