Sometime in the 2000s, my buddy Dave acquired a 20Q. It was a translucent red egg shaped device with a small LCD screen and a handful of buttons, about the same feel technologically as those Mattel handheld electronic football games from the late ‘70s.
The player thought of a physical object and the device asked you a series of questions. Is it alive? Can you hold it? Is it used for recreation? You answered yes, no, sometimes, or unknown. Eventually it guessed what object you were thinking of. A tennis racket. A refrigerator. An elephant. And more times than not, it was right.
Dave and I spent hours feeding it different objects and watching how its questions changed, trying to work out the pattern or how its internal algorithm worked. I always assumed it was running a simple decision tree, a big flowchart of branching questions somebody had typed in by hand.
An early neural network
A Canadian developer named Robin Burgener started 20Q in 1988 on a DOS computer. He taught it about objects by having people play Twenty Questions with it.
The program was a neural network. Burgener didn’t script the questions. It learned associations between objects and the answers people gave about them, and stored each one as a numerical weight. Most people agree a tennis racket has strings, gets used in sports, and fits in your hand. The network learned that, along with thousands of other relationships.
As you answered, it used those weights to narrow the field. It also worked backward, looking at the remaining candidates and picking the question that would split them best.
In the mid-’90s Burgener put it online for free. When it guessed wrong, players told it the right answer and it built new associations. By May 2006 it had been played 35 million times and knew about 10,000 objects.
Every one of those games was training data.
The handheld game was a compressed version of that online network: the 2,000 most popular objects and the 250,000 most useful associations between them, burned onto a chip. Radica sold it in toy stores, and it was one of the best-sellers of the 2005 holiday season. While the online version had about 10 million connections, the offline version was comparatively small, but worked for the most commonly guessed objects.
The egg didn’t learn from new games. Its knowledge was fixed, but it still picked each question based on your last answer, with no internet connection.
Twenty Questions meets ChatGPT
20Q and a modern large language model both use neural networks that learn numerical relationships from data, and both can handle situations nobody explicitly programmed.
20Q did one job: guess the object. An LLM is trained on enormous amounts of text and other data, and it writes code, analyzes documents, and holds a conversation about almost anything.
They also learn differently. The online 20Q folded every finished game into its network. When you talk to ChatGPT, the model’s weights don’t change. It can use what you’ve said during the conversation, and memory features can carry information into later sessions, but the underlying network stays the same until the company trains a new version.
Back to the egg
ChatGPT launched in November 2022. Burgener started 20Q thirty-four years earlier, and researchers had been building neural networks for decades before that.
The egg could identify a tennis racket. I asked an AI to explain how the egg worked and compare it to the model I was talking to, and it did, which is how this post started.
When Dave and I were feeding it objects and watching the questions shift, we were trying to find the logic in a neural network trained by millions of strangers.