It’s fantastic to have a confluence of random things that seem to sync to a particular topic. Netflix now has Arrival, the fantastic film based on a really amazing science fiction short story by Ted Chiang.
I wasn’t familiar with Ted Chiang’s work when the movie first hit theaters — it was mostly: sci fi? Count me in. I was surprised at the focus on the idea of language as the principal tool for these advanced aliens.
Instead of rocket ships and lightsabers, it was intensely focused on, essentially, alien grammar. I won’t say much more about this for people who haven’t seen the movie yet.
Working at Goldman Sachs years ago, it drove me nuts how a hedge fund like Renaissance Technologies was the most successful quant / AI shop in the world, and by far. It always haunted me the hints they gave about how their fortunes in trading turned when they hired the IBM speech recognition software team in the early 90s. There was one cryptic comment by the hedge fund as to why they would hire people from left field, many of whom have never traded a stock in their life.
I am paraphrasing here, but essentially it comes down to: “In trading and in speech recognition, you are always doing the same thing — trying to predict what is going to be said next, whether it’s a price or a word.”
Fast forward 30 years and the basis of success for AI Large Language Models and algorithmic trading is becoming clearer and clearer. As a math major I thought very differently at the time, but it now strikes me that human language itself is a much richer description than the mathematical boolean logic and set theory that underpins it. Both have structure but human language is different. The stories it can tell are richer.
Arrival as the movie, was incredibly ahead of its time — two dueling scientists, one who believes in the power of science and the other believing the glue to civilization is language. It’s an interesting debate that I would have dismissed years ago as the “science guy is right,” but you have to acknowledge that they are called “Large Language Models” and not “Large Math Models” for a reason. And those very, very good at math have no idea why they work so well. That says something.
I’ll posit one final idea for those who have seen Arrival or for those who will see it soon. When people are talking about the advanced GPT styles of AI and their prompts, we always talk about it predicting what word is going to be said next by the user.
The cooler question isn’t what word GPT4 predicts next. The cooler question is when it expects a period, in the sentence.
Is the story over? When does a theme, a story, come to a close? As the sentence and saga unfolds, is the next token going to be a comma, a hyphen, or a period? It’s something humans, and now LLM AIs, seem to determine extremely well.
There might be more to be said about this, but it seems I’ll end it here.
