Saturday Links: Smart Glasses, Novelty, and Stochastic Parrots

Happy weekend to everyone, and here are the week's most interesting links:

• A Critique of Pure LLM Reason. With the great tagline, "It's parrots all the way down" This article by Eryk Salvaggio has been fairly widely shared (and I encourage you to read it). To me, it's both a very useful description of what's going on inside an LLM (and in particular in OpenAI's new Strawberry model) and at the same time very off-base about the implications. It makes the correct point that any claims of reasoning in the new OpenAI models are rather overblown since what's really happening is a form of prompt iteration. I agree with this, and yes, this means it's "parrots all the way down." In other words, the system is still only doing follow-on token generation, just now in a layered way. What I disagree on is the value of this. As regular readers will know, I suspect that we'll ultimately find out that humans are also just very good, very nuanced stochastic parrots. In other words, we're also collections of knowledge, memories, emotions, etc., and we navigate them when we react to situations.
• Thinking through the future for LLM companies... and what this means for B2B AI startups. This is a great post by Benchmark's Sarah Tavel on startups as they navigate an expanding and improving LLM landscape. I very much agree with her core point that, as a founder of an AI startup (or any startup) today, you need to think through the thought experiment of "what happens if, in 3-5 years, the foundation models are so good, they can readily take on any general human task that's digitized" (my paraphrase). Her answer is: look for network effects, look for proprietary data, and execute fast. All good advice!
• Meta’s Ray-Bans will now ‘remember’ things for you. Meta's Ray Ban smart glasses are still hands down my gadget of the year (though my Rabbit R1 arrived, so we'll see!). A new software update will make it possible to have the glasses take a photo and send you a reminder on your phone via a note later on. (All voice controlled.) These are baby steps, but I think people are underestimating how this gradual bottom-up to rolling out AI tech in real-world settings could put Meta in a very strong position. The are also potentially seeding the user base for their, still "very prototype", AR glasses.
• OpenAI keeps losing executives. It's been another turbulent week at OpenAI, with CTO Mira Murati leaving. Media speculation links the latest departures to changes in the profit / non-profit structure of the firm. It's hard to imagine this is the real trigger, however, since the change has been brewing for a long time. This is all great fodder for press talk, however one has to hope for everyone still there, an AI in general, that the company can get into calmer waters. People love to criticize, but it's hard to keep a high-functioning team together when the stakes are high. It's in everyone's interest that OpenAI manages to do that.
• Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers. This new paper was also referenced by Eryk Salvaggio above and describes a study in which 100+ researchers and LLM-driven systems separately generated research ideas. The researchers then evaluated the ideas for novelty and feasibility without knowing how each was generated. The main conclusion was that some of the LLM results indeed scored high for novelty but also a little lower in feasibility. This really isn't a surprise. LLMs after wall give us a way to explore the space of possible ideas, but they are not yet strong at filtering the results. Alpha Go's famous move 37All the best, and a happy weekend!) no matter how much our human egos want it to be the case that they "can't be creative."

All the best and happy weekend!