Saturday Links: Weights & biases acquisition, emergent misalignment, and AI SaaS

March started off with just as much AI news as February, but there seems to be something of a vibe shift: away from big lab announcements for a while and onto usages, infrastructure, and applications. Here are this week's stand-out links:

• CoreWeave acquires AI developer platform Weights & Biases. The Information reports that CoreWeave (the up-and-coming AI Data Centre provider) acquired the MLOps company for $1.7B. Weights & Biases was one of the leading companies providing tracking and process control for Machine Learning, so it's a little surprising to see a sale. On the other hand, the MLOps process has become very crowded with a lot of venture-funded players (Robust Intelligence was acquired by Cisco in September 2024). CoreWeave is rumored to be closing in on an IPO, so it perhaps makes sense that they want to build out the process management around their infrastructure offering (AWS, Google Cloud, and Azure all have layers and layers of tooling for Machine Learning). As Machine learning starts to form part of most software, everyone will need better ways to manage all the models they have in use.
• Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs. One of the most fascinating papers this week. The researchers show that by fine-tuning an LLM on a narrow domain (in their case, numbers or code) containing deliberately false responses, the LLM as a whole could sometimes begin giving malicious answers (even for questions in other domains). It's unclear why this happens, but the high-level effect is something like teaching someone to lie and cheat in a narrow domain leads to similar behavior in all domains. The effect was not always present and varied between LLMs. However, all LLMs that were tested showed some degree of this behavior. The work raises a lot of questions about how to preserve and assess truthfulness. Fine-tuning is common. Even though it is not generally done with false data, we are in a world where a lot of fine-tuned models are available for specific purposes. The trustworthiness of those sources will be key in trusting the models themselves.
• The Klarna // Salesforce Story. Klarna has been one of the most vocal large companies on it's use of AI, including using it to replace expensive software and SaaS products. One of the most quoted items from last year was their announcement that they had replaced their usage of Salesforce.com with a fully in-house solution built with AI. In an X.com post this week, one of the company founders Sebastian Siemiatkowski clarified that they, in fact, did move off Salesforce but also explained how and why. He is at pains to point out that many companies will not do this. However, looking at the pattern, there is clear value in the approach Klarna took, which was to first unify as much company data in one place and then rethink the interfaces and workflows they needed on top of that data. This pattern is what I've mentioned for a while on this blog. SaaS companies either need to become key data repositories for their customers or risk being cut off as companies dynamically recreate just the parts of the workflow they need.
• In a counter move, some people are building entirely new SaaS solutions using AI. Adam Silverman's fun Linkedin post gathers together a few recent examples built with looks like Replit, Lovable, and Bolt.
• Naturally occurring molecule rivals Ozempic in weight loss, sidesteps side effects. A healthcare story, but also an AI story. The researchers use AI algorithms to explore huge numbers of prohormone combinations to find candidate molecules. There is not much information on the exact method used, but results like this are going to increasingly shine through to the population at large that AI has benefits. There's also a long way to go for this treatment to turn into a viable therapy, but exploring nature for more secrets might help ween us off artificial compounds.

Wishing you a great weekend!