Saturday Links: AI self-reflection, compilers and critics

Here are this week's AI links:

• The AI developer... A great step-by-step analysis post by Marie Brayer of Fly Ventures on the progress towards Code writing AI in its many forms. AI and code generation go hand in hand because you have a perfect execution environment against which to check outputs. There's still a long way to go before we have true, reliable, autonomous code generation, though.
• Meta's Compiler LLM: Meta has released two open-source LLMs trained on compiler and assembly code that can compile and decompile software code of various types. This is a big deal since it should help with potential compiler optimizations (though ... I'd add a big note of caution -> errors in a compiler can be extremely subtle and hard to verify; this will need to be coupled with heavy verification), but also maintaining legacy code through analysis.
• CriticGPT, a model based on GPT-4, writes critiques of ChatGPT responses to help human trainers spot mistakes during RLHF. Error correction and self-reflection are heavy on the AI agenda this week. Part of the LLM creation process is the use of layers of human feedback to tune outputs. It turns out humans also make mistakes, especially small, subtle errors. What to do about that? How about giving those humans an extra layer of AI help? OpenAI's CriticGPT does exactly this - flagging subtle errors that humans may have missed. One has to wonder how this will affect the humans in the process. If I know, I'm the ultimate arbiter of correctness and take my job seriously, that may feel very different when I know there is an automated system to step in and catch things. Will I still pay as much attention?
•  Designing Prompts for LLM-as-a-Judge Model Evals. As it happens, The Sequence has a nice guest post this week from Nikolai Liubimov on using LLMs to evaluate LLM outputs (HumanSignal produces a product in this area). Some useful tips and tricks here with some insight into how LLMs work. I'm starting to think we need to build some network of human-verified ground truth if the future of knowledge is within LLMs that are verified by other layers of LLMs.
• Anthropic's Artefacts trends towards new LLM interfaces. VentureBeat covers Anthropic's new UI launch for Sonnet 3.5. This essentially enables users to modify created output side-by-side with the chat window. It's a simple change but important since it requires the model to be more aware of the operating context (and what's been created before); it's also a big step beyond the current LLM interaction mode of (often) retyping the same prompt with variations. VentureBeat seems to herald this as a potential leadership opportunity for Anthropic. I'm not so sure you won't simply see the same UI coming to OpenAI and Google products in a month or two. UI innovations in this space are likely hard to defend.

Wishing you a great weekend!