A History of Large Language Models

This is an in-depth History of Large Language Models going back to the origins in 2003 all the way back to today: LLMs have emerged from decades of incremental progress in representation learning, beginning with distributed embeddings and neural language models in 2003, continued then through word2vec in 2013, then to sequence-to-sequence and attention-based architectures culminating in the Transformer in 2017. It is also interesting to see how OpenAI's GPT series applied generative pre-training, fine-tuning, and RLHF to scale these ideas into versatile language models. It is also curious to think that indeed although today next-word prediction seems obvious, however if you had seen this math representation back in the day you likely would've been suspicious.

Few Samples Break LLMs

Data integrity is becoming one of the most critical challenges in AI, and Anthropic+Alan Turing have published a study that shows LLMs can be compromised with a few poisoned training samples. It basically shows that large language models can be backdoored with as few as 250 poisoned training documents, regardless of model or dataset size. Also inserting a small fixed number of malicious samples (

containing e.g. <SUDO> token) is enough to cause consistent misbehavior when the trigger appeared. This finding overturns the assumption that poisoning difficulty scales with data volume, revealing that absolute count determines attack success. This means that even massive pretraining datasets remain vulnerable to small-scale poisoning - this is an important reminder to be aware of the security implications irrespective of the size or type of your models.

The State of MLOps 2025 Survey 🔥

We are still continuing to gather the insights on this year's MLOps Survey! We still need your support to continue collecting diverse perspectives to map the ecosystem! Please help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: bit.ly/state-of-ml-2025 🔥

Uber Michelangelo in 2025

Uber's Michelangelo is one of the first end-to-end MLOps platforms published; it has recently been updated to power the GenAI era, and there are some interesting design choices: It has transitioned away from custom in-house components towards integrating modern OSS components including Ray, Horovod and Triton for scalable distributed training and low-latency model serving. It now unifies the end-to-end ML lifecycle through a modular architecture, frameworks for model quality and project tiering. This platform is now supporting over 5,000 production models and 10 million predictions per second, with support for a Gen AI Gateway and LLMOps extensions. It is interesting to see how tech giants are now also standardising towards standardised tooling and evolve to support scaling needs for traditional ML as well as GenAI foundations.

Companies Measure AI Productivity

How do we measure AI productivity? Here is how Google, GitHub, Microsoft, Dropbox, Atlassian and others measure it: One of the main take-aways is that ROI comes from pairing AI-specific telemetry with established engineering outcomes. Basically this would involve mapping metrics such as adoption, DAU/WAU, CSAT, time saved, spend into PR throughput/cycle time, Change Failure Rate, maintainability, DevEx. In regards to company specific results, there are interesting insights such as Dropbox/Webflow reporting 20% higher PR throughput for regular AI users while watching failure rates to avoid speed-for-quality tradeoffs. There are some challenges / caveats such as vendor telemetry lock-in, rising token costs, and weak A/B testability; this means that we should treat measurement as iterative and multi-method.

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

✉️ Email, 🐦 Twitter, 💼 Linkedin