Raschka on DeepSeek v3.2

Sebastian Raschka has put together the best breakdown of DeepSeek's recent V3.2 model release, and there are some pretty interesting architectural choices that make it clear why it's so far ahead: This release was quite exciting as it's starting to feel like one of those moments where open-weight models are now a serious replacement for frontier closed-source models in real production stacks, and we are also hearing news like OpenAI declaring code red due to lack of moat. I really liked how Sebastian Raschka introduced the architecture choices by actually going through breakthroughs in previous models that actually have now enabled by bringing these all together. The most interesting one was the concept of sparse attention, which he explains how it's different to windowed / sequential attention in that it allows the model to focus on sparse historical tokens instead of being restricted to say the last n tokens. Similarly there are some interesting breakthroughs on the Mixture-of-Experts backbone built on a Multi-Head Latent Attention trick for cheap KV caching, and pushing hard on the RL side mixing classic RLVR for math/code with LLM-as-a-judge rewards and domain-specific GRPO tweaks. Reading through this really shows how this release has been truly achieved by a small series of breakthroughs from previous iterations, and does make it clear that releasing fast really can give the competitive advantage (which many of the larger players are missing - although not google surprisingly!)

DeepMind Nobel Prize Documentary

The recent DeepMind documentary is actually a must-watch! I remember the good old days working in AI back in the mid 2010s, watching how AlphaGo blew minds with move 37... back then there wasn't much hype for the field - funnily enough, the hype was all about crypto 😂 This documentary seems to really capture the excitement on how the hype delivered into one absolutely exciting breakthrough from DeepMind on protein folding, which so far has not yet been processed widely across the mainstream. There are quite a lot of really nostalgic, and particularly exciting details that are covered in the documentary that I wasn't aware of; definitely recommend the watch! Back in these days is when I started working on Responsible AI given how critical it was becoming, I am very much glad I did as although there's been progress, there is still quite a lot to do (and arguably more with the accelerated pace of innovation in the field!).

The State of MLOps 2025 Survey 🔥

In 2025, about 23% of organisations are training using Databricks to train their ML models (vs 18% in 2024), followed by 13% using AWS Sagemaker (vs 14% in 2024), closely followed by a tie of 12% for both Google Vertex AI (vs 8% in 2024) and Azure ML Studio (vs 6% in 2024). It is interesting to see in our 2025 survey on the State of Production ML that there's both a continued trend towards DBX; similarly there is an opposite trend on organisations building their custom in-house training platforms down to 11% in 2025 vs 28% in 2024. The space of model training platforms seems to have become the most consolidated domain across the entire end-to-end ML lifecycle as part of the survey, which aligns with my anecdotal experience. 🥳 The results are shared as open source like last year! You can access them directly at: https://ethical.institute/state-of-ml-2025 🔥

The Race of DuckDB / Polars / Spark

In production ML we're all feeling the pain of "cluster fatigue" - I am glad we are not taking the run-in-my-laptop challenge to the next level with benchmarks of tools like DuckDB / Polars / Spark which are very promising: This is a great experiment - namely asking, can I just process a 650GB S3 bucket on a simple 32GB / 16-CPU machine? The answer is (as always) "it depends - but likely yes" with the right tools, which is great to see. This use-case shows how DuckDB, Polars, and Daft can happily chew through a full-table aggregation without blowing up within ~12–16 minutes for Polars/DuckDB and ~50 minutes for Daft. The PySpark job (untuned) was also able to chew through it in a bit over an hour, but certainly there can be further optimizations that can be introduced. I myself have been playing around with DuckDB in various random projects and I have to say I'm impressed on the performance, and seeing also the integrations that it's providing into data lakes makes it increasible more feasible, definitely recommend checking it out!

Advent of SadServers 2025

In production ML, the real outages usually come not from your models but from the Linux, Docker, and web infrastructure holding them up - this Christmas we can dive into the Advent of DevOps: This is a great initiative similar to the traditional Advent of Code, but focusing on DevOps channels, and although it's not directly and MLOps set of challenges, from experience most of the challenges in production ML systems infra will require these type of drills to address. Some of these challenges are things like debugging an nginx reverse proxy in Docker, restoring a broken stack so the frontend can talk to its backend, fixing file permissions so a normal user can reliably create and edit files across sessions, etc. What better way to spend your Christmas than feeling like you're debugging in production!

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

✉️ Email, 🐦 Twitter, 💼 Linkedin