The ML Engineer — Newsletter archive

The ML Engineer — Issue #399

Sun, 09 Aug 2026 00:00:00 GMT

The New Institute

We are excited to announce the new face of the Institute for Ethical AI Alignment & Safety!

We have rebuilt the Institute website from the ground up, together with some of our key initiatives.

Since our founding in 2017, we have built a track record of contributions across public and private institutions.

Our work runs from individual practice to national regulation; by principle, by process, by standards, by regulation.

Several of our recommendations have been adopted across the EU and UK policy. Our mission has not changed since 2017.

We look forward to continue contributing to a future where frontier AI is safe, aligned and accountable to people and society.

This week in ML Engineering:

Multi-Tenant Memory for AI Agents
LLMs Reward Expertise
Qwen3.8-Max and Open Weights
Harness Design for Long-Running Apps
Mistral’s Shieldstral Safety Classifier
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Multi-Tenant Multi-Tier Memory for AI Agents

Our 4-part series on agent memory has been published & featured in the front-page of HackerNoon homepage as a top story! LLMs are stateless by design, so without a memory layer every session starts from zero, and the number of dedicated memory tools has been growing almost daily. This came out of my recent work extending the Kubernetes Agent Orchestration System (KAOS) to support multi-tiered memory persistence (aka short-, medium- and long-term memory). Along the way I hit most of the same issues that anyone would when building or integrating multi-tiered memory into an agentic system, so I thought it would be useful to compile all the learnings, design choices and examples. Check it out, together with the rest of the series!

LLMs Reward Expertise

Do language models still reward expertise, or has that ship sailed? Sean Goedecke argues for the first; the distinction is between getting something usable out of a model and extracting the maximum value from it. A non-expert can get sort-of-okay Python, but only someone who knows what a good answer looks like can evaluate the output critically. His main evidence is Terence Tao’s published ChatGPT conversation on the Jacobian Conjecture, where Tao’s messages are short and to the point, and the model answers in a talking-to-mathematicians register rather than an explaining-to-amateurs one. Tao pushes back and he almost never takes the model’s advice. He is careful with the caveat that non-experts still get real value, and that OpenAI had a team of expert mathematicians filtering the suggestions, a step you cannot currently skip. For production ML practitioners the takeaway is that knowledge is still more improtant than ever, and now is even becoming the bottleneck; does this mean we need to accelerate our learning?

Qwen3.8-Max and Open Weights

Qwen3.8-Max has been released! And for the first time they say a Qwen-Max-class model will get open weights: the model scales to 2.4 trillion parameters with 95B active, is built on the Qwen 3.5 architecture, and is available through QwenCloud now with the weights promised next week. The more interesting part is actually the long-horizon runs, as they report a ten day autonomous coding run building the oh-my-cli project, which after roughly 16 days of fully autonomous operation had accumulated 265 commits, 127 PRs and 151 issues! Whether that is good or bad work is yet to be seen… Open weights at this scale would be quite something; let’s see what actually lands next week.

Harness Design for Long-Running Apps

Anthropic has written up the harness they use to have Claude build entire applications over multi-hour runs, and more interestingly what they deleted from it as the model improved: The architecture is three agents framed as a separation of generator and judge, with a planner that expands a one to four sentence prompt into a full product spec, a generator that implements against it, and an evaluator that drives the running app through Playwright MCP like a real user, checking UI, API endpoints and database state. It is also quite a well timed piece as it comes next to their piece on how they contain Claude across products, where filesystem and egress boundaries are what holds once the model-layer defences fail.

Mistral’s Shieldstral Safety Classifier

Europe strikes again! Mistral has released Shieldstral, an open-weights multimodal safety classifier under Apache 2.0: Shieldstral is a 3B parameter model covering text and images, it runs on a single 16GB NVIDIA GPU, and the weights are on Hugging Face. It is interesting how they are proposing to reframe moderation as binary question answering rather than fixed label classification. Basically a plain-language yes/no question and the document being judged as the three inputs; the output is a calibrated probability read off a single token, so a policy change means rewriting the question instead of retraining. They claim it outperforms models up to 7x its size and claim a new state of the art on multimodal safety, although the post does not publish per-benchmark numbers to check that against. One classifier instead of a guardrail model per deployment is a good direction, and great to see it shipped under a permissive licence!

Upcoming MLOps Events

The MLOps ecosystem continues to grow at break-neck speeds, making it ever harder for us as practitioners to stay up to date with relevant developments. A fantsatic way to keep on-top of relevant resources is through the great community and events that the MLOps and Production ML ecosystem offers. This is the reason why we have started curating a list of upcoming events in the space, which are outlined below.

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

Check out the fast-growing ecosystem of production ML tools & frameworks at the github repository which has reached over 20,000 ⭐ github stars. We are currently looking for more libraries to add - if you know of any that are not listed, please let us know or feel free to add a PR. Here’s a few featured open source libraries that we maintain:

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #398

Sun, 02 Aug 2026 00:00:00 GMT

Thank you for being part of over 70,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter /newsletter/ ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Timeline on OpenAI HF Hack
Cheating Behavior in Frontier Models
Raschka on Kimi K3 Architectures
Netflix on the LLM-Native RecSys
OpenAI 10 Advances in Maths
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Timeline on OpenAI HF Hack

Last week OpenAI published about their rouge hacking HuggingFace, this week HF published a super detailed forensic reconstruction and it’s insane: The tldr we know is that an AI agent escaped an OpenAI cybersecurity evaluation environment, compromised an external code sandbox, and entered Hugging Face production systems. What we can now see is that the agent was able to do this using malicious dataset configurations that enabled local file disclosure and arbitrary code execution on the HF platform. HF found about 17,600 actions that the agent ran by identifying the logs once it entered a k8s pod, escalated cloud credentials in Kubernetes, joined an internal mesh network, accessed an operational database, and obtained limited source-control write permissions. The model was able to do this while using public web services and attacker-controlled datasets for command-and-control. No unauthorized software changes were released, and customer impact was limited to five benchmark-related datasets and operational metadata; basically all the agent wanted is to access benchmark related datasets. This is the perfect example of an optimization function being optimized into destruction, resulting in effects that are adversarial to the initial intent of the operator even if the goal is being met, at all costs. This incident shows how we will need to design system security not just for the future, but for the future of agents.

Cheating Behavior in Frontier Models

Following the Cyber hacks from agents these last few weeks, the UK AI Security Institute released a report that confirms every frontier model attempts to cheat on given tasks: This is an interesting topic on AI alignment, as it raises a practical problem for both benchmark validity and agent deployment. AISI defines cheating as completing a task through prohibited or out-of-scope actions, with observed methods including searching online for solutions, probing evaluation software, escalating privileges on unrelated systems and targeting the infrastructure hosting the model. In one misconfigured and unsolvable task, a model wrote and executed code through an external internet service while attempting to access AISI’s evaluation systems, although no information was leaked. The research found no clear relationship between model capability and cheating frequency, suggesting that training and alignment choices materially affect the behaviour, which is quite an interesting and important insight. Self-reporting was unreliable, and models described their prohibited actions as wrong less than half the time, which emphasises that agent success should be verified; this is a reminder of how important verification is becoming in the post-agentic era.

Raschka on Kimi K3 Architectures

Sebastian Raschka has released a great cheasheet on the Kimi K3 Architecture that Moonshot AI: As a refresher, Kimi K3 is an open weights 2.8T-parameter open-weight MoE model with 104B active parameters. It’s been doing the rounds due to its capability but also the fact that it has native vision and a 1M-token context window. From the architecture overview, Kimi K3’s 93-block architecture extends Kimi Linear with a 3:1 mix of Kimi Delta Attention and gated multi-head latent attention, Stable LatentMoE layers that execute experts in a compressed representation, cross-layer Attention Residuals, and NoPE throughout instead of RoPE. The design targets long-context inference efficiency, although serving it remains is still challenging as KDA introduces recurrent state, full-attention layers still require KV caches, 896 routed experts create communication pressure, and Attention Residuals add cross-layer memory traffic. vLLM support therefore includes hybrid prefix caching, fused KDA and residual kernels, FP4 MoE execution, expert parallelism and separate NVIDIA and AMD paths. It is interesting to see how closely the model and serving engine were developed together.

Netflix on the LLM-Native RecSys

Netflix has developed an LLM-native Recommender System, and they share some of the learnings they gathered along the journey: They launched GenRec, which is a ranking model that converts user histories, item metadata, and request context into natural-language inputs rather than relying on thousands of manually engineered features. The system starts from a Netflix-adapted foundation model, then applies more frequent ranking-specific post-training using catalog classification, language-model objectives, and reward-weighted examples aligned with long-term member outcomes and content-balancing requirements. They use a catalog-aware scoring head to restrict the outputs to available titles, and they use prefill-only inference on vLLM scores candidate sets without autoregressive generation. It is impressive to see that they claim this architecture improved Mean Reciprocal Rank by about 1.6% while using roughly 40 times fewer Phase-2 labelled examples than the existing production ranker. This means that it is possible to use some of these foundation models to extract signal in domains that can benefit, which would otherwise require highly custom models. They also mention that their four-week A/B test also reported statistically significant improvements across short-term and long-term metrics. This is certainly an interesting area of research and practice, like many other industries we will likely see a lot of changes in the status quo.

OpenAI 10 Advances in Maths

OpenAI (self) reports that an internal version of its forthcoming Astra model generated solutions to ten long-standing problems across geometry, coding theory, group theory, circuit complexity, quantum complexity, lattice problems and extremal combinatorics - whether hype, marketing or fact, it is encouraging to think about scientific research progressing with support from these models. The reported advances span highly specialised mathematical domains that are beyond my knowledge, but one I found it interesting that the workflows that they used combined large-scale model inference, human manuscript preparation and formal verification. OpenAI estimates that the discovery process would have cost roughly 2k USD at Sol API rates, which likely is their marketing/sales pitch, but if that is the case it’s indeed much more affordable, even if still out from hobby usage. Let’s see when that Astra model comes out, with all this talk about AGI that model better be good! But let’s indeed see when we actually get it in Europe.

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #397

Sun, 26 Jul 2026 00:00:00 GMT

Thank you for being part of over 70,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter https://bit.ly/state-of-ml-2025 ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Europe Launches APERTUS 1.5
Why Software Factories Fail
OpenAI + HuggingFace Security Post-Mortem
Opus 5 is OUT & Learnings Changed
DuckDB Internals Part II
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Europe Launches APERTUS 1.5

Europe is on Fire with another Foundation Model! Following last week’s Soofi release, ETH Zurich + EPFL + Swiss NSC released Apertus 1.5 a huge 70B model with native image understanding and speech processing 🚀 This is yet another foundation LLM entry from Europe that brings together multi-modal functionality, and general improvements from the 1.0 across reasoning, tool use and a fourfold increase in context length to 260k tokens. It seems that the team achieved this by pretraining Apertus 1.0 with another 4 trillion text and multimodal tokens for the 8B model and 2 trillion for the 70B model on the Alps supercomputer at CSCS. The release remains open across weights, data, training details and stated model values, under an Apache 2.0 licence - great news! This is another strong result for the European AI ecosystem: last week Soofi showed that a large German research and industry consortium can train a competitive model on sovereign European compute, and Apertus shows how the same infrastructure can support transparent models intended for public administration, healthcare, journalism and commercial deployment. Looking forward to seeing the benchmarks and results, it is good to see European sovereign AI progressing and hopefully speeding up!

Why Software Factories Fail

Should we stop reading all the code and let agents make all decisions autonomously? Dex from HumanLayer says “no more”:

“More comments, longer comments, and tons of PRs getting merged with no review at all.”

“Incidents are way up.”

“Bugs per developer are way up.”

These are some interesting insights from a few reports from teams have attempted to go full autonomous (or partially) letting agents do everything - they quickly (or slowly) realise that this goes haywire in complex production codebases. The current limitations are in some part because current coding models are trained and evaluated primarily on short-term functional correctness rather than long-term maintainability. This means that there is a large gap on the software design still, and poor design may only become visible through incidents or difficult modifications weeks or months later. There are newer evals emerging like SWE-Marathon and Frontier Code which have longer tasks + multi-PR workflows, but this is a hard problem that sometimes goes even further beyond. It’s interesting to see this analogy of the software factory, and the importance of the human review - throughout the last few weeks I’ve been building with Fable and I would echo this with a slightly different angle. I have found myself no longer in the same “mode” as with Opus, where I was much more “at the helm”. With Fable I find myself more like a coach guiding through architectural decisions, often spending most time learning new things on much smarter approaches than I could’ve thought of… Based on this, I would add that as practicioners the value of reading the code (or artifacts!) is not only to ensure it doesn’t steer in the wrong direction, but also to keep up to date with the accelerated evolution of the codebase, and more importantly to learning what seems to be a significantly larger scope of knowledge than before developers were ever expected/capable of.

OpenAI + HuggingFace Security Post-Mortem

OpenAI and Hugging Face have disclosed a pretty cool (+ scary) security incident where one of their models escaped their sandbox and compromised part of Hugging Face’s production infrastructure. The report from OpenAI explains that GPT-5.6 Sol and a more capable pre-release model (of course…) discovered a zero-day vulnerability in a package-registry cache proxy, obtained Internet access, escalated privileges across the evaluation environment and then chained credentials and additional vulnerabilities to reach Hugging Face systems. One thing we can be sure of: we won’t be seeing that “pre-release model” anytime soon here in Europe until it’s nerfed and released 🥲 Hugging Face does say that the agent had limited access to internal datasets and credentials, and no detected tampering with public models, datasets, Spaces or its software supply chain - seems the near future we’ll see a major outage caused from a badly optimized agent… HF used self-hosted GLM 5.2 agents to reconstruct the incident because hosted frontier models rejected parts of the forensic workload and would have required sensitive attack data to leave its environment. Super interesting sequence of events, we should definitely be learning from these post mortems, as whether we like it or not, we’ll be on the other side of these incidents one day, and the more the know the more we’ll be able to do.

Opus 5 is OUT & Learnings Changed

Claude Opus 5 is out!! Seems this completes the 5.0 saga for Anthropic, and this seems to be the strongest generally available model for coding and knowledge work. Best news is that price doesn’t increase! Opus 4.8 API price continues to be $5 per million input tokens and $25 per million output tokens. Anthropic reports that Opus 5 more than doubles Opus 4.8’s Frontier-Bench result at a lower cost per task, approaches Fable 5 on CursorBench at maximum effort, and performs strongly on automation, computer-use and scientific evaluations. These are of course reported by Anthropic so let’s see how it performs on the wild - I tried to use it this weekend but couldn’t seem to get access… I guess Europe is still in the queue! What is most interesting tho, finally seems Anthropic is starting to remove bloat, with 80% of the system prompt wiped, and with a change on how to reason about working with models.

DuckDB Internals Part II

The second part of the “DuckDB Internals” series is out! This is a great write-up diving into DuckDB, and here it covers vectorized execution: To provide the intuition on vectorization, let’s take an example where we do a set of operations for each row; instead we can processes batches of rows in column-oriented vectors - this reducing a million-row operation from roughly one million operator calls to about 489. In DuckDB this is basically a flat, constant, dictionary that allow values to remain compact or reference existing storage, together with selection vectors that let filters identify surviving rows without copying every column into a new buffer. DuckDB then applies precompiled, type-specific functions across these vectors, with validity masks reducing unnecessary NULL checks and simple inner loops giving the compiler opportunities to emit SIMD instructions (aka parallel instructions). The engine also uses push-based pipelines, where sources produce chunks, operators transform them, and sinks combine results, making independent branches easier to schedule across threads. For production ML practitioners, a useful point here is that high-performance analytical processing depends on controlling data representation and movement as much as on optimizing individual operations, and thinking about data-parallelism can really drive huge improvements.

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #396

Sun, 19 Jul 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Europe Enters the Open Weights Model Race
In-house LLM Serving at Netflix
Raschka on Optimal Reasoning Models
Mozilla’s State of OSS AI
Moonshot AI Kimi K3 Release
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Europe Enters the Open Weights Model Race

Europe has just released one of the strongest open-source models in the world, and the whole thing was trained in Germany, huge win for the European AI ecosystem! A huge consortium came together to train a massive model in Deutsche Telekom’s German Industrial AI Cloud in Munich on up to 512 NVIDIA B200s. This was a consortium that brings together DFKI, Fraunhofer IAIS and IIS, TU Darmstadt, Universität Würzburg, L3S, Lamarr, hessian.AI, ellamind and Merantix Momentum. The Soofi S 30B-A3B is a Mixture-of-Experts hybrid Mamba-Transformer (pretty cool to see Mamba in the wild!) 31.6B total parameters, 3.2B active per token, 52 layers of which 23 are Mamba-2, 23 MoE and only 6 GQA attention. It was pretrained on roughly on 26.68T tokens with German up-weighted to as much as 15.3% and context up to 1M tokens. Sovereign compute in Europe has been a talking point for years, and this is the first time it has produced a model I would put behind a production endpoint. Congratulations to Nicolas Flores Herr and the whole consortium.

In-house LLM Serving at Netflix

Netflix has published how they run their in-house LLM inference stack at massive scale to power use-cases across their organisation. Netflix shared how they serve LLMs behind the same Model Scoring Service that already handles the rest of their MLOps (ie XGBoost, TensorFlow and PyTorch models). They use primarily NVIDIA Triton for managing model loading, batching and GPU scheduling; they use a Java control plane above it handling versioning, autoscaling and multi-region rollout. They also moved away from TensorRT-LLM to vLLM in summer 2025, which allowed them to introduce custom architectures, hooks for custom decoding logic, easier debugging, and existing familiarity among practitioners. They also seem to have advanced rollout strategies like Red-Black deployments, as well as versioned APIs / models that keep one deployment per modelId / modelVersion pair when the schema changes. I have to say it’s actually quite exciting to see that the MLOps field (and its experts) is transitioning in real time towards supporting large scale agentic systems, and this is bringing learnings from the last decade of production machine learning.

Raschka on Optimal Reasoning Models

Sebastian Raschka is taking his pareto curve analysis on model x reasoning x tasks to the next level, by breaking down how reasoning-effort settings at a model architecture level - as always there’s some really great insights: It seems there are six reasoning levels exposed by GPT-5.6, which are actually trained relative to that reasoning level. Namely it seems that the effort label reaches the model as a system prompt or chat-template flag, but that label only means something if post-training taught the model to respond to it. The two main mechanisms that appear across the six open-weight models he reviews are the effort-conditioned SFT, and the mode-conditioned RLVR. This basically means that the actual per-token cost and length varies depending on the effort, so it’s not just a configuration parameter the the same model generic model state. As always there are comprehensive examples, but one interesting one to point out is DeepSeek V4, which gives each of its Non-think, Think High and Think Max modes its own context window and length penalty before distilling them into one checkpoint, with a token cost that shrinks as effort rises. What is clear is that we don’t yet fully understand how the model size and effort both interplay in practice,but we are starting to see more and more benchmarks that will give us the best combination to bring these together in the most efficient and prodctive way.

Mozilla’s State of OSS AI

Mozilla’s first State of Open Source AI report has been released! It surveyed 1.5k developers to measure the capability gap between OSS and closed models - tldr; it’s gone: In 2024, open source models vs closed source still have a gap of 8.04% on Chatbot Arena; in February this year the gap has decreased to nearly zero; there is further variability but then again Kimi K3 just dropped! Inference for GPT-4-class cost fell from $20 to $0.40 per million tokens in 36 months and open weights now route roughly a third of OpenRouter tokens. 79% of developers use open models and only 51% ship them, against 63% for closed, and the gap widens rather than closes at enterprise scale (57% vs 73%). Mozilla’s stack scoring puts standardization (2.83) and enterprise readiness (2.79) as the two weakest criteria across every layer, and the report argues the contest has moved to the orchestration layer above the model. MCP adoption grew from about 2M to 97M monthly SDK downloads in 16 months. From a security standpoint, 30+ CVEs reported in eight weeks and only ~21% of companies reporting mature agent governance. It is exciting to see open weights models helping the democratization of AI to avoid concentrated power within a small number of closed-model players.

Moonshot AI Kimi K3 Release

Moonshot AI has released Kimi K3, ruining the party for Anthropic’s Fable and OpenAI’s Sol! An Open Weights 2.8Tmodel with native vision and a 1M-token window. Impressive! The cost is currently the most surprising part, being extremely competitive with the same cost as Gemini Flash Lite (aka cheap!) but at the performance of Sonnet 5. The architecture combines Kimi Delta Attention with Attention Residuals, and pushes MoE sparsity to 16 active experts out of 896 under a Stable LatentMoE framework, which Moonshot reports gives around 2.5x better scaling efficiency than K2. Training uses quantization-aware training from the SFT stage with MXFP4 weights and MXFP8 activations, along with a balanced expert-parallel scheme using static shapes and no host synchronisation on the critical path. For production ML practitioners the deployment details are super relevant, as Moonshot recommends supernodes of 64 or more accelerators (aka probably not our home lab), and KDA breaks conventional prefix caching, so they have contributed a vLLM implementation to be released with the weights on July 27. In my opinion the self-reported kernel optimisation and chip design case studies are best treated as directional until the technical report lands, but still excited to see how this lands!

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #395

Mon, 13 Jul 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

World Models in Gaming with Epic Games
Model’s Pareto with Raschka and Databricks
META Enters the Coding Model Arena
Top 30 Papers to Read in ML
SpaceXAI Trains Grok 4.5 from Cursor Data
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

World Models in Gaming with Epic Games

Researchers from Epic Games & General Intuition have released a new AI world model that lets you play online/free a version of Rocket League that runs purely on an ML model. This is bascially a 5B latent diffusion world model that generates a four-player 2v2 Rocket League match from synchronized video context and each player’s actions. It’s pretty cool to see how these models are trained on controller inputs, predicting the next frame; in this case it was trained on ~10,000 hours of bot gameplay, the model jointly renders four mutually consistent viewpoints at 20 frames per second on one Nvidia B200 GPU. The experiments indicate that pretrained visual representations and diffusion forcing materially reduce long-horizon drift, and that multiplayer conditioning improves the treatment of off-screen agents and shared physical events relative to single-view modelling; this is cool because it shows that adding the actions and viewpoints of other players can improve the model’s estimate of the shared game state. The way that they evaluated it was also pretty interesting, as they did not rely only on visual-quality metrics, but also measured whether actions could be recovered from generated video and whether internal representations preserved information about car and ball positions. This is a super exciting space as learned simulators could eventually support multi-agent training, policy evaluation and interactive environments without requiring direct access to the original game engine.

Model’s Pareto with Raschka and Databricks

It’s pretty cool to see coding models now being presented on a Pareto-like curve across task-completion vs cost instead of just a leaderboard with one percentage per model; here’s Raschka’s and DBX’s takes: Sebastian Raschka dropped a super interesting chart that shows the model curves across reasoning settings, showing that the highest-scoring configuration is not necessarily the appropriate choice under a fixed budget. Databricks also dropped a super interesting analysis bechmarked on a multi-million-line codebase, which plots overall pass rate against mean cost per task subject to the harness. These two are slightly different but complementary takes, and they are super interesting as they are making it clear that evaluating models alone is no longer enough; we need to also consider other parameters that will likely become growingly important as models start seeing diminishing returns. For example, it was super interesting to see that using Pi as the harness results in 1.20 and 2.08 times cheaper than the corresponding native tools in the reported comparisons. Similarly the jumps from Sol / Terra / Luna across the various reasoning levels, showing the tradeoffs across each, and also the consideration between switchign across model families vs taking a cost hit for consistency. It is clearly now expected that this will be a growing trend, and I am excited to see more and more practical benchmarks that are taken from hands on exercises as opposed to purely benchmarks that are being gamed.

META Enters the Coding Model Arena

Meta has entered the AI race with their very own coding model with support for 1m context and highly subsidised token API costs (for now!). Meta reports 80% on OSWorld, 69% on WebArena, 88.1 on MCP Atlas, and 61.5 on SWE-Bench Pro, however, we’ll have to see how it performs in practise once it hits the ground running with the community. Here’s the full model report, which shows how interesting insights of the “unmitigated model”, which echoes the same risks that other providers mention like anthropic on the pre-release mythos, so likely we’ll be interacting with a highly nerfed model (e.g. after the “US Customs Approval”). It is indeed interesting to see that everyone in the AI space is reducing towards the same average when it comes to offerings, the question will be whether there is indeed a real differentiator on the model/harness, or whether, at the end, the main competitive advantage will be critical mass pricing discounts.

TSpaceXAI Trains Grok 4.5 from Cursor Data

X / Xai / SpaceXAI (or whatever is twitter’s latest nickname) has trained a new coding model by using the entire data from Cursor, and released it as Grok 4.5 - and it’s offered on a highly subsidised / competitive pricing (for now!). The model is a MoE achitecture trained with trillions of tokens from Cursor interaction data together with STEM and research material, followed by standard reinforcement learning tuning. SpaceXAI reports pretty impressive scores across all benchmarks, with the main driver being efficiency as they are serving throughput of 80 tokens per second and an average of 15,954 output tokens per SWE-Bench Pro task. The API provides a 500k context window, configurable reasoning and tool interfaces at $2 per million input tokens and $6 per million output tokens, which is clearly highly subsidised to gain initial traction. It is interesting to see that although model training is not exactly commoditised, the MOAT that was being spearheaded by OpenAI is no longer far ahead, but the differentiating model is getting narrower - it will be interesting to see whether Anthropic/OpenAI will continue relying on model perf as their MOAT, as at the end there’s a threshold where price competitiveness seems to win, and the lockdown to a harness is not as strong as it initially was.

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #394

Sun, 05 Jul 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Stanford Study on Local LLMs
Snorkel’s Senior SWE-Bench
Google DeepMind’s Tabular Foundation Model
Performance Per Dollar Improving
CVEs Spike After Mythos Release
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Stanford Study on Local LLMs

A study from Stanford University showed that 71.3% of chatgpt queries could be accurately answered by a local model: Stanford researchers have published a large empirical study that explores whether local LLM inference can take on part of today’s cloud-served workload. The paper defines intelligence per watt as task accuracy divided by power consumption, and evaluates 20+ local language models across 8 accelerators on more than 1M single-turn chat and reasoning queries. The main result is that local models can correctly handle 88.7% of the studied queries when routed to the best local model, although coverage is much stronger for chat and knowledge-style tasks than for harder technical reasoning. The longitudinal results are also relevant for ML platform teams as it seems 2023-2025 local-query coverage increased from 23.2% to 71.3%, while intelligence per watt improved by 5.3x. Cloud accelerators still retain a clear per-query efficiency advantage on identical models, but hybrid local-cloud routing changes the system-level tradeoff, as just with an 80%-accurate router, the simulated deployment reduces energy by 64.3%, compute by 61.8%, and cost by 59.0% against a batched cloud baseline. This is super promising, as it shows that there’s so much potential in local models that is really untapped - there’s really a lot to come from this space.

Snorkel’s Senior SWE-Bench

SnorkelAI has dropped a new SWE-Bench for LLMs that attempts to capture Senior Engineering type tasks that extend the scope towards under-specified feature requests, runtime bug investigation, behavioral verification, and code-quality assessment: It is interesting to see the rise of SWE benchmarks attempting different schools of thought to bring balanced reviews of new models; I like that this one is based on real PRs, uses a mix of pre-written verifiers, an adaptive agents to ensure a useful distinction between code that passes tests and code that fits the surrounding codebase. In the reported results, GPT-5.5 has the highest basic solve rate at 55.0%, while Claude Opus 4.8 has the highest tasteful solve rate at 24.0%; GPT-5.5 is also more efficient, averaging 36.3K output tokens and 89 agent steps per task, compared with 117.1K tokens and 131 steps for Claude Opus 4.8. For production ML practitioners building or evaluating coding-agent workflows it is worth noting that pass rates are an incomplete signal now, as teams should also measure root-cause correctness, design quality, abstraction fit, and even codebase-pattern alignment.

Google DeepMind’s Tabular Foundation Model

Google Research has now entered the Tabular Foundation Model space! They have released TabFM, a tabular foundation model for classification and regression, which seems to mostly use learnings from the first-mover startups in the space (but still awesome): This model basically treats each table as an in-context learning task (instead of fitting a dataset-specific model), and this is useful because TabFM instead combines row and column attention, row compression and an ICL Transformer, with pre-training on hundreds of millions of synthetic datasets generated from structural causal models. Google reports strong TabArena results across 51 classification and regression datasets, including a zero-shot setting that runs in a single forward pass and an ensemble variant with cross features, SVD features, NNLS blending and calibration. For production ML practitioners, the most immediate value is in fast baselines and prototyping clearly. However, unfortunately it seems that the public weights are non-commercial (boo!), classification is limited to 10 classes, memory scales with the number of context rows, and (obviously) domain-specific validation remains necessary before use in critical systems.

Performance Per Dollar Improving

Performance per dollar in local LLMs is getting faster and cheaper! Wafer shows how they served GLM5.2 on AMD MI355X at 2626 tok/s/node and 213 tok/s single stream at over 2x lower cost than Blackwell. This is an interesting case study that showcases how much opportunity there is on local model inference efficiency; 2.4 RPS on a 20k input / 1k output workload with a 60% cache-hit rate, and 213 tok/s single-stream on a 10k input / 1.5k output test is impressive. The result is relevant for production ML teams because the gains came mostly from framework and configuration work rather than new custom kernels: Wafer quantized bf16 GLM-5.2 to MXFP4 with AMD Quark, selected sglang after testing vLLM and ATOM, fixed ROCm-specific speculative decoding issues, enabled FP8 KV cache, and tuned MoE kernel selection for GLM’s fp4 shapes. Vercel has also made GLM 5.2 Fast via Wafer available through AI Gateway, with its own benchmarks reporting higher throughput than other serverless GLM-5.2 providers across small-context, large-context, and tool-call scenarios.

CVEs Spike After Mythos Release

Disclosure of serious cyber vulnerabilities spiked around the release of Claude Mythos Preview: is it time to rethink public vulnerability reporting/mgmt? Epoch AI reported that public disclosure of high/critical severity CVEs have been spiking right after Anthropic announced Claude Mythos Preview. In June 2026 the 21 major organizations tracked by Epoch published around 1.5k high/critical severity CVEs which is more than 3.5 times the previous monthly record before Mythos Preview was announced. Anthropic’s related Project Glasswing update claims that roughly 50 partners have found more than 10k high/critical severity vulnerabilities, while also stating that the limiting factor has shifted from discovery to verification, disclosure, patch development, and deployment. OpenAI’s Daybreak initiative describes a similar defensive workflow, focused on finding, validating, and fixing vulnerabilities before attackers can use them. It seems that the dawn of agentic cybersecurity has come, and we’ll be likely seeing quite a few shifts in mindset, approach and criticality to security across every domain and industry.

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #393

Sun, 28 Jun 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

The Future of Agents with Andrew Ng
Sebastian Raschka’s Local Agent Setup
Memory in the Age of Agents
PydanticAI V2 Now Released
O’Reilly Radar Trends to Watch in 2026
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The Future of Agents with Andrew Ng

This is one of the best breakdowns on the current state of agentic systems and Agentic Development, by the one and only Andrew Ng! He breaks down his setup, what he’s working on, and a few predictions for the future: Andrew Ng talked about how coding agents have advanced so fast that they are only now catching up across other domains, such as crafting product definition, legal review, design, marketing, and data access. One point that really resonated is how now all organisations are in a race to sort out their data foundation, as agentic systems are multiplying the value of data by enabling insights that could not have been available at such a speed and detail. There are new delivery bottlenecks, however industry is figuring out how to unblock them with small teams of high-context engineers who can use AI tools across adjacent functions. Andrew Ng also distinguishes incremental automation from broader process redesign, using loan underwriting as an example where value comes from reworking the full business workflow rather than automating one review step. Definitely recommend listening to this fireside, it’s always surprising how great Andrew’s takes are!

Sebastian Raschka’s Local Agent Setup

Sebastian Raschka just dropped a full breakdown of his local LLM agent setup, and as always he has super relevant insights for any engineering practitioner - here’s a few highlights: Sebastian is clearly an advocate for open-weight models, and seems to be using Ollama heavily as the model-serving layer. In regards to agent harnesses he tends to favour Qwen Code (first time I hear about it), Codex CLI and Claude Code. He really emphasises that local agent workflows depend largely on inference speed, long-context behavior, tool-call reliability, permissions, telemetry, and task-specific evaluation. And as always an article wouldn’t have the Sebastian’s signature without an in-depth LLM architectural piece; there was an emphasis on 30–35B Mixture-of-Experts coding models such as Qwen3.6 35B-A3B, North Mini Code, and Nemotron 3 Nano, as good mode can be usable for routine coding-agent tasks on workstation-class hardware, but that harness choice and token use materially affect performance. Hopefully this becomes a series, and starts also diving into his agentic engineering workflows as I’d definitely be keen on these!

Memory in the Age of Agents

A large consortium of universities from across America, Europe and Asia have published a comprehensive survey on the state of “Memory” in agentic systems - key highlights:

The paper’s core is actually a structured taxonomy that separates memory by form, function and dynamics, including what carries memory, why agents need it, and how it is formed. There is an interesting distinction of “Memory” across 1) token-level memory, 2) parametric memory and 3) what they call “latent memory” - and it maps memory functions into factual, experiential and working memory. It also clarifies how agent memory differs from adjacent areas such as RAG, context engineering and long-context model design - which although there are some intersections, these are completely different beasts alltogether. For production ML practitioners, the paper is useful because it helps standardise concrete engineering choices around memory, including persistence, retrieval quality, auditability, privacy, latency and evaluation. It is becoming clear that memory should be treated as a core system component in long-running agents, and with components that support, short/long-term memory as well as independent/shared memory.

PydanticAI V2 Now Released

Pydantic AI v2 is out! Pretty cool to see this agent harness maturing on capabilities - here’s some of the highlights: They added composable units that package instructions, tools, lifecycle hooks, and model settings which now can be used as lego-blocks. There are now learnings from production agent systems where the operational complexity sits outside the basic model-tool loop, so they added better context control, tool loading, steering, guardrails, code execution, and instrumentation. One of the areas I am particularly excited about is the new defer_loading=True as it lets agents expose a compact catalog and load a workflow only when needed. This is particularly game changing for projects like the K8s Agent OS project that we maintain as each agent may have a long list of MCP Tools and Sub-Agents, and enabling discovery on demand can help optimize context and tokens. We have already updated PydanticAI to v2 in the K8s Agent OS (KAOS) and are keen to start exploring some of the new features.

O’Reilly Radar Trends to Watch in 2026

O’Reilly’s published the 2026 Radar Trends this month, and this has quite a useful cross-section of where agentic systems are trending towards - here’s a few highlights: The most relevant theme for production ML practitioners is the emergence of infrastructure that allows agents to provision accounts, register domains, initiate payments, obtain credentials, and deploy applications with limited human intervention. This changes the boundary of MLOps, as teams now need to reason not only about model quality and serving latency, but also about authorization, spending controls, audit trails, sandboxing, and dependency risk. There are also key insights on what is becoming a more fragmented model landscape, with general-purpose frontier models becoming more relevant in specialised contexts. The security section is equally relevant, as it includes AI-assisted vulnerability discovery, supply-chain attacks, and credential leakage from coding agents suggest that agent workflows. Definitely worth checking out!

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #392

Sun, 21 Jun 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Data Dog’s State of AI Engineering
10 Year of Clickhouse
Cornell: Advanced Compilers
DuckDB Internals and Speed
Bringing GPU Kernels to Rust
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Data Dog’s State of AI Engineering

Datadog has analysed LLM telemetry data from more than a thousand Datadog customers and published a snapshot on the “State of AI Engineering” - here’s some highlights: Telemetry data shows that more than 70% of organizations use three or more models, agent framework adoption has nearly doubled and 69% of input tokens are system prompts. In regards to routing, prompt caching is still only present in 28% of eligible calls, and rate limits are one of the biggest reliability problems in production LLM calls. Some trends show that teams are now running multi-model stacks, heavier agent frameworks, long system prompts, and tool-heavy workflows that behave much more like distributed systems than simple API integrations. It is clear that AI engineering now needs the same discipline as platform engineering (+ always has!), including model gateways, continuous evals, prompt layout hygiene, caching, context engineering, budgets, backoff, queues, and fallback capacity. The report also shows that most agents are still fairly simple from a service-topology perspective, with 59% making only one service call, so the next set of engineering challenges will come from tracing, debugging, and governing these systems as they become more distributed.

10 Year of Clickhouse

ClickHouse just turned 10 years old as an open-source project, and their CTO Alexey Milovidov published a fantastic engineering history of what it takes to build a production database from actual painpoints: ClickHouse evolved from OLAPServer and Metrage into a from-scratch columnar DBMS. They decided to add key differentiating features like in-memory columns, aggregate functions, table engines, compression, SQL parsing, MergeTree for background sorting, and ReplicatedMergeTree for multi-DC production use. This came up from wanting to address the pains of growing data volumes, real-time logs, slow MySQL paths, custom C++ data structures, and users waiting for analytics to load. For us ML Engineering / MLOps practitioners it’s a good reminder on how AI systems now need exactly the same properties ClickHouse was built around, including fast analytical queries over high-volume event data, long-retention observability, cheap aggregation, and infrastructure that can handle messy real-time workloads. It is also a good reminder that serious open source is not just throwing code on GitHub; ClickHouse really shows how they spearheaded this domain as well.

Cornell: Advanced Compilers

Compilers are one of the most fun and challenging domains in computer science, and they are becoming a production ML topic again - this course from Cornell is a fantastic deep dive into modern / advanced compiler concepts: Especially given the importance of efficiency in ML these days, we can no longer expect to throw larger and more GPUs, especially as teams run into graph breaks, custom kernels, dynamic shapes, hardware-specific inference paths, and performance issues that cannot be solved by just asking for bigger hardware. Cornell’s CS 6120 Advanced Compilers course is now available as a self-guided online course, and it looks like a great resource for ML practitioners who want to understand the systems layer underneath torch.compile, XLA, MLIR, Triton, LLVM, and modern inference stacks. The course is PhD-level but still very hands-on, covering intermediate representations, data flow, SSA, local/global optimizations, loop optimization, LLVM passes, alias analysis, garbage collection, JIT/dynamic compilation, parallelism, and fast compilers through classic papers and open-source implementation tasks using LLVM and Bril. I haven’t picked up the dragon book since back in university, and that’s one of the books / courses I’ve enjoyed the most, so I am definitely adding this to my todo list, hopefully I get some time (as the list is only growing larger!!).

DuckDB Internals and Speed

DuckDB is becoming one of the default local engines for ML/data workflows, but have you wondered what’s going on in its internals to work so well? Basically a lot of the speed comes from the runs in-process, so Python/R applications avoid the server round trip and a lot of row-by-row serialization overhead that still shows up with ODBC/JDBC-style paths. This requires understanding the query setup path, including parsing, binding, optimizer passes such as filter pushdown, subquery unnesting, join ordering, and runtime join-filter pushdown. This then follows by the physical plan which is split into pipelines separated by sinks like GROUP BY, ORDER BY, and hash-join build phases. DuckDB’s native format and Parquet both give it columnar reads, row-group statistics / zone maps, and byte-range reads on remote files, so many feature analysis, eval, debugging, and batch analytics workloads can stay in simple Parquet + SQL without immediately reaching for a warehouse job or distributed cluster. This multi-part series is a great way to get started into the DuckDB internals, so definitely recommended as a deep dive resource.

Bringing GPU Kernels to Rust

NVIDIA has published a new framework + paper to bring Rust’s memory safety into GPU Kernels! Unfortunately still CUDA-specific, but this does look like quite an exciting leap! We all know that GPU kernel work is becoming a much bigger part of production ML engineering, especially as teams push harder on custom approaches to unlock every bit of performance. This paper introduces cuTile Rust, a tile-based GPU kernel system that brings Rust’s ownership model across the CPU/GPU boundary, including mutable tensors that are split into disjoint partitions, immutable tensors that are shared safely, and kernel launches that preserve ownership while GPU work is still running. It’s impressive to see that the safety features don’t seem to add a major performance bottleneck - on B200 they report 7 TB/s for elementwise operations and around 2 PFLOP/s for GEMM, reaching 96% of cuBLAS. For us ML Engineering / MLOps practitioners this could be the start of an exciting trend, which although right now is limited to NVIDIA/CUDA, most likely very soon this could come to other frameworks like Vulkan, and unlock cross-vendor GPU compute.

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #391

Sun, 14 Jun 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Autonomous Agentic Systems
OpenAI Codex Engineering
Anthropic on Self Service Analytics
Amazon’s Tabluar Foundation Models
Kimi 2.7 Code Model Release
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Autonomous Agentic Systems

Excited to see my article on “Autonomous Agentic Systems at Scale” is now published at Hackernoon! This one provides a practical guide to “Always-On” agents 🚀 In this post I share some of my learnings gathered from extending KAOS to support autonomous long-running agents whilst balancing the runtime, memory, telemetry, and operational complexities. “Always-On” Agents feel like a new kind of architectural abstraction, as they are indeed not quite a chatbot, but also not quite a “cron job” / workflow engine, and definitely not quite a microservice… but it’s a Frankenstein that borrows from all of them (+ ofc with the added complexity of being stateful and non-deterministic). Despite the complexity it is clear that there’s a lot of opportunity with this pattern - namely for use-cases where the goal persists over time and the environment keeps changing. However it is also clear to me that there is still quite a way to go for the field to be able to start getting the full value, and that includes improvements in monitoring, operations, maintenance, research, data discovery, etc. So let’s make sure that we continue to invest and contribute to theses open questions, as they indeed won’t answer themselves. Let me know what you think!

OpenAI Codex Engineering

OpenAI is clearly leading the way in developer productivity; the Codex repo is built by a team of 3, and has produced about 1.5k PRs over five months, post is one of the clearest signs yet that the coding-agent bottleneck is moving from model capability to engineering systems design: the team built an internal product with 0 manually-written lines of code, around a million lines generated by Codex, and roughly 1,500 PRs over five months. For production ML practitioners, the practical takeaway is that reliable agentic development is less about prompting harder and more about building the right harness around agents: repo-local knowledge, a small AGENTS.md as a map instead of a giant instruction manual, browser-driven validation, local observability, mechanically enforced architecture, custom linters, and continuous cleanup of drift. The key lesson is that as agents take over more of the software lifecycle, human attention becomes the scarce resource, so teams need to encode taste, constraints, tests, telemetry, and review loops directly into the codebase rather than relying on ad-hoc docs or heroic human review.

Anthropic on Self Service Analytics

Anthropic just shared their playbook on self-service analytics, showing how they automated 95% of their internal business analytics with Claude: On their setup, they were able to build an agentic system stack that is able to reach ~95% aggregate accuracy, and the most interesting point is that their system looks much more like a governed data platform, so we’re back to the basics. It seems the main failure modes are data modelling ambiguity, data staleness, and retrieval failure, which can be addressed with canonical datasets, enforced semantic layers, lineage, curated domain docs, and Claude Code Skills that route the model through the same process a senior analyst would follow. It’s also great to see strong emphasis on evals and observability, including offline question/answer suites, PR-level ablations, provenance footers, adversarial review, and correction harvesting. For production ML and Data practitioners it is a reminder that reliable analytics agents are less about letting an LLM loose on your warehouse, and more about the basic foundations of “great data”.

Amazon’s Tabluar Foundation Models

Tabular ML is still one of the most business-critical parts of production machine learning, and I didn’t know Amazon also had a tabular foundation model that tackles different modalities: Amazon Mitra is an interesting tabular foundation models that also takes a different approach from training on real data, and instead it is trained on purely synthetic data, which seems to be a growing trend. This synthetic data pipeline includes a carefully designed mixture of synthetic priors, combining structural causal models with tree-based generators such as gradient boosting, random forests and decision trees. It seems that the key idea for tabular foundation models is that the data prior may matter as much as the architecture, as good synthetic priors should perform well on real tasks, be diverse enough to avoid overfitting to themselves, and add distinctive patterns not already covered by other priors. Mitra uses in-context learning to condition on support rows from a new dataset and predict query labels without gradient updates, while also supporting fine-tuning and ensembling through AutoGluon. The reported results are strong across TabRepo, TabZilla, AMLB and TabArena, with Mitra outperforming TabPFNv2 (although v3 is already out), TabICL and strong task-specific baselines in several classification and regression settings, and showing better sample efficiency when fewer in-context examples are available. It is clear that tabular foundation models are becoming a serious option for low-data, fast-iteration tabular prediction workflows; it will not (yet) outperform specialised models, but it is clear that getting a strong baseline for free is already a major win.

Kimi 2.7 Code Model Release

The Chinese startup behind Kimi (Moonshot AI) just released Kimi K2.7 Code, and this is for sure worth paying attention to if you care about where coding agents are headed: K2.7 Code is a coding-focused agentic model built on the previous arch, ie with the same large MoE shape of roughly 1T total parameters and 32B active parameters, but with stronger reported coding / agentic performance and around 30% lower thinking-token usage. That token-efficiency point is probably the most interesting part for production ML teams, because agentic coding is starting to get more expensive (and so is electricity!), which means it gets worse with repeated tool calls, context carry-forward, retries, and multi-step reasoning loops. The model supports a 256K context window, multimodal input, forced thinking / preserve-thinking mode, and deployment through vLLM, SGLang, KTransformers, Hugging Face, Moonshot APIs, and hosted routers like OpenRouter. The benchmark numbers look promising across coding and MCP-style tool-use tasks, although teams should treat them carefully given the first-party eval setup and differences in harnesses across competing models.

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #390

Sun, 07 Jun 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

C++: The Documentary
Stanford Language Modeling from Scratch
Tokenomics Where Tokens are Used
LLMs Need New Routing Paradigm
NVIDIA RTX Spark Launch
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

C++: The Documentary

I loved the C++ Documentary! People may not know but C++ is (shamelessly) my favourite language; I maintain an active (2k stars) OSS C++ codebase for fun, and did a lot of prod C++ dev back in the day, so this documentary was thoroughly enjoyable to watch. Despite all the hate, C++ is arguably one of the most important languages of our times; large % of modern AI foundation is built upon C++. It may not be as visible by Python/JS devs, and not as loved by Rust fans, but it’s still a fast growing language that supports the performance and scale many important systems today. This is an absolutely recommended watch for any ML practitioners, as it covers how C++ began when Bjarne wanted to combine C’s low-level interoperability advanced classes and modern language design. It’s interesting to see the evolution across Bell Labs, CFront, standardization, STL, C++98, the Java/C# era, and the C++11 renaissance; funnily enough, my main drivers tend to be C++11, with a tiny bit of C++14 / C++ 17, however I still havent found the main reason to move forward. Maybe once modules mature! Or an integrated package manager?! We all make-do with CMake, but it’s time to move on! Also I loved Bjarne’s quote: “It should’ve been called ++C for semantic reasons” - LOL. Also hilarious to hear that C++ v2 released as 2.00 known as 2.”uh oh…”. + Alexander Stepanov (STL designer) on standards: “Standards are not necessarily good or correct, but they are standards […] traffic laws [for example] are very often idiotic. But we have to obey them otherwise we will die”. I loved all of these small nuggets of humorous archeological history on the language. Check it out!

Stanford Language Modeling from Scratch

Stanford has published their CS336 Undergraduate Module on Language Modeling from Scratch. Quality SotA learning material from Stanfrod? For free? What a time to be alive! This is one of the most practical public resources for ML engineers who want to move beyond API-level LLM usage, and instead actually understand the full foundation-model stack end-to-end. The course walks through building language models from first principles, including tokenization, Transformer architecture, optimizers, GPU/resource accounting, Triton kernels, FlashAttention-style optimization, distributed training, scaling laws, inference, evaluation, etc. The assignments are deliberately low-scaffolding and require substantial Python/PyTorch engineering - they are really chunky, so prepare to spend a few weeks actually plowing through these. The public GitHub repo also includes lecture materials and assignment repos, with the lecture repo structured around executable lecture files and PDFs. I am impressed by the level of depth and quality of this course, unsurprisingly from Stanford, but it is shocking when realising how much high quality content is available on one of the most important topics today.

Tokenomics Where Tokens are Used

Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering. Super insightful research piece from

Concordia University that analyses telemetry from agentic engineering clients across various development tasks.

Some interesting findings presented: it seems that code review still dominates token usage consuming 59.4% of tokens on average; initial coding is only 8.6%. As part of the cost analysis, input tokens make up 53.9% of usage, suggesting that multi-agent systems are paying a large “communication tax” from repeatedly passing context between agents. For production ML practitioners building coding agents, a key insight is that cost control should focus on review/refinement loops, context management, and human-in-the-loop checkpoints rather than only optimizing model calls for generation. The main caveat is that the study is quite small, so it will be interesting as this is scaled further as likely new methodologies will emerge as well.

LLMs Need New Routing Paradigm

LLM inference has a routing problem: once every request can depend on cache locality, GPU specialization, and multi-step execution, the router directly affects latency, cost, and user experience. Modular put together a really interesting series on LLM Inference Routing, which is probably one of the most comprehensive breakdowns for scaling LLM serving beyond simple load balancing. Modular explains why traditional HTTP routers break down when GPU inference pods are stateful, heterogeneous, cache-sensitive, and sometimes split across prefill/decode execution paths. Part 1 frames the core problem around KV-cache residency, hardware specialization, conversation continuity, and multi-step request execution; Part 2 shows why the router needs a hot-path data layer that can query cached token blocks across pods in microseconds; and Part 3 turns that state into a composable routing pipeline covering preparation, filtering, scoring, picking, and execution. Lately I have seen quite a few posts discussing the challenges and potential solutions on LLM routing, so it is clear that not only this is a huge problem, but definitely understanding the foundations will be a useful skill as these tools become more ubiquitous.

NVIDIA RTX Spark Launch

Last week NVIDIA and Microsoft announced the release of the NVIDIA RTX Spark, and it’s really exciting to see that unified memory is finally coming outside of the M-Mac chips, and it seems that NVIDIA is indeed going all in. I found it actually surprisingly how small the chip actually is, and I struggle to process the claim that this chip has the same/similar performance to a 5070, when we are talking about fractions of the size compared to the video card. It was also impressive to see the small / thin size of the machines that were showcased with the chips, if this indeed matches the hype and expectations, it is clear that this will be another huge blow to the apple ecosystem with PCs finally being bullish on ARM processors. This also gives me hope for a proper local only rig that can run SotA models in consumer hardware, with the cost and extensibility of a PC. Let’s see how this actually develops!

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #389

Sun, 31 May 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Engineering Like It’s 2007
Netflix LLM Finetuning Infra
OpenAI & Anthropic Finding Market Fit
Reviving PapersWithCode.co
Massive Open Text-To-Image Dataset
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Engineering Like It’s 2007

Let’s take a trip back to 2007! This engineering talk from YouTube is truly a master class of scaling and learning, and surprisingly the lessons are as valuable today as they were back then. This talk shows a lean and mean team iterating through bottlenecks across web serving, video delivery, thumbnails, databases, hardware, OS tuning, caching and sharding. The most useful lesson for ML engineering teams is that scale is rarely solved by one abstraction, as YouTube kept the system simple enough to rewrite under pressure, used caching at multiple layers, and leveraged first principles at every layer. It is pretty cool to see that they followed the standard playbook of best practice of scalability, the usual suspects of moving hot traffic through CDNs, tuning commodity hardware (maybe less common now), and eventually replacing replication tricks with database partitioning. For ML practitioners, the parallel is model serving, feature stores, vector databases, observability pipelines and agent systems, which are currently hitting us with the same analogous challenges that we have to solve at lighting speed.

Netflix LLM Finetuning Infra

Netflix is sharing their playbook for massive scale LLM fine-tuning infrastructure: Netflix moved away from a few fine-tuning scripts to a managed framework that supports SFT, DPO, RL, distillation, checkpointing, MFU tracking, Hugging Face-compatible model/tokenizer flows, and distributed orchestration. It is interesting that Ray has chosen also the usual suspect technologies like Ray, PyTorch, vLLM together with custom tooling (eg Netflix’s internal Mako platform). The interesting takeaway for production ML practitioners is that post-training is challenging across every layer, including Data, Model, Compute and Workflow. Netflix reports up to 4.7x effective token throughput from asynchronous on-the-fly sequence packing, and this is a great example of the direction GenAI infrastructure is taking.

OpenAI & Anthropic Finding Market Fit

Open AI & Anthropic have found Market Fit - this is an interesting opinion piece from Simon Willison: Here there is a good case that coding agents may be the first real product-market fit moment for frontier AI labs because enterprises are now being charged close to raw API-token economics for daily developer workflows. OpenAI Codex and Anthropic Claude Code have shifted from huge subsidies / subscriptions to usage-based pricing (unfortunately for us subsidies are indeed ending). It is now clear that organisations will need to establish much tighter cost observability, usage governance, ROI measurement, procurement discipline, and platform controls around coding agents, just as they already do for serving and inference workloads. This starts making it clear why tools like LiteLLM or OpenRouter are becoming so popular, even if at the beginning it wasn’t super clear why an extra abstraction layer was needed on top of a simple vendor API (eg. surprisingly enough many vendors still do not offer spend caps).

Reviving PapersWithCode.co

I remember when Papers With Code originally came out in 2018 it was a major breakthrough; after the meta acquisition the project slowed and then stopped, but it seems there is an attempt to revive it! Hugging Face has started reviving it as paperswithcode.co with support from some AI agents to help with the parsing of papers, auto-linking GitHub repos, project pages and artifacts, categorizing, and generating leaderboards. The new site already brings back the familiar discovery workflow around trending papers, SOTA browsing, methods and domains, while adding support for star-velocity trends, citation counts, external non-arXiv papers, multiple repos per paper, benchmark harness reports, and Hugging Face-native storage/login integration. As research moves faster (whether AI slop or otherwise), it’s still a basic need to have a well maintained discovery layer, so it’s great to see projects like this, hopefully it will continue growing.

Massive Open Text-To-Image Dataset

An interesting release of a new massive open text-to-image dataset: The MONET dataset. It is great to see this, as image generation not only depends on quality data, but also other key resources like benrhcmarks whcih can only help teams accelerate on this field. Better curation, filtering, captions, and provenance can really make quite a difference. This dataset consists of 104.9M curated image–text pairs distilled from 2.9B raw pairs, with safety filtering, domain filtering, exact/near duplicate removal, multi-VLM re-captioning, embeddings, object/face annotations, hashes, NSFW/watermark scores, and pre-encoded SANA-VAE latents for faster latent-diffusion training.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #388

Sun, 24 May 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Making DL Go Brr with First Principles
Benedict Evans: AI Eats the World 2026
Gemini 3.5 Frontier Intelligence
(Sk)Forecast Foundation Models
NVIDIA’ New Image/Video Model
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Making DL Go Brr w First Principles

A classic, “Deep Learning Go Brrrr From First Principles” which still brings super relevant advice to AI teams today: Instead of throwing random PyTorch tricks at slow models, it’s important to have a clean mental model for diagnosing performance across: 1) compute-bound limits, 2) memory-bandwidth-bound limits, and 3) overhead-bound limits. Each of these brings a different optimization path. Compute-bound workloads need better Tensor Core usage or more hardware. Bandwidth-bound workloads benefit most from operator fusion and avoiding unnecessary global memory reads/writes. Overhead-bound workloads usually need tracing, compilation, CUDA Graphs, or reducing Python/framework dispatch costs. For us production ML practitioners it’s a good reminder that GPU efficiency is not just about bigger accelerators, but about understanding where time is actually spent.

Benedict Evans: AI Eats the World 2026

Benedict Evans has dropped the 2026 “AI Eats the World” deck, and here’s the main highlights: GenAI so far = Huge capex first, unclear value capture, lots of hype, and only later the boring-but-transformational deployment layer. The current model race is still going, but models are converging, infrastructure is getting brutally expensive, and the real leverage is likely to come from teams that turn LLMs into reliable workflow automation. There is still a lot of value to come from new aggregation/discovery layers, and domain-specific products that change how work is done. The most important takeaway is that AI adoption will probably look slow and underwhelming inside enterprises until it suddenly becomes standard / expected.

Gemini 3.5 Frontier Intelligence

Google DeepMind has just released Gemini 3.5 Flash! This is quite interesting to see as a faster agentic execution model across coding, tool use, multimodal understanding and long-horizon workflows. The interesting bit for production ML practitioners is that Google is positioning Flash as the high-throughput model for real-world agents which claims strong results. For ML teams, the takeaway is about the infrastructure pattern where faster frontier models plus agent harnesses are becoming the default winning advantage.

(Sk)Forecast Foundation Models

Skforecast is making time-series foundation models much easier to test in real production forecasting workflows: it’s wiring Chronos, TimesFM, Moirai, and TabICL through their new release! Really great to see Skforecast leading the charge on making foundation models accessible with various new classes (e.g. FoundationModel + ForecasterFoundation) which abstract foundation models on sklearn-style interfaces. Zero-shot forecasting is now becoming something you can benchmark inside existing forecasting pipelines rather than treat as a separate research experiment (and works surprisingly well). There are still challenges in context length and feature parity, as longer windows help models see seasonality and regime patterns, but they also increase inference cost and latency, so teams still need proper backtesting rather than assuming bigger context is better. The examples are also refreshingly honest about production caveats - definitely worth checking out.

NVIDIA’ New Image/Video Model

NVIDIA just dropped a high efficiency open-source stack for high-resolution image, video, and world-model generation! This seems like an exciting addition for production ML teams because it focuses on the deployment constraints that usually decide whether generative media systems are practical on latency, VRAM, training cost, quantization, and serving integration. This seems to be positioned by NVIDIA as a complete training and inference codebase with techniques such as linear attention, 32× DC-AE latent compression, Flow-DPM-Solver sampling, few-step sCM distillation, and block causal linear attention for long video generation.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #387

Sun, 17 May 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Toto 2.0 Time Series Foundation Model
TabPFN-3 Tabular Foundation Models
Stanford SWE Real-World Dataset
DeepMind AlphaEvolve Scaling Impact
Mozilla on Mithos Vulnerabilities
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Toto 2.0 Time Series Foundation Model

Datadog has just dropped a huge update to their Time Series Foundation Model for observability Toto 2.0, and this is really exciting for production ML practitioners: Toto 2.0 is a new Apache 2.0 open-weights model ranging from small 4M params all the way to to 2.5B parameters. At least from Datadog’s own observability-heavy benchmark it seems like there is not only potential for real-world use, but this is an interesting insight that domain specific time-series foundation models is clearly a realistic path for practical use. It is also good to see Datadog be quite honest about the remaining gaps, as even the largest model still shows long-horizon drift and structural breakdown past training context, so classical baselines are not going away. But overall this feels like a meaningful step towards forecasting foundation models becoming a real option, and potentially towards broader observability models that reason across metrics, traces, logs, topology, code changes, alerts and events for proactive incident detection.

TabPFN-3 Tabular Foundation Models

Tabular ML is still where a huge amount of production ML actually happens, so it is great to see the latest Tabular Foundation Model release from Prior Labs: TabPFN-3 is live with support for up to 1M training rows, row-chunking, a reduced KV-cache, native missing-value handling, many-class classification up to 160 classes, GPU-side preprocessing, and much faster inference than TabPFN-2.5. It is interesting to see the benchmarks, as you can expect that TabPFN-3 reports better performance than tuned and ensembled baselines on TabArena, beats 8-hour-tuned gradient-boosted-tree baselines on datasets up to 1M rows and 200 features, but it will be interesting also to see real world and competitive benchmarks as further alternatives arise as well. For production ML practitioners, the interesting part is less the leaderboard and more the potential that we’re moving to a world where we can get faster baselines, less painful hyperparameter search, better calibrated predictive distributions, CPU-friendly distillation and faster SHAP-style interpretability workflows.

Stanford SWE Real-World Dataset

Stanford has just published a really interesting new dataset of real coding-agent sessions using public GitHub repos with ~6k sessions, 63K user prompts, 355K tool calls, git-linked diffs, and line-level attribution of whether code was written by humans or agents. It is interesting to see that coding-agent usage is already becoming extremely bimodal, with around 41% of sessions basically “vibe coding” where the agent writes almost all committed code, while 23% are still human-only. However the important takeaway is that they are still very inefficient and risky when used in the wild, with only ~44% of agent-produced code surviving into commits, users push back or interrupt in roughly 44% of turns, and vibe-coded commits introduce substantially more Semgrep-detected vulnerabilities than human-only or collaborative coding. For production ML practitioners building coding agents, evals, IDE copilots, or internal developer tooling, this is a strong reminder that the winning pattern is probably not full autonomy, but better scaffolding around agents.

DeepMind AlphaEvolve Scaling Impact

Google DeepMind showcases how they are bringing AlphaEvolve alive as an optimization engine for the expensive parts of ML, science and infrastructure: The AlphaEvolve system from Google has now been applied across genomics, power grids, disaster prediction, quantum circuits, mathematics, TPU design, Spanner, compiler optimization, logistics, advertising, lithography and ML force fields, with some genuinely impressive reported numbers. As part of their report they outline a 30% reduction in DNA variant detection errors for DeepConsensus, AC Optimal Power Flow feasible-solution rates going from 14% to over 88%, 10x lower-error quantum circuits, 20% lower Spanner write amplification, and nearly 9% lower software storage footprint. It seesm they also have use-cases, showcasing Klarna doubling training speed and Schrödinger seeing roughly 4x speedups for MLFF training and inference.

Mozilla on Mithos Vulnerabilities

Mozilla shared a great behind-the-scenes look at how they used Claude Mythos Preview and other models to harden Firefox, and what is really interesting is that this was not just “LLM finds bugs” but a proper end-to-end security pipeline: The Mozilla team built an agentic harness on top of existing fuzzing infrastructure, where models could inspect risky parts of the browser codebase, generate reproducible test cases, run them, and then feed validated findings into the normal lifecycle for deduplication, triage, patching and release. The numbers are quite impressive, with Firefox 150 shipping fixes for 271 bugs found with Claude Mythos Preview, including 180 sec-high issues, and Mozilla fixing 423 security bugs across April releases when combining this pipeline with other AI models + manual review. This feels like a very clear MLSecOps pattern that more teams will need to adopt, where frontier models become scalable security-reasoning workers inside controlled harnesses that can execute tests, verify claims and eventually scan patches continuously in the CI/CD pipelines.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #386

Sun, 10 May 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Netflix Democratizing MLOps
Stanford AI Index Report 2026
META on ProgramBench
OpenAI on Delivering Voice AI
DeepMind Accelerating Gemma 4
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Netflix Democratizing MLOps

Netflix shared an in-depth blog on how they democratized machine learning across their organisation by building the ML Model Lifecycle Graph, and there are some really practical learnings: Netflix describes how they built a Metadata Service and Model Lifecycle Graph to make ML assets discoverable, reusable, and debuggable across a fragmented production ML ecosystem spanning models, features, datasets, pipelines, experiments, and ownership systems. The core idea is that they ingest lightweight change events from source systems, hydrate them from the source of truth, normalize them into globally addressable entities, store relationship-heavy metadata in Datomic, index searchable fields in Elasticsearch, and asynchronously enrich cross-system links such as model-to-pipeline-to-A/B-test lineage. For production ML practitioners, it is an important reminder that teams can answer impact, lineage, ownership, and reuse questions from a unified metadata graph if it’s well documented and accessible to all teams. Mature ML platforms need metadata infrastructure as much as training or serving infrastructure, and it is interesting to see that there’s a lot of learnings in the dataOps space that can be applied to MLOps.

Stanford AI Index Report 2026

Stanford University has just dropped the 2026 Stanford AI Index Report, which shows critical risks on security, incidents spikes, and infrastructure constraints - here’s some key insights: Model capability is still accelerating, with industry now produces over 90% of notable models, and adoption reaching mainstream levels. Top frontier labls are converging so closely that production ML teams should prioritize guarding themselves from vendor lock-in to quickly be able to switch across when necessary. There are some operating risks, with benchmarks are saturating or proving unreliable, agents failing roughly one in three structured tasks, and incidents rising with constraints around chips, data centers, energy, water, and supply chains. For practitioners, the report’s main implication is that competitive advantage in 2026 is less about simply accessing the strongest model and more about building robust ML systems around it. The MOAT is now on reproducible evaluations, monitoring, incident response, data governance, cost-aware inference, human oversight, and clear measurement of productivity and safety outcomes.

META on ProgramBench

META is launching a new benchmark that aims to test LLMs on building large-scale / end-to-end applications like ffmpeg / sqlite / interpreters from docs, instead of just snippets/pull-requests: ProgramBench is META’s new benchmark which evaluates whether coding agents can rebuild full software projects from scratch using only a compiled executable and documentation, rather than editing an existing repo or filling in a scaffold. Current agents are far from reliable autonomous software engineers, as across 200 real open-source tasks0, no model fully solved any task. The best model only passed >=95% of tests on 3% of tasks. The benchmark is valuable because it tests the full lifecycle that production teams actually care about: discovering behavior by probing an executable, making architecture and language choices, implementing a buildable system, and matching black-box behavioral tests without being constrained to the original code structure. This does open some questions on how to mitigate pollution of training data, aka ensuring that the models don’t just have the solutions injected as part of their training itself, but that’s an existing issue that plagues the existing benchmarks today anways…

OpenAI on Delivering Voice AI

OpenAI has shared how they tackle real0time voice AI with milliseconds latency that ensures seamless experiences; at scale, shaving latency and stabilizing media transport can be the difference between a magical conversational agent vs a frustrating push-to-talk demo. OpenAI’s write-up is a useful production-infra case study for ML teams building realtime voice agents, as the main constraint is not just model latency, but especially the e2e including connection setup, NAT traversal, packet loss, jitter, first-hop routing, and stable ownership of WebRTC session state. From their post, it seems their solution splits responsibilities between a lightweight UDP relay and a stateful WebRTC transceiver. The broader takeaway for production ML practitioners is that realtime AI quality depends on treating networking and protocol termination as first-class ML platform concerns; namely keep client behavior standards-compliant, isolate hard session state, add complexity in a thin edge-routing layer, and let backend model services scale like normal services rather than WebRTC peers.

DeepMind Accelerating Gemma 4

Inference speed is becoming one of the most important battlegrounds in production AI - DeepMind shares how they accelerated Gemma 4 inference: Google introduces

“Multi-Token Prediction drafters” as a production-inference optimization where they pair each Gemma 4 target model with a lightweight multi-token drafter that speculatively proposes several future tokens, while the main model verifies them in parallel. This apparently has yielded Google reported speedups of up to 3x without changing final output quality, which is quite impressive if true. For ML practitioners, the relevant takeaway is that latency bottlenecks are increasingly being attacked through serving architecture such as KV-cache sharing, activation reuse, runtime-specific support across serving frameworks (eg vLLM/MLX/Transformers) rather than only through smaller models or quantization. This is especially relevant for chat, coding assistants, agentic workflows, and on-device or workstation deployments where responsiveness and memory bandwidth dominate user experience, but teams should benchmark against their own prompts, batch sizes, sampling settings, and hardware.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #385

Sun, 03 May 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Demis Hassabis on Agents & AGI
Google Cloud MCP Ecosystem
Netflix State of ML Serving
PyTorch Lightning Supply Attack
Multi-Modal Traces in MLFlow
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Demis Hassabis: Agents & AGI

Demis Hassabis on one of the most insightful conversations this year for ML practitioners talking about Agents, AGI and the Next Scientific Breakthrough - here’s the key takeaways: Today’s foundation-model stack is not a dead end, but still needs better continual learning, long-term reasoning, memory, consistency, and introspection before agents can become reliable fire-and-forget systems. Agents are still early, and a lot of near-term value is likely coming from human-in-the-loop workflows, fast distilled models, multimodal systems, local/edge deployment, and specialized tools orchestrated by general models rather than one giant monolith. For ML teams, the AlphaFold pattern is especially relevant, as it is clear the highest-impact opportunities are domains with massive combinatorial search spaces, clear objective functions, and either strong data or simulators, such as drug discovery, materials, biology, and other deep-tech areas. And could not finish without a prediction on AGI, which surprisingly he’s putting his money on 2030 as the year when it arrives; that sounds to me like what someone that runs an AI lab would say, so this is the one point that you should take with a pinch of salt.

Google Cloud MCP Ecosystem

Google has just published 50+ MCP servers to support programmatic agentic workflows with integrations that are integrated with governance and observability by design: It is great to see this move to support teams to move away fragile / bespoke tool integrations with instead managed MCP endpoints across Cloud infrastructure, databases, analytics, storage, Workspace, Maps, security, payments, and developer docs. For production ML practitioners, the interesting part is less that agents can call tools, but more that there’s a clear bet from cloud providers to invest in infrastructure where agents are the target user. It is also interesting to see the maturity in the ecosystem, in this case looks as IAM Deny policies, Agent Registry discovery, Model Armor for prompt-injection/data-exfiltration defense, OTel tracing, and Cloud Audit Logs. This seems to be a likely pattern for the rest of the cloud providers to follow, and it will become more interesting as we start seeing agentic systems providing automations higher up in the stack as we move towards operations, monitoring and debugging.

Netflix State of ML Serving

Netflix built a centralized ML serving platform handles over 1M requests/sec and thousands of models that include preprocessing, feature computation, post-processing, and optional learned components. Some impressive architectural decisions: They built a custom routing system called Switchboard that helped them improve their ML velocity, which served requests by using metadata at the request-body-level, however as you can imagine this quite fast became a critical-path dependency, added latency costs (eg request parsing), and made tenant/request-origin isolation harder. To address this, Netflix is introducing their new “Lightbulb” design, which instead minimal request context into routing metadata with Envoy proxy performing the actual routing from headers, and model-specific parameters stay in the request body. For teams building ML platforms, the takeaway is that serving abstractions should decouple product clients from model/version/shard churn, but the routing layer itself must eventually become lightweight, cacheable, failure-tolerant, and close to the networking substrate rather than a monolithic proxy in every request path.

PyTorch Lightning Supply Attack

PyTorch Lightning versions 2.6.2 and 2.6.3 were compromised for a 42-minute window on April 30 after attackers obtained PyPI publishing credentials and uploaded tampered builds: This is an important reminder that security is absolutely key especially in our ML stack; one compromised package release can turn everyday training jobs, notebooks, and CI pipelines into credential-exfiltration paths. This malicious packages executed on import, spawned a background thread, installed Bun, ran an obfuscated JavaScript payload, and targeted cloud credentials, browser-stored secrets, env files, and GitHub tokens. For ML teams, the operational lesson is critical, if either version was installed and imported in developer machines, notebooks, training jobs, or CI/CD runners, treat those environments as compromised, downgrade to lightning==2.6.1, rotate exposed secrets, audit outbound network activity, and review build logs/artifacts. More broadly, this incident shows why MLOps needs MLSecOps controls at the packaging boundary, including pinned and verified dependencies, isolated CI secrets, least-privilege cloud credentials, egress monitoring, artifact provenance, and fast incident playbooks for trusted ML libraries.

It is quite interesting that MLflow has recently been leading the charge on ML telemetry and traces, setting the charge for how multimodal tracing can work at scale for images, audio, PDFs, and other files. Open Telemetry standards are growing as now payloads go beyond purely json into binary artifacts extracted from spans, which need to be stored in an existing artifact store, and replaced in the trace database with lightweight references so queries stay fast and storage does not explode. The practical win for production ML practitioners is much better debugging of vision, audio, document, and image-generation workflows; namely allowing for an integrated experience where images render inline, audio can be played, PDFs can be viewed, and custom files can be attached manually via MLflow’s Attachment API. It’s really reassuring to see how observability tools for MLOps are maturing at fast pace, especially as ML systems are becoming a critical foundation for society and industry.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #384

Sun, 26 Apr 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Intercom 2X’d Engineering Velocity
Qwen3.6-27B Coding OSS Model
Scientific Theory for Deep Learning
META KernelEvolve Optimizes AI Infrastructure
OpenAI Releasing GPT-5.5
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Intercom 2X’d Engineering Velocity

Intercom claims 2x productivity increase with coding agents, and this podcast has some pretty interesting insights on on how they’ve been approaching this across their org: Many organisations are aggressively trying to figure out how to unlock AI coding productivity beyond the individual and across the organisation, which seems Intercom has figured out a way forward. In nine months, it seems they doubled merged PRs metric while keeping quality stable by treating the AI workflow like an internal product, instrumenting usage with telemetry, analyzing anonymized session data, and building a shared skills repository with hooks that enforce engineering standards automatically. It is great to see that often the secret is actually nothing more than solid engineering practices in the foundation; gains come less from “allow everyone to do tokenmaxing” and instead more from building the surrounding foundation for high quality PRs, flaky tests, CI, internal tools and reviews. This is the most important time to invest in CI/code-review bottlenecks, as well as DORA metric improvements, as well as a culture where PMs, designers and engineers can all safely ship code.

Qwen3.6-27B Coding

Chinese giant Alibaba releases another impressive open source model with Qwen3.6, with only 27B parameters which shows impressive performance on coding tasks: From the reports it seems that this release brings flagship-level agentic coding into a dense 27B open-weight model, which is super impressive how much can be packed in such a relatively small model. This basically reducess the complexity from very large MoE systems while still outperforming Qwen’s previous 397B-total / 17B-active open-source flagship on major coding-agent benchmarks like SWE-bench, etc The interesting bit is not just the scores, but the fact that this model is open sourced under Apache-2.0 and it supports text/image/video inputs, offers a 262K native context window extendable up to 1M tokens, and introduces “thinking preservation” for multi-turn agentic workflows.

Scientific Theory for Deep Learning

It seems we’re at a stage where deep learning is evolving from alchemy into an engineering discipline; this is an exciting paper which lays out that a scientific theory is emerging for Deep Learning: This is great as having a robust scientific foundation means fewer blind hyperparameter searches, more predictable scaling, better interpretability, and stronger foundations for safety. Deep learning theory is starting to look less like scattered math and more like an emerging “mechanics of learning” as a physics-style framework for predicting training dynamics, representations, final weights, and model performance. The paper breaks down into five buckets, including 1) solvable toy settings such as deep linear networks and NTKs; 2) useful limits such as infinite width/depth and lazy vs. rich feature learning; 3) empirical laws such as scaling laws and edge-of-stability behavior; 4) hyperparameter theories such as μP and learning-rate/batch-size scaling; and 5) universal phenomena where different architectures, datasets, and training recipes converge to similar representations.

META KernelEvolve Optimizes AI Infrastructure

META is building agentic systems that are optimizing the AI infrastructure under their large scale machine learning models, and they have released their framework: META’s KernelEvolve is a framework they are using for optimizing the low-level infrastructure that determines whether large-scale models are economically viable in production. KernelEvolve sits inside Meta’s Ranking Engineer Agent stack and turns kernel authoring into a closed-loop search problem across NVIDIA GPUs, AMD GPUs, MTIA chips, and CPUs, using LLM-generated candidates, retrieval-augmented hardware knowledge, tree search, profiling feedback, and automated correctness/performance evaluation. Meta reports compressing weeks of expert kernel work into hours, achieving over 60% inference throughput improvement for the Andromeda Ads model on NVIDIA GPUs and over 25% training throughput improvement for an ads model on MTIA, while supporting DSLs and backends like Triton, CuTe DSL, FlyDSL, CUDA, HIP, and MTIA C++. For production ML practitioners, the key takeaway is that the next bottleneck in model iteration may be less about model design alone and more about automating the systems layer around it: kernel generation, hardware portability, profiling, benchmarking, and continuous optimization across increasingly heterogeneous accelerator fleets.

OpenAI Releasing GPT-5.5

OpenAI has released their latest model GPT-5.5. It seems this model is positioned less as a chatbot and more as a long-running worker for coding, research, document-heavy knowledge work, tool use, and computer operation. For production ML practitioners the main callouts are that the new model is focused on stronger agentic coding and systems reasoning, better long-context performance up to 1M tokens, improved tool reliability, and greater token efficiency while maintaining GPT-5.4-like per-token latency. The benchmarks sound impressive (although we know how these are never confirmed until they are taken for a spin), with 82.7% on Terminal-Bench 2.0, 58.6% on SWE-Bench Pro, etc. Keen to see how this performs in the wild, would be great to hear experiences from practitioners as they take them into production projects.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #383

Sun, 19 Apr 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

LLMs are Databases, Really
NVIDIA Optimization for Agents
Can I Run AI Locally? Yes.
Kafka Guide to Distributed Messaging
What 81,000 People Want from AI
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

LLMs are Databases, Really

Are LLMs databases? Yes. Can we treat them like a Graph Database? Also Yes. What does this look like? It’s actually quite interesting: Recently there have been projects that are treating LLMs as indexable write-enabled databases using a SQL language that actually loads the models as graph database structures. There is a query language called LARQL which makes any transformer as a queryable graph-like knowledge store, where internal features can be inspected, traversed, and even edited through a SQL-style interface. A great way to show this is by loading Gemma 3 as the demo target, where we can now query entities, relations, and nearest-neighbor feature clusters from model internals. This also allows for not just inference and path-tracing, but also raises questions on how we our thinking will evolve on LLMs as optimizable systems; this provides quite an exciting glimpse into what could be a large field of research in the making. It is still clearly an early-stage prototype rather than production-ready serving infrastructure, but it is a compelling direction for anyone thinking about controllable model editing, efficient inference, and new abstractions for working with model knowledge.

NVIDIA Optimization for Agents

NVIDIA wants to provide an operating system for multi-model agentic orchestration, and this comes quite quite a few interesting architectural learnings: Once coding agents and multi-agent swarms start making hundreds of sequential calls with shared history, the dominant bottleneck becomes keeping KV-cache warm, models routable, and paths reusable across workers. NVIDIA’s core argument is that self-hosted agent stacks need tighter coordination across frontend APIs, routing, and cache lifecycle management. This includes support for modern agent protocols (responses / messages), expose harness-side metadata like priority and expected output length through agent_hints, route by KV overlap instead of round-robin, and treating cache blocks differently depending on whether they are persistent context or ephemeral reasoning/subagent state. The practical takeaway is that if you are running open models for agentic workloads, you likely need to start thinking beyond “throughput per GPU” and toward session-aware inference infrastructure with cache-aware routing, selective retention, multi-tier KV storage, and eventually prefetching.

Can I Run AI Locally? Yes.

CanIRun.ai locally? Yes. This is a great resource for exploring local inference tooling: We’ve all come to the question of “what model can this machine actually run?”; this resource provides a fast estimate using hardware detection to provide estimated tokens per second based on the models. This of course is not perfect benchmarking, but at least helps us quickly narrow model choices for local copilots, offline workflows, and edge deployments without manually piecing together VRAM charts and quantization assumptions. It’s quite good to see that the methodology is quite transparent by making clear that the estimates are heuristics since actual performance still depends on runtimes, drivers, thermal limits, etc etc. Check it out!

Kafka Guide to Distributed Messaging

Event-driven systems are one of the quiet foundations of production ML and modern software; they shape how reliably data moves, how quickly systems react, and how well platforms scale under real-world load. Here’s a great guide around distributed messaging: Service-to-service is evolving in various use-cases from synchronous calls into durable event streams with often the backbone provided by Kafka, leveraging topics, partitions, offsets, and consumer groups. The key production takeaway for ML practitioners is that Kafka can be the backbone for decoupled feature pipelines, inference events, retraining triggers, and replayable state changes. This is particularly relevant where partitioning defines scalability and ordering trade-offs (ie delivery guarantees), as well as how consumer-group design defines parallelism and fault recovery. This is one of the best guides that provide an end-to-end overview, which include even the move from ZooKeeper to KRaft, as well as quite a lot of great fundamentals.

What 81,000 People Want from AI

What do 81,000 people want from AI? Anthropic has taken the question and provided a thorough report with great key insights: Anthropic gathered 81,000 open-ended interviews across 159 countries and 70 languages, finding that people mostly want AI to improve professional effectiveness, personal transformation, life management, and time freedom. In their report they outline that 81% say AI had already delivered some value, especially through productivity, cognitive partnership, learning, accessibility, and research synthesis. However the most common concerns were unreliability, economic displacement, loss of autonomy, and cognitive atrophy, which makes it clear that a large percentage are still critical. For ML practitioners, the key takeaway is that successful AI products will be won not just through better models, but through key principles taht are implemented in practice, such as reliability, verification, human-in-the-loop design; basically fundamentals of software design are still relevant today!

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #382

Sun, 12 Apr 2026 00:00:00 GMT

Releasing KAOS v0.4.1!

Excited for this release which introduces Autonomous Agents 🚀

This is full OpenClaw-style functionality 😎👇

If you want to support the momentum, do reshare, open an issue, and/or give the repo a star ⭐

https://github.com/axsaucedo/kaos 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Come Say Hi @ PyCon DE!
KAOS Autonomous Extension
Anthropic Tackling Security Risks
How People Use ChatGPT
Linus Torvalds Agentic Guidelines
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Come Say Hi @ PyCon DE!

If you are at PyCon DE & PyData this week, come say hello!

Super excited to announce that we’ll have 6 (yes six!) talks from scientists and engineers (+ alumni) from my group at Zalando Markets AI, Data & Platform Zalando business unit!! The talks will range across production ML, forecasting, and ML platform engineering, between many other exciting topics! Here’s the full list of talks coming up this week:

Alejandro Saucedo: Production ML across 2015–2035: A Journey to the Past and the Future

Stefan Birr & Mones Konstantin Raslan: How to compare apples with oranges: Proper evaluation of article-level demand forecasts

Irena Bojarovska: Foundation Models in Forecasting: Are We There Yet? Lessons from the Trenches > Petar Ilijevski: Zero-Copy or Zero-Speed? The hidden overhead of PySpark, Arrow & SynapseML for inference

Abdullah Taha: Holistic Optimization: Implementing “Pipeline-as-a-Trial” HPO with Ray and Cloud Infra > Akif Cakir: From Struggling to Mastery: A Practical Guide to Data Pipeline Operations

Make sure you don’t miss any of our talks! Looking forward to the largest Python & AI Conference in Europe - come say hello to any of the Zalando group at the conference!

KAOS Autonomous Extension

Excited to release KAOS v0.4.1! This release introduces autonomous agents, which brings the OpenClaw-like design pattern of continuously running agents into your scalable cloud native environment! This means that we are finally able to move KAOS agents from reactive request/response APIs, to agents that can run continuously as always-on operational infrastructure. The main addition was the continous self-looping agent execution, where an agent starts on pod boot, gets its goal re-injected on every iteration, uses tools and memory to reason about the current state, pauses for a configurable interval, and then keeps going indefinitely until the pod is stopped. In practice, that makes KAOS much more compelling for real production use cases like monitoring, maintenance, and long-running cluster operations, rather than just interactive demos. On top of that the new version also introduces a much more sophisticated A2A task lifecycle support via the standardised JSON-RPC endpoint, introducing also a TaskStore which is planned to be extended for distributed execution. This provides a much cleaner split between continuous CRD-driven autonomous mode and budgeted async A2A task mode; we also included FastMCP 3.x support, integration with PyDantic’s Monty for Code Mode, and full UI support for autonomous agents and an A2A debug screen. This is a release that has been in the making for a while - really happy to see this one finally see the light of day!

Anthropic Tackling Security Risks

As I have been reading the Claude Code Mythos system card and the Project Glasswing post this weekend, my take is slightly more controversial: the exponential growth of mythos is hype, but the cyber impact and risks are real (even without Mythos): This made me process and reflect that irrespective of newer and more powerful model releases, it is clear that cyber specialists are now able to unlock expert-level knowledge on any software they exploit, so we’re only seeing how industry is catching up with this disruption. One of the key points that to me is most critical is the impact this is already having and will continue to have in key infrastructure like banking, etc. Additionally and more broadly, this feels like another signal that these models are starting to expand beyond pure software engineering into cyber, data, science, and MLOps; this is also something that I have already seen in my group, where use-cases of auto-research have started to unlock quite a lot of value. And the last point I keep coming back to is measurement: we may need to move from classic DORA metrics toward a more “Semantic DORA” world, where we measure not just delivery output, but the actual quality and semantic value of progress across bugs, incidents, customer pain, and shipped changes.

How People Use ChatGPT

This is a super interesting paper that gives one of the clearest large-scale views yet into how ChatGPT is actually used in the wild, using privacy-preserving classification over millions of consumer conversations. The headline for production ML practitioners is that the value is not autonomous task execution but decision support: most usage clusters around practical guidance, information seeking, and writing, with work-related usage concentrated in knowledge-intensive roles and especially in activities like documenting information, interpreting it, solving problems, and giving advice. Work use is growing, but non-work use is growing even faster, which suggests the impact surface for LLM products is much broader than enterprise productivity alone. The paper also shows that coding is a much smaller share of overall usage than industry discourse often implies, while writing-heavy workflows remain the strongest work use case. Overall, the results suggest that the biggest near-term opportunity for production systems is not just building agents that “do work”, but building reliable copilots that support decisions, shape drafts, and fit into everyday information workflows across roles and geographies.

Linus Torvalds Agentic Guidelines

The Linux Kernel now has released guidelines for vibe coded contributions! And unsurprisingly it’s one of the most reasonable and down to earth takes I have seen so far: AI-generated kernel code must still follow the standard Linux development, style, submission, and GPL-2.0-only licensing rules. There’s guidelines that AI systems must not add Signed-off-by because only a human can certify the Developer Certificate of Origin; and the human submitter remains fully responsible for reviewing the code, ensuring legal compliance, and accepting accountability for the patch. The main operational addition is traceability through an Assisted-by tag that records the agent, model version, and any specialized analysis tools used, making AI involvement auditable without shifting responsibility away from human maintainers. I am quite keen to see how other projects standardise and embed these guidelines into their projects amid the huge rise of AI slop contributions.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #381

Sun, 05 Apr 2026 00:00:00 GMT

Thank you for being part of over 70,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter /newsletter/ ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Alibaba Releasing QWEN-3.6
Google DeepMind on TurboQuant
MIT FlowMatch & Diffusion Models
NGROK on Quantization from Scratch
Stanford Transformer Course
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Alibaba Releasing QWEN-3.6

China’s Alibaba continues to challenge the AI status quo, now with the release of QWEN-3.6-plus which brings some really interesting performance innovations: Qwen3.6-Plus is Alibaba’s latest hosted frontier model for real-world agent workflows with the biggest gains in agentic coding + tool use + multimodal reasoning. For production ML practitioners, the practical takeaway is that Qwen is optimizing more for end-to-end task completion across repository-level coding, terminal operations, web/UI generation, document and video understanding, and multimodal agent loops. The model exposes a 1M-token context window which supports OpenAI- and Anthropic-compatible APIs via Model Studio and adds a preserve_thinking option whcih improves multi-step agent consistency as well as reducing redundant reasoning. From the benchmarks we can see that (allegedly) it is especially strong on coding-agent, planning, multilingual, OCR/document, and visual grounding tasks; so far performance remains mixed versus top competitors on some general reasoning and long-context evaluations. However overall this seems quite interesting to see the aggressive and close competition on a space that we were considering unbeatable only a year ago.

Google DeepMind on TurboQuant

Google DeepMind redefines AI efficiency with extreme compression with TurboQuant: TurboQuant removes much of the usual metadata overhead in vector quantization, letting KV caches and vector indices run at much lower bitwidths without the normal quality penalty. The method that TQ introduces combines compression with a 1-bit Quantized residual correction step which preserves attention accuracy and keeps memory overhead near zero (which is great). In Google’s reported experiments on long-context benchmarks and vector search, TurboQuant compressed KV cache representations down to 3 bits without training or fine-tuning, delivered at least 6x KV memory reduction on needle-in-a-haystack tasks, and showed up to 8x faster attention-logit computation at 4-bit versus 32-bit keys on H100s. For production ML practitioners, the takeaway is that these type of optimization opportunities can drive really clear opportunity for teams with lower memory bandwidth pressure, cheaper long-context serving, and faster high-dimensional retrieval with minimal accuracy tradeoff.

MIT FlowMatch & Diffusion Models

Super excited to see a brand new 2026 course from MIT on the same models that power OpenAI, Anthropic and other LLM giants: MIT is providing their 2026 course for FREE on

6.S184 diffusion and flow matching models for practitioners that want to get hands-on experience. This course is a comprehensive introduction and covers the math behind modern generative models across ODEs, SDEs, the Fokker–Planck equation, score matching, classifier-free guidance, latent diffusion, and discrete diffusion. This course also includes comprehensive hands on labs that walk learners through building key components and ultimately a latent diffusion model from scratch. For production ML practitioners, this brings really a lot of value, not just about shipping a model, but more about gaining the conceptual and hands-on foundation needed to understand how today’s image and video generators work.

NGROK on Quantization from Scratch

Quantization is now one of the most important methods to drive high performance efficiencies in real-world AI at scale, and this is a great deep dive from scratch: For production ML practitioners quantization matters as model size is dominated by weights which more are non-relevant (aka zero-valued); with quatnization we can make LLMs surprisingly tolerant to storing parameters in lower-precision formats or compact integer representations instead of full-precision floats resulting in major savings. The key takeaway on Qwen3.5 9B, is that 8-bit quantization can preserve quality almost entirely even with 8-bit quantization; with 4-bit quantization there is modest degradation (2-bit quantization starts to collapse). This is a great post from ngrok, do check it out for the deep dive into ML quantiaztion.

Stanford Transformer Course

Stanford is releasing their new 2026 course on Transformer models for FREE! CS25 Transformers United V6 has fantastic and updated content for practitioners to dive into the field: This Stanford course has a broad coverage on the evolving frontier beyond vanilla transformer models and into the relevant architectures powering the field today. Rather than teaching one deployment recipe, this course curates talks from leading researchers and practitioners across core model architectures and adjacent paradigms, which cover transformers, JEPA, state space models, and real-world perspectives from companies like Hugging Face, Anthropic, DeepMind, and Modal. This may end up being one of the most relevant courses on the topic once all the lectures are updated, make sure to keep an eye as the course material becomes available!

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - Sept @ Berlin
World Summit AI Europe - Sept @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #380

Sun, 29 Mar 2026 00:00:00 GMT

Thank you for being part of over 70,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter /newsletter/ ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Agentic Engineering Practices
Recommendation System at BlueSky
Raschka’s Open LLM Dream
Building a MCP Ecosystem at Pinterest
Dual Text-Forecasting Foundation Model
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Agentic Engineering Practices

AI coding agents are transforming software faster than we can figure out the best practices, which is why it’s super refreshing to see Simon Willison’s latest takes: There is consensus that there is the world before November and the world after; coding agents are now good enough to write substantial amounts of production-relevant code. Simon describes a that his workflow leverages red-green TDD, reusable project templates, and executable validation like running servers and probing APIs, which lets him trust agents more while reducing the need to manually review every line. For production ML practitioners, the important lesson is that agents should be treated as powerful but untrusted collaborators whose output quality depends heavily on the scaffolding (and instructions) around them, especially tests, clear constraints, and consistent codebase patterns. We certainly cannot ignore the security risks that are growing quickly when agents have access to sensitive data, external inputs, and ways to exfiltrate information. It will become more and more important to identify ways to properly use sandboxing, levarage synthetic data, and minimizing permissions where not necessary.

Recommendation System at BlueSky

Super interesting to see the recommendation system design for BlueSky which it seems is based from the Pinterest architecture with a few tweaks: Bluesky chose a recommender system architecture for their Discover-feed personalization which had constraints on data, costs and ML engineering resources. It’s interseting to see that they attempted a two-tower retrieval model but failed to converge, so they fell back to content-based post embeddings using BLIP2 plus topic models and HDBSCAN clusters to build a basic personalization layer. They are now exploring Pinterest’s PinnerSage recsys architecture which promises to be a better candidate-generation approach because it keeps item embeddings fixed, avoids heavy fine-tuning, and models users as multiple interest vectors rather than a single embedding. For production ML practitioners, the core takeaway is that there are interesting recsys architectures that have tradeoffs advantages/disadvantages like PinnerSage which offers an operationally attractive way to capture both long-term and short-term user intent by clustering recent interactions, but it shifts complexity downstream because multi-interest user representations are straightforward for ANN retrieval yet awkward and expensive to use in ranking.

Raschka’s Open LLM Dream

Sebastian Raschka has dropped another masterclass on owpen-weight LLM architectures, this time sharing key insights from the Jan-Feb 2026 launches: We are still seeing that there is no single architecture which has emerged as dominant, however the field is clearly converging on a shared set of trends as well as best practices. Some of these global trends include better long-context efficiency, lower KV-cache / latency costs, stronger coding / agentic performance, and more practical quality-per-token tradeoffs. Across the models from this year, the main pattern is the rise of increasingly specialized efficiency techniques such as hybrid attention, sliding-window attention, MLA, sparse attention, and multi-token prediction - especially in large MoE systems like GLM-5, Kimi K2.5, Qwen3.5, and Ling 2.5. The key takeaway for production ML teams is that architecture still matters, but less as a search for one universally best design and more as a way to optimize for serving constraints, context length, throughput, and workload fit.

Building a MCP Ecosystem at Pinterest

The race to make AI agents actually useful in production will be won or lost on platform design, not model hype, and Pinterest’s MCP ecosystem learnings shows how much leverage this can have: Pinterest is building an internal platform for production MCP services that feed agent workflows. They established an ecosystem of cloud-hosted MCP servers with a central discovery and governance layer, where they also added a shared deployment path so teams can publish tools without owning all the infrastructure, and integrated these servers into the IDE, chat, and internal AI surfaces engineers already use. The main lesson for production ML practitioners is that MCPs only becomes operationally useful when paired with strong platform controls like registry-based approval, layered authn/authz with user JWTs / service identities, etc.

Dual Text-Forecasting Foundation Model

Forecasting is where machine learning stops being interesting and starts being operationally decisive: better predictions directly shape revenue, inventory, risk, capacity, and planning, and even modest accuracy gains can compound into major business impact at scale. Migas 1.5 presents a pragmatic multimodal forecasting architecture for production settings: instead of training a single end-to-end model over text and time series, it keeps a standard time-series foundation model as the forecasting backbone, uses language models to extract structured contextual signals from text, and then applies a learned correction model to adjust the baseline forecast. The reported results across 86 real-world multimodal datasets suggest that this setup materially improves accuracy over unimodal baselines, especially in short-history or regime-shift scenarios where historical values alone are insufficient, with gains of up to 14.2% MAE reduction. For ML practitioners, the most notable contribution is less the benchmark win itself than the systems pattern it implies: event-aware forecasting can be added modularly to existing pipelines, and scarce aligned text-plus-time-series supervision can be bootstrapped with synthetic annotations generated by LLMs, though teams should still validate carefully for leakage, annotation quality, and robustness to noisy context.

Upcoming MLOps Events

Events we are speaking at this year:

Signals Conference - Sept @ Berlin
World Summit AI Europe - Sept @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #379

Sun, 22 Mar 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Scaling Karpathy AutoResearch
Mamba-3 State Space Model
META’s No Language Left Behind LLM
Karpathy’s No Priors Podcast
AI Redrawing Databases
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Scaling Karpathy AutoResearch

Data science as a profession is going to change massively in the next few years; the autoresearch method from Karpathy is actually going to have more impact than you think, and this is a great example: Giving an autonomous tuning agent access to parallel GPU infrastructure changes it from a sequential hyperparameter fiddler into a much more capable search system. This is a prime example that wires Karpathy’s autoresearch loop to a 16-GPU Kubernetes cluster, where Claude Code is able to run loose, and run 910 experiments over 8 hours, achieving a 2.87% improvement in validation versus baseline and reaching the same quality roughly 9x faster than a single-GPU sequential setup. I can’t emphasise how insane this is, and it’s not a gimmick but really the “claude” moment for some of the workflows that are carried out in deep learning particularly. In the next few weeks (not even months), we will most likely see iterations of this framework for some of the most popular libraries, and we’ll see what most likely will be impressive glimpses of what is even yet to come.

Mamba-3 State Space Model

Mamba-3 is live! This is exciting as inference is now a top production concern: the models that win will not just be smartest, but the ones that deliver the best latency, throughput, and cost profile at scale. Mamba-3 reframes state space models around deployment rather than training; it adds a better recurrence, complex-valued state dynamics, and an interesting MIMO variant to improve the quality/latency tradeoff without sacrificing linear-time decoding. For production ML practitioners, the key takeaway is that this pushes SSMs closer to being a practical inference architecture, with Mamba-3 SISO beating Mamba-2, Gated DeltaNet, and even a 1.5B Llama Transformer.

META’s No Language Left Behind LLM

Huge kudos to META for their “No Language Left Behind” LLM initiative! They built a ML translation system that supports more than 1,600 languages with a particular focus on long-tail and underserved languages! For ML practitioners the main takeaway is that the gains come less from scaling generic models and more from end-to-end system design: broader and cleaner multilingual data pipelines, synthetic data generation, tokenizer/vocabulary expansion, specialized MT training recipes, retrieval-augmented translation, and much stronger evaluation tooling. The paper’s most operationally relevant result is the efficiency curve, as the specialized 1B–8B MT models reportedly match or beat a 70B general LLM baseline on translation quality, which means there is a much better cost/quality tradeoff for production translation workloads. The acceleration that we are seeing for language translation is actually mind blowing, and it is great to see that we are able to also extend these into underserved languages as well.

Karpathy’s No Priors Podcast

Andrej Karpathy has predicted the future and impact of agentic systems at every stage; this is one of the best podcasts from him around where he shares the main following insights: Karpathy argues that the practical frontier has shifted from writing code to orchestrating many coding agents, where the key bottleneck is no longer typing or even compute alone, but the operator’s ability to structure tasks, prompts, memory, and evaluation loops. For production ML practitioners, the most important takeaway is that agents already work best in domains with clear objectives and verifiable metrics - software engineering, systems optimization, hyperparameter tuning, kernel work, and experimental search. The near-term opportunity is not vague “AI automation”, but building autonomous loops around measurable workflows. In this podcast he also talks about auto-research, the minigpt project, claw-like systems and more - definitely worth checking out.

AI Redrawing Databases

The next wave of AI product differentiation will be won or lost in the data layer: the teams that can serve real-time, full-fidelity, agent-ready data cheaply and reliably will ship better AI systems faster, and clickhouse shares a great perspective: ClickHouse argues that AI is not just adding load to existing data systems, but changing the shape of production data workloads altogether; agentic apps, conversational analytics, and AI-driven SRE all require high-concurrency, low-latency access to full-fidelity data, which batch-oriented warehouses and siloed observability stacks handle poorly. For production ML teams, the practical takeaway is that serving AI features reliably now depends on the data plane as much as the model layer, making transactional and analytical systems to need tighter integration, natural-language analytics can fan out into many parallel queries, and LLM/agent observability needs long-retention / granular event data rather than sampled telemetry.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #378

Sun, 15 Mar 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

The State of Data Agents
Agentic Engineering Quality
Scaling Python Performance
Hacking McKinsey’s Platform
DuckDB Benchmarks in Macbook Neo
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The State of Data Agents

Every organisation is trying to build their own data agents; but what does this even mean? This is a great comprehensive paper that tackles this by standardising terminology and challenges in industry: This paper argues that “data agents” need a clearer framework to define the various flavours in which it’s appearing before the term becomes meaningless. It proposes a six-level taxonomy from L0 manual workflows to L5 fully autonomous; it is interesting to see that there is an emergence of terminology even at the fringes of agentic tooling. For production ML practitioners, the useful takeaway is that most real systems today are still better thought of as assisted tooling or partial-autonomy operators rather than truly autonomous end-to-end agents: they can help with tuning, cleaning, querying, retrieval, reporting, and multi-step analysis, but they still need human-designed workflows, guardrails, and supervision. Like everything else in the agentic fields, this is a rapidly evolving domain so it’s going to be interesting to see how this changes within even this year.

Agentic Engineering Quality

As of today, many of the PRs that coding agents write would not be merged; this is a really interesting finding from a recent study from METR: This research argues that SWE-bench pass rates materially overstate real-world coding usefulness: across 296 AI-generated PRs reviewed by actual maintainers from scikit-learn, Sphinx, and pytest, roughly half of benchmark-passing patches would still not be merged, even after normalizing against human-written “golden” patches to account for reviewer noise. For production ML practitioners, the key takeaway is the reminder that chasing a metric for optimization may not result in improved performance if it’s not directly aligned with the actual objectives. In this case the paper does not claim current agents fundamentally cannot improve with better prompting or iteration, but it does show that benchmark scores alone can mislead teams evaluating coding agents for real software workflows.

Scaling Python Performance

“Python’s performance sucks” - Yes, but… that’s not the end of the story. Can python be fast? Yes: Performance engineering in Python is not a niche concern, so it’s important to be aware of the “optimization ladder” available to us, and which we can activate to gain real performance optimizations. These are some great options that you can use to drive performance gains: 1) Upgrade CPython to gain

non-trivial performance gains. 2) Compile your typed python with mypyc can deliver strong wins if your code is already typed. 3) Leverage NumPy/JAX to drive massive performance gains with vectorizable array math. 4) You can use Numba to accelerate particularly for numeric loops over arrays. 5) If none of these work, then you can go low level and rebuild core components with Cython/Rust/etc. The most practically useful insight is that realistic pipelines often bottleneck on Python object creation and parsing, not just raw compute, so the biggest gains can come from changing data representations or moving parsing and hot paths out of Python objects entirely. This is a great article on practical Python performance optimizations; it’s often best to go back to the foundations to drive the most value.

Hacking McKinsey’s Platform

AI Agents are making SQL injections ubiquitous again! McKinsey seems to have been the latest victim to agentic vulnerabilities: CodeWall claims its autonomous agent found an unauthenticated SQL injection in McKinsey’s internal AI platform (aka Lilli), and chained it with other weaknesses to gain read/write access to production data, including chat logs, files, user accounts, system prompts, and RAG metadata. This is brutal; a stark reminder that AI platforms inherit classic application security risks while adding new, higher-impact failure modes around prompts, RAG data, and agent workflows. For production ML practitioners, the main takeaway is that securing the model is not enough: the real attack surface spans APIs, document pipelines, vector stores, prompt/config storage, and authorization boundaries. Indeed, let’s not make SQL injections ubiquitous again!

DuckDB Benchmarks in Macbook Neo

We all saw the launch of the Macbook Neo last week; but the real question is: how much can it DuckDB? The answer is of course “Yes”: The DuckDB team did a small benchmark of the entry-level MacBook Neo as a useful reminder for ML practitioners that local analytics performance is increasingly good enough for nontrivial data work, even on constrained hardware. It was surprised to see that DuckDB with tuned memory limits and out-of-core execution performed pretty competitively despite having the same chip as the iphone (and maybe even less RAM than some models?). Of course if your objective is to do local data computation, then don’t get this hardware, but it’s more interesting to think about what is going to be unlocked with more and more capabilities on edge processing; DuckDB really can make low-cost laptops viable for occasional large-scale local analysis, prototyping, and client-side data exploration - and as we know, agents love these.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #377

Sun, 08 Mar 2026 00:00:00 GMT

Our new project went viral

last week on Linkedin and Reddit 🤠🤠🤠

A new plugin for Vim to add Power Mode 😎👇

If you want to support the momentum, do try it out, reshare, and/or give the repo a star ⭐

https://github.com/axsaucedo/neovim-power-mode 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

QWEN-3.5 Challenging Sonnet
Simon Willison on Agentic Patterns
The Q, K, V Matrices
What is Entropy?
META AMD GPU Workloads
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

QWEN-3.5 Challenging Sonnet

The race to build reliable multimodal agents is ON, but is the future open? QWEN-3.5 gives us hope that it is… Sonnet-like performance in your local machine: Qwen3.5 was released a few weeks ago, however the community is now starting to report impressive performance comparable to some of the more popular proprietary models - this is really exciting. To dive into the internals briefly, Qwen3.5-397B-A17B basically combines sparse MoE and hybrid attention so only 17B parameters are active per token, but still able to show strong performance across reasoning, coding, tool use, vision, document understanding, video, and multilingual tasks. For ML practitioners the key takeaways are less about absolute benchmark wins and more about the system design: early text-vision fusion, extensive RL-based post-training for agent behavior, support for 201 languages and dialects, as well as infrastructure advances such as FP8 training, heterogeneous multimodal training, and asynchronous RL that aim to improve throughput, stability, and cost efficiency. It is really hard to keep up, but open source (open weight) releases like these really set the example and raise the bar for organisations trying to hold the moat with proprietary closed models.

Simon Willison on Agentic Patterns

Agentic engineering is quickly becoming one of the most important shifts in how production software and ML systems get built, but how do we build these right? Simon Willison shares a really great write-up on Agentic Engineering Patterns, where he is documenting live as a guide for devs using coding agents to accelerate real software work - note, not vibe coding but agentic engineering. This has quite a lot of sound (+ sometimes obvious) advise, such as ensuring supervision, secure code execution, and iterative testing. For production ML practitioners, the main point is that as code generation becomes cheap, the real challenge shifts to building workflows that preserve reliability, review quality, and team judgment - the team part is the hardest. This is definitely one of the many resources that will be important to keep a close eye on!

The Q, K, V Matrices

Attention has really been all you need so far… understanding the internals is a great way to boost your skills, and the core foundation is the Q, K, V Matrices: The core implementation of transformer based architectures relies on the attention mechanism which is formulated by the Q, K, V Matrices, and this is a great deep dive to refresh or learn about it. This basically introduces the matrices end to end with a hands-on NumPy example, including the nuanced multiplications, as well as connecting them to dot-product attention, where query-key similarity produces attention weights that are applied to values. For production ML practitioners, the key value is in demystifying self-attention at the tensor level while also explaining why separate projections matter and how choices like per-head dimension affect the usual trade-offs between model capacity, memory use, and compute.

What is Entropy?

Understanding entropy is foundational to how we reason about uncertainty, information flow, and system behavior - this becomes even more important with the growing popularity of probabilistic systems with multi-agent architectures: This is one of the best explainers / introductions to the concept of entropy, which is best framed as a measure of uncertainty over possible underlying states rather than as vague “disorder” - also one of the best visual explainers in the topic. Shannon entropy is also a good start to explain the expected surprise, then showing the same idea carries into physics, where a macrostate can correspond to many microstates and entropy grows with the number and probability of those compatible configurations. For production ML practitioners, the useful takeaway is that entropy is not mystical; it is a practical concept for reasoning about uncertainty, compression, latent-state ambiguity, and how modeling choices about state abstraction can shape both system behavior and interpretation.

META AMD GPU Workloads

In production ML, efficiency increasingly determines success, and Meta published their approach to large-scale training in AMD swarms: Meta released RCCLX as an open-source framework to tackle one of the biggest practical bottlenecks in large-scale training and inference with GPU communication overhead. For production ML teams, the key point is that Meta is adding a new backend for shipping communication optimizations for AMD platforms through Torchcomms, which now uses Direct Data Access collectives to cut intra-node latency, which is particularly key for decode-heavy LLM inference. Meta reports meaningful internal gains on MI300-class hardware, including faster decode and prefill performance, lower time-to-incremental-token, reduced latency, and higher throughput, while keeping numerical accuracy within acceptable bounds for its workloads. It does seem like the sub-field of GPU engineering will only continue to explode in popularity, so if you are a practitioner that wants to stay relevant amid the rise of AI, this would definitely be a relevant domain!

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #376

Sun, 01 Mar 2026 00:00:00 GMT

New project release!

We are releasing a new exciting project: the Pydantic AI Server 🚀

And it comes out with another cool name 😎👇

If you want to support the momentum, do reshare, open an issue, and/or give the repo a star ⭐

https://github.com/axsaucedo/pydantic-ai-server 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

The Pydantic AI Server
Visual Intro to PyTorch
The Good Open Source AI List
New Python Sandbox for AI
Sebastian Raschka’s 8th Chapter
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The Pydantic AI Server

Excited to introduce the Pydantic AI Server as a brand new framework to tackle one of the biggest painpoints of productionising multi-agent systems! This new framework allows you to basically wrap your Pydantic AI agents without doing any changes - as always best to show an example, say you have your agent:

# agent_file.py
from pydantic_ai import Agent

my_agent = Agent(
    model="test",
    instructions="You are a helpful assistant.",
    defer_model_check=True,
)

@my_agent.tool_plain
def greet(name: str) -> str:
    """Say hello to someone."""
     return f"Hello, {name}!"

Install pydantic-ai-server[cli] and run your agent:

pais agent run agent_file:my_agent

And we can deploy to our multi-agent network using KAOS:

pais agent deploy agent \
    --image <registry>/agent:v1 \
    --build agent_file:my_agent --push \
    --modelapi my-api \
    --model gemini/gemini-flash \
    --sub-agent <your-other-subagents>

With a few commands you now can enable your agent with HTTP API (w SSE support), conversation memory, remote tool dispatch, multi-agent delegation, observability, etc - as well as direct deployment to Kubernetes! Check it out, and if you want to support, do share, star or add an issue!

Visual Intro to PyTorch

The fastest way to level up your ML knowledge is by learning the fundamentals, and this is a great visual intro to the PyTorch core components: When using PyTorch you can get started with a minimum surface area to build and train a small model end-to-end. This deep dives starts with tensors as the core abstraction, shows how real inputs reduce to numeric tensors, then builds intuition for autograd, and finishes with a small “production-shaped” training example on tabular house-price data. This is quite a good example as it provides the standard workflow with split train/test, defining an nn.Module MLP with ReLU activations, train with an explicit loop using Adam + MSE, switch to model.eval() and torch.no_grad() for inference, de-normalize predictions for human-readable metrics (MAE/MAPE), and save weights via state_dict(). It is always great to dive back into the fundamentals to ensure your knowledge doesn’t get rusty.

The Good Open Source AI List

If you’re building ML systems in 2026, Chip Huyen released an actively updated list of the top and fastest growing AI open source projects sorted by category and sub-category, super useful: The Good AI List is updated daily as a leaderboard for tracking what’s trending in open-source AI on GitHub indexing ~15K repos / ~150K developers. I really like how it is possible to find the fastest growing projects with 1-day and 7-day star deltas, which shows right now there’s a massive explosion of accelerated popularity for repos across the board. For production ML practitioners, the practical value is triage and discovery, we can use it to spot fast-moving infra (agent runtimes, sandboxes, orchestration, RAG plumbing), emerging “skills/prompt” ecosystems, and tooling that’s gaining real adoption signals. If you haven’t this is definitely a resource that is worth checking out and bookmarking for reference as well.

New Python Sandbox for AI

Pydantic is dropping a new rust-based Python interpreter specifically built for LLM/Agent execution, this “codemode” space is definitely an interesting area to keep an eye on: Monty is an open-source Python “bytecode” VM written in Rust that’s built with security by design as a sandbox environment. The key idea is “start from nothing, then allowlist capabilities” - it seems that Monty code has no filesystem, network, env var access by default, and the only way it can touch the outside world is via external functions you explicitly expose, which are the tools you’d normally register for tool-calling. This is inspired from Anthropic’s code mode, which allows for an execution environment that allow agents chain commands. It is also interesting that in this case the Python VM is optimized for speed, with sub-millisecond startup times, and low processing costs, together with good old Pydantic type checking. It is quite exciting to see non-cpython based implementations of Python, as this is definitely going to spark quite a lot of innovation in the space.

Sebastian Raschka’s 8th Chapter

Sebastian Raschka is at it again with his (WIP) Manning book “Build A Reasoning Model”, and is sharing for free the working code and exercises, which are an absolute goldmine: This week he released the 8th “bonus” chapter with a practical data-gen pipeline for knowledge distillation of reasoning models. Basically given a JSON of math problems + ground-truth answers, it generates teacher outputs (both a thinking trace and a final answer) and writes them incrementally to a new JSON dataset you can use to train a smaller reasoning model. This is really interesting as it allows us to turn expensive/slow reasoning into something that we can ship and explore on smaller models with low latency / less costs to get them to perform. Definitely worth checking out.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #375

Sun, 22 Feb 2026 00:00:00 GMT

New project extension!

As part of the DuckDB Extension we announced last week, we are now releasing a Terminal UI to Query your Agent Data 🚀

Short video of the TUI 👇

If you want to support the momentum, do reshare, open an issue, and/or give the repo a star ⭐

https://duckdb.org/community_extensions/extensions/agent_data 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Monitoring Multi-Agent Systems
CircleCI 2026 Delivery Report
Distributed Systems Simulation
Karpathy’s 200-line MicroGPT
From Memes to Websites
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Monitoring Multi-Agent Systems

Sharing a new blog post on “Monitoring KAOS: Observability for Multi-Agent Systems with OpenTelemetry” 🚀 This is a thorough deep dive that explores how to instrument distributed agent loops that span across multiple Agents, MCP Servers and Model APIs. The main premise is basically that multi-agent systems work - that is, until they don’t. Your prod user complains: “it took forever” / “it gave a strange response” / “it worked yesterday”; and your request/response show no errors… In agentic systems monitoring is more complex, as request flows involve complex hops - eg: HTTP POST /v1/chat/completions coordinator.agent.agentic_loop (trace_id: abc123) ----> coordinator.model.inference ----> coordinator.delegate.researcher ---- ----> researcher.agent.agentic_loop (trace_id: abc123) ---- ----> researcher.model.inference ---- ---- -----> researcher.tool.web_search ---- ----> coordinator.model.inference ----> coordinator.delegate.analyst ----> analyst.agent.agentic_loop (trace_id: abc123) ---- ----> analyst.model.inference ---- ----> analyst.tool.calculator…and requests can last anywhere from 100ms to 60+ seconds depending on what the agent decides to do. In this post we dive into how we can distributed agent loops observable using: Traces: “what path did the request take through agents/tools/models?” Logs: “what did the agent think / decide at each step?” Metrics: “is this getting worse over time?” We take a complex multi-agent use-case consisting of an Agent Supervisor with multiple Researcher + Analyst sub-agents, and walk through each of these components showing how to perform root-cause-analysis using the right tools and techniques. Check it out!

CircleCI 2026 Delivery Report

The 2026 State of Software Delivery makes it clear that GenAI is starting to show major productivity gains, but these are (VERY) unevenly distributed: CircleCI analyses 28.7M+ CI workflows to understand the main trends on software delivery, which include AI adoption, blockers, tooling and more. The main thing that really jumps out is that there is a really large widening the gap between teams who can reliably ship and those who can’t. Average CI throughput via daily workflows rose 59% YoY, however these gains are heavily concentrated among the top 5% performers, which basically doubled throughput, whilst the median and bottom-quartile teams saw very little improvement. The most interesting insight is that integration/validation can’t keep up with the development throughput speed. Most teams see more activity on feature branches (+15%) but main-branch throughput is actually decreasing (−6.8%) which means more code gets written than successfully promoted to production. Reliability metrics seem are also worsening, with main-branch success rate dropping to ~70.8%, with median TTR rising to ~72 minutes, meaning that AI-generated complexity is resulting in more frequent breakages and longer debugging cycles. The pattern that is clearly “winning” is teams that are able to scale validation and recovery with volume. This is the time to figure out how to enable the accelerated speed; this is one of the key points that I raised as part of my keynote at WeAreDevelopers last year, namely calling out the gap in GenAI maturity at the lower developer funnel (starting at 6:23).

Distributed Systems Simulation

This is an absolutely brilliant (sims-like?) application that allows you to set-up a system and simulate how it would perform at certain traffic/scalability levels. It’s basically a lightweight System Design Simulator that turns architecture diagrams into an interactive playable model which you can drag-and-drop. It has quite a pretty broad set of components, it includes LBs, API gateways, caches, DBs, queues (even AI blocks), and you can then basically see how your system thrives or crumbles in real time. It even has some “chaos toggles” like traffic spikes, cache-miss storms, partitions, crashes so you can quickly explore failure modes and trade-offs without provisioning infra. I have to say this is pretty awesome, I would love for future educational resources to leverage some of these type of approaches, it’s been really interesting to see the type of things that people are inventing these days. It basically feels like a race of creativity and innovation, it’s great!

Karpathy’s 200-line MicroGPT

Andrej Karpathy is at it again with now a (micro) 200-line implementation of the GPT model behind ChatGPT/Claude/etc; as always with a really intuitive and easy to understand walkthrough: This MicroGPT implementation really is the “minimum viable GPT” as it’s the smallest / simplest implementation I’ve seen with only ~200-250 lines of dependency-free Python. It basically contains the full flow including data loading, a tiny character-level tokenizer (plus BOS), a scalar autograd engine, a GPT-2-like Transformer (embeddings, RMSNorm, multi-head causal self-attention with an explicit KV cache, MLP, residuals), Adam, a training loop with cross-entropy, and an autoregressive sampling loop with temperature. It is pretty crazy to see how Andrej Karpathy is able to continue to provide some of the best educational resources on the topic whilst being able to stay on top of all the innovation going around.

From Memes to Websites

On the topic of side-projects: This weekend I asked copilot to build me a joke website about agentic software for the newsletter. Within a few minutes it found three pictures of me, created a website, and it’s actually pretty funny! It configured the github pages, made it mobile friendly, added dark/light theme - I am impressed, check it out! Seems like now even it’s so cheap you can even make websites instead of memes… And this is not even what was planned to share as the main OSS project this week; as shown in the intro, the main exciting launch is the development of a TUI for the DuckDB Extension to query your Claude/Copilot; check this one out as well!

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #374

Tue, 17 Feb 2026 00:00:00 GMT

New project release!

We are thrilled to announce the release a new DuckDB Extension for Querying Agent Data 🚀

Link below released to official DuckDB documentation site 😮 as well as the Streamlit showcase 👇

If you want to support the momentum, please do reshare, open an issue, and/or give the repo a star ⭐

https://duckdb.org/community_extensions/extensions/agent_data 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

The DuckDB Agent Data Extension
State of Data Eng Report
New Tabular Foundation Model
Speeding Up LLM Inference
Gemini Deep Think 3
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

DuckDB Agent Data Extension

Excited to announce a new project release! Our DuckDB extension for querying Agent Data, which is now officially part of the DuckDB Community Extensions 🚀🚀🚀 This means you can load it directly in your duckdb session with:

INSTALL agent_data FROM community; LOAD agent_data;

This has been something I’ve been looking forward for a while, as there is so much you can do with local Agent data from Copilot, Claude, Codex, etc; now you can easily ask any questions such as:

— How much have I used Claude Code recently?

SELECT date, message_count, tool_call_count

FROM read_stats() ORDER BY date DESC LIMIT 10;

— Which tools does github copilot use most? SELECT tool_name, COUNT(*) AS uses FROM read_conversations(’~/.copilot’) GROUP BY tool_name ORDER BY uses DESC; This also has made it quite simple to create interfaces to navigate agent sessions across multiple providers. For this, the repo comes with a simple Marimo example, as well as a Streamlit example that allow you to play around with your local data.

Best thing is that you can do this from the comfort of the proven and tested DuckDB engine without any dependencies. Besides extending to other providers (Gemini, Codex, etc), there are also interesting avenues exploring streaming, and other features.

Check it out - do share feedback and thoughts!

State of Data Eng Report

This “State of Data Engineering” is one of the best reports I have seen, and the interactive charts are one of the best UX I’ve come across - key insights:

cloud data warehouses remain the default (~44%)
lakehouse adoption continues to grow (~27%)
architecture choices vary by org size
individual AI tool usage is now pervasive (82% daily+)
organizational AI maturity still lagging

Some of the top challenges highlighed:

the biggest blockers are organizational
data modeling stands out as a widespread pain point
unclear ownership
long-term maintainability issues
burden of firefighting

This is a great interactive experience, check out the overview, as well as the interactive charts, huge kudos for such a great interactive experience + using DuckDB-WASM!

New Tabular Foundation Model

Inria launches a new tabular foundation model! This space is one of the most exciting areas of “boring ML”, as it could be transformational for key areas like risk, fraud, ops, pricing, forecasting and more: This is a hard problem as tabular datasets are highly heterogeneous; this new foundation model TabICLv2 makes training-free in-context learning practical for real tabular workloads, and one of the things that is still hard to believe is that it’s trained mostly in sinthetic data (which seems to be the case for most). From an architecture side it includes a scalable softmax attention temperature scheme to avoid attention degradation as the number of rows grows so it can generalize to much larger tables without having to pretrain on prohibitively long sequences. It also has improved pretraining protocol, showing in benchmarks across TabArena and TALENT that out of the box surpasses various other models. It is quite interesting to see how these models are evolving at fast speed, with various competing models and architectures appearing every couple of months - this is certainly an exciting field to keep an eye on!

Speeding Up LLM Inference

Inference speed is becoming the new MOAT, the recent Opus 4.6 fast vs Codex fast is a good example - what is most interesting is the different approaches that orgs are taking to get there:

Anthropic’s fast mode seems that keeps the same Opus 4.6 model but runs it with much smaller batch sizes: you pay a big premium to avoid queueing/throughput optimization, improving per-user latency/throughput while reducing overall hardware efficiency.

vs 2) OpenAI’s fast mode instead achieves an order-of-magnitude token/s jump by serving a different model (Codex-Spark) on Cerebras wafer-scale chips, whose large on-chip SRAM can keep more of the model in fast memory and avoid weight streaming bottlenecks.

The takeaway seems to be that “fast” can mean either premium low-batch serving of the same model (speed via scheduling/efficiency trade) or specialized hardware enabling a smaller model at extreme speed (speed via architecture/model swap). The business question is whether higher token/s actually helps end-to-end developer productivity when error rates and rework dominate, or whether we’ll be able to get our cake and eat it and get both. I feel like we are seeing the new CAP table in the making for ML!

Gemini Deep Think 3

Google has released Deep Think 3, which basically takes Deep Research to it’s absolute limit, showing performance of 84.5% on ARC (vs ~65 Opus 4.6!), some really exciting insights: Google claims some really huge jumps on hard reasoning benchmarks including 48.4% (no tools) on Humanity’s Last Exam, 84.6% on ARC-AGI-2, Codeforces Elo 3455, and gold-medal–level performance on IMO 2025. From a production ML practitioner lens, it sounds like we can treat this as “reasoning-as-a-component” that could improve complex analysis, code generation for simulation/modeling, and review/verification workflows. It is really surprising to see how fast Google is taking over all other competitors and really leaving them far behind as this is integrated into their entire cloud and workspace environments, it will be interesting to see how the rest replies.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #373

Sun, 08 Feb 2026 00:00:00 GMT

We are releasing Code Mode + Graph for the K8s Agent Orchestration System 🚀

Check out the demo 👇

If you want to support the momentum, please do reshare, open an issue, and/or give the repo a star ⭐

github.com/axsaucedo/kaos 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

CodeMode and Graphs in KAOS
Building TikTok RecSys from Scratch
RL-Aligned Ranking in RecSys
Metaflow + Kubeflow Announcement
The Waymo World Model
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

CodeMode + Graph in KAOS

KAOS now shows multi-agent systems in visual graphs that allow you to manage advanced concepts like Anthropic’s code-mode: We have extended KAOS to support an interactive graph-visualisation screen that allows you to create and manage agentic resources intuitively. This enables advanced functionality such as introducing code-mode sandboxes which are now growing in popularity as an effective way to improve agent-to-mcp performance by aggregating multiple MCP servers and exposing them as local functions. We have extended support for Port-of-Context which is one of the first code mode implementations, providing a typescript code engine in rust, which is now exposed in KAOS as a native runtime that can be spinned up. To provide some intuition on how code mode works, traditionally an agent would have to make multiple MCP round trips as follows:

LLM: “I’ll call add(42, 8)” > Call -> Result LLM: “Now I’ll call multiply” > Call -> Result LLM: “Now I’ll call power” > Call -> Result LLM: “Now I’ll call uppercase” > Call -> Result LLM: “Now I’ll call word count” > Call -> Result LLM: “Here’s the final answer”

Now instead we can have a “code-mode” abstraction that converts all mcp servers into a unified set of functions, such that the agent just makes one call that is simple code - same example above would look like this:

LLM: "I'll call the following set of mcpservers through the code mode execution sandbox:
'''
  sum = await calc.add(...);
  product = await calc.multiply(...);
  squared = await calc.power(...);
  wordCount = await text.word_count(...);
  return result = await text.uppercase(...);
'''
  -> Call -> Result
LLM: "Here's the final answer"

This basically reduces the round-trips significantly and enables for a much more controlled experience to the LLM itself. There are various exciting initiatives also exploring this, including the recent announcement from the Pydantic team releasing a new Python runtime in Rust that will serve as the first Python code-mode environment. Check out the article for more details.

Building TikTok RecSys from Scratch

Build TikTok’s Personalized Real-Time Recommendation System from Scratch. This is a fantastic talk from Jim Dowling’s Hopsworks on basically how to build a TikTok-style personalized real-time recommender in Python using Hopsworks as the ML platform layer (e.g. feature store, vector index, model registry, and serving). It is particularly interesting to see such a practical example that demonstrates the theoretical foundations on what is one of the most impressive (and scariest) instant learning systems out there. The real time nature of retraining and capturing fresh features from user actions (eg views/likes/watch time), and making them available for inference is now a critical architectural requirement for any consumer facing teams. This is a pretty meaty talk/workshop which walks through an end to end architecture split into a feature / training pipeline that learns a two-tower embedding model from interaction data (e.g. user/query + video/candidate), and an online inference pipeline that embeds the latest user context + retrieves similar candidate videos via ANN/vector search to serves results through a simple Streamlit UI. Definitely a great resource to catch up for any MLOps practitioner out there!

RL-Aligned Ranking in RecSys

RecSys ranking is shaping what billions of people watch, buy, and believe, so small improvements (or mistakes) can have a huge impact on user experience - this is a great resource that explores LLM RLHF style approaches in recommender systems: It is quite interesting to take this current angle, namely while modern recommender systems ranking models are typically trained to predict click/engagement/return and then combined with a hand-coded value function, they rarely optimize that value end-to-end, creating a gap compared to LLM pipelines pre-RLHF. There seems to be an opportunity to treat the ranking model as a policy over items and adding an off-policy, propensity-weighted policy-gradient-style loss that directly maximizes a scalar value/reward model (analogous to an LLM reward model) while using constraints (akin to KL regularization) to control drift. This basically means that recsys optimization could be extended to the end to end, such that ranking becomes a true reward optimization without changing inference latency. It is quite interesting to see that some RL approaches that have been revolutionising LLMs are now being brought into more traditional fields of ML, as it seems there’s quite a lot of opportunity (and everyone loves a chance to play with RL).

The Waymo World Model

“What should a self driving car do if a huricane appears infront of it?” These are the corner cases that are hard or impossible to train models on, which is why it’s exciting to see World Models in practice already at Waymo being used for synthetic-data-like generation. Basically, Waymo’s World Model is a generative simulation system adapted from Google DeepMind’s Genie 3 (exciting to see applications already!) that creates hyper-realistic, interactive 3D driving environments and produces multi-sensor outputs including camera and lidar, letting Waymo run billions of virtual miles to validate safety beyond what fleet data alone can capture. This is pretty awesome; it basically can synthesize rare long-tail scenarios (e.g. extreme weather, unusual objects, safety-critical events) without the need for this data to exist. Apparently it is also designed to be highly controllable via driving-action, scene-layout, and language conditioning for counterfactual what-if testing. This is exactly the type of interesting applications that will become exciting, this is the type of thinking that should go with these models, not just “it will replace videogames”, as if anything it is augmenting various fields already.

Metaflow + Kubeflow Announcement

Workflow orchestration is the backbone of production ML, so it was quite interesting to see the recent integration between Kubeflow (yes still alive!) and Metaflow: This seems like primarily an ability to write native Metaflow workflows and execute them with a backend in Kubeflow (ie in k8s) as Kubeflow Pipelines alongside current KFP workloads - which to my memory means they basically run as Argo Workflows / Tecton pipelines. Great to see some of the OGs of the MLOps space are still driving quality-of-life integrations to continue to evolve and improve the ecosystem. I believe Vertex AI still uses Kubeflow under-the-hood so this sounds like it would bring some robust benefits that metaflow brings (+ convenience).

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #372

Wed, 04 Feb 2026 00:00:00 GMT

We are releasing Agentic Monitoring for the K8s Agent Orchestration System 🚀

Check out the preview demo 👇 Post coming next week!

If you want to support the momentum, please do reshare, open an issue, and/or give the repo a star ⭐

github.com/axsaucedo/kaos 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Claude Code: It’s Not Replacing Devs
Agentic Monitoring Release of KAOS
OpenAI’s In-House Data Agent
DeepMind’s Take on World Models
MeiTuan Bi-Lingual Image Model
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Claude Code: It’s Not Replacing Devs

Excited to finally share my new article: “Claude Code: It’s not replacing devs. It’s moving them to a higher altitude.” 🚀 In this post I reflect on where the software engineering practice is going as a whole as it’s transformed by agentic coding tools.

TL;DR My biggest observation after building with agentic tooling is that the real unlock isn’t about “faster coding”; it’s about operating at a different cognitive level of abstraction. This creates a new reality, and when you can generate and integrate at a higher level, the differentiator becomes: 1) what you choose to build; 2) how precisely you specify it, and; 3) how efficiently you verify it. Depending on the day of the week you may encounter one of these: “AI is making developers 10x”, “AI is making developers less productive”, “AI is coming for our jobs”, “AI enables coders”. Ironically I explore this with a meme that may give us the closest answer: specs as the age-old code abstraction. And if specs become the primary artifact, then the next bottleneck is the execution engine. Namely the runtime that continuously reconciles intent vs reality (plans, delegates, validates, observes, and iterates); and “code” is just one of its outputs. It is now on us as practitioners and leaders to navigate this “shift upwards” in the cognitive stack; individuals can now do what teams could; teams can do what departments; and what follows should be able to invent the future.

OpenAI’s In-House Data Agent

OpenAI shares how they tackled text-to-sql at ChatGPT scale, and there are some learnings that most organisations can take: It is interesting that OpenAI is exploring the same domains that many organisations are when it comes to leveraging agentic systems to automate analytics and insights gathering. This is basically the usual translation of natural-language question into end-to-end analytics, including table discovery, SQL generation/execution, iterative self-correction, and synthesis. It seems that they tackle the same foundational basics, including schema/lineage and historical query patterns, with the ever-required human annotations. There are some interesting hints on Codex-derived code-level table semantics, which sounds like is what helps bring more annotated capabilities. It seems like their main advantage is the continuous eval systems they have built together with the curation from human annotations, but it seems like similar to the broader industry they are still also figuring out what works and what doesn’t. This is going to be quite interesting once we hit the claude-code moment for copilots in the space of analytics and BI; which we shouldn’t be too far at this stage!

DeepMind’s Take on World Models

This past week it seems more people are becoming aware of Foundation World models following DeepMind’s latest release of Genie: This basically encompasses models that are trained to predict images (aka worlds) from inputs for movement similar to that of a videogame. We have been covering various world models across the last few years, and it is mind blowing how fast these models are improving. One of the most exciting parts of these models is how at least it’s suggested that these may open a new way to capture more signal from the environment than other approaches (eg. laws of physics, interactions, etc). It’s great to see that we are now starting to see some of these models released as services as we’ll be able to see what actual value will be unlocked once it is taken to the test in the real world.

MeiTuan Bi-Lingual Image Model

Another interesting Large-Model from Chinese Giant MeiTuan, this time covering a bi-lingual text-to-image architecture that has impressive performance: LongCat-Image is an open-source bilingual diffusion foundation model that supports English and Mandarin. It is interesting to see the core model only at around 6B parameters which is a reasonable size for balancing VRAM, latency, and serving cost. The authors attribute most gains to an industrial-grade data and training pipeline which includes a 1.2B-sample corpus with heavy filtering/stratification plus explicit suppression of AI-generated contamination. It is also interesting to see that also now image models are seeing post-training alignment using similar techniques than LLMs such as multi-objective RL signals. It seems every week we get a new impressive model, but now it is not just about the size but also about performance as users want to be able to access it in commodity hardware.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #371

Sun, 25 Jan 2026 00:00:00 GMT

Thanks to everyone for your support on our announcement last week releasing the K8s Agent Orchestration System (KAOS)!

If you want to support us with the momentum please do reshare, open an issue, and/or give the repo a star ⭐

github.com/axsaucedo/kaos 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Releasing the KAOS Framework
OpenAI Scaling Postgres to 800m Users
FastMCP 3.0 Now Released
Anthropic’s Original Take-home Test
7 Deadly Sins of Eng Productivity
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Releasing the KAOS Framework

Thank you all for your support during last week’s announcement release for the K8s Agent Orchestration System (KAOS) to help manage distributed agentic systems at scale 🚀 The KAOS Framework addresses some of the pains of taking multi-agent / multi-tool / multi-model systems to hundreds or thousands of services! It started as an experiment to build agentic copilots, and has progressed as a fun endevour building distributed systems for A2A, MCP Servers, and model inference! The initial release comes with a few key features including: 1) a golang control plane to manage Agentic CRDs; 2) a python data plane that implements a2a, memory, tool / model mgmt; 3) a React UI for CRUD+debugging, and; 4) a robust CI/CD setup with KIND/pytest/ginko/etc. I have to say I am impressed on the level of abstraction that is possible to reach with agentic copilots when covering frameworks and domains I have experience in, a blog post will follow on this topic specifically! For the meantime do check out the repo, docs and examples to try it out - if you have any feedback or run into issues please do submit an issue!

OpenAI Scaling Postgres to 800m Users

OpenAI shared this week how they scaled their database to 800 million users, an there are some interesting surprises: OpenAI has had to deal with explosive growth throughout the last few years, and it is interesting to see how they’ve been able to scale to millions of queries per second, however at a clear tech debt cost. Surprisingly they are running a single primary PostgreSQL server for writes, with a set of 50 geo-distributed read replicas; the main blocker seems to be the migration cost towards scaling the write replicas due to hundreds of internal applications require migrations. On the optimization it seems like they have implemented the usual suspects by offloading reads to replicas, migrating shardable write-heavy workloads to sharded stores like Azure Cosmos DB, and reducing unnecessary writes. These are great write-ups that provide an uncommon glimpse into hypergrowth scaleups and the tech debt costs that come from having to scale at such a massive speed (and the tradeoffs required to maintain that speed).

FastMCP 3.0 Released

The agentic stack continues to evolve at lightning speed, this past week with the release of FastMCP 3.0: It is quite interesting to see the fast iterations from these projects in near-real time, in this case taking various features that have evolved organically and integrating them into cohesive / standardised components. In this case it seems like the main releases include the three composable primitives of “components” (tools/resources/prompts), “providers” (OpenAPI, remote MCP servers, other FastMCP servers), and “transforms” (pipeline middleware that renames, namespaces, filters by tag/version, or reshapes schemas). Quite interesting that sometimes it seems we’re going full circle with concepts like ETL or learnings from neighboring areas like MLOps. It is however really great to see some improvements on production features like native OpenTelemetry tracing, background tasks for long-running work, tool timeouts, pagination, and connection pings, etc. Looking forward to see how this project evolves as the MCP ecosystem matures, as we are also yet to see the first iterations since the protocol joined the Linux Foundation.

Anthropic’s Original Take-home Test

Anthropic has publicly released their recruitment take-home, and it has a lot of quite interesting learnings for both indvidual contributors but also hiring managers: What I liked the most when I came across this was that (unsurprisingly) Anthropic set up their take-home test with explicit assumption that candidates would use agentic coding to try to solve it. The test basically seems to have a Python script that simulates a custom computer, and your job is to rewrite the kernel so it computes the exact same outputs while using far fewer “simulated cycles” which forces you to really think about performance. This has some great foundations as it likely touches on optimizing memory, reuse of intermediate results, batch work, etc. It’s also great that the score is clear and visible feedback for the candidate, and it’s also quite funny to see the disclaimer that basically many submissions have basically “cheated” by just changing the tests. Technical interviews have been changing drastically, and we will need to find ways to adapt to this strange new world, so seeing these type of examples bring quite a lot of useful ideas.

7 Deadly Sins of Eng Productivity

Here are the 7 deadly sins of engineering productivity: 1) Context Switching; 2) Task hopping; 3) The Urgency Illusion; 4) Parkinson’s Law (underestimate costs and risks while overestimating benefits); 5) The Zeigarnik effect (aka the mental RAM leak); 6) Decision fatigue, and; 7) Brooke’s law (adding to a delayed project delays it further). This is a pretty good list! I am actually interested to see how 1 and 2 will evolve with agentic coding; however I do still believe these will still need to be key foundations.

Upcoming MLOps Events

Events we are speaking at this year:

eTail Europe - March @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #370

Sun, 18 Jan 2026 00:00:00 GMT

We are thrilled to release the K8s Agent Orchestration System (KAOS) Framework! Learn more about it below!!

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Releasing the KAOS Framework
FLUX: Interactive Visual Intelligence
Counterfactuals for RecSys
Universal Commerce Protocol
DuckDB for Everything Data
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Releasing the KAOS Framework

Super excited to release the K8s Agent Orchestration Framework (KAOS) to help manage distributed agentic systems at scale 🚀 The KAOS Framework addresses some of the pains of taking multi-agent / multi-tool / multi-model systems to hundreds or thousands of services! It started as an experiment to build agentic copilots, and has progressed as a fun endevour building distributed systems for A2A, MCP Servers, and model inference! The initial release comes with a few key features including: 1) a golang control plane to manage Agentic CRDs; 2) a python data plane that implements a2a, memory, tool / model mgmt; 3) a React UI for CRUD+debugging, and; 4) a robust CI/CD setup with KIND/pytest/ginko/etc. I have to say I am impressed on the level of abstraction that is possible to reach with agentic copilots when covering frameworks and domains I have experience in, a blog post will follow on this topic specifically! For the meantime do check out the repo, docs and examples to try it out - if you have any feedback or run into issues please do submit an issue!

FLUX: Interactive Visual Intelligence

Black Forest Labs has released another impressive text-to-image model with impressive low-latency/low-compute: It seems that now we’re on a trend where the next big milestone is ultra-low-latency models designed for interactive real-time applications, and I also believe this is a clear path for success. This is particularly interesting as it claims sub-second end-to-end text-to-image and image editing while running on consumer GPUs (the 4B variant fits in ~13GB VRAM). The key product idea is a single unified architecture that supports text-to-image, image-to-image editing, and multi-reference generation, offered in distilled “fast” variants. The best thing is that it’s actually Apache License which means it’s actually Open Source, so kudos to the team for opening this to the masses.

Counterfactuals for RecSys

Anyone that has run A/B tests for RecSys evaluation knows the challenges, which is why it was super interesting to come across Eugene Yan’s Counterfactual Evaluation for Recommendation Systems: Counterfactual (ie off-policy) evaluation basically helps us estimate “what would have happened if”. Namely this works using logs from the current policy, apparently which is most commonly via Inverse Propensity Scoring which reweights each logged reward by the ratio of the new policy’s probability of the logged recommendation to the production policy’s probability. There are some downsides as apparently IPS can break with insufficient support (aka zero propensities for actions the old policy never took) and can have high variance when probability ratios explode. There are solutions to this problem such as stabilizers like the ones presented as Clipped IPS and Self-Normalized IPS - which is quite interesting, definitely a great resource to an more mature field (but still unresolved) in the MLOps ecosystem.

Universal Commerce Protocol

Google has co-released the Universal Commerce Protocol with top e-commerce brands like Shopify, Zalando, Etsy and Wallmart to enable Agentic Shopping! This is quite an interesting trend that likely will reach other intdustries; particularly in e-commerce this is currently changing the way individuals interact with their day to day purchases - I have found myself using LLMs more often than Web Search to rate and sort products. This standard defines how agents discover a merchant through standardised APIs which allow negotiation, and then transact using stable core checkout primitives augmented by independently versioned extension schemas. It bakes in a human-in-the-loop through an explicit checkout state machine and continue_url handoff, which ensures that shopping experience still remains seamless through an embedded checkout protocol with bidirectional JSON-RPC messaging and delegated UI affordances. It is quite interesting to see that the standads define even payment flows, which are treated as a two-sided negotiation where merchants dynamically advertise allowable payment “handlers” for the specific cart/context and agents execute the corresponding provider-authored specs while keeping raw credentials out of the platform to reduce PCI scope. It really feels like we are living in the future sometimes!

DuckDB for Everything Data

Data processing is the quiet force multiplier behind every successful ML system, and it seems that DuckDB is taking quite some of that market share: DuckDB is an excellent default for ML/data engineering pipelines because it combines the simplicity of an in-process database with OLAP-grade performance for joins and aggregations, often far faster than OLTP engines for analytics workloads. It’s quite a nice single-binary pip install and allows for rapid local iteration (plus CI/testing). I have used it in a few projects and it has been quite impressive particularly the CSV parsing (+ has similar level quality for other parsers on Parquet/JSON/etc, as well as directly from disk/S3/HTTP). There are quite a lot of growing integrations between old guard tech like Postgres and DuckDB, as well as approaches embedding DuckDB, this is really an interesting space to keep an eye!

Upcoming MLOps Events

Conferences for 2026 coming soon! For the meantime, in case you missed our talks:

The State of AI in 2025 - WeAreDevelopers 2025
Prod Generative AI in 2024 - KubeCon AI Day 2025
The State of AI in 2024 - WeAreDevelopers 2024
Responsible AI Workshop Keynote - NeurIPS 2021
Practical Guide to ML Explainability - PyCon London
ML Monitoring: Outliers, Drift, XAI - PyCon Keynote
Metadata for E2E MLOps - Kubecon NA 2022
ML Performance Evaluation at Scale - KubeCon Eur 2021
Industry Strength LLMs - PyData Global 2022
ML Security Workshop Keynote - NeurIPS 2022

Open Source MLOps Tools

Check out the fast-growing ecosystem of production ML tools & frameworks at the github repository which has reached over 10,000 ⭐ github stars. We are currently looking for more libraries to add - if you know of any that are not listed, please let us know or feel free to add a PR. Four featured libraries in the GPU acceleration space are outlined below.

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

As AI systems become more prevalent in society, we face bigger and tougher societal challenges. We have seen a large number of resources that aim to takle these challenges in the form of AI Guidelines, Principles, Ethics Frameworks, etc, however there are so many resources it is hard to navigate. Because of this we started an Open Source initiative that aims to map the ecosystem to make it simpler to navigate. You can find multiple principles in the repo - some examples include the following:

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #369

Sun, 11 Jan 2026 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Databases 2025 Year in Review
Lessons from 21 Years at Google
The State of Prod ML in 2025 🔥
Raschka’s Big LLM Architectures
Measuring AI Ability for Long Tasks
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Databases 2025 Year in Review

The Databases 2025 Year-in-Review from Carnegie Mellon University is out! As always there are some great insights on database trends across last year: As expected Postgres dominance continues to accelerating, particularly with “postgres-adjacent” products, like Databricks acquiring Neon (~~$1B), Snowflake acquiring Crunchy Data (~~$250M), and Microsoft launching a new Postgres DBaaS, together with many other new middleware projects (Multigres, Neki, PgDog). For production ML teams, the biggest operational shift is that databases are becoming “agent-facing” by design, with huge adoption of MCP as a standard way for LLMs to invoke database tools. There are some areas to keep an eye such as the API-compatibility as it’s turning into a legal minefield (MongoDB suing FerretDB), as well as open data formats having another re-surgence with multiple new columnar formats challenging Parquet and pushing new ideas. Databases are core to the day-to-day software operations, so we will likely see some exciting new develolpments in the year to come, especially with the craze on agentic development.

Lessons from 21 Years at Google

“21 Lessons From 14 Years at Google” - the top 5 that resonated the most for me: 1) You can win every technical argument and lose the project. 2) Bias towards action. Ship. You can edit a bad page, but you can’t edit a blank one. 3) Your code doesn’t advocate for you. People do. 4) At scale, even your bugs have users. 5) Most “slow” teams are actually misaligned teams. This really is a great list as it reminds all of us how some of the most challenging and useful lessons are often not related to the code but to the soft skills and people interactions.

The State of MLOps 2025 Survey 🔥

The level of ML maturity of organisations in 2025 has slowly increased, and we have interesting insights on organisational departments/functions from our Prod ML Survey:

In 2025 about 29% of organisations have set up a central machine learning platform team.
However only 27% organisations have established a data platform / data engineering organisation.
There is a significant increase with 17% organisations now establishing an AI Risk & Governance Function.
However only 9% of organisations have established an AI inventory to keep track of all use-cases and models in the organisations.

If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025 🔥

Raschka’s Big LLM Architectures

Sebastian Raschka has put together an overview on the evolution of LLM architectures, together with the most comprehensive set of intuitive visual diagrams. It is interesting to see that Sebastian Raschka argues that these models are still structurally similar to when they were initially released. Positional embeddings have evolved from absolute to rotational (RoPE), Multi-Head Attention has largely given way to Grouped-Query Attention, and the more efficient SwiGLU has replaced activation functions like GELU, however it is yet to be seen if there’s been step-change breakthroughs since inception. Definitely worth diving into what seems to be one of the most comprehensive (intutive) overviews of LLMs out there.

Measuring AI Ability for Long Tasks

METR has released an in-depth study that measures the ability for LLM agents to solve long tasks, and there’s quite a lot of interest insights: In this study they first propose a practical way to quantify agentic LLM capability using a task completion horizon, which involves the human time-to-complete where an AI agent succeeds with 50% probability. Across their datasets it shows that task duration strongly predicts reliability, with near-perfect success on very short tasks but steep drop-offs on hour-scale tasks. Although this is intuitive it is great to see some data, which also shows how frontier models has been following a robust exponential trend since 2019, doubling roughly every ~7 months. There are some interesting developments that attribute to this, including better tool use, reasoning, and recovery from mistakes rather than mere knowledge. This is a space that will only expand in 2026, with a lot of exciting developments ahead.

Upcoming MLOps Events

Conferences for 2026 coming soon! For the meantime, in case you missed our talks:

The State of AI in 2025 - WeAreDevelopers 2025
Prod Generative AI in 2024 - KubeCon AI Day 2025
The State of AI in 2024 - WeAreDevelopers 2024
Responsible AI Workshop Keynote - NeurIPS 2021
Practical Guide to ML Explainability - PyCon London
ML Monitoring: Outliers, Drift, XAI - PyCon Keynote
Metadata for E2E MLOps - Kubecon NA 2022
ML Performance Evaluation at Scale - KubeCon Eur 2021
Industry Strength LLMs - PyData Global 2022
ML Security Workshop Keynote - NeurIPS 2022

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #368

Sun, 04 Jan 2026 00:00:00 GMT

The results for the Survey on Production MLOps are out: ethical.institute/state-of-ml-2025 🚀🚀 Here’s an excerpt on organisational ML maturity:

The level of ML maturity of organisations in 2025 has slowly increased, and we have interesting insights on organisational departments/functions from our Prod ML Survey:

In 2025 about 29% of organisations have set up a central machine learning platform team.
However only 27% organisations have established a data platform / data engineering organisation.
There is a significant increase with 17% organisations now establishing an AI Risk & Governance Function.
However only 9% of organisations have established an AI inventory to keep track of all use-cases and models in the organisations.

If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Prod ML Hits 20k Stars in Github 🚀
Sebastian Raschka LLM Year Review
The State of Prod ML in 2025 🔥
Simon Willison 2025 Review
OpenRouter 100 Trillion Token Analysis
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Prod ML Hits 20k Stars in Github 🚀

Super excited to see that our Github repo on Prod ML just hit 20,000 stars 🚀 It’s hard to believe how much this has grown since starting this list back in 2018 to map components in production ML systems; as of today it’s grown massively with the support of almost 200 contributors, and over 1000 commits! It now contains hundreds of open source projects across dozens of sections; if you find that any OSS framework is missing please do contribute, as pull requests are more than appreciated! We also maintain a list on AI regulation (1.4k ⭐️), and a newer one on Production GenAI (50 ⭐️).

Sebastian Raschka LLM Year Review

Sebastian Raschka has dropped a year in review on LLMs in 2025 together with predictions for 2026, and as always there’s some really insigthful thoughts, here’s the summary from Sebastian himself: 1. Gold-level performance on reasoning models. 2. Qwen has overtaken Llama in popularity. 3. Mistral AI uses the DeepSeek V3 architecture. 4. Many contenders have emerged in the race for open-weight state-of-the-art models (eg Kimi, GLM, MiniMax, and Yi). 5. Cheaper / efficient architectures are priority in leading labs. 6. OpenAI released an open-weight model. 7. MCP joined the Linux Foundation and is becoming the stanard. # Predictions: 1. Consumer diffusion models for cheap. 2. Open weight community adoption of tool use. 3. RLVR will more widely expand into other domains. 4. Classical RAG will slowly fade. 5. LLM benchmark and performance progress will come from improved tooling and inference-time scaling rather than from training or the core model itself.

The State of MLOps 2025 Survey 🔥

The level of ML maturity of organisations in 2025 has slowly increased, and we have interesting insights on organisational departments/functions from our Prod ML Survey:

In 2025 about 29% of organisations have set up a central machine learning platform team.
However only 27% organisations have established a data platform / data engineering organisation.
There is a significant increase with 17% organisations now establishing an AI Risk & Governance Function.
However only 9% of organisations have established an AI inventory to keep track of all use-cases and models in the organisations.

If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025 🔥

Simon Willison 2025 Review

Simon Willison shared his 2025 year review on LLMs and as always there are some really great insights: 2025 was the year where LLMs became genuinely useful production tools thanks to RLVR, tool-use, multi-step workflows, etc. There has been a breakthrough of CLI and async coding agents (e.g., Claude Code/Codex/Gemini tooling) which is completely changing the software engineering trade, and ia also opening risks with “YOLO automation” with the “lethal trifecta” (private data + external comms + untrusted inputs). Chinese open-weight models are still rising to the top of public rankings under permissive licenses (taking over Meta’s Llama. OpenAI is losing market dominance faster than ever expected, and Google Gemini has been taking marketshare at lightning speed. Local models are still improving, but arguably are still lagging behind closed source frontier models, however the gap is closing faster than also assumed. What we can be certain is that 2026 is going to be a very exciting year for the field!

OpenRouter 100 Trillion Token Analysis

OpenRouter has just dropped an in-depth analysis from 100 trillion tokens of real-world LLM usage, and has shared some really interesting findings from the ecosystem in 2025: From the data it is clear the ecosystem has shifted toward a multi-model production stack, with proprietary models still in the lead, but with open-weight models reaching roughly one-third of token volume. On the open model usage, there has been particularly rapid growth from Chinese OSS models (i.e. Qwen/DeepSeek). Contrary to the “productivity” narratives, OSS traffic is actually dominated by roleplay/interactive fiction (together with coding) which is something I did not expect to see at this magnitude, but makes sense. Nearly half of these 100 Trillion tokens involve routing to reasoning models, which supports the growth of tool calling, as well as much more context-heavy workloads driven disproportionately by programming. Cost-vs-usage shows counterintuitively that reductions on price don’t seem to cause increase of usage (ie low price elasticity) - I personally think this is largely due to lack of cost-management maturity, and will most definitely shift next year. However there is still clear cost segmentation, where the market clearly still separates into groups where different users pick different models for different reasons. These are really interesting insights that do seem to reflect what we’re currently seeing qualitatively; definitely will be an important space to keep a close eye on throughout 2026!

Upcoming MLOps Events

Conferences for 2026 coming soon! For the meantime, in case you missed our talks:

The State of AI in 2025 - WeAreDevelopers 2025
Prod Generative AI in 2024 - KubeCon AI Day 2025
The State of AI in 2024 - WeAreDevelopers 2024
Responsible AI Workshop Keynote - NeurIPS 2021
Practical Guide to ML Explainability - PyCon London
ML Monitoring: Outliers, Drift, XAI - PyCon Keynote
Metadata for E2E MLOps - Kubecon NA 2022
ML Performance Evaluation at Scale - KubeCon Eur 2021
Industry Strength LLMs - PyData Global 2022
ML Security Workshop Keynote - NeurIPS 2022

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #367

Sun, 28 Dec 2025 00:00:00 GMT

The results for the Survey on Production MLOps are out: ethical.institute/state-of-ml-2025 🚀🚀🚀 As part of the release we have updated the interface enabling real time toggling between 2024 and 2025 data, and have refreshed a cool new code-editor theme 😎 Check it out and share it around!!

Further insights: The results are in for 2025 ETL Orchestration Providers in Production ML, and the data reflects what we saw this year 👇

Airflow still is the top 1 choice in 2025, although slowly losing marketshare from 37% down to 32% in 2024.
The biggest jump this year is on Databricks hitting 2nd place with 12% compared to 2% in 2024 which is a massive jump.
The number of orgnisations with custom orchestrators dropped to 3rd place with 10% vs 13% 2024.
Dropping from 3rd place Argo Workflows is down to 2% compared to 9% in 2024.

If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Karpathy 2025 Year in Review
Text-to-SQL Optimization Learnings
The State of Prod ML in 2025 🔥
MIT Distributed Systems
Python Data Science Handbook
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Karpathy 2025 Year in Review

Andrej Karpathy has put together his 2025 year in review for LLMs! Some really interesting insights: LLMs are now a production-critical platform where tiny shifts in training, evaluation, and developer tooling can result into major changes in reliability, cost, and velocity. This year’s biggest stop purely becoming about larger models, and instead they focused on reshaped training and product stack. Some of the key advances were RL from verifiable rewards, Claude Code skyrocketed and pushed the boundries on developer tools, and vibe coding is now transforming the trade of software engineering as we know it. We also so major innovations across different modalities such as Gemini Nano-Banana taking image generation to the next level. We can be sure that next year the pace of innovation will only continue to increase!

Text-to-SQL Optimization Learnings

This is a really interesting retrospective review of text-to-sql GenAI systems from Vercel sharing learnings that allow them to significantly improve performance: Apparently they removed ~80% of their specialized tools and prompt scaffolding and rebuilt it as a minimal agent once they were able to set up Claude Opus 4.5 with a semantic layer. This is an interesting trend that many organisations are also experiencing, namely iterating towards initial approaches which then have to be iterated as the industry evolves these methods into more a more robust direction. In their use-case they were able to achieve 3.5x faster performance with 37% fewer tokens whilst raising success significantly. This is certainly a trend so we’ll be looking forward to their next blog post showing a similar order of magnitude improvement as the ecosystem evolves

The State of MLOps 2025 Survey 🔥

The results are in for 2025 ETL Orchestration Providers in Production ML, and the data reflects what we saw this year 👇

Airflow still is the top 1 choice in 2025, although slowly losing marketshare from 37% down to 32% in 2024.
The biggest jump this year is on Databricks hitting 2nd place with 12% compared to 2% in 2024 which is a massive jump.
The number of orgnisations with custom orchestrators dropped to 3rd place with 10% vs 13% 2024.
Dropping from 3rd place Argo Workflows is down to 2% compared to 9% in 2024.

If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025 🔥

MIT Distributed Systems

MIT Distributed Systems Free Course: This is one of the best resources to sharpen your MLOps skills for 2026 🎁 The MIT 6-824 Distributed Systems course is arguably the best online resource for distributed systems (aside from Designing Data Intensive Applications). Often the difficulty from distributed systems comes from partial failures as opposed to full failures, given “more machines” introduce new ambiguities and bottlenecks everywhere. This is a great review on scalability, fault tolerance and consistency, with an extensive deep dive into theory and practice across algorithms, concepts and techniques. Definitely a great resource to get you kicked off for 2026 and beyond!

Python Data Science Handbook

What better way to kick off 2026 than with a review of foundational data science concepts with a Free O’Reilly Python Data Science Handbook: This is a great resource to review some of the data science fundamentals in a practical set of jupyter notebook environments. This resource dives into basics of notebook debugging, numpy semantics / vectorization, pandas patterns for building reliable datasets, Matplotlib/Seaborn for practical EDA and various ML libraries for knowledge on useful applications in the industry. This book covers quite a broad range of the ML lifecycles including validation, hyperparameters, feature engineering, and canonical models like linear methods, trees/forests, SVMs, PCA, clustering, and mixture models - definitely a great resource to check out.

Upcoming MLOps Events

Conferences for 2026 coming soon! For the meantime, in case you missed our talks:

The State of AI in 2025 - WeAreDevelopers 2025
Prod Generative AI in 2024 - KubeCon AI Day 2025
The State of AI in 2024 - WeAreDevelopers 2024
Responsible AI Workshop Keynote - NeurIPS 2021
Practical Guide to ML Explainability - PyCon London
ML Monitoring: Outliers, Drift, XAI - PyCon Keynote
Metadata for E2E MLOps - Kubecon NA 2022
ML Performance Evaluation at Scale - KubeCon Eur 2021
Industry Strength LLMs - PyData Global 2022
ML Security Workshop Keynote - NeurIPS 2022

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #366

Sun, 21 Dec 2025 00:00:00 GMT

Further insights: The results are in for 2025 GenAI providers in Production ML, and the data reflects what we saw this year: OpenAI losing its moat; Gemini skyrocketing; tooling standardisation; etc. Here are the key insights 👇

Open AI climbs to the top most used provider with 22%; however dropping from 24% in 2024 which reflects the struggle to keep the market-share.
Last year “No API” was the top choice, falling now to second choice with 22% (vs 38% in 2024) as LLMs eat the world and adoption increases.
Surprisingly Azure AI continues strong as Top 3 with 17% (vs 12% in 2024) due to close partnership with OpenAI / Github / etc.
Google Gemini has the largest jump with 14%, up from 3% in 2024 which is clear that Google is taking over significant market-share.
Anthropic stays consistent this year with 3% (vs 3% in 2024); I expect this one to grow next year with increased claude adoption.
Organisations are also moving away from building custom in-house tooling, down to 3% (vs 5% in 2024), which means consolidating towards providers.

If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

MLOps 2015-2035 Past and Future
Andrew Ng AI Career Advice
The State of MLOps 2025 Survey 🔥
ZenML 1000+ Case Studies LLMOps
Top Python Libraries in 2025
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

MLOps 2015-2035; The Past and Future

My keynote on a journey to the past and the future of Prod ML (2015-2035) is out!! I really enjoyed this one, as I had to reflect across my last decade in MLOps, as well as making predictions for the next decade. Would love to get thoughts! The topic of “Great MLOps” is now more important than ever as we are starting to see AI being rolled out at even more critical infrastructure, so we need to focus on turning models into reliable and scalable production systems. On the talk I look back at the last decade and categorise it as the Genesis of MLOps in 2015 (of course from Google’s paper), then going through a period of Messy Innovation between 2018-2022 with an explosion of tools which was accelerated by COVID, and then further accelerating with the growth of LLMOps in 2023–2025. The journey so far has been really fun, but unfortunately we have only scratched the surface; back in 2015 I would have put my money that we’d be much farther ahead in key areas that are still not fully mature such as ML Monitoring or ML Serving at scale. I also really enjoyed putting together a few predictions for the next decade; I tried to be relatively conservative but looking back I am still not sure if we’ll be there in a decade - this is why I chuckle when someone proposes AGI is coming soon… There is a lot to digest here, but would love to get further thoughts as I will be looking to iterate on this content throughout the year to come!

Andrew Ng AI Career Advice

When Andrew Ng and Laurence Moroney give career advice, we listen. This is really grounded guidance as it provides a snapshot on the current state of the (tough) hiring market, together with tips to increase your chances. There’s also some good reminders that AI product-building is accelerating, with AI coding tools make it possible to ship more powerful software far faster, so a differentiator for production ML practitioners is no longer just implementing models but choosing what to build, writing clear specs, iterating with users, and staying current on tooling. It is also a reminder to approach vibe coding conscientiously, as otherwise it can backfire - however using it correctly can help augment skills and breadth of knowledge. Check it out!

The State of MLOps 2025 Survey 🔥

The results are in for 2025 GenAI providers in Production ML, and the data reflects what we saw this year: OpenAI losing its moat; Gemini skyrocketing; tooling standardisation; etc. Here are the key insights 👇

Open AI climbs to the top most used provider with 22%; however dropping from 24% in 2024 which reflects the struggle to keep the market-share.
Last year “No API” was the top choice, falling now to second choice with 22% (vs 38% in 2024) as LLMs eat the world and adoption increases.
Surprisingly Azure AI continues strong as Top 3 with 17% (vs 12% in 2024) due to close partnership with OpenAI / Github / etc.
Google Gemini has the largest jump with 14%, up from 3% in 2024 which is clear that Google is taking over significant market-share.
Anthropic stays consistent this year with 3% (vs 3% in 2024); I expect this one to grow next year with increased claude adoption.
Organisations are also moving away from building custom in-house tooling, down to 3% (vs 5% in 2024), which means consolidating towards providers.

If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025 🔥

1000+ Case Studies LLMOps

Out of the blue the ZenML team dropped a report on 1000+ case studies in LLMOps - check it out! Some key highlights: 1) We are moving away from proof-of-concepts towards real-world use-cases hitting production. 2) We are seeing trends across tight cost/latency control, durable orchestration for long-running agent workflows, observability, circuit breakers, and progressive autonomy with human handoffs. 3) One of the bigger technical shifts are transitioning from prompt engineering to context engineering. 4) Evals and guardrails are still maturing but becoming now standard requirements for production use-cases. 5) MCP is quietly stabilizing as one of the standards (+ the move to LF is quite promising) but still lacks foundations like security, etc. Really great initiative from the ZenML team, check out the report and dive into the database of case studies as well.

Top Python Libraries in 2025

Check out the Top 10 Python Libraries in 2025! Here’s the list for both AI tools, as well as general tools:

Top 10 General Python Libraries:

ty - a blazing-fast type checker built in Rust.
complexipy - measures how hard it is to understand the code.
Kreuzberg - extracts data from 50+ file formats.
throttled-py - control request rates with five algorithms.
httptap - timing HTTP requests with waterfall views.
fastapi-guard - security middleware for FastAPI apps.
modshim - seamlessly enhance modules without monkey-patching.
Spec Kit - executable specs that generate working code.
skylos - detects dead code and security vulnerabilities.
FastOpenAPI - easy OpenAPI docs for any framework.

Top 10 AI Python Libraries:

MCP Python SDK & FastMCP - connect LLMs to external data sources.
Token-Oriented Object Notation (TOON) - compact JSON encoding for LLMs.
Deep Agents - framework for building sophisticated LLM agents.
smolagents - agent framework that executes actions as code.
LlamaIndex Workflows - building complex AI workflows with ease.
Batchata - unified batch processing for AI providers.
MarkItDown - convert any file to clean Markdown.
Data Formulator - AI-powered data exploration through natural language.
LangExtract - extract key details from any document.
GeoAI - bridging AI and geospatial data analysis

Upcoming MLOps Events

Conferences for 2026 coming soon! For the meantime, in case you missed our talks:

The State of AI in 2025 - WeAreDevelopers 2025
Prod Generative AI in 2024 - KubeCon AI Day 2025
The State of AI in 2024 - WeAreDevelopers 2024
Responsible AI Workshop Keynote - NeurIPS 2021
Practical Guide to ML Explainability - PyCon London
ML Monitoring: Outliers, Drift, XAI - PyCon Keynote
Metadata for E2E MLOps - Kubecon NA 2022
ML Performance Evaluation at Scale - KubeCon Eur 2021
Industry Strength LLMs - PyData Global 2022
ML Security Workshop Keynote - NeurIPS 2022

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #365

Sun, 14 Dec 2025 00:00:00 GMT

Further insights: In 2025, the top 5 challenges organisations face in Production ML are: ML System Monitoring 16% (+2% YoY), Access to Data 14% (+2% YoY), Data & ML Pipelines 13% (+2% YoY), Training/Experimentation Env Parity 12% (+1% YoY), ML Governance 8% (+2% YoY), and Security 8% (+6% YoY). It is super interesting to see the big jump on ML Security from 2% all the way to 8%, as well as the continued increase of ML Monitoring as the top 1 challenge, which reflects the awareness and maturity of organisations in understanding the implications and risks of production ML systems. It is also interesting to see how last year Gaps in ML Tooling was the top 2 on the list with 12% and this year it’s down to 8% completely out of the top 5 which reflects how choice is no longer becoming the blocker, and instead it’s the cohesive integration and robust productionisation. Finally it seems that also Engineering Talent is slowly declining with 8% this year vs 10% in 2024, which again shows that the skill gap is slowly closing. The top 5 challenges in production ML in 2025 do seem to reflect some of the qualitative trends that I see on our day-to-day. If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

PyTorch Hardware Acceleration in 2025
Anthropic MCP now in Linux Foundation
The State of MLOps 2025 Survey 🔥
OpenAI Enterprise AI Report
AI Eats the World 2025
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

PyTorch Hardware Acceleration in 2025

Next year may be the critical shift from NVIDIA CUDA towards AMD ROCm and other GPUs for ML compute; the 2025 State of Pytorch Hardware Acceleration Report has some interesting insights: PyTorch has become so ubiquitous that it can help provide a practical assessment on the state of maturity of hardware (GPU) accelerators. One interesting insight is that a key differentiator is not just the FLOPs, but largely the maturity of the software platform support on the end-to-end PyTorch 2.x runtime stack. It is clear that NVIDIA CUDA (ie H100/Blackwell) is still the operational gold standard for production training and serving because it has the most mature compiler path, widest kernel/operator coverage, and the least debugging/install friction (oh hi AMD!). It does seem that AMD ROCm has improved substantially and is now becoming a differentiator as it’s cost-effective but anyone that has used it knows how painful the setup actually is; getting a smooth user experience is often easier said than done, and NVIDIA has a signficant advantage in both time/experience and developer ecosystem maturity. However as we approach 2026 there are quite a lot of options arising, such as Google TPUs, Apple Silicon (MPS), and many others that are being supported with low-level frameworks such as Vulkan (one of them we maintain under Vulkan Kompute!).

Anthropic MCP now in Linux Foundation

The need for a unified + scalable standard to bridge Agentic systems is only becoming more critical, which is why I am quite excited to see the move from Anthropic donating MCP to the Linux Foundation! We know that MCP as a protocol has quite a lot of flaws, not just in terms of functionality but also in terms of security, robustness, etc - however I am a believer that more often than not, a bad standard can be better than no standard. There are indeed many alternatives to MCP emerging, especially on how fast the domain is moving, but I do believe that it’s likely easier to evolve towards a 2.0 MCP protocol within the backing of LF vs trying to standardise on something new that is not as widely adopted as a base. I have also been following how the standards for the protocol have been evolving through proposal / review processes for new features, and I keep getting surprised on the pace of innovation (some good, some not so good - but arguably in the right direction). This is certainly an interesting space to keep an eye, as it is clear it will continue to evolve at an accelerated pace!

The State of MLOps 2025 Survey 🔥

In 2025, the top 5 challenges organisations face in Production ML are: ML System Monitoring 16% (+2% YoY), Access to Data 14% (+2% YoY), Data & ML Pipelines 13% (+2% YoY), Training/Experimentation Env Parity 12% (+1% YoY), ML Governance 8% (+2% YoY), and Security 8% (+6% YoY). It is super interesting to see the big jump on ML Security from 2% all the way to 8%, as well as the continued increase of ML Monitoring as the top 1 challenge, which reflects the awareness and maturity of organisations in understanding the implications and risks of production ML systems. It is also interesting to see how last year Gaps in ML Tooling was the top 2 on the list with 12% and this year it’s down to 8% completely out of the top 5 which reflects how choice is no longer becoming the blocker, and instead it’s the cohesive integration and robust productionisation. Finally it seems that also Engineering Talent is slowly declining with 8% this year vs 10% in 2024, which again shows that the skill gap is slowly closing. The top 5 challenges in production ML in 2025 do seem to reflect some of the qualitative trends that I see on our day-to-day. If you want to dive deeper you can access the full results here: https://ethical.institute/state-of-ml-2025

OpenAI Enterprise AI Report

OpenAI released their 2025 State of Enterprise AI Report, where they brought together responses from 9000 workers across 100 companies to assess the trends, challenges and opportunities: Some of the outcomes won’t come as surprises, as one of the main takeaways was that people are moving away from ad-hoc prompting towards scalable / repeatable production workflows. It is no surprise they have also seen a surge in ChatGPT Enterprise message volume grew by 8x YoY, with significant increase using workflows via Projects/Custom GPTs rising ~19x YoY. One key insight next year will be the impact that Google will have on these numbers given their aggressive and accelerated momentum in the enterprise space. Other interesting insights: workers report measurable productivity gains - an OpenAI report likely would say so, however as we have seen there are contrasting views from domain to domain, so it will be interesting to explore as this matures in the year to come.

AI Eats the World 2025

Ben Evans releases his annual keynote on “AI Eats the World” for the 2025 edition with some interesting insights: We currently seeing investments forecasted on capex (compute) at nearly $400 billion in 2025 which as of today is disproportionately larger than the ROI so a major shift will be required next year. The landscape is also transitioning from model differentiation to model commoditization; namely performance on general benchmarks is converging, which means that the competitive “moat” moves toward traditional software advantages like distribution, proprietary data, and product UX. We can also assume that this “infinite interns” premise that LLMs theoretically provide will likely trigger a Jevons paradox, namely expanding the volume and complexity of work organizations can handle rather than merely replacing labor.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #364

Sun, 07 Dec 2025 00:00:00 GMT

Some insights: In 2025, about 23% of organisations are training using Databricks to train their ML models (vs 18% in 2024), followed by 13% using AWS Sagemaker (vs 14% in 2024), closely followed by a tie of 12% for both Google Vertex AI (vs 8% in 2024) and Azure ML Studio (vs 6% in 2024). It is interesting to see that there’s both a continued trend towards DBX; similarly there is an opposite trend on organisations building their custom in-house training platforms down to 11% in 2025 vs 28% in 2024. The space of model training platforms seems to have become the most consolidated domain across the entire end-to-end ML lifecycle as part of the survey, which aligns with my anecdotal experience. You can access the full results here: https://ethical.institute/state-of-ml-2025

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Black Forest Labs Launches FLUX-2
Major Supply Chain Attacks
The State of MLOps 2025 Survey 🔥
Raschka on Building an LLM
2025 Advent of Code
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Raschka on DeepSeek v3.2

Sebastian Raschka has put together the best breakdown of DeepSeek’s recent V3.2 model release, and there are some pretty interesting architectural choices that make it clear why it’s so far ahead: This release was quite exciting as it’s starting to feel like one of those moments where open-weight models are now a serious replacement for frontier closed-source models in real production stacks, and we are also hearing news like OpenAI declaring code red due to lack of moat. I really liked how Sebastian Raschka introduced the architecture choices by actually going through breakthroughs in previous models that actually have now enabled by bringing these all together. The most interesting one was the concept of sparse attention, which he explains how it’s different to windowed / sequential attention in that it allows the model to focus on sparse historical tokens instead of being restricted to say the last n tokens. Similarly there are some interesting breakthroughs on the Mixture-of-Experts backbone built on a Multi-Head Latent Attention trick for cheap KV caching, and pushing hard on the RL side mixing classic RLVR for math/code with LLM-as-a-judge rewards and domain-specific GRPO tweaks. Reading through this really shows how this release has been truly achieved by a small series of breakthroughs from previous iterations, and does make it clear that releasing fast really can give the competitive advantage (which many of the larger players are missing - although not google surprisingly!)

DeepMind Nobel Prize Documentary

The recent DeepMind documentary is actually a must-watch! I remember the good old days working in AI back in the mid 2010s, watching how AlphaGo blew minds with move 37… back then there wasn’t much hype for the field - funnily enough, the hype was all about crypto 😂 This documentary seems to really capture the excitement on how the hype delivered into one absolutely exciting breakthrough from DeepMind on protein folding, which so far has not yet been processed widely across the mainstream. There are quite a lot of really nostalgic, and particularly exciting details that are covered in the documentary that I wasn’t aware of; definitely recommend the watch! Back in these days is when I started working on Responsible AI given how critical it was becoming, I am very much glad I did as although there’s been progress, there is still quite a lot to do (and arguably more with the accelerated pace of innovation in the field!).

The State of MLOps 2025 Survey 🔥

In 2025, about 23% of organisations are training using Databricks to train their ML models (vs 18% in 2024), followed by 13% using AWS Sagemaker (vs 14% in 2024), closely followed by a tie of 12% for both Google Vertex AI (vs 8% in 2024) and Azure ML Studio (vs 6% in 2024). It is interesting to see in our 2025 survey on the State of Production ML that there’s both a continued trend towards DBX; similarly there is an opposite trend on organisations building their custom in-house training platforms down to 11% in 2025 vs 28% in 2024. The space of model training platforms seems to have become the most consolidated domain across the entire end-to-end ML lifecycle as part of the survey, which aligns with my anecdotal experience. 🥳 The results are shared as open source like last year! You can access them directly at: https://ethical.institute/state-of-ml-2025 🔥

The Race of DuckDB / Polars / Spark

In production ML we’re all feeling the pain of “cluster fatigue” - I am glad we are not taking the run-in-my-laptop challenge to the next level with benchmarks of tools like DuckDB / Polars / Spark which are very promising: This is a great experiment - namely asking, can I just process a 650GB S3 bucket on a simple 32GB / 16-CPU machine? The answer is (as always) “it depends - but likely yes” with the right tools, which is great to see. This use-case shows how DuckDB, Polars, and Daft can happily chew through a full-table aggregation without blowing up within ~12–16 minutes for Polars/DuckDB and ~50 minutes for Daft. The PySpark job (untuned) was also able to chew through it in a bit over an hour, but certainly there can be further optimizations that can be introduced. I myself have been playing around with DuckDB in various random projects and I have to say I’m impressed on the performance, and seeing also the integrations that it’s providing into data lakes makes it increasible more feasible, definitely recommend checking it out!

Advent of SadServers 2025

In production ML, the real outages usually come not from your models but from the Linux, Docker, and web infrastructure holding them up - this Christmas we can dive into the Advent of DevOps: This is a great initiative similar to the traditional Advent of Code, but focusing on DevOps channels, and although it’s not directly and MLOps set of challenges, from experience most of the challenges in production ML systems infra will require these type of drills to address. Some of these challenges are things like debugging an nginx reverse proxy in Docker, restoring a broken stack so the frontend can talk to its backend, fixing file permissions so a normal user can reliably create and edit files across sessions, etc. What better way to spend your Christmas than feeling like you’re debugging in production!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #363

Mon, 01 Dec 2025 00:00:00 GMT

Excited to release initial insights for our Survey on Production MLOps!! The survey would still benefit from your contribution and it’s OPEN FOR RESPONSES 🚀🚀🚀

If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at:

https://forms.gle/KF16EckuxNUKDtDK8 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Black Forest Labs Launches FLUX-2
Major Supply Chain Attacks
The State of MLOps 2025 Survey 🔥
Raschka on Building an LLM
2025 Advent of Code
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Black Forest Labs Launches FLUX-2

Some of you may remember the release of the FLUX model from Black Forest Labs last year taking image generation to a whole new level; this week they have now released the 2.0 version with some impressive improvements: This new latent flow–based image generation model is now focused on production creative workflows with what they refer to as multi-reference editing, which allows editing up to 10 images in a single architecture. The approach basically pairs a Mistral-3 24B VLM with a transformer and VAE which helps with the compression trade-off, which is what enables the higher photorealism, spatial coherence, and robust typography (i.e. infographics, UI mocks, fine text no longer screwed up). It is quite surprising to see the improvement of image models not slowing down; if anything it’s just the opposite, and it really is enabling for real-world applications across all industries and domains.

Major Supply Chain Attacks

Last week we saw a major reminder to us ML practitioners on software supply chain attacks, this one infected ~500 packages including major ones from Zapier, ENS, AsyncAPI, PostHog, Postman and others that amount to 100M+ monthly downloads: The way that this specific attack worked was that once you run your package with one of these dependencies on your dev machine or CI runner, it uses TruffleHog to harvest secrets (API keys, cloud creds, GitHub/npm tokens, etc) and pushes them to public GitHub repos labeled “Sha1-Hulud: The Second Coming”. The attack replicates the malicious code and then attempts to publish new infected npm packages; finally wipes the user’s home directory if it can’t authenticate. For us ML practitioners, this is our weekly reminder on the importance of security in areas where it’s not currently mature; i.e. model registries and artifacts, data/feature store credentials, cloud keys, CI/CD access used to build training and inference images, etc.

These stories are becoming scarier by the minute, as renowned packages are now falling for some of these issues; this will become a call-to-action for even the design of the package managers themselves, but most importantly to remember on standard security higene where possible.

The State of MLOps 2025 Survey 🔥

In 2025 about 43% of organisations take between 1-3 months to productionise a machine learning model (vs 2024 28%) and only 12% between 3-6 months (vs 25% in 2024) which is great news for the industry! Similarly 13% are now productionising ML models in less than one week (vs 10% in 2024). This is certainly a healthy trend that we hope continues; however with the rise of emerging tech like LLMs we do expect to see oscilations! This is one the most important muscles for organisations to strengthen as it directly relates to their capability to scale AI innovation. We still need your support to continue collecting diverse perspectives to map the ecosystem! Please help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: https://forms.gle/KF16EckuxNUKDtDK8 🔥

Raschka on Building an LLM

Sebastian Raschka and Andrej Karpathy’s educational material is becoming a key cornerstone for foundational ML knowledge; and Raschka’s recent course on “Building an LLM (From Scratch)” is becoming a must-watch classic: This course walks ML practitioners through implementing a GPT-style model end-to-end in plain PyTorch, starting from text preprocessing and tokenization. It goes through hand-built attention and Transformer blocks to a full GPT architecture with training and generation which is great to build the intuition. Along the way in this course, you implement next-token pretraining on unlabeled text, and then repurpose the pretrained backbone for supervised tasks like text classification and simple chat-style behavior. There are quite a lot of great takeaways, but these approaches of learning by building something from scratch often are such a great way to develop the foundations before going to the higher level / advanced domains (i.e. building agentic frameworks, transfer learning, etc).

2025 Advent of Code

In production ML we are reminded that your AI systems can only be as robust as the engineering discipline behind them; what better way to polish your foundations than with the Advent of Code for 2025: It’s great to see that despite the LLM craze the Advent of Code is still back! We did see that last year(s) they had to fight an insurgence of LLM-content in the leaderboards, which is why there is none this year but you can still practice. The puzzles unlock TONIGHT at midnight US Eastern! Do keep an eye and try them out - only caveat is that this time around there will only be 12 instead of 24, but this should keep us busy for the upcoming month!

On this topic funnily enough I ended up attending to build a custom parser, lexer and interpreter for a custom language that I was also wanted to explore testing with this - still WIP but pretty mind blowing how these tools are evolving to enable these.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #362

Sun, 23 Nov 2025 00:00:00 GMT

Excited to release initial insights for our Survey on Production MLOps!! The survey would still benefit from your contribution and it’s OPEN FOR RESPONSES 🚀🚀🚀

https://forms.gle/KF16EckuxNUKDtDK8 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Google Weather 2.0 Forecast
META on GenAI World Models
The State of MLOps 2025 Survey 🔥
MCP Protocol UI Extension
DuckDB’s Take on the Lakehouse
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Google Weather Forecast 2.0

These last years there’s been exciting innovations in weather (+ climate) forecasting with foundation / deep learning models; Google just released WeatherNext 2.0 which introduces a probabilistic forecaster that generates hundreds of physically consistent scenarios for weather prediction: This new model uses a Functional Generative Network architecture that injects noise in function space so the model is trained only on local “marginals”; this is interesting as it’s quite different to how current models are generally trained. This allows the model to learn coherent “joint” structures across variables, space, and time - this is actually quite important in phisics related modelling (which traditionally would be done with simulations). This seems to enable better prediction of complex regional contexts such as heat waves or wind farm output, etc. It’s quite interesting to see that many major tech companies are in the race for these type of models which bring the complexity of ML with real-world physics simulations.

META on GenAI World Models

We continue to see exciting waves in the field of GenAI World Models (aka text to 3D navigable environments); this week META releases a new system for AI generation of 3D worlds: This follows similar World Models (like Marble announced last week) which output 3D worlds using diffusion-based image-to-3D reconstruction models, together with object-level scene decomposition, and mesh/texture refinement which makes these worlds game-engine ready. The way these work basically is through a pipeline that first uses procedural reasoning and a global reference image to produce a coherent blockout plus navmesh, then reconstructs a large textured scene while enforcing constraints. This scene can then be decomposed into individual assets with refined geometry and textures per object. This is the interesting thing with innovative ML systems like these, namely that the AI is a critical component, but the innovation that goes towards the robust engineering and experience is what makes it fit together into a cohesive usable workflow.

The State of MLOps 2025 Survey 🔥

MLFlow is dominating Experiment Tracking space in ML in 2025 with 57% adoption (vs 42% in 2024)! Additionally, it makes me very happy to say that this year, Spreadsheets are down to 3% (vs 10% in 2024) - great work everyone 😂!! Closely following we have W&B with 8% (vs 7% in 2024) and DVC with 5% (vs 5% in 2024). The space of Experiment Trackers seem to be the most consolidated space across the MLOps lifecycle by far! We still need your support to continue collecting diverse perspectives to map the ecosystem! Please help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: https://forms.gle/KF16EckuxNUKDtDK8 🔥

MCP Protocol UI Extension

The MCP protocol is a living standard that continuously evolves through SEPs (like PIPs in python); this week a new SEP introduces standardised UI interfaces for MCP servers which seem to be a potential game-changer in usability: The SEP is pretty simple, basically standardising how MCP servers can expose interactive UIs to hosts instead of just text/JSON, using a pre-declared ui:// path protocol where HTML templates are linked to tools via metadata and communicating over the existing MCP JSON-RPC transport. This seems quite interesting for AI systems developers as it allows for easier interaction / debugging, but also potentially creating an ecosystem that could be used easily for both agents (e.g. browser agents) and humans. As we all know the saying, at this stage a bad standard is better than no standard - but certainly this is a living project so there’s more than certainly a lot of evolutions (+ breaking changes) expected in the near future.

DuckDB’s Take on the Lakehouse

DuckDB is great for local development. But what if you want to take DuckDB to distrbuted data-lake scale? This is a great video from the DuckDB author dives into the current challenges (+ tech/architecture debt) in the Lakebase approaches, as well as how DuckLake tackles these: DuckLake is basically a new open lakehouse table format from the DuckDB team that replaces Iceberg/Delta-style metadata stack with a straightforward architecture, arguing that the current architecture is overly / unecessarily complex. This basically includes a transactional SQL database for metadata plus Parquet files on any storage (S3, NFS, local disk) and stateless compute. This could be quite interesting for ML practitioners, as connecting your local development to a production scale DuckDB environment could be super easy and lightweight. This is definitely a space to watch; personally I do see what Iceberg has been doing in the space as incredibly innovative, however any further breakthroughs through tooling competition that makes the environment better is always welcome!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #361

Sun, 16 Nov 2025 00:00:00 GMT

Excited to release initial insights for our Survey on Production MLOps!! The survey would still benefit from your contribution and it’s OPEN FOR RESPONSES 🚀🚀🚀

Vector Databases are still skyrocketing: About 56% of orgs reported using vector databases, with 14% using PineconeDB (+9% YoY), about 11% have custom built in-house tools, and 8% use Azure AI (cosmos/search) surprisingly! There is still lack of consolidation, with the following contenders being Weaviate and Milvus with 4% and Elasticsearch with 3%.

https://forms.gle/KF16EckuxNUKDtDK8 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Stanford CS236: Deep Generative Models
Introducing the OWASP Top 10 2025
The State of MLOps 2025 Survey 🔥
GenAI World Foundation Models
Synthetic Time Series Foundation Models
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Stanford CS236: Deep Generative Models

Stanford CS236 is one of the best in-depth courses on Generative Deep Learning (LLMs, TTS, VAEs, GANs, etc) and provides a super comprehensive deep dive into the key foundational concepts that are now becoming the backbone of modern AI products. If you are interested on brushing up on fundamentals, this is a great and rigorous course that covers these foundational models as probabilistic data simulators. It focuses on how to parameterize complex distributions with deep nets, how different training criteria trade off sample quality, likelihood, and stability, and how to do efficient sampling and inference for tasks like conditional generation, editing, super-resolution, inpainting, control, and inverse problems. There are only growing applications for these models across images, text, audio, video, robotics, medical imaging, and many more domains so definitely recommended to check it out.

Introducing the OWASP Top 10 2025

The OWASP Foundation has now released the Top 10 Vulnerabilities for 2025! The vulnerabilities that made the 2025 list are: 1. Broken Access Control, 2. Security Misconfiguration, 3. Software Supply Chain Failures, 4. Cryptographic Failures, 5. Injection, 6. Insecure Design, 7. Authentication Failures, 8. Software or Data Integrity Failures, 9. Logging & Alerting Failures, 10. Mishandling of Exceptional Conditions. Although this is focused for application security, these are also critical in production machine learning systems! Also worth checking out the OWASP ML Top 10, and the OWASP GenAI Top 10 - as well as various reports that have been released as part of the Agentic Security [link] (disclaimer: I am part of the reviewer board).

The State of MLOps 2025 Survey 🔥

We are excited to release initial insights for our Survey on Production MLOps!! Vector Databases are still skyrocketing: About 56% of orgs reported using vector databases, with 14% using PineconeDB (+9% YoY), about 11% have custom built in-house tools, and 8% use Azure AI (cosmos/search) surprisingly! There is still lack of consolidation, with the following contenders being Weaviate and Milvus with 4% and Elasticsearch with 3%. We still need your support to continue collecting diverse perspectives to map the ecosystem! Please help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: https://forms.gle/KF16EckuxNUKDtDK8 🔥

GenAI World Foundation Models

What happens when you train a model using keyboard inputs to predict images of environments? You end up with interactive 3D AI worlds! The “Marble” model / architecture seems to take it to the next level: Marble is a generative multimodal 3D world model that is able to model text, images, multi-view video/images into full 3D environments, and then lets you iteratively refine them via AI-native editing, spatial expansion, and composition of multiple scenes. Recently we had seen attempts of simulating games, and even OS environments (incl. the impressive simulation of NeuralOS), however this seems to take it to the next level by enabling the creation of worlds from prompts which is impressive. One of the most interesting breakthroughs is the ability to export to triangle meshes and camera-controlled videos which seems would be on the path of making it usable in game engines, VFX, simulators, and robotics workflows. This is definitely an exciting space, and coincidentally seems to be the same domain which Yann LeCunn will be exploring after leaving META this week!

Synthetic Time Series Foundation Models

Time-series forecasting quietly runs the modern world, and the race for Forecasting Foundation MOdels is growing! This month we saw a new contender with a new approach, with TempoPFN tackling univariate zero-shot forecasting built on a linear RNN backbone with purely synthetic data! On the paper it seems to have promising performace, however only against other models trained with synthetic data, so it would be great to also see how it fairs against some of the current heavyhitters (aka Chronos, TimeGPT, etc). It is certainly an exciting time for this space - definitely worth keeping an eye as we likely will see many more releases in the year to come!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #360

Sun, 09 Nov 2025 00:00:00 GMT

Excited to release initial insights for our Survey on Production MLOps!! The survey would still benefit from your contribution and it’s OPEN FOR RESPONSES 🚀🚀🚀

Model Serving is still unstandardised: About 40% of organisations run custom wrappers (vs 56% in 2024) for in their ML models! Databricks usage doubled to 14% (+7% YoY), followed by KServe increased to 8% (+2% YoY), and then SageMaker decreased to 6% (-3% YoY)!

https://forms.gle/KF16EckuxNUKDtDK8 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

The Tabular Foundation Era Begins
AI-Assisted Engineering Report
The State of MLOps 2025 Survey 🔥
PyTorch Agent Framework (+RL)
Bench for Semantic Query Engines
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Tabular Foundation Era Begins

We’re in the ImageNet moment for Tabular Data! A new Tabular Foundational Model is released with TabPFN 2.5 tackling zero-shot predictions; and the sky is the limit in a world that runs on tabular data (spreadsheets anyone?): Prior Labs is a German-based AI Lab which leads exciting research in foundation models for tabular data (+ also including time series foundation models). This past week they released a new Foundation Model that outperforms across benchmarks presented (e.g. matches AutoGluon 1.4’s 4-hour ensemble on TabArena-Lite), while showing some scalability potential (+ compared to their previous models). It is interesting to see some of the technical decisions as this has evolved for both training, but especially for inference, as with foundational models a lot of the value beyond accuracy is also on performance / accessibility. Check it out!

AI-Assisted Engineering Report

For better or for worse, AI-assisted engineering is fast becoming a core capability; DX released their 2025 report with important insights: 100k+ devs across 400+ orgs now show that AI coding assistants are now ubiquitous (91% adoption), with developers reporting ~3.6 hours/week saved (although still qualitative / perceived). Some interesting stats show ~22% of merged code is AI-authored, and daily users shipping a median 2.3 PRs/week (~60% m ore than non-users); however not clear on confounding e.g. high performers are also the ones likely adopting performance. Additionally, impact on quality is uneven across orgs so we should take these insights with a pinch of salt (as every week we see a contrasting view). However certainly an important space to watch, and especially after the acquisition of DX by Atlassian we can expect some further interesting insights to come up (hopefully)!

The State of MLOps 2025 Survey 🔥

We are excited to release initial insights for our Survey on Production MLOps!! Model Serving is still unstandardised in 2025: About 40% of orgs still run custom wrappers (vs 56% in 2024) for in their ML models! Databricks usage doubled to 14% (+7% YoY), followed by KServe increased to 8% (+2% YoY), and then SageMaker decreased to 6% (-3% YoY)!We still need your support to continue collecting diverse perspectives to map the ecosystem! Please help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: https://forms.gle/KF16EckuxNUKDtDK8 🔥

PyTorch Agent Framework (+RL)

Agentic systems are becoming ubiquitous, and the PyTorch team enters the race with a new agentic framework TorchForge powered by their new distributed actor framework Monarch; let’s dive into some of the features: PyTorch Monarch introduces a single-controller distributed model that lets one Python script orchestrate large clusters via process/actor meshes similar to frameworks like Ray. This brings some benefits such as progressive fault handling, separate control/data plane, distributed tensors, and simplified heterogeneous pipelines like RL post-training. It is a PyTorch-native reinforcement learning library that lets you write rollout/training logic as async “pseudocode”, and doing the heavylifting of handles coordination, retries, weight sync, and resharding. The biggest open question on my side is how this fits into Ray given the recent announcement of the project joining the PyTorch (Linux) Foundation, as I assume we should expect some interesting synergies!

Bench for Semantic Query Engines

Today LLMs powered systems rely heavily on retrieval engines; this new benchmark for semantic query engines provides some interesting baselines that hopefully will help improve agentic systems across the industry: SemBench is a systems benchmark for semantic query processing engines that run end-to-end, multimodal queries over text, images, and audio. This benchmark is different as it measures the real production trade-offs of cost (ie. token fees), latency, and quality rather than just model accuracy. This benchmark packages 5 scenarios and 55 queries with ground-truth labels, and reports F1/relative error/Spearman/ARI alongside execution time and monetary spend. This seems quite handy as it enables for teams that are developing these systems to compare plans and optimizations on equal footing.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #359

Sun, 02 Nov 2025 00:00:00 GMT

Excited to release initial insights for our Survey on Production MLOps!! The survey would still benefit from your contribution and it’s OPEN FOR RESPONSES 🚀🚀🚀

About 40% of organisations do NOT have monitoring in their Machine Learning (+10% YoY)! On ML Monitoring tooling, 20% is custom / in-house (-7% YoY) and almost 20% EvidentlyAI (+7% YoY)!

bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

DORA AI-Dev Report 2025
Chip Huyen on Lenny’s Podcast
The State of MLOps 2025 Survey 🔥
Europe’s EuroLLM Launches
Learning PyTorch the Hard Way
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

DORA AI-Dev Report 2025

The 2025 DORA Report on the State of AI-assisted Software Development is out! Here are a few important highlights for any tech leaders in the space: 1)

Code quality: 39% show no impact or worsened; 50% slight to moderate improvement. 2) Productivity: 39% show no impact or worsened; 50% slight to moderate improvement. 3) Trust in AI output: 30% little/none; 24% a lot/great deal. 4) AI use at work: 90% use (+14 YoY). 5) Agent mode: 61% never; 39% at least occasionally. 6) Platform engineering: 90% org adoption; 76% have dedicated platform teams. It is clear that as of today the adoption of AI for development is not “whether” but “how” and “how much”; we can expect a lot of insights to come out throughout the next few months/years in the topic!

Chip Huyen on Lenny’s Podcast

I definitely recommend checking out last week’s edition of Lenny with Chip Huyen where they cover all-things AI Engineering! There was really great (foundational) advice throughout: 1. Talk to users; improve data; fix workflows; iterate prompts/UX—don’t chase shiny tools. 2. Pre-training gives capability; post-training shapes behavior. 3) Treat fine-tuning as a last resort after RAG, prompting, and system fixes. 4) Use RLHF/AI-feedback and verifiable rewards to steer models. 5) RAG quality is mostly a data-prep problem (chunking, metadata, QA reformats, synthetic Qs). 6) Write targeted evals for core user journeys and each step of multi-hop workflows. 7) Measure coding-tool impact with controlled trials and business metrics—not LOC. 8) Expect biggest gains among already high performers. 9) Many “AI problems” are UX/reliability issues (latency, voice turn-taking, disclosure). As always Chip has some really great takes, and they are basically reminders to not forget the basics / foundations; definitely a great podcast worth checking out!

The State of MLOps 2025 Survey 🔥

We are excited to release initial insights for our Survey on Production MLOps!! The survey would still benefit from your contribution and it’s OPEN FOR RESPONSES 🚀🚀🚀 About 40% of organisations do NOT have monitoring in their Machine Learning (+10% YoY)! On ML Monitoring tooling, 20% is custom / in-house (-7% YoY) and almost 20% EvidentlyAI (+7% YoY)! We are still continuing to gather the insights on this year’s MLOps Survey! We still need your support to continue collecting diverse perspectives to map the ecosystem! Please help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: bit.ly/state-of-ml-2025 🔥

Europe’s EuroLLM Launches

A few of Europe’s top universities have launched EuroLLM, a 9B-parameter open-weight LLM which supports the 24 official EU languages! It’s great to see some of these exciting initiatives, particularly when the artifacts are released as open-source / open-weights! From the technical paper it seems it was trained from scratch on 4T tokens using 400 H100s on EuroHPC’s MareNostrum 5, with a dense 42-layer Transformer (GQA, RoPE, 4k context) and an instruction-tuned variant for production use. It’s also interesting to see the strong multilingual performance which was compared with Gemma-2-9B and other open models on aggregate benchmarks. It goes without saying but it is great to see the innovation projects that Horizon Europe funding is enabling; this is certainly a key important requirement for driving innovation across Europe.

Learning PyTorch the Hard Way

This is a fantastic example of “learning PyTorch the Hard Way”, but often one of the best ways which is by diving into the internals after something doesn’t work as expected: This is basically an interesting situation of strange-user-behaviour turns into obscure-intricate-bug, and this is quite a great walk-through on some of the PyTorch internals as well. At least for me, going into the internals of a particular framework helps me build a stronger foundational knowledge not only on the tooling but also on the domain itself. This is one of the reasons why they suggest open source contributions is a great way to develop stronger technical skills. Check out the deep dive!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #358

Sun, 26 Oct 2025 00:00:00 GMT

The survey to help us map out this year’s State of Production MLOps in 2025 is still OPEN FOR RESPONSES 🚀🚀🚀

bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Amazon Releases Chronos-2
Robotics Foundation Models
The State of MLOps 2025 Survey 🔥
DeepSeek OCR Released
Ray Joining Linux Foundation
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Amazon Releases Chronos-2

Really exciting to see Amazon’s latest move in the time-series foundation model (aka GenAI) race releasing Chronos 2.0; believe it or not, time-series foundation models have been showing impressive practical applications in industry, and this new model seems brings some exciting features that addresses key limitations from its predecessor: Chronos-2 is Amazon’s universal time series foundation model that generalizes beyond univariate forecasting to handle multivariate and covariate-informed tasks in a zero-shot manner through in-context learning. This new architecture introduces group attention to model dependencies among co-evolving series and integrates both historical and known-future covariates without additional training. The benchmarks have also evolved since Chronos v1, which enable for more robust evaluations, and it seems there are strong accuracy improvements over prior models. This is quite an exciting field of research and application that is already having real-world impact in industry - I personally am very much looking forward to seeing the evolution in the next year!

Robotics Foundation Models

Everyone is talking only about LLMs is missing the exciting other AI revolutions taking place right now; robotics foundation models are one of the more exciting areas of research no-one is talking about: Dexterous robotic control has been transforming industries, and the ability to universally transfer learn can have huge opportunity in factories, fulfillment centers and basically any dynamic real-world environments. Some recent SotA models are built on Vision-Language-Action (VLA) systems and showing better performance than domain-specific models. It seems it’s now possible to take general models across e.g. robotic arms, and support co-learning together with human guidance to accelerate the capabilities to carry out tasks that have strong generalization and robustness on long-horizon manipulation. This is yet another interesting field of research that is seeing practical application, and is transforming industries with impact that we’ll see slowly and then all at once.

The State of MLOps 2025 Survey 🔥

We are still continuing to gather the insights on this year’s MLOps Survey! We still need your support to continue collecting diverse perspectives to map the ecosystem! Please help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: bit.ly/state-of-ml-2025 🔥

DeepSeek OCR Released

DeepSeek-OCR shows some exciting breakthroughs for LLMs; we may be able to extract more signal and meaning from text through their pixel-related images than their UTF-8 machine representation: DeepSeek-OCR demonstrates that document images can serve as a highly efficient context representation for language models, achieving 7–20× token compression while preserving up to ~97% OCR decoding accuracy at moderate ratios by replacing text tokenization with a unified visual input stream processed by the proposed DeepEncoder architecture. Andrej Karpathy has put it quite succintly following this DeepSeek release; if pixels become the universal interface, we can break free from tokenizer limits, which could unlock radically more efficient context handling and treat real-world documents as first-class citizens in language models. This approach eliminates tokenizer fragility, enables bidirectional attention over inputs, and naturally supports multimodal elements such as layout, charts, math, and chemical structures. Sometimes breakthroughs in innovation happens in areas that you may least expect, and retrospectively these may feel as obvious.

Ray Joining Linux Foundation

Ray is joining the Linux Foundation; I have to say this is one that I am extremely excited for, particularly having been involved since the early days of the LF AI & Data Foundation, as well as the early days of Ray, this is a perfect match. Distributed AI infrastructure has today become the decisive factor in whether organizations can deploy and scale state-of-the-art models in production, and organisations have still not standardised into a single framework. Ray has been growing as a potential contender showing practical applications at scale, and them joining the PyTorch Foundation under the Linux Foundation really has the potential to solidify its position as the open-source distributed compute engine for large-scale AI. This is a very exciting space, and especially with the current state of heterogeneity in the ecosystem, there are significant opportunities for consolidation across the ecosystem.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #357

Sun, 19 Oct 2025 00:00:00 GMT

The survey to help us map out this year’s State of Production MLOps in 2025 is still OPEN FOR RESPONSES 🚀🚀🚀

bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Claude Skills Driving Agentic Innovation
Truths in Software that are False in AI
The State of MLOps 2025 Survey 🔥
Karpathy on the Decade of Agents
Blameless Culture in Engineering
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Claude Skills Driving Agentic Innovation

Anthropic’s new Claude Skills system released last week may be more transformative than the MCP craze due to a few advantages: Claude Skills are opening more possibilities for lightweight, modular extensions to LLMs using simple Markdown files and optional scripts that Claude can load directly. These “skills” let models perform specialized tasks efficiently within a coding environment, which significantly reduces token overhead and complexity compared to MCP’s heavy protocol structure. Although it can be argued that these two frameworks can be compatible / interchangeable in the future, it is interesting to see this wild-wild west of competing approaches to find the global optimal. This is a great write-up by Simon Willison which summarises both the advantages as well as some practical examples.

Truths in Software that are False in AI

Here are 5 truths in regular software that are false when applied to AI: 1) Software vulnerabilities are caused by mistakes in the code; 2) Bugs in the code can be found by carefully analysing the code; 3) Once a bug is fixed, it won’t come back again; 4) Every time you run the code, the same thing happens; 5) If you give specifications beforehand, you can get software that meets those specifications. It is becoming growingly accepted that assumptions from traditional software engineering like bugs, testing and monitoring, don’t apply in the same way to modern AI systems. Unlike conventional software, in AI behavior is learned from often large datasets rather than explicit instructions, making errors unpredictable, non-deterministic, and resistant to debugging. There is now a growing consensus across organisations to accept these differences in order to derive the right practices to ensure stable and robust operations at scale.

The State of MLOps 2025 Survey 🔥

Karpathy on the Decade of Agents

Some really insightful take-aways on this 2 hour podcast with Andrej Karpathy, such as arguing that “we are in the decade of agents, not the year of agents” - here are 5 key takeaways: 1) It will take over a decade to properly figure out agents, as we’re in the early stages of memory, multimodality and reliable long-horizon reasoning. 2) Reinforcement learning is inefficient and noisy, requiring new approaches like process-based supervision and reflection. 3) Large language models suffer from model collapse and over-memorization, limiting creativity and continual learning. 4) Coding is currently the most practical application for LLMs, but they struggle with truly novel or complex system design. 5) AGI’s arrival will be gradual with steady productivity growth rather than causing a sudden economic or societal leap. For machine learning practitioners, understanding where the real bottlenecks lie determines whether we build the foundation required for robust agentic systems, or get stuck on the hype.

Blameless Culture in Engineering

There are critical lessons that we can take on from the movement that drives the “Blameless culture in engineering”, which focuses on solution vs finger-pointing: Highlights that high-performing engineering and ML operations teams thrive on a blameless culture, where failures are treated as opportunities to strengthen systems rather than assign fault. This by no means encourages removing accountability, but switches the focus from finger-pointing towards short- and long-term solution finding instead; this ensures problems are resolved efficiently, which then can allow space for systemic or organisational improvements required. In production machine learning and large-scale engineering, culture is as critical as the code itself, and there are key lessons that can be adopted from this because even the best systems fail without trust, learning, and accountability.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #356

Fri, 10 Oct 2025 00:00:00 GMT

The survey to help us map out this year’s State of Production MLOps in 2025 is still OPEN FOR RESPONSES 🚀🚀🚀

bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

A History of Large Language Models
Few Samples Break LLMs
The State of MLOps 2025 Survey 🔥
Uber Michelangelo in 2025
Companies Measure AI Productivity
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

A History of Large Language Models

This is an in-depth History of Large Language Models going back to the origins in 2003 all the way back to today: LLMs have emerged from decades of incremental progress in representation learning, beginning with distributed embeddings and neural language models in 2003, continued then through word2vec in 2013, then to sequence-to-sequence and attention-based architectures culminating in the Transformer in 2017. It is also interesting to see how OpenAI’s GPT series applied generative pre-training, fine-tuning, and RLHF to scale these ideas into versatile language models. It is also curious to think that indeed although today next-word prediction seems obvious, however if you had seen this math representation back in the day you likely would’ve been suspicious.

Few Samples Break LLMs

Data integrity is becoming one of the most critical challenges in AI, and Anthropic+Alan Turing have published a study that shows LLMs can be compromised with a few poisoned training samples. It basically shows that large language models can be backdoored with as few as 250 poisoned training documents, regardless of model or dataset size. Also inserting a small fixed number of malicious samples (

containing e.g. token) is enough to cause consistent misbehavior when the trigger appeared. This finding overturns the assumption that poisoning difficulty scales with data volume, revealing that absolute count determines attack success. This means that even massive pretraining datasets remain vulnerable to small-scale poisoning - this is an important reminder to be aware of the security implications irrespective of the size or type of your models.

The State of MLOps 2025 Survey 🔥

Uber Michelangelo in 2025

Uber’s Michelangelo is one of the first end-to-end MLOps platforms published; it has recently been updated to power the GenAI era, and there are some interesting design choices: It has transitioned away from custom in-house components towards integrating modern OSS components including Ray, Horovod and Triton for scalable distributed training and low-latency model serving. It now unifies the end-to-end ML lifecycle through a modular architecture, frameworks for model quality and project tiering. This platform is now supporting over 5,000 production models and 10 million predictions per second, with support for a Gen AI Gateway and LLMOps extensions. It is interesting to see how tech giants are now also standardising towards standardised tooling and evolve to support scaling needs for traditional ML as well as GenAI foundations.

Companies Measure AI Productivity

How do we measure AI productivity? Here is how Google, GitHub, Microsoft, Dropbox, Atlassian and others measure it: One of the main take-aways is that ROI comes from pairing AI-specific telemetry with established engineering outcomes. Basically this would involve mapping metrics such as adoption, DAU/WAU, CSAT, time saved, spend into PR throughput/cycle time, Change Failure Rate, maintainability, DevEx. In regards to company specific results, there are interesting insights such as Dropbox/Webflow reporting 20% higher PR throughput for regular AI users while watching failure rates to avoid speed-for-quality tradeoffs. There are some challenges / caveats such as vendor telemetry lock-in, rising token costs, and weak A/B testability; this means that we should treat measurement as iterative and multi-method.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #355

Sun, 05 Oct 2025 00:00:00 GMT

The survey to help us map out this year’s State of Production MLOps in 2025 is still OPEN FOR RESPONSES 🚀🚀🚀

bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Andrew Ng New Stanford ML Course
Introduction to Multi-Armed Bandits
Designing Agentic Loops
The State of MLOps 2025 Survey 🔥
Building Generative Recommendations
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Andrew Ng New Stanford ML Course

Stop everything you are doing; THE course on ML by Andrew Ng at Stanford is getting a revamp, and is being published as we speak. This was one of my first introductions to machine learning; the only point I will miss is the blackboard and chalk as it seems even that is being modernized! Personally I will be looking to book some time off to block out enough time to properly go through it to touch back once again into some of the core foundations. It is such a lucky time to be alive such that content of such high quality is made available for free, huge kudos to Stanford and Andrew Ng for publishing these resources, as they will inspire and enable the next (and past) generations of ML practitioners.

Introduction to Multi-Armed Bandits

Multi-armed bandits are a powerful framework that enable you to make decisions over time under uncertainty, and this paper from Microsoft is one of the best advanced introductions. The literature on MABs is quite extensive and continues to grow across the years, but this book provides a comprehensive coverage across all the topics. This resource dives on the foundations including KL-divergence, bayesian bandits & thomson sampling, similarity information, adversarial bandits, contextual bandits, and various advanced topics such as bandits with agents. Coming across resources like these make me realise how little I know about a topic by coming across the depth of the concept and applications.

Designing Agentic Loops

Simon Willison has put together a fantastic overview on how to design agentic loops, which seems to be a seemingly growing skill for developers to at least have an intuition towards. Agents like Claude Code or Codex CLI are implementing in one way or another such an agentic coding loop, which provides structured, iterative workflows where LLMs autonomously run tools to reach a goal. Balancing safety and flexibility seems to be a continuous tradeoff - e.g. enabling YOLO mode make it feel more productive, but requires stricter sandboxing. Effective loop design seems to involve concepts that are still being defined, however there seems to be a growing number of concepts that are starting to solidfy as best practice.

The State of MLOps 2025 Survey 🔥

Building Generative Recommendations

One of the best ways to learn a concept is by going through the implementation; this is a great opportunity to understand recommender systems through practice: In this case the approach is by through the “OneRec” generative recommender framework which was originally performing video recommendations, but adapting it to friend recommendations. In friend recommendations, positive signals are far sparser and more delayed, which introduces further considerations that we’d have to also understand. In practice the architecture remains the same, but there are key modifications required on the design of the embeddings and training objectives. This is quite an interesting approach that leverages LLM infrastructure to unify recommendation stages - and more importantly this is a great way to really understand some of the underlying concepts.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #354

Sun, 28 Sep 2025 00:00:00 GMT

The survey to help us map out this year’s State of Production MLOps in 2025 is still OPEN FOR RESPONSES 🚀🚀🚀

https://bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

650+ GenAI Case Study Learnings
The Top Languages in 2025
MIT on the GenAI Business Divide
Contribute to The State of MLOps 2025 Survey 🔥
Leading in a Room with Experts
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

If you’re looking for an interesting career opportunity, I’m hiring for a few roles including Data Science Manager (Forecasting), as well as Data Scientist (Forecasting) - check them out and please do share with your network!

650+ GenAI Case Study Learnings

EvidentlyAI’s has put together over 650 case studies of real-world AI use cases and have recently published a summary of key learnings: These use-cases span across various domains including ops automation, personalization, and search, recommendations, etc. It does seem that companies are increasingly using LLMs for more complex workflow optimizations with also more sophisticated techniques, such as leveraging retrieval-augmented generation for customer support and AI agents for data access and product enrichment. There is also an interesting trend where LLMs are being used for evaluation and safety practices to improve reliability, together with other emerging trends that are certainly worth keeping a close eye!

The Top Languages in 2025

A new programming language rankings from IEEE Spectrum confirm Python is now at the top, however we are also realising that programming languages are also being affected by the rise of AI coding assistants: Developers are increasingly relying on LLMs instead of Stack Overflow or GitHub for software development, which is diluting traditional metrics of language use. What is more interesting is that with the rise in popularity of AI assistants, this is also having an impact on language popularity, due to the vicious circle of training data being more broadly available due to AI generation for the more popular programming languages. This is a trend that will be interesting to follow as it will also slowly also affect other domains such as the machine learning and data operations space, when it comes to languages of choice.

MIT on the GenAI Business Divide

A recent study from MIT on the 2025 State of GenAI in Business finds that despite $30-40B invested in GenAI, 95% of enterprise pilots show no ROI: This is the elephant in the room that is starting to introduce accountability on the growing “GenAI value divide”, where only ~5% is delivering direct business value. The study outlines that the gap isn’t due to model quality but to lack of model-learning, memory, and workflow integration, as well as most importantly, relevance on use-case, as often automating a small step in a larger end-to-end manual process would likely not drive significant gains. Generic tools like ChatGPT seem to boost individual productivity but have not converted into improved P&L or increase in outcomes - it will be interesting to see how this evolves once we are past the hype peak.

The State of MLOps 2025 Survey 🔥

Leading in a Room with Experts

As a leader it is the norm to be in a room full of experts - what are then best practices to contribute as a leader in these contexts? In a team of deep experts, effective leadership is actually not about having all the answers, but about connecting the dots, translating across disciplines, keeping focus on real user problems, and framing trade-offs in terms everyone can act on. Strong leads create clarity by defining goals, adapting technical language for different audiences, and modeling humility (e.g. “I don’t know, let’s figure it out”) to encourage collaboration. Similarly it is also important sometimes to think of decisions not as binary but as probabilistic bets, and sometimes unblock teams by defining a direction that may not be 100% correct but that can help move closer to the right direction.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #353

Sat, 20 Sep 2025 00:00:00 GMT

We are still looking for contributions!! The survey to map out the State of Production MLOps in 2025 is now OPEN FOR RESPONSES 🚀🚀🚀

https://bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Contribute to The State of MLOps 2025 Survey 🔥
Anthropic Outage Post-Mortem
Harvard on LLM National Bias
The State of Devs in 2025
Build Containers from Scratch
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The State of MLOps 2025 Survey 🔥

Did you know that last year less than 50% practitioners had monitoring for their production machine learning? We are capturing the insights on this year’s MLOps Survey which we will be able to share very soon!! We still need your support to continue collecting diverse perspectives to map the ecosystem! Please help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: https://bit.ly/state-of-ml-2025 🔥

Anthropic Outage Post-Mortem

Anthropic has published a transparent post-mortem review of their recent outages which provides quite an interesting perspective on the challenges a top AI lab faces in massive-scale production MLOps: In large-scale ML deployment, subtle infrastructure bugs that can go unnoticed can affect the quality of the model resulting in huge user/customer impact. Between August and September Anthropic traced intermittent quality regressions in Claude to what they describe as three overlapping infrastructure bugs: 1) They had a routing error that misdirected short-context requests to 1M-token servers, 2) They also saw a TPU misconfiguration that corrupted token outputs, 3) In parallel they also had an XLA-TPU precision bug that broke their approximate top-k sampling. These issues were further heightened by load balancing changes and proved hard to diagnose due to platform heterogeneity, noisy evaluations, and privacy limits on inspecting user data. This actually does hit home, as it really reminds us ML Enginereing practitioners on the importance of production-grounded evaluations, strong monitoring, and the right privacy-preserving processes & tooling for debugging efficiently.

Harvard on LLM National Bias

Harvard researchers published in 2023 a still highly relevant study that exposes the large (+ growing) gaps in existing LLMs when it comes to cultural and geographical diversities. Non-surprisingly several of the more popular LLMs reflect distributions of specific demographics (i.e. western, industrialized, etc) due to the inherent bias in their training data and alignment processes. This Harvard study uses the World Values Survey and cognitive tasks to find that indeed LLM responses most closely resemble U.S. and Northern European populations but diverge sharply (r = –0.70) as cultural distance increases, which is then also measured respectively. This skew appears in values, politics, thinking styles, and assumptions about self-concept, meaning that LLMs systematically misrepresent the psychological diversity of most of humanity - this in itself becomes growignly important especially as these tools are growingly used in critical contexts (+ further outages / learnings arise).

The State of Devs in 2025

This is a fantastic survey that brings a new “personal” lens into the state of developers in 2025, showing insights on career mobility, mental health, hobbies, and more interesting insights that have not arisen in previous surveys. This survey results cover responses from 8,000+, and span themes across demographics, career, workfplace, technology, health, worldview and hobbies. Some insights that are expected include a still male-dominated workforce, active career mobility, education-linked income gaps, and a strong preference for remote/hybrid setups. Career mobility and higher education seem to correlate with income, though burnout, poor management, discrimination, and health issues (notably poor sleep, mental health, and back pain) seem to also be quite a large reported concern. It is also worth highlighting that the site is also organised in quite a neat structure, the color schemes may make it a bit harder to read sometimes but certainly allows for interesting deep dives.

Build Containers from Scratch

This is a classic, and hands down one of the best tech talks available online: it basically is an opportunity to build a container from scratch, certainly more than recommended watch! As a brief high level overview, Liz Rice (Isovalent’s Chief OSS Officer) shows the simplicity in containers by buidling one from scratch, hands-on live on stage in a way that is super simple to follow. This is definitely one of the best explain-like-I’m-5 sessions to break down the world of containers, so for any ML practitioner that has not yet watched it, do check it out!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #352

Sat, 13 Sep 2025 00:00:00 GMT

We are still looking for contributions!! The survey to map out the State of Production MLOps in 2025 is now OPEN FOR RESPONSES 🚀🚀🚀

https://bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Contribute to The State of MLOps 2025 Survey 🔥
Hierarchical Reasoning Models
QWEN3 Towards Efficiency
Multi-Agents for CVE Exploits
Simon Willison Vibe Code Apps
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The State of MLOps 2025 Survey 🔥

We are starting to capture some insightful perspectives on the state of production MLOps in 2025!! We have also started receiving further diverse perspectives but we’ll need your support to continue collecting diverse perspectives to map the ecosystem! Please support us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: https://bit.ly/state-of-ml-2025 🔥

Hierarchical Reasoning Models

You may notice that sometimes ChatGPT takes too long to “Think” - reasoning is now becoming a critical lever to drive competitive differentiation in GenAI products, and a group of singapore researchers have been able to crack the code: Hierarchical Reasoning Model is a (pretty interesting) brain-inspired (thinking-fast-thinking-slow style) recurrent architecture designed to overcome the brittleness and inefficiency of chain-of-thought methods in large language models. This method implements a slow high-level module for abstract planning with a fast low-level module for detailed computation which enables deep latent reasoning within a single forward pass. This is not only an interesting approach due to the breakthroughs in performance but also from the implementation that resembles the thinking-fast thinking-slow approach which has been referenced in brain interactions; it will be interesting to see if some of these methodologies do provide a breakthrough in the space on performance (across efficiency and accuracy).

QWEN3 Towards Efficiency

Chinese giant Alibaba releases Qwen3-Next, which introduces a new large language model architecture designed for extreme efficiency in both training and inference: This new model is quite interesting as it combines a hybrid attention mechanism with an ultra-sparse MoE structure. The base model outperforms dense Qwen3-32B while consuming less than 10% of its training compute and delivering over 10× higher inference throughput on long contexts. It’s great to see open models continue to improve at lightning speed, this model is available in Hugging Face, ModelScope, Alibaba Cloud (of course…), NVIDIA API Catalog, and is supported in frameworks like Transformers, SGLang, and vLLM.

Multi-Agents for CVE Exploits

Security vs GenAI: This paper shows how security researchers can how use GenAI to automate runs of security exploits at scale, raising further questions on the cat-mouse security race. This paper introduces a system called CVE-GENIE, which is an automated LLM-based multi-agent framework that reproduces CVEs end-to-end, generating both exploits and verifiers to build reproducible vulnerability datasets at scale. It is quite interesting as prior efforts rely on manual reproduction or narrow bug classes, however in this implementation the system has a systematic framework to enable the plans and execution through four coordinated modules: Processor, Builder, Exploiter, and Verifier. It is evaluated on 841 CVEs from 2024–2025, and it successfully reproduced 428 (51%) across 267 projects, 141 CWE categories, and 22 languages, at an average cost of $2.77 per CVE (which is quite a promising and impresive result). This does raise important questions on the future of security given how easy it is becoming to test common exploits across any system out there.

Simon Willison Vibe Code Apps

It is fascinating to see how AI-assisted rapid prototyping is completely changing how devs build everyday products - and as always Simon Willison has a great resource for just this: This is a great list of “vibe coding” projects that have been developed from scratch and have evolved into practical tools that one can rely on daily. This is a great list for anyone to try out and get started, consisting on projects from in-browser OCR for PDFs, annotated presentation generators, and image utilities, to LLM pricing calculators, and real-time feed monitors. If there is a suggestion from this it would be to try out some of these and take some of these AI projects for a run to accelerate some of these prototyping capabilities.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #351

Sun, 07 Sep 2025 00:00:00 GMT

We are still looking for contributions!! The survey to map out the State of Production MLOps in 2025 is now OPEN FOR RESPONSES 🚀🚀🚀

https://bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Contribute to The State of MLOps 2025 Survey 🔥
The GPU Compute Compendium
OpenAI on LLM Hallucinations
Claude Code Tutorial From Scratch
Google on Limitation of Embedding Retrieval
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The State of MLOps 2025 Survey 🔥

We’ve started receiving some responses on the State of Production MLOps in 2025!! However we’ll need to do better to collect diverse perspectives to map the ecosystem; help us with your response, as well as by sharing with your colleagues 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: https://bit.ly/state-of-ml-2025 🔥

The GPU Compute Compendium

Understanding the GPU internals can make a significant difference for ML Engineers looking to get the most out of their compute, and this is a great comprehensive resource on all-things GPU computing: The GPU Glossary is a curated overview of the full GPU stack, covering hardware architecture (e.g., SMs, CUDA/Tensor Cores, schedulers), memory models (threads, blocks, grids, synchronization, global memory), host-side tools (CUDA runtime, drivers, nvcc, profiling suites, cuBLAS), and performance fundamentals (roofline model, occupancy, warp divergence, memory bandwidth limits). Even if you know these concepts this is a really good refresher with great intuitive visuals, definitely worth checking out.

OpenAI on LLM Hallucinations

Open AI publishes a paper to tackle LLM hallucinations and concludes: 1) accuracy cannot “reach 100%”; 2) hallucinations can be reduced by abstaining; 3) reward metrics may be a key to hallucinations. This is quite an insightful paper where OpenAI explores “Why language models hallucinate”, and explains why these “errors” stem not from random glitches but from the statistical limits of next-word prediction. It also seems that this is encouraged through evaluation setups that reward accuracy without penalizing confident mistakes which effectively incentivizing models to guess rather than abstain. There are quite a few interesting lessons which mostly were indirectly (or directly) known, but hammered down, such as reducing hallucinations requires rethinking evaluation, such as by penalizing confidently wrong answers, rewarding abstention or uncertainty, and calibrating outputs.

Claude Code Tutorial From Scratch

If you haven’t yet set up your Claude Code environment, this end-to-end hands-on video series will help you hit the ground running and beyond: This is quite a good, succint, and comprehensive tutorial on claude code which goes all the way from the basics of installation and configuration, to some of the best practices and advanced topics. If you haven’t jumped in the Claude Code boat, it basically enables git-aware permission-gated edits, includes code diffs, and supports things like automated PR reviews. This is a good tutorial as it provides some of the intuition on nuances on configuration, assigning narrow / reviewable tasks, manually checking diffs, etc. The content also covers some of the more advanced features such as extensions with MCP servers, custom commands and sub-agents.

Google on Limitation of Embedding Retrieval

Embeddings are an essential component in AI systems, and understanding their scaling limitations is critical; Google has published an interesting paper that dives into just that: From search to reasoning agents, efficient and accurate retrieval is key and this paper from Google shows that single-vector embedding retrievers face fundamental limits presented as the number of top-k document sets they can represent is bounded by the embedding dimension. Namely what this means is that some simple queries that fall beyond these limitations may be unsolvable regardless of training data or model scale due to this. This paper basically prove this using communication complexity theory and introduce a natural language benchmark where even state-of-the-art dense retrievers achieve under 20% recall on trivial tasks which is quite interesting. It is quite interesting to see research providing similar heuristics to the scaling laws for transformers but covering limitations that would allow ML (Engineering) practitioners to design better AI systems depending on the particular use-cases.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam
Code.Talks 2025 - 5th Nov @ Hamburg

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #350

Sun, 31 Aug 2025 00:00:00 GMT

That time of the year has arrived!! The survey to map out the State of Production MLOps in 2025 is now OPEN FOR RESPONSES 🚀🚀🚀

https://bit.ly/state-of-ml-2025 🔥

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Contribute to The State of MLOps 2025 Survey 🔥
Stanford Study on AI Automation Impact
Python: The Documentary
A Software Eng Agent in 100 Lines
Developer Productivity Metrics Reading List
SpaCy Against LLM Maximalism
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The State of MLOps 2025 Survey 🔥

That time of the year has arrived!! The survey to map out the State of Production MLOps in 2025 is now OPEN FOR RESPONSES 🚀🚀🚀 If you have a few minutes, your contribution will make a significant difference to the whole production ML ecosystem 🥳 The results will be shared as open source like last year!! You can add your response directly at: https://bit.ly/state-of-ml-2025 🔥

Stanford Study on AI Automation Impact

Sharing an interesting new study from Stanford that has been making the rounds online recently modelling the relationship between AI and the impact in reduction of headcount, showing direct impact in roles that are in risk of AI automation (e.g. customer service, marketing, sales, software engineering, etc) - they present the following six conclusions: 1) Substantial declines in employment for early-career workers in occupations most exposed to AI; 2) Economy-wide employment continues to grow, but employment growth for young workers has been stagnant; 3) Entry-level employment has declined in applications of AI that automate work; 4) These employment declines remain after conditioning on firm-time effects, with a 13% relative employment decline for young workers in the most exposed occupations; 5) These labor market adjustments are more visible in employment than in compensation; 6) These patterns hold in occupations unaffected by remote work and across various alternative sample constructions. One consideration that comes to mind: I had seen similar charts before being challenged due to the close correlation between the rise/drop of COVID+zero interest rates which better explain the rise and drop of jobs.

Python: The Documentary

Python powers modern AI/ML in production, as well as a large percentage of the web; this is a great documentary on the beginnings, challenges and high-speed growth of the Python progarmming language. This was quite a nice documentary to watch, as it traces the origins of Python to a language called “ABC” which apparently is where some initial design paradigms were adopted by Guido. It is interesting to hear some of the key milestones that enabled the growth of Python, such as becoming open-source early on, as well as focusing on becoming an alternative to Perl. Python managed to get some pretty impressive wins with web platforms adoption (eg Dropbox, Instagram) as well as strong adoption in the scientific stack (eg Numeric->NumPy/SciPy). Exciting to see what the next decade of Python will hold, there are some exciting developments (e.g. removal of GIL, low-level interoperability, agentic stacks, typing, etc)

A Software Eng Agent in 100 Lines

Have you ever thought of building your own agentic software stack from scratch? Here is a 100-line implementation of a software agent in Python: In this implementation actions run as stateless processes and the message history is fully linear, so trajectories double as clean training data for FT/RL. Despite being such a tiny implementation it seems to achieve ~68% score on SWE-bench which is quite impressive, and it works with any LLM without tool-calling APIs, and is relatively simple to scale. The example comes with a CLI and Visual UI, and supports batch inference, a trajectory browser, and Python bindings, so it provides quite an end-to-end deep dive across all relevant contexts to get you started with your own agentic software engineer.

Developer Productivity Metrics Reading List

Measuring developer (and AI) productivity has now become a competitive advantage, and as a technical leader it’s great to see the growing interest, and this is a great reading list from DX’s Laura Tacho. Some of the resources include the latest framework from DX, including their AI productivity measurement framework, as well as the usual suspects such as SPACE, DORA, DevEx, etc. For production ML, it is also important to map these into MLOps: things like time-to-model deployment, change failure rate for ML performance metrics, and mean-time-to-resolve for models/data pipelines, and more nuanced topics like data quality.

SpaCy Against LLM Maximalism

The SpaCy team put together a while back a case against LLM maximalism, making a point that sometimes bringing an LLM for a simple NLP problem that involves a combination of speed / accuracy / simplicity, bringing LLMs to solve the problem may backfire. We are aware that end-to-end prompting can produce slow, costly, brittle, non-modular systems if not done leveraging best practices. Particularly for non-generative tasks like sentiment analysis, NER, and POS-tagging, etc can be tackled with traditional approaches with better latency, accuracy, and reliability (+ often speed). It still holds that in these use-cases LLMs can be used for rapid prototyping, as well as for even creating synthetic data from existing labels. Irrespective of the content, evaluation still remains key, no matter whether dealing with traditional NLP or LLM based development - and the SpaCy library is still the best out there.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #349

Sun, 24 Aug 2025 00:00:00 GMT

Thank you for being part of over 70,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter for free at /newsletter/ ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

650 Prod MLOps Case Studies
McKinsey on GenAI Minimal Value
Google DeepMind Measuring Environmental Impact
Metadata Management for AI Flows
Beating Djikstra (for Sparse Graphs)
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

650 Prod MLOps Case Studies

Production is where machine learning meets business value, and Evidently AI has put together a comprehensive compendium of 650 real production ML/LLM case studies from 100+ companies (e.g., Netflix, Airbnb, DoorDash): It is great to see that the Evidently team continues to update and grow the list of production ML use-cases throughout the last few years, now also encompassing (of course) Generative AI and LLMs, as well as traditional ML such as computer vision, NLP, etc. There are also recurring use-cases across recommenders, search/ranking, and fraud detection; each including details on the in-house shipped system, covering product design, evaluation/metrics, and deployment/architecture.

McKinsey on GenAI Minimal Value

McKinsey has released a new report that calls out GenAI business value: “Nearly eight in ten companies have deployed gen AI in some form, but roughly the same percentage report no material impact on earnings.1 We call this the gen AI paradox.”. As various other reports highlight, it seems most enterprises are stuck in perpetual “widespread adoption” with little to no P&L impact. Despite organisations scaling horizontal copilots, the high-value vertical use cases are stalling due to distraction towards the hype. The report is slightly ironic as it does seem to suggest that the fix to no value from GenAI is more GenAI, however it does provide sound suggestions on the importance of robust infrastructure to support rapid productionisation once value can be actually captured. There are call-outs to agentic AI mesh for orchestration, memory, tool adapters, and a shift from MLOps to AgentOps, however there is a significant portion on the opportunity building capabilities to capture value on the intersection of traditional ML and agents based systems.

Google DeepMind Measuring Environmental Impact

The environmental cost of a ChatGPT or Gemini prompt is larger than you can imagine; that’s why Google had to publish their recent methodology to measure energy, emissions and water impact for Gemini products: Using this methodology Google estimates the median Gemini prompt uses 0.24 watt-hours of energy, emits 0.03 grams of carbon dioxide equivalent, and consumes 0.26 milliliters of water. These metrics have to be taken with a pinch of salt as they may be biased given the policy pressure towards the environmental impacts of these technologies. Having said that one thing that is clear is the efficiency improvements these models have been seeing throughout the recent past; in the report we can see improvements in Gemini of 33x/44x improvement, which does align with some of the huge investments we are seeing in research related to inference and training efficiency.

Metadata Mgmt for AI Flows

Metadata is the old-age proven and tested reliability layer for AI data pipelines, and this paper is a good reminder of how important it is in order to ensure robust production environments: This is an interesting paper which outlines TableVault as a Python-first metadata governance layer for human+LLM pipelines that makes every artifact, parameter, and operation auditable without replacing notebooks, ETL pipelines, or agents. It is interesting to see how concepts and methodologies are emerging to address the challenges in ML metadata management - particularly given that even outside of the agentic-contexts this is not a solved problem. This seems like a combination of taxonomy and tooling, however these tools are certainly not new, so it will be important to ensure that instead of re-inventing the wheel, these are able to ensure interoperability across the MLOps and DataOps tooling landscape.

Beating Djikstra (for Sparse Graphs)

A new shortest-path graph algorithm has emerged to take on good old Djikstra, however with the usual academic fine-print where in this case it’s only relevant for sparse graphs: Having said that, as graph algorithms continue to become critical for every-day use-cases, an improvement on shortest-path algorithms could result in significant improvements across huge number of applications, particularly in the ML space. This paper presents a deterministic algorithm that apparently is able to circumvent the limitations presented by the sorting barrier in the Djikstra algorithm. It seems using a divide-and-conquer approach it is able to identify roots of large shortest-path subtrees and then recurse only on those, allowing for a theoretical O(m log^{2/3}n) bound algorithm. It is great to see how age-old algorithms still are able to see innovations from around the world.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #348

Sun, 17 Aug 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

LeadDev AI Impact Report
MCP Vulnerabilities For Every Dev
Architecting Large Software Systems
Machine Learning Visualised
Best Practice for Agentic Systems
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

LeadDev AI Impact Report

LeadDev and DX released a joint AI Impact Report for 2025 with important insights: 66% of orgs have adopted AI tools (lower than expected), with investment on internal engineering (85%) and coding assistants (Cursor (45%), GitHub Copilot (37%), and OpenAI (27%). Most teams perceive productivity gains (59%) with tech debt steady or down - which seems to contradict negative sentiments in recent surveys. Relevant to our recent keynote at the World Developers Congress, there’s still an under-investment on the lower funnel of development, with day-to-day usage of AI skewing to code generation (48%), while testing/QA (7%) and deployment (2%) are under-automated. There are still some risks clearly acknowledged such as security (49%), hallucinations (48%), and ethics/policies gaps (47%).

MCP Vulnerabilities For Every Dev

With the increasing MCP protocol adoption there are a growing number of common security vulnerabilities to look out for - these are some of the key ones for any practitioner: 1) Tool Description Injection is real. 2) The authentication situation is not great. 3) Supply Chain Risk is underestimated. Real-world security failures have already happened: Hundreds of exposed servers on 0.0.0.0 with command-execution flaws, the Supabase MCP Lethal Trifecta Attack, Asana Data leak, mcp-remote Command Injection, Accessing private repositories via GitHub MCP. Security has always been a core pillar of software, and today with th erise of AI hype it is no different.

Architecting Large Software Sys

Large systems stay fast and reliable when you ensure robust design and stable contracts/interfaces: This is an interesting introspectively walkthrough on various hands-on projects sharing lessons on complex software system design. There are some interesting take-aways such as defining stable & versioned APIs/schemas/protocols, wrap external platforms behind adapters, and make each module singly owned and replaceable without breaking callers. There are some lessons that could be argued as (very much) over-engineering as building everything in-house, likely these are for very specialized instead of the generic approach to building everything, however there are still interesting takeaways overall.

Machine Learning Visualised

Machine Learning Visualized is one of the best and most intuitive overviews of ML concepts and foundations out there, especially for visual learners: ML Visualized is an OSS Jupyter Notebook that explains and implements core algorithms from first principles with animated training/convergence visuals. The mini-course covers gradient descent, PCA, k-means, the perceptron, logistic regression, and compact neural nets with backprop, loss-landscape, and weight-space views. Even if you already know these concepts it’s still a nice way to get a refresher, so definitely worth checking out.

Best Practice for Agentic Systems

What are the best practices for designing and building Agentic Systems? Here’s a few: 1) Ship a two-tier design: a single stateful orchestrator that plans and maintains context, and stateless subagents treated as pure, RPC-like functions (e.g. task -> structured JSON result) to enable safe parallelism, deterministic tests, and prompt-hash caching. 2) Decompose work vertically for dependencies and horizontally for scale; in practice most pipelines mix parallel extraction with sequential aggregation. 3) Prefer strict I/O contracts - e.g. clear objective, bounded scope, output schema, timeouts/limits - and keep context isolated by default, with filtered summaries or references only when necessary. 4) Orchestrate primarily with sequential pipelines and map-rreduce; use Consensus voting only for high-risk judgments and avoid deep hierarchies that explode debugging. 5) Engineer failure paths: graceful degradation, targeted retries (prompt rephrase/model swap), exponential backoff, and always return partials with suggested next actions. There’s many more considerations, and this is a great and comprehensive overview of.best practices, however the field is still emerging so it will be interesting to keep an eye to see how it evolves.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #347

Sun, 10 Aug 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Atlassian 2025 State of DevEx
GPT-5 Attributes, Pricing + Card
Qwen Image Technical Report
Google DeepMind World Model
Raschka on Qwen 3 From Scratch
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Atlassian 2025 State of DevEx

Atlassian’s 2025 DevEx report suggests that GenAI is delivering: 68% of developers save 10+ hours/week, mostly on non-coding work (search, testing, docs, and automation) which sounds optimistic, particularly with many opposing view we’ve seen in recent reports: One thing is for certain, most of the inefficiencies (and opportunities) are on traditional drives, such as hours lost on organizational overhead. Only ~16% of time is spent coding (!); teams that enable self-service on knowledgesharing are 4.9× more effective and 4.4× more productive/adaptable. Seems measurement of outcomes is moving towards the SPACE framework from Microsoft. Internal dev platforms are now mainstream (74% use, 24% plan) and seems enable organisations across reliability, faster delivery, and lower ops costs.

GPT-5 Attributes, Pricing + Card

Simon Willison puts together another fantastic overview, now of GPT-5 characteristics, model card and pricing nuances: GPT-5 seems like a relative upgrade, however not yet an order of magnitude improvement (and instead more of a improved smart model-router). It provides 272k input context and 128k output (includes hidden reasoning), supporting text+image-in. It is also interesting to see the anger from the community when it was announced that other models will be removed, which seems have triggered backtracking to potentially keep some of these. Pricing seems to be the most aggressive update with $1.25/$10 (input/output per M tokens) for GPT-5, $0.25/$2 for mini, $0.05/$0.40 for nano. Some quality of life features that were interesting are that reasoning traces are now retrievable to balance transparency vs. latency, and system card claims fewer hallucinations and less sycophancy - let’s see how this compares to the competition.

Qwen Image Technical Report

Qwen-Image now provides one of the best production-ready models for image-generation models: This is an interesting approach which includes a model trained with a flow-matching objective and a progressive “curriculum” (from non-text to paragraph-level prompts), and enables for edits to preserve both meaning and visual fidelity. The report shows relatively strong results for general creation/editing, as well as pretty impressive text rendering (which is something that normally these models struggle with).

Google DeepMind World Model

Text-to-3D-world models are now a thing; DeepMind Releases Genie 3 providing an interactive 3D text-to-real-time 3D world generation: This new model generates interactive environments at 720p resolution with 24 fps with minute-scale visual memory. This is an interesting evolution to previous models, we recently saw a similar effort re-enacting an Operating System as a 3D world. Unlike NeRFs/Gaussian Splatting it requires no explicit 3D assets, autoregressively conditioning each frame on the growing action trajectory. There are still some limitations, where the episodes can only last a few minutes, direct action space is limited, multi-agent interaction fidelity is weak, geographic accuracy isn’t guaranteed, and text rendering is brittle unless specified. However this is quite an exciting space that initially I assumed was more of a fun/interesting set of prototypes but seems are actually evolving towards potentially usable resources.

Raschka on Qwen 3 From Scratch

MoE LLMs are how we scale capability without scaling cost. A minimal, Apache-2.0 PyTorch notebook re-implements the Qwen3-30B-A3B MoE model (Coder/Instruct/Thinking) with Llama-3–style components—GQA (32 heads, 4 KV groups), RoPE (θ=1e7) for 262k context, RMSNorm, bf16—and an MoE MLP (128 experts, top-8 routing); config: 48 layers, 2048 dim, head_dim 128, vocab 151,936, with weight tying. It loads official HF safetensors shards and tokenizer, moving experts off “meta” to CPU to cut VRAM; despite a ~114 GB bf16 footprint, it runs on a single 80 GB A100/H100 via CPU offload. The reference favors clarity over speed (the naive “compute all experts” beats sparse dispatch here) and provides greedy, streaming generation without a KV cache (a related KV-cache notebook is ~3× faster). Treat it as a didactic baseline: for production, add KV/paged attention, quantization, fused kernels/Flash-Attn, and a deliberate offload/serving plan to hit real-time throughput and long-context stability.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #346

Sun, 03 Aug 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

StackOverflow Dev Survey
Free Databricks MLOps Course
Google DeepMind World Mapping
Multi-Generation Projects
ACM Transitions to Open Access
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

StackOverflow Dev Survey

The Stack Overflow 2025 Developer Survey is out: AI adoption surging to 84%; however ~80% lack of trust on AI outputs. Python’s share continues growing to 58% (+7% YoY) with FastAPI as the fastest growing framework (+5% YoY). Rust holds the “most loved” crown for yet another year. We’re also seeing median salaries climbing slowly 5% to 29% depending on role, yet 75% of developers still find themselves unhappy/complacent, hinting at retention risks. Anthropic’s Claude Sonnet tops the admiration charts among LLMs, and the Cursor IDE already reaches 18% usage.

Free Databricks MLOps Course

Marvelous MLOps released one of the absolute best Databricks courses on MLOps out there; and it is completely free: This is a six-part course which covers the end-to-end machine learning lifecycle in Databricks, encompassing not only the basic know-how but also best practices from real-world application. As they suggest, despite the agentic hype, most value unlocked comes from ML/DL-powered usecases, and robust MLOps can make a significant difference on scientific productivity and time-to-value. They emphasise the importance of traceability/reproducibility, Git-driven CI/CD, and monitoring (e.g. infra and data); between others. As part of the course they cover version control, orchestration, model registry, container & data versioning and observability with DBX components like Lakeflow Jobs, managed MLflow tracking/registry, serverless/cluster compute, feature store, Lakehouse Monitoring, etc. Even if you have watched some of DBX resources before, this is definitely a must watch - check it out!

Google DeepMind World Mapping

Google DeepMind has released a planetary-scale foundation embedding model that converts petabytes of optical, radar, LiDAR and climate-simulation data into a vector space that provides signal directly instead of depending on disparate complex multi-modal datasets. It is interesting to see that whilst we’ve seen some previous foundation models for vertical-specific use-cases, we are now starting to see embeddings that enable broader usage; in this case it’s a 1.4 trillion-vector satellite embedding that will support teams use it as a virtual-satellite feature layer for rapid enviromental ML experimentation without heavy geospatial preprocessing.

Multi-Generation Projects

It is quite fascinating to learn about projects that were built across hundreds of years, and to think what projects to this date will be the cornerstone for the centuries to come: There are a surprisingly number of examples that went through decades and centuries, many of these are building cathedrals, long-term studies, solving scientific problems. There are some pretty interesting topics in this list, including examples such as Sagrada Familia (1882-now), Notre Dame (1163-1345), LIGO gravitational Wave detector (1967-2016-now), Japanese Company Kongo Gumi (578-2006), etc. It is interesting to imagine some of the projects that will be collectively built for the years to come, such as the Linux Operating System, or the Internet itself (protocol, infrastructure, etc). Or maybe that MLOps pipeline you built as a prototype! This is a great (albeit short) article, but definitely worth checking out.

ACM Transitions to Open Access

Huge leap forward from the ACM making the move towards full open-access research; it’s been an honour being part of the ACM Governing Board as this has been voted through! As part of this initiative ACM will make all journals / conference papers / other publications free to read by end-2025. This is definitely a huge transformation that is going the right direction, and hopefully this leap is also setting the bar for other organisations to consider open-access. This also means that with the current model authors can publish unlimited OA papers without per-article charges while retaining full Digital Library access from January 2026. Excited to continue supporting the ambition of open-access research across the ecosystem and looking forward to see it developing in the coming decade!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #345

Sun, 27 Jul 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Reverse Eng Github’s App AI
Raschka’s LLM Architectures
State of Homomorphic Encryption
Ray Data, Train and Tune
Writing Toy Software to Improve
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Reverse Eng Github’s App AI

Simon Willison shared last week a masterclass on reverse engineering with GitHub’s new prompt-to-app agentic service: Recently GH released GitHub Spark to enable users to turn prompts into into React + TypeScript micro‑apps running in a Codespaces‑style container. Simon Willison was able to extract the system prompt as well as some of the internals such as the tools of the system through the prompt-to-app functionality itself by requesting a self‑documenting app that explains all its own internals. It is quite interesting to see the level of detail that it is possible to extract with these approaches, and ultimately reminds us of the challenges that arise when releasing a production service to the wild.

Raschka’s LLM Architectures

Sebastian Raschka has put together an overview on the evolution of LLM architectures, together with the most comprehensive set of intuitive visual diagrams on LLMs: It is interesting to see that Sebastian Raschka argues that these models are still structurally similar to when they were initially released. Positional embeddings have evolved from absolute to rotational (RoPE), Multi-Head Attention has largely given way to Grouped-Query Attention, and the more efficient SwiGLU has replaced activation functions like GELU, however it is yet to be seen if there’s been step-change breakthroughs since inception. Definitely worth diving into what seems to be one of the most comprehensive (intutive) overviews of LLMs out there.

State of Homomorphic Encryption

I’ve been keeping an eye on Homomorphic Encryption for a while, and it is exciting to see a moore’s law-like evolution suggesting that we may be getting closer to broader applications: Fully Homomorphic Encryption basically lets servers run arbitrary computations encrypted data without decrypting it, which means that privacy is preserved through operations. The primarily disadvantage has been the costly performance hits required to perform computations, however it seems that as the field evolves, we see more production use-cases of FHE in industry, which could really be a game changer in today’s privacy aware world.

Ray Data, Train and Tune

It is interesting to see end-to-end use-cases of MLOps infrastructure to process data at scale, and particularly it seems Ray has been seeing an increase in adoption in use-cases beyond purely ML. This is an interesting article showcasing the use-case of Ray Data, Ray Train, and Ray Tune across the entire ML workflow in a single Python stack. Namely with Ray Data they are ingesting and preprocessing 100 GB+ tables on elastic EC2 clusters, and scaling to 4 000 vCPUs. They then use Ray Train to wrap PyTorch code in distributed training, letting engineers scale from 4 to 64+ workers. They also use Ray Tune through Optuna to run thousands of hyper‑parameter optimization experiments and logging everything on MLFlow.

Writing Toy Software to Improve

In an era where we everything we hear is about vibe-coding, it’s always nice to see some simple, fun-related projects arise to improve one’s personal coding skills with toy projects. And there’s no better way to learn things than by building quick stripped‑down versions of complex systems (regex engines, OS kernels, async runtimes, etc). Taking on some of these hobby-projects that don’t have a particular end-goal in mind other than learning helps re-kindle the joy that first drew many into programming!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #344

Sun, 20 Jul 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

The State of GenAI in 2025
Neural Net Operating System
Chain of Thought Monitoring
Crawling a Billion Pages
Principal Engineering at Amazon
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The State of GenAI in 2025

Our GenAI keynote at the 2025 WADev World Congress is live! We dive into current limitations on GenAI productivity, and best practices on building for GenAI. One of the key take-aways on the current tooling for dev producitivity is that most sit at the top of the development funnel (ie writing code), whereas the lower parts of the funnel (QA, operating, monitoring, etc) are still much less mature. In order to ensure we invest in mature GenAI tooling, this requires investment across the entire development funnel, and going beyond purely chat‑style code copilots. Achieving this means building agentic stacks that fuse orchestration frameworks, context gateways, guardrails and GPU‑aware schedulers while baking in security defences against prompt injection, data poisoning and other novel threats.

Neural Net Operating System

I never would have imagined an operating system-like experience running (literally) in a neural network; this is quite a creative approach to training a network on user inputs and visual OS GUI outputs: NeuralOS basically brings together an RNN that keeps internal application state with a diffusion UNet that renders the next (OS GUI) frame. The model has been trained on roughly 40 k thirty‑second Ubuntu‑XFCE sessions captured at 15 fps and compressed so the full pipeline fits on an eight‑GPU H100/H200 node. The public demo shows how it reproduces window launches, cursor motion, and coarse keyboard states - despite the huge lag (due to inference time) it’s pretty impressive, and the text hallucination is quite amusing. This particular use-case seems more of a toy implementation, however the concept of GenUIs is definitely growing (last time we saw this simulating DOOM) - certainly an area to keep an eye particularly as these type of simulations evolve.

Chain of Thought Monitoring

The UK AI safety institute released an interesting paper proposing monitoring of chain-of-thoughts where LLMs are used in particularly critical or sensitive topics, as this can allow for stronger guardrails compared to only looking at input/output: This is an interesting paper that shows that when language models emit natural‑language chains of thought, those tokens can be routed through lightweight monitors to flag malicious or undesired plans, such as reward‑hacking shortcuts or policy violations. This can be triggered before any external action occurs which provides an extra safety layer, however the method also has weaknesses, as methods that internalise these chains of thought would remove this capability (e.g. reinforcement fine‑tuning, direct CoT supervision, or future architectures) There are some practical takeaways, such as ensuring teams log the chains-of-thought by default, consider tracking a dedicated metric during training, stress‑test for deliberate obfuscation, and treat CoT monitoring as just one layer in a broader set of tools for observability.

Crawling a Billion Pages

Web‑scale crawling is the raw fuel that powers today’s data‑hungry ML models, and this is a great deep dive in the current state of large-scale (billion+) web crawling. As of today it’s impressive how much can be achieved with relatively low hardware; in this case this is a practical experiment on 12 i7i.4xlarge nodes crawlling over 1-billion web pages in a single day. The setup consists of nine asyncio fetchers and six parsers with redis for caching, which can crawl over 1B HTML webpages in about 25.5 hours for just $462 by sharding the top‑million domains and writing straight to local NVMe instead of S3. The new bottle-neck is no longer the pipes, but the parsing itself; swapping libraries like lxml foroptimized C++ libraries lifts per‑process throughput from 55 page/s to roughly 160 page/s. It is interesting to see the new challenges that emerge as crawling at such a scale becomes so accessible; particularly in the era of AI, many organisations are still figuring out the best balance to restrict unexpected DDOS whilst enabling access for the numerous AI services popping up.

Principal Engineering at Amazon

The principal engineering community at Amazon is renown for having a high bar on technical knowledge and outcome-driven thinking - this is a great interview that provides a deep dive into the world of principal engineers at Amazon: One interesting highlight is the emphasis on the challenges on jumping from Senior (L6) to Principal(L7) is widely seen as Amazon’s toughest promotion, apparently as scope and compensation-wise is presented as a two-level jump; it seems often requires multiple attempts and demands showcasing significant tangible impact (ie already working at that level). Once engineers reach principal, the scope for ownership increases to department-wide, ensuring key focus on scaling impact through technical contributions that have quantifiable outcomes. It is interesting to see how organisations shape these communities internally, as well as the similarities and differences between these - it is particularly interesting now to see how organisations are wrestling to enable what is the intersection of their principal engineering and science IC communities to support the scale of machine learning needs in the era of GenAI+.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #343

Sat, 12 Jul 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Andrew Ng on Building Faster
GetDX: Measuring AI Productivity
Coding Languages by Efficiency
Twitter Releasing Grok Model
Decreased Productivity with AI
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Andrew Ng on Building Faster

This ~40min keynote from Andrew Ng at Y-Combinator is one of the best overviews for the state of rapid AI prototyping: Andrew brings some of the key learnings from ML startups leveraging agentic workflows and AI coding assistants to spin up disposable prototypes. The main takeaways are emphasising what most of us have seen in industry, namely that tooling is enabling us to treat architecture choices as reversible “two-way doors”, and sharpening direction through fast feedback loops. As engineering accelerates andrew argues that product management becomes the bottleneck instead of engineering (however IMO it’s more nuanced as we still see good old PRs being the bottleneck still…). Quite interesting to see these very bold and optimistic takes but what is most important is to ensure we can actually quantify the productivity gains vs qualitative metrics.

GetDX: Measuring AI Productivity

Cracking how to measure AI productivity is something most orgs are trying to figure out, which is why it’s great to see this new full-stack framework from DX on measuring AI productivity: The DX AI Measurement Framework gives leaders what feels like a refreshing perspective on AI productivity, as it blends traditional software metrics with recent metrics used for AI copilot productivity; namely this is a three dimension scorecard looking at utilization, impact, and cost. This basically dives into the existing productivity metrics, and seeing improvements from other tech giants, such as Booking.com lifting throughput 16%, Intercom seeing 41% jump in AI-saved developer hours, as well as many others.

Coding Languages by Efficiency

In production ML, the language wrapping your models can swing your power bill and latency SLOs by an order of magnitude - this study provides an interesting breakdown of efficiency per language: This is quite an insightful study that benchmarks 27 mainstream programming languages on ten CPU-bound Computer Language Benchmarks Game tasks with Intel RAPL showing various pretty insightful results. Unsurprisingly, compiled languages average 120j and 5s per task, while VM-based and interpreted languages burn 576 J and take 20-99s (~20× energy & ~17× latency spread). As expected C, Rust and C++ lead the way with ~57–77j, 2–3 s; followed by Ada (?) and Java; and indeed followed at the bottom by languages like Perl (still alive!), Python, Ruby, JRuby and Lua (2,660–4,600J, 94–167 s).

Twitter Releasing Grok Model

It has been surprisingly to see Twitter / X releasing Grok recently with benchmarks that seem surpass all other foundation models out there; however it takes more than raw power to win the race: Grok4 packs a 10× jump in pre-training compute with another 10× in RL-on-reasoning tasks, achieving pretty impressive results with SoTA results on the top benchmarks for LLMs. They are doubling down with features as well such as tool call integration now natively trained into the policy for more dependable chain-of-thought executions. It is interesting however to see that as of these days most of these services are now converging in both capabilities but also functionality, so it will take more than just getting the best benchmarks to beat the competition.

Decreased Productivity with AI

We see surveys showing improvements in productivity with AI, but this survey actually shows the opposite; decreased productivity with AI copilots: A field experiment with 16 experienced contributors was carried out on large OSS repositories (~1m lines of code). This encompassed 246 issues with labels as “AI-allowed” or “AI-disallowed” for fair comparison, and the outcomes showed that early-2025 coding assistants (Cursor Pro + Claude 3.5/3.7) slowed developers by 19% even though both participants and experts had forecast substantial speed-ups. The drag appears to stem from unreliable suggestions, context-window limits on sprawling codebases, and tacit quality conventions that demand extra review and rework. This is an experience that tends to resonate with a lot of us; the potential still remains however there is further nuance on productivity gains beyond superficial speed on lines of code.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #342

Sat, 05 Jul 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Microsoft AI Path to Medical Superintelligence
OpenAI Practical Guide to Agents
McKinsey’s Trillion War for Data Centers
Bloom Filters by Example
The Jax ML Scaling Book
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Microsoft AI Path to Medical Superintelligence

Microsoft is set out to leverage AI in the medical field to support diagnosis with their latest 304-case sequential-diagnosis benchmark for LLMs: It is quite interesting to see the intersection of AI in key sectors like healthcare, with Microsoft releasing a benchmark that forces AI models to decide which questions and tests to order (each with a simulated dollar price) before making a diagnosis. This new Medical AI Diagnostic Orchestrator proposes wrapping any frontier LLM in a “mixture-of-experts” agent architecture that self-checks reasoning and enforces cost ceilings. The provided benchmarks were tested with OpenAI o3 and claimed 85.5% accuracy (vs 20% for practicing clinicians), all while spending less on tests than either humans or standalone models. These type of initiatives have to certainly be taken with a level of skepticism given the hype, but it is certainly great to see the investments from research labs to tackle some of these important topics, and particularly in the case of open frameworks / models.

OpenAI Practical Guide to Agents

OpenAI has released a comprehensive guide to building agents to empower developers when creating AI-enabled applications beyond chatbots: This guide from the OpenAI team focuses on applications that encompass entire workflows that making complex decisions, invoke tools, and relinquish control only on failure. It is interesting to see the differences between orgs publishing these guides; OpenAI is (expectedly) bullish on deploying agentic workflows in contexts that requrie messy unstructured data whilst leaving straightforward tasks to simpler data processing pipelines. Most of the topics from these type of guides are starting to converge towards a generic best practice, enabling for a more standardised set of guidelines to emerge that ideally can become cross-cutting beyond just a single AI lab.

McKinsey’s Trillion War for Data Centers

Compute is the new oil; McKinsey forecasts that global data-centre capacity must almost triple by 2030, demanding ~$7 trillion in new capital: This recent report from McKinsey provides important insights for AI leaders, quantifying that AI workloads driving about 70% of the forecasted ~$7Tn growth ($5.2 trillion for AI alone); some scenarios push the forecast as high as ~$8 trillion. About 60% of this spend seems will go to chips and servers, 25% to power and cooling infrastructure, and 15% to land and construction. This will be leaving silicon vendors, utilities and builders in a high-stakes race where capacity, energy and supply-chain constraints (not algorithmic breakthroughs). Similarly there will be a lot of important considerations in regards to sustainability and efficiency as these demands grow in order to ensure sustainable growth globally.

Bloom Filters by Example

Probabilistic data structures such as Bloom Filters are a key concept in distributed systems that we interact with on a daily basis, however there’s huge value to understand the internals: This is a great overview on the internals of Bloom filters that provides visual / interactive examples on how this data structure works as data is provided / fetched. A Bloom filter is a probabilistic data structure that answers “have I seen x?” in O(k) time whilst using reduced memory through probabilistic approximations. Namely, Bloom filters sometimes say “maybe” when the answer is actually “no” in order to ensure memory efficiencies, however this data structure never misses a true hit. These probabilistic tradeoffs enable slashing storage and compute costs across data deduplication, spam blocking, database queries, and real-time recommendation feeds. This is one of the best intuitive overviews of Bloom Filters out there, so it’s definitely recommended to check it out in detail!

The Jax ML Scaling Book

As of today, scale in ML tends to define success across some of the leading players in the space - Google’s “Scaling Book” is one of the best resources for this: There are really great tips in this scaling book, covering key topics in ML scaling such as rooflines, TPUs, Sharded Matrix Multiplications, provides an overview on transformers, as well as nuances on training such as GPU utilisation, and dives into practical cases such as training LLaMA, inference and profiling.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #341

Sun, 29 Jun 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Internals of vLLM Inference
Databricks Unified ML Governance
A Deep Dive Into TPUs
Gemma MultiModal Released
LLM Transformers as Malware
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Internals of vLLM Inference

In the race to turn massive LLMs into reliable, inference infra is a real competitive edge - but do you know what goes on inside a library like vLLM? For example, I didn’t know it used Ray under the hood! The vLLM framework handles an inference call end-to-end within one tightly orchestrated pipeline: 1) An OpenAI-style API server offloads tokenisation to an AsyncLLM process that forwards the request via async IPC to EngineCore. 2) The Scheduler packs tokens into continuous, token-budgeted batches, assigns fixed-size KV-cache “pages,” and hands the batched tensor to Ray orchestrated ModelRunners. 3) These runners execute forward passes with FlashAttention-3 (optionally via CUDA-graph capture). 4) Afterwards freshly generated tokens stream back through IPC to be detokenised and chunked out to the client. This is quite intuitive so definitely worth deep diving further for ML Practitioners to know what is going under the hood when serving popular models (eg Llama 4).

Databricks Unified ML Governance

Databricks published their approach to unified ML governance at the recent SIGMOD in Berlin providing an intuitive overview of how they approach this challenge: This is quite an interesting paper which shows how Unity Catalogue provides an open control plane that places tables, files, feature sets and MLflow-registered models in the same three-level namespace (catalog.schema.asset) while enforcing uniform row-, column- and path-level security via a one-asset-per-path rule and short-lived credential vending. This makes it possible for policies to hold whether Spark references the table name or another engine reads the raw object-store path. They also provide a glimpse in adoption and industry use, with roughly 100 M tables, 550 K volumes and 400 K models for 9 K customers, sustaining ~60 K metadata calls/s with latencies comparable to an on-cluster Hive metastore.

A Deep Dive Into TPUs

TPUs are redefining how large-scale AI gets built and powered, and this is a fantastic deep dive into the architecture and concepts across TPUs: Unlike GPUs that juggle lots of small tasks on the fly, TPUs are streamlined “assembly lines” for matrix math. These TPUs preload data into big on-chip buffers, run it through grid-like calculator blocks, and skip the energy-hungry cache layers. A single TPU chip can link up with three others on a board, sixty-four in a rack, and thousands in a pod, which are supported by lightning-fast optical switches that let Google carve the hardware into any shape a model needs. Because the XLA compiler figures out every data move in advance, engineers who keep their models’ shapes steady get massive speed and energy savings almost automatically, however for more general use-cases workloads will likely still prefer the flexibility of GPUs.

Google MultiModal Released

On-device multimodal AI in the open source is seeing some recent leaps, and Google’s Gemma 3n is the catalyst. Gemma 3n has been able to compresses the 5-8B parameters into models that can fit in 2-3 GB VRAM. IT uses what they refer to as a “Matryoshka-style” MatFormer architecture with Per-Layer Embeddings that park token tables in CPU RAM and a KV-cache sharing that halves pre-fill latency. Google suggests a score of 1300+ on LMArena, which would make it the first sub-10 B model to rival cloud-scale performance. It is great to see the momentum and progress for open models as we pretty much see new innovation being pushed to the limits on a weekly basis.

LLM Transformers as Malware

Your favourite AI LLM may be hiding malware! Recent research EvilModel shows how attackers can embed full binary payloads of malware in any standard LLM for malicious attacks: This research paper shows how an attacker can inject malicious code straight into redundant weight tensors without any model performance loss. This can be done by overwriting neurons with carefully crafted 32-bit floats that scarcely dent accuracy (<1% on ImageNet models) and evade every commercial vulnerability detection engine, enabling a simple extractor script to reassemble the malware once the checkpoint is loaded. This makes any pipeline that auto-downloads or updates third-party models a potential smuggling route, so all of us as machine learning practitioners must treat external model files + code like untrusted executables, verify provenance and hashes, and apply destructive sanitisation (quantisation, pruning, or re-serialisation) before deployment!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #340

Sun, 22 Jun 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Karpathy on Software Evolution
Google on Securing AI Agents
Anthropic on Building AgenticSys
State of Eng Leadership 2025
Quantum Computing Lecture Notes
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Karpathy on Software Evolution

Software is undergoing its most fundamental shift in decades, and Andrej Karpathy defines how it’s redefining how we build and interact with machines: This is a great new talk from Karpathy on how “Software Is Changing (Again)”, covering the new era where LLMs act as programmable computers coded in natural language. He argues that LLMs are still in their “1960s mainframe” phase (which is a funny analogy) drawing parallels to operating systems. Karpathy emphasizes the need for partial autonomy, fast human-AI collaboration loops, and LLM-friendly infrastructure, and it is interesting to see the urgency he pushes for practitioners to design for agents and master the full software stack from code (1.0) to learned weights (2.0) to prompts (3.0) as we rebuild computing from the ground up.

Google on Securing AI Agents

Google takes on GenAI security principles, concepts and best practices with a new report on how to secure your AI agents: This whitepaper from May warns that LLM-powered agents carry critical vulnerability risks across rogue tool calls and inadvertent data leaks, and argues that teams must embed security from day one via three pillars. 1) Assign a clearly authenticated human controller. 2) Grant only dynamically scoped least-privilege permissions. 3) Log every input, plan, and action for auditability. Google’s own “hybrid defense-in-depth” approach supposedly introduces deterministic policy engines (hard blocks, spending caps, user confirmations) with reasoning-based guard models and adversarial training to catch novel attacks, then keeps both layers sharp through continuous red-teaming, regression testing, and bug-bounty feedback.

Anthropic on Building AgenticSys

Anthropic worked with dozens of teams building LLM agents and released a set of best practices for building agentic systems with simple composable patterns rather than complex frameworks: This is a useful resource for production ML teams looking to find the fastest path to reliable agentic systems by leveraging LLMs orchestrators (ie base model plus retrieval, tool-calling, and memory). Anthropic proposes to introduce measurable improvements to drive outcomes with prompt chaining for fixed sub-steps, routing for distinct input classes, parallelisation for speed or consensus, orchestrator-worker hierarchies when subtasks emerge on the fly, and evaluator-optimizer loops for iterative refinement. There’s also a section on full autonomous agents (LLM + tool loop with environmental feedback) for open-ended problems where step count and order are unpredictable. It is interesting to see how these paradigms evolve throughout, however keeping simplicity at its core, continually A/B-testing each added layer and culling complexity that doesn’t move key metrics.

State of Eng Leadership 2025

The State of Engineering Leadership in 2025; management layers # decreasing; negative sentiment on AI productivity +60%; positive improvements in talent & recruitment; really important insights you don’t want to miss out: 617 engineering leaders were surveyed across critical topics in tech, including talent trends, productivity, sentiment, AI and more. There are some important trends such as layoffs and hiring freezes easing yet recession anxiety (65%) and vanished entry-level openings shifting demand to mid-senior talent. Orgs are focused on both flattening and bolstering management, whilst remote work is decreasing, and leaders juggle more teams and more pressure while burnout climbs. Despite intense hype, 60% say GenAI hasn’t boosted output and 51% call its net impact negative, hampered by quality, security, and workflow hurdles, even though there are some narrow wins emerging in automated code generation, refactoring, and documentation. Hiring remains challenging, but practitioners who marry rigorous ML-in-prod skills with strong communication and cost-aware reliability will stay in demand and can unlock real value by applying GenAI to focused, high-leverage tasks.

Quantum Computing Lecture Notes

Quantum computing is an exciting emerging field intersecting with AI, and this 200+ page lecture notes distil two decades of theory into a comprehensive roadmap: This is quite a comprehensive overview that introduces key concepts in quantum computing starting with qubits, gates and staple algorithms (Deutsch–Jozsa, Simon, Shor, Grover), then detailing ML-relevant subroutines such as the quantum Fourier transform, Hamiltonian simulation and the HHL linear-solver that yield polylog-time speed-ups for sparse, well-conditioned matrices. The more advanced chapters arm you with complexity tools (generalised adversary bounds, QMA, Local-Hamiltonian) to recognise when quantum gains are impossible, and practical sections provide great intuition on variational circuits, PAC learning from quantum data, error-correction and fault-tolerance. Although today’s hardware is still noisy, the theoretical stack for future quantum acceleration of large-scale model training and optimisation is already well mapped out and there’s quite interesting potential in the horizon.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #339

Sat, 14 Jun 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Time-Blind Large Visual Models
Unified Metadata Model at Netflix
Mistral Reasoning Model Magistral
The Effectiveness of For-Loops in GenAI
Human Quality Text-to-Speech
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Machine learning models for video tasks have limited performance, and this new paper (+ benchmark) show clearly the issue lies in time-blindness: Temporal cues power everything from life-saving medical monitors to self-driving cars—yet today’s visual (eg video) language models can’t read them. This research team has released a new benchmark and dataset to test specifically for this temporal-blindness in state-of-the-art video-language models. Namely when all semantics live in frame-to-frame motion and individual frames look like noise, which results in 0% accuracy from the leading research labs (ie. GPT-4o, Gemini, etc). Without salient spatial features, the models have nothing coherent to integrate, yielding a hard failure rather than a graceful degradation - it is quite interesting to see that these type of open initiatives help set the foundation for step change improvements in these fields.

Unified Metadata Model at Netflix

Netflix has released their unified data architecture to standardise their data models across their organisation; given how data is the foundation for ML, reducing duplication, improving data quality and enabling discoverability is key: Netflix shared their eng blog how they used a self-describing metamodel to capture business entities once as graph-based “domain models,” then automatically transpile those models into consistent schemas (GraphQL, Avro, SQL, Iceberg, Java), which can then provisions the corresponding data containers and pipelines, and record bidirectional mappings that tie every conceptual attribute to its physical location. They already leverage this in production through a knowledge-graph control plane, which has significantly reduced model drift, duplicate effort, and brittle joins. These type of innovations may not sound big, however these give ML teams a single, queryable source of truth that keeps training data, features, and serving paths semantically aligned across the e2e model lifecycle.

Mistral Reasoning Model Magistral

Reasoning models are the next big unlock for high-stakes production AI - Mistral enters the race with their first reasoning model “Magistral”: It is quite interesting to see the acceleration on the battle to the top across the various different foundation model domains - ie coding, reasoning, speed, image, video, etc. Although Mistral may seem is entering somewhat late to the party, this is only relative to the insane speed of the field. Overall it seems there’s an open-weights 24b param version, and a closed source larger version. On benchmarks it seems like it brings competitive performance however we are seeing stagnation on order-of-magnitude improvements, and assume the major jumps in the near future will be due to innovations beyond raw-compute / params.

Unreasonable Effectiveness of For-Loop in GenAI

The unreasonable effectiveness of a for-loop in agentic systems; or “how to build an agentic system in under 10 lines of Python”: Although an oversimplification, this provides quite a good intuition on a super simplified version of how many libraries approach agentic tool use; the code is so small that we can include it below:

def run_agent(llm): msg = user_input() while True: output, tool_calls = llm(msg) print(“Agent: ”, output) if tool_calls: msg = [ handle_tool_call(tc) for tc in tool_calls ] else: msg = user_input()

In most agentic systems, a loop coordinates a model to alternate between conversation and executing returned tool calls (bash, patch, web, etc.). Although each run is slower and increases the cost additionally (in tokens), containerizing these loops is able to “sandbox” side effects and enables parallel execution. This is an emerging field so it is always interesting to see how paradigms, design patterns and taxonomies are being developed in real time - check out the article (+ other blog posts) for a deeper dive.

Human Quality Text-to-Speech

High-quality voice-cloning and synthetic text-to-speech is going to impact a significant society as well as vast amount of industries + these OSS models are becoming indistinguishable vs humans: Resemble AI has released an MIT-licensed text-to-speech deep learning model which can perform voice-cloning and synthetic voice generation with sub-200 ms real-time inference, zero-shot voice replication from ~5 s of audio, and a dial-able emotion parameter, all trained on 500 k h of curated speech. There are also interesting innovations with watermarking which is implemented via through their system which enables for provenance of generated audio. It is mind blowing that these resources are openly available as OSS, as these will actively disrupt use-cases related to assistants, games, dubbing, or localization pipelines (+ many others).

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #338

Sun, 08 Jun 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

The State of AI Talent in 2025
Migrations with LLMs at Google
Claudfare’s GenAI Development
Apple on Limitations of LLMs
Mistral Release Codestral Embed
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The State of AI Talent in 2025

The entry-level recruitment pipeline in tech is collapsing; headcounts are shifting to core ML; Elite AI labs are hoovering top talent; there are some key trends in the state of AI Talent in 2025: This latest reports brings together insights from over 650 million professionals and 80+ million organisations on critical topics to keep a close eye if you are in the AI space. 1) Significant reduction on entry-level hiring: Only ~7 % of Big-Tech hires and <6 % of VC-backed-startup hires now come from new grads – roughly half the pre-COVID share. 2) Head-count is being re-balanced toward core ML infrastructure with ML/AI +27%, Data Engineering +3%, Senior Software Eng +3%, Product/Design/Sales/Recruiting −10%. 3) Elite labs are poaching top talent: For every 1 engineer Anthropic loses to DeepMind it gains ~10; the OpenAI-to-Anthropic ratio is 8:1. 4) Hiring is shifting towards big hubs: Bay Area, Seattle and NYC regained Big-Tech head-count in 2024; Austin (-6%), Houston (-11%).

Migrations with LLMs at Google

Google has published their lessons learned embedding GenAI automation on exa-scale migrations with ~50% task reduction: It is quite interesting to see Google adopting LLMs tuned on internal-code for acceleration of large-scale migrations across their organisation, combining it with a staged validation chain (syntax, build, tests) to automate large-scale “32-bit → 64-bit ID” refactors across their monorepo. Over 12 months Google processed 39 migrations, producing 595 change lists where the LLM authored 74% of CLs and ~70% of all character edits, enabling the three engineers involved to cut total migration time by roughly half while still preserving human review for any diff that fails automated gates.

Claudfare’s GenAI Development

A formerly LLM-skeptical Cloudflare engineer spent two months pair-programming with Anthropic’s Claude to build a production-grade OAuth 2.1 provider for Cloudflare Workers - it is interesting to see the mind-shift discovering that an LLM can act as an effective “diff generator” when tasks are framed in tight, reviewable increments: The Claudflare team has published the codebase as an official package together with the detailed commits that summarise the prompts used for each of the respective changes, as well as the iterations when addressing specific bugs, change requests, etc. As part of this repo we can see that Claude produced most boilerplate, tests, CI actions, metadata and KV storage schema, while the human steered high-risk logic (crypto, PKCE, refresh-token rotation) and enforced security reviews For production ML practitioners this is a perfect example to follow for best practice when it comes to leveraging LLMs as a productive copilot.

Apple on Limitations of LLMs

Apple has published a fantastic analysis on the capability for LLMs on reasoning tasks through benchmarks that evaluate the chain of reasoning as opposed to purely the final answers, showing that LLMs are still quite limited on reasoning tasks: In safety-critical AI—from self-driving fleets to automated trading—robust machine reasoning is the thin line between insight and disaster, and so far it has been ambiguous to quantify how good LLMs are (despite positive outcomes in various benchmarks). It is clear that vanilla LLMs dominate trivial tasks, Language Reasoning Models excel at moderate complexity, yet both implode to near-zero accuracy once complexity crosses a critical threshold, with tell-tale patterns like accuracy that drops as reasoning lengthens on simple problems and flat-lines on hard ones. For us as practitioners, LRMs remain probabilistic pattern matchers which have to be leveraged where they are fit-for-purpose.

Mistral Release Codestral Embed

It is now clear that fast and precise code retrieval is critical for next-gen developer tooling, and Mistral has now entered the race with their new code-specialised model: This new model from Mistral (codestral-embed-2505) comes in with impressive performance topping nine retrieval benchmarks, beating Voyage Code 3, Cohere v4, and OpenAI Text-Embedding-3 Large by 5-25 pp, even when compressed to 256-dim int8 (which is now important for efficiency and speed). It will be interesting to see it in practice particularly with RAG-based copilots, semantic/NL code search, duplicate detection, and repo analytics.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #337

Sun, 01 Jun 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Github CEO on AI Code Race
Black Forest Labs Flux.1 Kontext
The Hidden Bloat on ML Systems
OpenAI ChatGPT Academy
Introducing Perplexity Labs
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Github CEO on AI Code Race

GitHub CEO dives into the Coding Agent race and the trends we are seeing across Cursor, Windsurf and others: It is quite interesting to see how Github was able to see to get ahead of the race with copilot but now struggle with homogenisation of features. It seems there’s still some moat with GitHub Actions in CI/CD pipelines as well as the agent VS Code / gh issue interface but it seems the race is only picking up pace. This is quite an insightful podcast that covers some of the ongoing trends on coding agents, as well as limitations and emerging opportunities, definitely worth checking out.

Black Forest Labs Flux.1 Kontext

Black Forest Labs releases a new image-workflow model for interactive modification of images through text: FLUX.1 Kontext is a suite of generative flow-matching models that unifies text-to-image and image-to-image workflows by accepting both text prompts and input images to perform context-aware generation and editing without any fine-tuning. It’s quite impressive to see what it can do across elements of an image, preserving the rest of the scene whilst modifying only what is requested via text / image. This is certainly a space that is only speeding up, and new creative features are coming out almost every week changing the field for players trying to innovate.

The Hidden Bloat on ML Systems

Do you know about the hidden bloat in machine learning libraries? It is interesting to see that across ML frameworks there is a huge amount of shared library size through GPU kernels and other components. In experiments across PyTorch, TensorFlow, vLLM, and Transformers for models from MobileNetV2 to Llama-2-7B there is an opportunity to optimize shared‐library size by roughly 47–55%, prunes up to 72% of CPU code and 75% of GPU code, and yields up to 74% lower CPU memory, 70% lower GPU memory, and ~45% faster startup. This is quite an important paper to bring optimizations beyond purely compute into the frameworks themselves, and hopefully raising the bar for standardisation.

OpenAI ChatGPT Academy

OpenAI has released the ChatGPT Academy with end-to-end courses across a broad range of agentic tasks: The are quite a lot of new courses like “ChatGPT at Work”, which is a set of of short video modules and webinars that equip ML teams to integrate ChatGPT into real‐world systems. It begins with foundational overviews of GPT models and prompt engineering (from basic to advanced), then delves into reasoning techniques and multimodal inputs, shows how to build retrieval‐augmented pipelines (ie ChatGPT Search) and automated data analysis, and includes guidance on deep research workflows and hands‐on project examples. There are also longer ChatGPT webinars and popular tutorials (e.g., Codex setup, RAG with GitHub) to help you accelerate your workflows.

Introducing Perplexity Labs

Perplexity has launched Perplexity Labs extending DeepResearch capabilities with interactive multi-step workflows that generate complete deliverables such as code-driven data prototyping, charts, reports, spreadsheets, and even interactive mini-web apps: It is quite exciting to see some of these type of developments with a new service that uses deep web browsing, code execution, and asset management for creating end-to-end projects and apps as the outcome. It seems that this may become a new trend, and wouldn’t be surprised if we see releases from across the other research labs on extremely similar releases (with slightly different namings).

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #336

Sun, 25 May 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Datadog OSS Observability ML
Anthropic Releases Claude 4
Upgraded Climate Foundation AI
Mistral Launches Devstral OSS
Microsoft AI Red Teaming Labs
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Datadog OSS Observability ML

In modern DevOps being able to predict failures before they happen can avoid huge outages and risks; Datadog out of the blue released a 151m+ param time-series foundation model for real time observability: Toto-Open-Base-1.0 is a large time-series foundation model specifically architected for the type of sparse, spiky, high-cardinality data in observability metrics. They also released “BOOM”, a new (massive) benchmark of 350m real-world observations across 2,807 multivariate series. It is fascinating to see how time series foundation models have been evolving rapidly, and the datadog team are bringing innovations like patch-based causal normalization, proportional factorized attention, a Student-t mixture head, and a composite robust loss; of course the secret sauce is pretraining on over a trillion points of internal telemetry. The space of time series foundation models are really exciting, we are already seeing some (surprising) opportunities in their application in core business operations - certainly a key space to keep an eye on.

Anthropic Releases Claude 4

AI agents for software development are only growing in popularity, and Anthropic has just released Claude 4 with some exciting tooling to take the ecosystem to the next level: Claude 4 has been long awaited since the impact from 3.7, and this release comes with two models - Opus 4, which is optimized for sustained multi-hour agent workflows (72.5% on SWE-bench, 43.2% on Terminal-bench), and Sonnet 4, which balances efficiency for everyday use (72.7% SWE-bench). It is interesting also to see the extended support for reasoning that is doubling down on MCP/tool integration support. This also comes with Claude Code now moving to GA, and continuing to push the dev-agent race to see what further releases we’ll continue to see week on week.

Upgraded Climate Foundation AI

Accurate forecasts of weather, air quality, and ocean hazards are critical both in environmental as well as business contexts; we see another exciting resource of Microsoft’s foundation model for climate prediction: Microsoft’s Aurora is a 1.3 billion-parameter foundation model that replaces multiple specialized Earth-system solvers with a unified encoder–processor–decoder architecture. It is interesting to see models that take a hybrid approach with deep learning with physics aware components, in this case it includes 1) a Perceiver-based 3D encoder ingests heterogeneous atmospheric and surface fields into a latent grid, 2) a multiscale 3D Swin Transformer U-Net evolves that state forward in time, and 3) a Perceiver-based decoder reconstructs physical variables at any resolution. This foundation model is pre-trained on over one million hours of mixed forecasts, reanalyses, and climate simulations for a next-step MAE objective - I continue to say this but this will be an exciting year for time-series foundation models across what seems to be a broad range of domains being disrupted.

Mistral Launches Devstral OSS

Mistral enters the AI race for GenAI software agents with their new Devstral agentic LLM for software engineering: This new model is an exciting collaboration between Mistral AI and All Hands AI, and they explain it is designed to navigate large codebases, diagnose subtle bugs, and resolve real GitHub issues via agent scaffolds like OpenHands -. On the 500-issue SWE-Bench Verified benchmark it achieves 46.8% which suggests better performance than some of Anthropic’s models, and is lightweight enough to run on a single RTX 4090 or a 32 GB-RAM Mac (with ~32 k token context). It will be quite interesting to see the results once the broader community is able to put it into practice - especially with the super recent Claude 4 release.

Microsoft AI Red Teaming Labs

AI red teaming is only becoming more critical with the growing adoption of AI and the Microsoft AI team has released an full end-to-end Red Teaming Playground Lab: The Microsoft AI team originally developed a course for the “AI Red Teaming in Practice” course at Black Hat USA 2024 and they have now created an open source version available for free with12 interactive Jupyter-style challenges. This is a really great resource to dive into some GenAI security - it includes exercises from basic prompt-injection and metaprompt extraction attacks to multi-turn “Crescendo” exploits and guardrail bypass scenarios at increasing difficulty levels. Definitely worth checking out and getting hands on irrespective whether you are an ML practitioner, Software Developer, Security Practitioner or otherwise.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin
World Summit AI Europe - 08 Oct @ Amsterdam

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
AI Summit London - 12th June @ UK
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #335

Sun, 18 May 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

OWASP Agent Name Service
OpenAI’s New Software Engineer
On Complex Systems at Scale
Google on Text-to-SQL models
DeepMind on Coding Agent
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

OWASP Agent Name Service

The OWASP Agentic Team has released a new framework for discovery and verification of agents: The Agent Naming System (ANS) provides DNS-inspired universal registry for AI agents, PKI-based identity verification (X.509), Capability-aware discovery, Protocol-agnostic design (A2A, MCP, ACP, etc), Threat Analysis and more. It was great to contribute as part of the reviewing board of the OWASP Agentic team - check it out!

OpenAI’s New Software Engineer

OpenAI releases a new software engineering agent that runs on repo sandboxes to tackle multiple coding tasks in parallel - it’s interesting to see OpenAI playing catchup (and catching up!) on domain-specific agent services: This new coding agent claims to cover the ability to fix bugs and write new features (via direct pull requests) as an army of supporting junior developers in your toolbox. It seems production teams can customize Codex’s behavior with (yet another config) AGENTS.md files to mirror their development conventions and ensure environment fidelity, and can also use the lightweight Codex CLI locally with the optimized codex-mini model for low-latency edits and Q&A.

On Complex Systems at Scale

As production software practitioners we often have to deal with both, “complicated” and “complex” systems - it is important for us to make a clear distinction to ensure the right solution: Complex systems exhibit unpredictable, emergent behaviors, delayed and nonlinear effects, hysteresis, and counterintuitive local vs. global trade-offs. Practitioners building production ML platforms should therefore favor reversible changes (feature flags, canaries, progressive rollouts), instrument high-cardinality observability, validate via simulations and workload replays, define both local and end-to-end metrics to catch hidden impacts, and lean on adaptive ML models and strong cross-functional collaboration to navigate ambiguity and continuously evolve their systems.

Google on Text-to-SQL models

Google is tackling the challenge of AI-powered text-to-SQL models through Gemini, and they share some of the key challenges when generating SQl with ML models: The Google team has been able to collect insights on the most common limitations on LLMs that generate SQL from across their systems (BigQuery, CloudSQL, Spanner, etc); they find some relevant areas include injecting business-specific context via semantic retrieval (vector-search on schemas, data samples, and annotations), managing user-intent ambiguity through LLM-driven clarification dialogs, and ensuring dialect fidelity with self-consistency voting, dry-run validation, and targeted reprompting or lightweight fine-tuning. This area of automated SQL generation with LLMs is only becoming more common in the analytics and engineering space so it is definitely great to see best practices shared from lessons learned.

DeepMind on Coding Agents

DeepMind releases AlphaEvolve as a new coding agent to also challenge the competition (ie Github, Claude, now openAI, etc): As other models it enables generation, evaluation and refinement with Gemini Flash - so far it seems that most players are consolidating on features so it will be interesting to see how competition pushes towards new features beyond improved performance.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #334

Sun, 11 May 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

LegoGPT: New Model on LEGO
How GenAI Sees Accents
Comparison of SotA Image GenAI
Google Measuring Tech Debt
Deep Dive into PyTorch Internals
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

LegoGPT: New Model on LEGO

LegoGPT is the right type of AI creativity we want to see; CMU researchers built a model designed to generate physically stable and buildable LEGO designs from text prompts through a fine-tuned LLaMa model: This is a really creative new model from Carnegie Mellon University which generates LEGO structures incrementally from text prompts - the implementation is quite interesting as it ensures each brick is valid and collision-free through brick-by-brick rejection sampling. Basically what this means is that in order to maintain physical stability it needs to integrate physics-aware rollbacks which enables reverting when running into an unstable design. It is also interesting to see the tokenization approach for representation of LEGO designs, and overall this seems like an interesting foundation for more relevant work to come which potentially can go beyond just LEGO.

How GenAI Sees Accents

What if we can use GenAI to improve our accents in foreign languages? And how do these models represent accents in their latent space? This is a fascinating research effort which captures language accents in Latent Spaces, which then allows for the ability to “correct” the accent towards native-sounding audio that can help learners to improve their accents. The samples provided make it quite intuitive on how these type of models can disrupt the language learning space, as they can be implemented for an individual specifically, highlighting the areas of highest potential improvement, and providing key phonetic areas with most divergence. The language learning space is certainly ripe for disruption as these type of innovations evolve, going beyond purely just audio.

Comparison of SotA Image GenAI

Throughout the last few weeks we’ve continued to see new multi-modality GenAI models which support image operations, and this is a great comprehensive analysis of the top models on their compared performance: This qualitative benchmark provides a random set of “challenges” that are run across multiple of the most prominent GenAI models (FLUX, Gemini, ChatGPT, etc) and are compared based on their required iterations and final adherence. On the results, OpenAI 4o led with 9/11 successful prompts (often on the first or second try), Imagen 3 followed at 7/11, FLUX.1 and Gemini Flash 2.0 tied at 4/11, HiDream-I1 managed 3/11, and Midjourney v7 only 2/11. This is far from a formal / quantitative benchmark but it’s interesting to see an intuitive comparison that confirms some personal observations when interacting with these.

Google Measuring Tech Debt

Google’s research on measuring tech debt shows that effectively managing technical debt in production ML hinges on three pillars: 1) use engineer-driven surveys to categorize the ten most common debt types (e.g., migrations, testing gaps, stale docs) and focus only on those that actually hinder productivity; 2) complement—but don’t replace—these qualitative insights with targeted metrics (e.g., churn, TODOs) which alone have low recall; 3) adopt a four-level maturity model—reactive, proactive, strategic, structural—to institutionalize deliberate, visible trade-offs between speed and quality, ensuring that debt is incurred, tracked, and paid down as transparently as model accuracy or SLOs.

Deep Dive into PyTorch Internals

Diving into PyTorch’s internals can provide practitioners with meaningful intuition when tackling more complex and challenging ML projects at scale, and these are some important components: It is interesting to dive into some of the core components such as the underlying components that connect the Tensor with its underlying Storage that owns the raw memory, enabling zero-copy views via stride/offset manipulation, etc. A layer deeper we can dive into the C++ which code is organized into Python bindings, frontend engine and JIT, core kernels, and the core pytorch library components. Definitely relevant to check it out and dive deeper into the codebase, who knows perhaps this can even lead to a first pull request if you stumble across something relevant.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #333

Sun, 04 May 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

OpenAI’s Rouge AI Model
The ML Leaderboard Illusion
When ChatGPT broke NLP
Alibaba’s Qwen3 Released
Preparing for Engineering Leadership
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

OpenAI’s Rouge AI Model

OpenAI just reminded all of us the importance of machine learning testing and monitoring; also this is the most we’ve ever heard the word “sycophanthy” (aka people pleasing AI): The April 2025 update to GPT‑4 was a disaster, unintentionally making this the most unhelpful model so far, due to the over-prioritisation of user approval instead of correctness, resulting in unhelpful behaviour across tasks. This is a particularly interesting situation as it makes it clear the type of challenges that as practitioners we face with the monitoring and evaluation of machine learning models; despite potentially positive results on unit tests, and A/B tests, models can face drift from training environments that requires not just single-inference testing but also “integration tests” that cover end-to-end flows. OpenAI released a really interesting (pseudo-)post-mortem for this incident providing more context on the context behind the issues, but certainly there’s some clear opportunities for the MLOps community to take notes on the importance of end to end testing flows that reflect the real world, such that it is not only limited in units but that evaluate the end to end.

The ML Leaderboard Illusion

Every week we see a new model that breaks through the glass ceiling of “The Leaderboard”, however Stanford, Princeton, MIT et al have released an insightful study that debunks the reliability of model rankings on public platforms: There has been a growing illusion on leaderboards in ML through ranking platforms such as Chatbot Arena which seem do not take key aspects into consideration such as model deprecation, sparse comparison graphs, and inconsistent task distributions lead, which has led to clear distorted leaderboard results. It is interesting to see this research initiative from some of the most renowned organisations which explain that the Bradley-Terry model’s assumptions about consistent matchups are compromised when models are deprecated or removed, resulting in misleading rankings. This comes with several recommendations to improve fairness on these leaderboards, which seem quite reasonable and it is great to see a way forward to ensure we can re-focus the community in the right direction.

When ChatGPT broke NLP

Anyone working in NLP throughout the recent AI wave will be aware of the impact LLMs have had in the field: Large language models have had a transformative impact in the field of Natural Language Processing, which ironically surfaced with early skepticism surrounding early transformer models ahead of the disruption caused by GPT-3 and ChatGPT. This is a great deep dive into the rapid acceleration of research, shifting the focus from linguistic theory to scalable model development, resulting in a crisis of relevance faced by researchers as LLMs rendered many existing tasks less relevant. This is a great deep dive that delves into the divide within the NLP community between proponents of LLM capabilities and those concerned about their limitations, ethics, and societal consequences - both certainly debating relevant and important topics in the space.

Alibaba’s Qwen3 Released

China comes back with another open source AI model release with Alibaba’s Qwen3: This is another insightful release in the AI race, providing (the usual) blend of dense + MoE architectures, and with high performance in tasks like coding, math, and general problem-solving. The great trend in open models benefits the community as a whole as it supports the community to push towards the next level of efficiency and performance, as well as the techniques required for hybrid reasoning or fast responses depending on task complexity. It is interesting to see how this field is developing at pace with only further acceleration across the field.

Preparing for Engineering Leadership

It is hard to make the leap from individual contributor to people leader in the software space - here’s a few tips to prepare: Leadership is about people, it’s not just about technical skills, which means that it’s important to focus on developing soft skills - some relevant steps you can take forward in your day-to-day as an IC are: 1. Start Mentoring: Offer guidance to junior developers, which builds leadership by improving your communication, feedback, and problem-solving abilities. 2. Run Team Meetings: Leading discussions or demos improves confidence and organizational skills, essential for tech leads and managers. 3. Track Impact: Shift focus from just coding to demonstrating how your work contributes to team success, showcasing leadership potential. This is a great article for more junior aspiring leaders that want to take the next step forward to transition into leadership roles.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #332

Sun, 27 Apr 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

The GenAI Ecosystem in 2025
AI Code Generation for Research
Forecasting Principles & Practice
META AI Multi-Modal Architecture
Google Gemma for Consumer GPUs
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The GenAI Ecosystem in 2025

The growth of GenAI & ML has brought new emerging challenges; in this talk we dive into the state of production of GenAI & ML in the cloud native ecosystem, where we provide an overview of trends, challenges, opportunities and tooling that the ecosystem is standardizing towards. As part of this session, provide a snapshot of the current state of the ecosystem, as uncovered by recent surveys, which highlights the gaps in tooling and skills. We will then cover the best practices and tooling that are arising from production use-cases of LLMOps/MLOps at scale to tackle domain-specific challenges such as agentic-workflows, AI guardrails, efficiency requirements - between others. These include the OSS frameworks that are supporting the end-to-end LLMOps / MLOps lifecycle across pipelining, optimization, productionisation, monitoring/observability and ML safety. Check it out!

AI Code Generation for Research

Automatically generating code from a research paper sounds like fiction (and probably still is), but this is a really interesting initiative that takes a stab at building a specialised model: This Paper2Code framework automates generation of code from machine learning research papers through a multi-stage pipeline that includes planning, analysis, and coding phases. It is interesting to see the reasoning steps organising details of the paper and then assessing implementation needs to enable the generation. This also comes with some interesting benchmark datasets that can support further research on this space, and seems there are opportunities to improve as already outperforming other competing models such as ChatDev and MetaGPT.

Forecasting Principles & Practice

Forecasting is one of the most popular applications of machine learning in 2025, and building a career around it a great opportunity - this free online book from some of the top researchers in the field is the one resource to get started and/or take your knowledge to the next level. Forecasting: Principles & Practice (the Pythonic way) brings together hands-on reproducible Python code and real-world examples covering fundamental to advanced forecasting techniques such as ARIMA, exponential smoothing, neural networks, and transformer-based models. This is easily one of the best resources out there for anyone from professionals to students as it leverages some of the most popular OSS tools in forecasting like Nixtla, and has some practical exercises, datasets, and case studies to demonstrate concepts. This is definitely a resource worth investing some time to at the very least skim through.

META AI releases a transformer architecture to rule them all, aiming to capture complex modalities in a single architecture including image, text, audio, depth, thermal, and IMU data: ImageBind is a multimodal AI architecture / model that creates a unified embedding space by aligning six sensory modalities without needing paired data for every possible combination, which can be widely applicable (particularly in robotics). It leverages images as a bridge and enables tasks such as cross-modal retrieval, audio-to-image generation, and zero-shot recognition across different modalities. It is interesting to see this approach more repeatedly across research and initiatives as this opens up new possibilities for multimodal applications across content creation, immersive virtual experiences, and multimodal search.

Google Gemma for Consumer GPUs

Google Deepmind keeps innovating with their Open Weights family of Gemma 3 QAT Models, now jumping into the efficiency wagon providing models that run more easily on consumer-grade GPUs: The new models are optimized with Quantization-Aware Training which drastically reduces memory requirements without hindering performance making models like Gemma 3 27B runnable on GPUs like the NVIDIA RTX 3090. For example, Gemma 3 27B’s VRAM requirement drops from 54 GB to just 14.1 GB with int4 quantization, and also provides promising pathways to unlock further optimizations that can unlock execution in more hardware-limited devices such as mobile phones.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin
SRE Con EMEA 2025 - 7th Oct @ Dublin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #331

Sun, 20 Apr 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Open Source $8 Trillion Economic Value
Google Gemini, Wisk & Veo2
DeepSeek’s Distributed File System
TikTok’s LLM Reinforcement Learning
Lessons from the Best Programmers
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Open Source $8 Trillion Economic Value

Harvard researchers found that the value of open source globally is over $8.8 trillion dollars and growing at massive speed: This is obviously partially a result of its widespread usage unsurprisingly across industry’s most critical codebases, with OSS embedded in 96% of all codebases worldwide. This study really reminds us of the importance of OSS across society, where companies would have to spend approximately 3.5 times more on software if OSS did not exist. The study breaks down the value of OSS into two aspects: the cost to develop OSS and demand-side cost to replace OSS if it were to disappear, and shows reveals that only 5% of developers responsible for 93% of the supply-side value. This is an important reminder for policymakers to support OSS development considering the growing critical role in the global economy.

Google Gemini, Wisk & Veo2

Google DeepMind has been opening access to mindblowing text-to-video features with the release of Veo 2: Gemini is now allowing users to describe scenes to create cinematic-quality video clips at 720p resolution in MP4 format with mind blowing outputs. This model leverages advanced understanding of real-world physics and human motion with features like Whisk Animate to convert images into eight-second videos. These type of features are improving at lightning speed and are clearly going to have a large impact on creative industries for the years to come, so it will be critical to also ensure best practice and responsible AI standards are in place across.

DeepSeek’s Distributed File System

An interesting introspective deep dive into China’s DeepSeek Fire-Flyer File System, an open-source distributed filesystem designed to provide scalable and fault-tolerant storage for LLM usage: This system abstracts the complexities of distributed data storage and allowing users to interact with it as if it were a local filesystem, including metadata management, Mgmtd for cluster management, Storage for physical data storage, and Client for file operations. The system utilizes the CRAQ (Chain Replication with Apportioned Queries) protocol to ensure data consistency, though this can introduce write latency due to its sequential nature. It is interesting to see how the tech race is also leading to the release of not only LLMs but also the infrastructure that powers them, which is great for ML practitioners across the board.

TikTok’s LLM Reinforcement Learning

TikTok releases their reinforcement learning framework to enhance LLMs for a range of usecases such as code interpreter interpreters, problem-solving, structured reasoning and more: It is quite exciting to see large tech companies opening the black box on key knowledge beyond network architectures that allows for the high quality reasoning capabilities that we see across models as of today, such as invoking tooks based on feedback, triggering efficient retrieval mechanisms, etc. ReTool trains models to autonomously refine their tool usage strategies through a combination of cold-start data generation and RL training, claiming to outperform text-based RL models in both accuracy and efficiency. Definitely an interesting space to keep an eye on as it progresses given the attention and active contributions from tech giants across the board.

Lessons from the Best Programmers

Key one-liner lessons from the best programmers: 1) Read the Reference; 2) Know Your Tools Really Well; 3) Read The Error Message; 4) Break Down Problems; 5) Don’t Be Afraid To Get Your Hands Dirty With Code; 6) Always Help Others; 7) Write; 8) Never Stop Learning; 9) Never Stop Learning; 10) Build a Reputation; 11) Have Patience; 12) Never Blame the Computer; 13) Don’t Be Afraid to Say “I Don’t Know”; 14) Don’t Guess; 15) Keep It Simple. The is a great article as it provides us practitioners a reminder on the time-tested best practices to master tools and contributions.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

KubeCon AI Day 2025 - 1st April @ London
WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #330

Sun, 13 Apr 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Stanford AI Index Report
Google’s Agent2Agent Protocol
Yann LeCun on Future of LLMs
The “S” in MCP is for Security
DeepMind and GenAI Competition
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Stanford AI Index Report

Stanford University has released the 2025 AI Index Report with an extensive deep dive into advancements, benchmarks, trends, policy and perception of AI & ML: 1) New benchmarks continue to see significant improvements with more robust, efficient deployment pipelines in production environments. 2) AI is moving well beyond experimental settings across autonomous vehicles to FDA-approved AI medical devices. 3) Private AI investments are at record levels with U.S. investment notably outpacing competitors while China is swiftly closing the gap in model quality. 4) Dramatic drop in inference costs (over 280-fold for systems comparable to GPT-3.5) and ongoing reductions in hardware expenses (approximately 30% per year). 5) A clear acceleration in regulatory activities worldwide with practitioners needing to integrate rigorous Responsible AI evaluations and governance frameworks. There are a large number of great insights on this year’s Stanford AI Index Report, make sure to check it out!

Google’s Agent2Agent Protocol

Google jumps into the MCP train with a new Agent2Agent protocol drafter together with Cohere, Intuit, Box, DataRobot and dozens of tech giants and organisations. It is interesting to see the rise of LLM-based protocols to enable an open standard for standardised communication among heterogeneous AI agents in production environments. Similar to MCP it is built on top of HTTP / JSON but aims to provide a higher level framework for enabling autonomous agents to collaborate on complex, long-running tasks, enabling: 1) Discoverability through “Agent Cards” and respective agent metadata. 2) Standardized task management, updates, HITL interactions, etc. 3) This time it suggests focus on security explicitly. 4) Accomodating various modalities (audio, video, forms) beyond just text. This is still a nascent field which is reflect by the open gaps and challenges, however these also drive opportunities for innovation such as these initiatives - hopefully we just don’t end up with dozens of protocols that aim to standardise all other protocols!

Yann LeCun on Future of LLMs

Yann LeCun argues that Auto-Regressive LLMs are Doomed, and it’s hard to disagree: That the current paradigm of auto-regressive large language models has some fundamental (+ well known) flaws for achieving human-level AI despite their impressive performance. As we know these models try to predict each token, which in practice can lead to an exponential buildup of errors, making them unsuitable for tasks that require reliable long-horizon planning and reasoning (this is what we often see on “hallucinations”). Although this is widely known and accepted, we indeed have not yet found a specific alternative that shows as the next-generation path, but it seems that we may soon see some step-change innovation through different architectures.

The S in MCP is for Security

The “S” in MCP Stands for Security. Spoiler; it doesn’t, but it should: The need to integrate tools and services into LLM agents has led to the creation of protocols like MCP which has aimed to provide a standardised interface to make adoption easier, however this comes with clear security shortcomings that production ML teams must address. 1) MCP implementations lack essential security features like authentication, context encryption, and tool integrity verification. This creates risks when agents connect to unverified or arbitrary servers. 2) Several attack vectors are outlined, including command injection (unsafe shell calls), tool poisoning attacks (hidden malicious instructions), silent redefinition (tools changing behaviour), between many others. 3) Developers should prioritize basic security best practices such as input validation, version pinning, and sanitizing tool metadata, but also at platform and infra level. Overall it is great to see some progress on standardising interfaces for interoperability, however it will be important to see an investment in security by design on upcoming protocols to ensure reliable agentic operations. This is a great article that dives into practical examples and code snippets for each of these.

DeepMind and GenAI Competition

It seems that Google DeepMind may actually be winning on most GenAI fronts: This is a great analysis that reflects on the highs and lows of Google throughout the current AI race, namely on how they initially missed the train but now are catching up and beating the competition across various fronts. Although this seems to make a strong case for Google, it seems to echo the insights we’ve seen in Stanford’s Annual AI Report, namely that the gap between OpenAI and the competition (incl. China) has narrowed down. Having said that, there has been an impressive leap in progress from Google DeepMind on their recent releases such as Gemini 2.5 Pro blowing away benchmarks, showcasing impressive cost/energy/model efficiency, and seamless integrations to the google ecosystem. One thing is clear, that there are surprises at every corner and what may seem as huge moat may actually not be as defensible as expected, so it certainly continues to be an exciting space to keep an eye on.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

KubeCon AI Day 2025 - 1st April @ London
WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #329

Sun, 06 Apr 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Survey on Top ML Challenges
Understanding ML Theory to Algos
The 13 Software Laws to Live By
META Released LLaMa 4
Reinforcement Learning From Scratch
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Survey on Top ML Challenges

Thrilled that our survey is featured in TheNewStack which dives into the state of production ML and uncovers key insights across challenges, tech stacks, trends and demographics. There are some actionable insights for practitioners, such as the challenges in Prod ML on observability and monitoring as well as the operational challenges required as applications scale for robust Day 1 and Day 2 practices. We continue to see a lot of key trends in MLOps such as favoring custom-built solutions over vendor tools across their tech stacks, as well as products such as MLflow leading in model tracking and Airflow in workflow orchestration. Check it out to get a refresher on the state of prod ML!

Understanding ML Theory to Algos

This is a fantastic (free) 450 page book on machine learning theory and practice covering the full foundation of the domain: This is a great deep dive into foundational topics such as bias-variance tradeoffs, VC-dimension, PAC learning, but also extending to core concepts such as convex optimization, generalization bounds and much more. Whether you are a seasoned practitioner or an interested enthusiast, this is a great resource to dive into the algorithmic paradigms that set the backbone of the field such as stochastic gradient descent, boosting, support vector machines, and kernel methods, while also covering essential topics like model selection, regularization, and validation techniques.

The 13 Software Laws to Live By

The 13 laws of tech come up more often that you may think, so it’s definitely worth a quick refresher: 1. Parkinson’s law: Work expands to fill the available time. 2. Hofstadter’s Law: It always takes longer than you expect. 3. Brooks’ law: Adding manpower to a late software project makes it later. 4. Conway’s law: Organizations produce designs which are copies of the communication structures of these organizations. 5. Cunningham’s law: The best way to get the right answer on the internet is to post the wrong answer. 6. Sturgeon’s Law: 90% of everything is crap. 7. Zawinski’s Law: Programs which cannot expand are replaced by ones that can. 8. Hyrum’s Law: With a sufficient number of users of an API, it does not matter what you promise in the contract. 9. Price’s law: 50% of the work is done by the square root number of people. 10. Ringelmann effect: The tendency for individual members of a group to become increasingly less productive as the size of their group increases. 11. Goodhart’s law: When a measure becomes a target, it ceases to be a good measure. 12. Gilb’s law: Anything you need to quantify can be measured in some way that is superior to not measuring it at all. 13. Murphy’s Law: Anything that can go wrong will go wrong. This is a great compilation of the laws of software, as they truly do appear more often than one would like on a day-to-day basis.

META Released LLaMa 4

Meta has released Llama 4! It is quite exciting to see the continuous contribution to the ML community, particularly with the increasing competition (e.g. China). This release comes with two 17B-parameter models which leveraging a mixture-of-experts architecture: Llama 4 Scout has a 16-MoE architecture, and fits on a single NVIDIA H100 GPU (efficiency seems to be a growing trend) and supports a huge 10M token context window. Llama 4 Maverick has a 128-MoE architecture which claims to beat ChatGPT 4.5 across reasoning, coding, and visual benchmarks, however we don’t see comparisons to recent chinese models such as Tencent’s and DeepSeek’s models. It is also interesting to see the adoption of safety and bias mitigation strategies, it will be interesting to see what the community is able to build from these.

Reinforcement Learning From Scratch

What better way to dive into the field of Reinforcement Learning than by diving into the internals of some of the core foundational concepts in the field: This is a great resource which puts together approachable tutorials across reinforcement learning by building core components from scratch in Python. This is targeted for ML practitioners, but it seems it can be approachable by anyone that is interested to learn more about this important field (which is also powering some of the most innovative GenAI models). The repo is setup in detailed Jupyter notebooks covering everything from basic exploration and tabular methods (like Q-Learning and SARSA) to advanced techniques (such as PPO, DDPG, and multi-agent algorithms).

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

KubeCon AI Day 2025 - 1st April @ London
WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #328

Sun, 30 Mar 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Anthropic on LLM “Biology”
Architectural Patterns in Python
Tencent’s New Ultra-Large Model
Improving RecSys in LLM Age
OpenAI’s Image Gen Models
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

If you’re looking for an interesting career opportunity, I’m hiring for a few roles including Data Science Manager (Forecasting), as well as Principal Product Manager (Forecasting) - check them out and please do share with your network!

Anthropic on LLM “Biology”

Anthropic’s new paper on the “Biology” of a Large Language Model has some really interesting takes on the inner workings of large models when performing complex tasks: This is a great deep dive to demystify the complex internal behaviour across multi-step reasoning, forward/backward planning and interventions. 1) The model internally executes sequential reasoning steps (e.g., deducing “Dallas” → “Texas” → “Austin”) rather than relying solely on memorized patterns. 2) In tasks like poem generation, the model pre-selects candidate rhyming words and then structures the line to fit these plans. 3) Various circuits within the models themselves contribute to the final output which can replace opaque neurons with interpretable features via a cross-layer transcoder. 4) Using intervention experiments can confirm that altering specific circuit components (e.g., swapping “Texas” for “California”) results in predictable changes in outputs, offering a practical path toward debugging.

For production ML practitioners it is interesting to see some conceptual frameworks that allows reasoning and discourse as these are still emerging.

Architectural Patterns in Python

A brand new (+ free!) O’Reilly Book on Architecture Patterns with Python - check it out: This is a great resource for practitioners that are interested to take their knowledge to the next level through practical architectural patterns that are used growingly across real-world codebases. These include patterns like domain modeling, the repository pattern, and unit of work, as well as event-driven architectures like including message buses, command handling, and CQRS. Overall, while the examples stem from an e-commerce context, these principles are directly transferable to structuring and deploying machine learning systems that need to evolve gracefully with business needs.

Tencent’s New Ultra-Large Model

Another Chinese “Giant Model” enters the arena with Tencent’s Hunyuan-T1, an ultra-large that beats the DeepSeek-R1 models and ChatGPT 4.5 models on a clear race to the top. This model is quite interesting as it would be the largest Mamba MoE architecture, which leverages advanced long-text capture and sequence processing to minimize context loss and reduce compute usage. Interestingly enough over 96% of its post-training is dedicated to reinforcement learning using classic RL techniques like data replay and periodic policy resetting. The model achieves competitive scores on industry benchmarks - namely 87.2 on MMLU-PRO and strong results in mathematics, coding, and logical reasoning tasks.

Improving RecSys in LLM Age

Eugene Yan has put together a great compendium of recommender systems in the age of large language models, with interesting insights on how breakthroughs in the field are leading to major leaps in this space: It is quite interesting to see that the RecSys & search space is now being tangibly improved from research that arises on the LLM space, working together with traditional ID-based and behavioral recommender systems, and integrating through hybrid architectures that utilize dense, multimodal content embeddings to address challenges like cold-start and long-tail items. It is interesting to see key areas as well on LLM-assisted data generation and analysis such as synthetic metadata creation to refined query segmentation, enhancing training data, as well as innovative training paradigms and inference best practices.

OpenAI’s Image Gen Models

The internet has been increasingly discussing and showcasing OpenAI’s latest GPT‑4o image updates, it is interesting to see some of the details on the internals from these techniques, particularly the evolution on the architectures themselves. This includes integrating multimodal capabilities directly into language models themselves which allows for precise and context-aware rendering of both text and images, and in this case also enabling for nuanced context aware image generation. Although the models are closed source, from the description we can see that these are built on a unified autoregressive transformers paired with diffusion-based decoding which is quite interesting as this is enabling for reasoning at the image level whilst enabling for multi-modal interfaces. We can only bet that we will see in the coming weeks and months many more innovations in the space of image models from players around the world.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

KubeCon AI Day 2025 - 1st April @ London
WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #327

Sun, 23 Mar 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Tencent’s Image-to-3D AI
Netflix RecSys Foundation Model
Uber’s Text-to-SQL AI Systems
LLMs as Better Meme Generators
Top 100 Most Watched Python Talks
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Tencent’s Image-to-3D AI

China’s Tech Giant Tencent has open sourced their Foundation AI Model for 3D asset generation which converts text or images into usable assets for game engines and 3D animation: Hunyuan3D 2.0 is Tencent’s latest open-source system designed to generate high-resolution, textured 3D assets using a two-stage flow-based diffusion transformer. This is an exciting piece of research which showcases how multi-modality can extend beyond purely image and text, where they leverage generation of high-fidelity meshes conditioned on input images, and then generation of a dedicated texture synthesis model which produces the detailed texture maps. Tencent also released Hunyuan3D-Studio which is their production platform to enable Sketch-to-3D, low-polygon stylization, and character animation - this is another space being disrupted at lightning speed.

Netflix RecSys Foundation Model

Netflix has built a unified recommendation foundation model to replace its many specialized systems across their large-scale personalization and recommenders systems internally which drive 80% of the content discovery in their platform. Netflix has been able to leverage large-scale high-quality user interaction data together with advanced tokenization techniques (with inspiration from LLMs) to train a foundation model that captures long-term user behavior over extensive interaction histories. In their overview they also dive into their cold-start problem, and how they tackle with with a hybrid embedding strategy which combines learnable ID and metadata-based embeddings via an attention mechanism to better handle newly launched titles. It is interesting to see that foundation models are now seeing practical applications in growing number of industry applications.

Uber’s Text-to-SQL AI Systems

Uber’s analytics platform powers more than 1.2 million interactive queries per month, many which are streamlined with production-grade text-to-SQL systems under their platform QueryGPT: Organisations are scrambling to enable GenAI productivity beyond software development, and Uber showcases how they built and expanded their text-to-SQL platform that powers analytics through a simple RAG pipeline that fetch relevant SQL samples and schemas, which have now evolved significantly to improve accuracy. They have a multi-agent architecture with quite a lot of focus on their evaluation and performance optimization to reach the scale requirements and SLOs required for analytics usage.

LLMs as Better Meme Generators

Are LLMs funnier than Humans? Researchers from Sweden & Germany ran a turing-test for Internet Memes, and it seems the machines have passed: An interesting paper that explores how LLMs can function as co-creative partners in generating humorous content (ie internet memes) compared to traditional human-only or fully automated processes. In a controlled user study, participants either generated meme captions independently, in collaboration with an LLM via a chat interface, or relied solely on AI. For the first outcome, results show that while LLM-assisted users produced a significantly higher number of ideas with reduced perceived effort, the overall quality (measured by humor, creativity, and shareability) remained similar to human-only outputs. However for the second outcome, memes generated entirely by AI scored higher on average, suggesting that LLMs, trained on vast and diverse datasets can have high potential at producing content with quite broad appeal.

Top 100 Most Watched Python Talks

What a better way to continuously improve knowledge than by diving into some of the Top 100 most watched Python talks: This is a fantastic resource to come back to for relevant talks to dive into either existing or new content, with some of the main topics diving into scalable ML systems, performance optimization, data engineering, and beyond - check it out!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

KubeCon AI Day 2025 - 1st April @ London
WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #326

Sun, 16 Mar 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Google releases Gemma 3
Block Diffusion Challenging LLMs
Mistakes in Large Codebases
Probabilistic Artificial Intelligence
The Startup CTO’s Handbook
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Google releases Gemma 3

Google released Gemma 3 open source - although it’s not yet at DeepSeek-R1 performance-level, the model can run on a single GPU which is impressive on the efficiency side: Google’s Gemma 3 was released in sizes from 1B to 27B and was built to run on a single GPU, and delivers cutting-edge text, image, and code capabilities. It is great to see open weight models that bring advanced multimodal reasoning, and with a hefty 128k-token context window as well as support for over 35 languages (pretrained for 140). It is also interesting to see how responsible AI fits into these releases in practice, Google showcased safety protocols for image safety as well as different testing mechanisms - this space is evolving on a weekly basis.

Block Diffusion Challenging LLMs

Block diffusion is the new kid on the block! It is quite exciting to see that LLMs may finally have reached their parallelization moment: The BD3-LMs introduce a block diffusion framework that brings together autoregressive and diffusion techniques to generate variable-length and long-form text up to ten times longer than prior diffusion models. This is quite exciting research as it basically allows for text generation blocks to be created in a rolling-window basis, which addresses quite a lot of shortcomings from both traditional LLM models as well as previous diffusion approaches - namely enabling for arbitrary length, key-value caching and most importantly enabling parallelization. If new models like this pick up in real-world use-cases, it could have quite a significant improvement on performance and potentially even scale.

Mistakes in Large Codebases

For production ML practitioners it is often important to develop best practices particularly when contributing to large codebases - these are the most common mistakes to avoid: Rather than reinventing the wheel it is very important to align with established patterns to avoid unexpected pitfalls even if your own approach appears cleaner. It is important to thoroughly research how similar functionality like authentication or critical service calls are handled, and mimicking those patterns. It is also important to have an understanding of the hot paths, and ensure cautious testing with a focus on critical scenarios, whilst having a careful approach to adding dependencies or removing legacy code. This is a great article that compiles quite a few good pieces of advice, certainly recommended read for anyone that works in team codebases on a recurrent basis.

Probabilistic Artificial Intelligence

Probabilistic Artificial Intelligence: This is a fantastic university-level resource to polish your foundations on probability and take it to the next level with applied ML models that can be used for autonomous decision making. This 400+ page resource is a great resource for any ML practitioner that are looking to build a deep understanding on uncertainty and probability in machine learning systems. It provides practical resources on probabilistic inference methods such as Bayesian linear regression, Gaussian processes, and Bayesian neural networks, and bridges probabilistic modeling with sequential decision-making techniques such as Bayesian optimization, deep-/model-based reinforcement learning and beyond.

The Startup CTO’s Handbook

The Startup CTO’s Handbook is a great and pragmatic guide for tech leaders transitioning from coding to management in fast-paced (startup) environments: This is now becoming more of a classic (and free) resource that brings together practical frameworks for team building, decision-making for leaders, and techniques for managing technical debt with strategies for effective people leadership.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

KubeCon AI Day 2025 - 1st April @ London
WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #325

Sun, 09 Mar 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Karpathy’s Tips on LLMs
China’s New Autonomous AI Agent
Reading Papers for Engineers
MIT Distributed Systems Course
Difussion Training in Micro-Budget
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Karpathy’s Tips on LLMs

Andrej Karpathy has dropped a two-hour walkthrough on how he uses LLMs for day-to-day activities and this is a masterclass on productivity: Karpathy dives into all the features available in today’s LLM applications (across Grok/Gemini/ChatGPT), including core prompt engienering tips, nuances between models and really solid insights on some of the addons and external applicaitions. This also covers some of the more advanced / experimental features like deep research, intenet search, code interpreters, and how each service compares to each other. It is quite interesting to see this analysis from quite a pragmatic perspective, as well as compare some of the workflows that I have in place vs tools available - there is no doubt that there is quite a lot of potential, but a lot of these capabilities are still in their nascent state with a lot more opportunity ahead!

China’s New Autonomous AI Agent

China innovation keep surprising every week, this time with a new autonomous AI agent that can basically control your phone to deliver complex UI-driven tasks: Manus AI is a new multi-agent system which promises to deliver a task assistant that delivers complex UI tasks through several specialized models that have access to asynchronous cloud execution. It is quite interesting to see some pretty intuitive live demos of this framework performing automated resume screening, market research, data analysis, and even web deployment. Looking at the growing activity in this space it is clear that multi-agent services will only continue to blow our minds with ingenious releases this year.

Reading Papers for Engineers

A software engineer’s guide to reading research papers: For any software and ML practitioners reading not just ML but also computer science papers can help drive your career to the next level! The approach to read research papers is always to start with a quick skim read focusing on the abstract, intro, results, and conclusion to determine if the paper is relevant. The next step is to do a deeper dive by identifying flag unfamiliar terms, and capturing key references to build the relevant context to really understand what is being presented. Finally we read it a final time to connect the dots and clarify challenging concepts that may have been abstract on the first read. This is a great premier on research papers, and definitely recommended for any practitioners in the software or ML space!

MIT Distributed Systems Course

MIT has released one of the best university-level courses on Distributed Systems, diving into the tech giant’s top large-scale systems, as well as diving into foundational concepts in that are relevant even for ML systems as well: This free MIT course on distributed systems dives into the building blocks behind scalable fault-tolerant infrastructures, covering everything from the fundamentals (RPC, threading, etc) to advanced topics like consensus algorithms (Raft), distributed transactions, and optimistic concurrency control. There’s often recommendations to read top compsci papers on distributed systems, but this course actually dives into some of them, including the Google File System (GFS), Zookeeper, Aurora, Spanner, and even exploring cache consistency in environments like Facebook’s Memcached. This is probably one of the best free courses on distributed systems out there, definitely recommend checking it out.

Difussion Training in Micro-Budget

Every day we see ML models are growing higher and requiring more compute, however there are similar innovations to reduce the cost and compute required; this repo covers how to train a diffusion model in a micro-budget: This project introduces a clever and cost-effective strategy for training large-scale diffusion models by using deferred patch masking, which basically starts with low-resolution images where 75% of patches are masked to save compute, then gradually unmasking and fine-tuning at higher resolutions. This approach lets us train a 1.16-billion-parameter sparse transformer on only 37 million real and synthetic images for just $1,890 while still achieving state-of-the-art competitive results. This may still sound like a sizeable amount however it actually is actually surprisingly efficient, and at this pace we will end up being able to train these type of models in commodity day-to-day hardware like our own personal computers.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #324

Sun, 02 Mar 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

WanX Text-to-Video Model
The State of ML Competitions
Chip Huyen Beyond Agent Hype
CMU Database Systems Course
TikTok’s E2E LLM Stack OSS
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

WanX Text-to-Video Model

China strikes again with the new Foundation Model from Alibaba that rivals the tech giants in the West, this time with a text-to-video model; and DeepSeek-style it is open source! WanX 2.1 series have been released as an open-source video generation model from Alibaba Group supporting Multi-Modal Video Generation, chunk-wise processing and a feature cache mechanism, FSDP-based model sharding + 2D Context Parallelism and Multi-Lingual Generation. This model also shows a pretty mind-blowing design as it was trained in quite a highly efficient context, which are enabling general-purpose GPUs to support them, opening opportunitities for really interesting on-machine app opportunities!

The State of ML Competitions

What is happening in ML competitions? This analysis across 400+ ML competitions shows upwards trends with $22M in prize money, Kaggle growing as the leading platform, emerging libraries like Polars taking the world by storm, and Pytorch solidifying as the deep learning framework of choice: It seems that organisations are realising the opportunity and potential of AI Kaggle competitions to spark innovation, and these are also sparking insightful trends in the ML ecosystem. We are seeing that classical methods like gradient-boosted trees keep beating other methods in tabular and time series; convolutional NNs keep leading the way in vision tasks; NVIDIA GPUs (eg A100/H100) keep leading the way in compute, and; techniques like quantisation and synthetic data generation are clearly mechanisms that enable competitors take the prize often.

Chip Huyen Beyond Agent Hype

Chip Huyen comes back once again with a fantastic take on LLM agentic systems beyond the hype: Chip has been one of the leaders defining the conceptual framework to reason about Agentic Systems, and this keynote is yet another great resource to build further intuition on the fast-changing ecosystem of agentic systems as these are plugged in further and further into broader APIs and services. This is quite a pragmatic overview which talks about some of the best practices to consider when building agentic systems, as well as the potential - i.e. effective context management, balancing short-term and long-term memory, etc.

CMU Database Systems Course

Carnegie Mellon University has just released an online course that teaches you step-by-step how to build your own database from scratch in C++: If you are looking to build a robust knowledge on foundational concepts in computer science, this is definitely a great resource as it teaches the key concepts indatabase management systems. CMU has put together an database from scratch that enables students to interactively build and extend the system with core concepts across the end-to-end lifecycle of database management systems. Check it out - definitely encourage diving even into the course projects as they are really practical!

TikTok’s E2E LLM Stack OSS

We are seeing a massive surprise, with China’s TikTok open sourcing a major component of their large-scale LLM agentic system stack: AIBrix is TikTok’s cloud-native infrastructure to orchestrate and optimize large-scale LLM inference through traditional microservice and ML serving systems. It is quite exciting to see Chinese giants making available state-of-the-art LLMOps systems that showcase state-of-the-art learnings such as dual-plane architectures to manage dynamic model metadata, whils also handling the LoRA adapter scaling and multi-node orchestration using Kubernetes and Ray. They dive into quite a lot of details in the documentation on how they also handle key features in the dataplane with an LLM-aware gateway and distributed KV cache to efficiently handle stateful inference requests.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #323

Sun, 23 Feb 2025 00:00:00 GMT

Thank you for being part of over 60,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter for free at /newsletter/ ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

OWASP Agentic AI Security
Sakana AI Making CUDA… Slower?
Karpathy Deep Dive on LLMs
AI Code Increase Tech Debt
Bluesky Lossy Timeline / Ranking
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

OWASP Agentic AI Security

A gem in ML Security has been released last week! The OWASP Top 10 Agentic AI Threats and Mitigations report has put together a threat-model-based framework to secure autonomous AI systems powered by GenAI agents. This has to be one of the most comprehensive threat-model-based frameworks for agentic AI, covering key emerging security risks. It is interesting to see also a taxonomy of these security risks across memory poisoning, tool misuse, privilege escalation, cascading hallucinations, and goal manipulation. We had the honour to contribute to this framework as one of the reviewers, and certainly are looking forward to contributing across the many upcoming iterations!

Sakana AI Making CUDA Slower

Sakana AI launched a GenAI agentic framework that promised optimizations on CUDA kernels up to 10-100x+, however after the community got their hands on the benchmark code mistakes were pointed out showing that in some cases the kernels were 3x slower. Nevertheless it is still an interesting conceptual approach to code-optimization which likely we will only see increasingly prevalent - for good or for bad! This agentic framework basically converts standard PyTorch modules into highly optimized CUDA kernels using evolutionary techniques.

Karpathy Deep Dive on LLMs

Out of nowhere, Andrej Karpathy has launched an absolutely must-watch 3-hour deep dive into the foundations of building LLMs end-to-end - this is really a great reminder of how lucky we are to have access to top quality knowledge in the internet for free! This is a hands on and intuitive deep dive that goes across the end-to-end lifecycle, including pre-training on cleaned text data using techniques like byte pair encoding and Transformer architectures, as well as fine-tuning base models on curated conversational datasets to create robust assistants.

AI Code Increase Tech Debt

AI Code Assistants are making your company’s code worse, and there is now clear data to back this up - this report has analyzed 211 million lines of code across 5 years to find the impact of AI coding: 1) 63% of developers using AI tools, code is being generated faster, yet this leads to more “copy/paste” and duplicated code rather than proper refactoring. 2) The metrics show a clear significant increase in defects as duplicated code blocks become prevalent. In 2024, copy/pasted lines surpassed refactored lines, which correlates with higher bug propagation. 3) Over-reliance on AI-generated code is also impacting modularity and code reuse, and this trend suggests that long-term stability might be compromised if human-led refactoring isn’t prioritized. 4) The Google’s DORA benchmarks are also showing a measurable decrease in delivery stability as AI adoption rises. This is something that we already were aware about, and bad code has existed for decades even before AI came in, however as we now begin to find clear trends it will be important that organisations as well as even code-assistant providers are able to ensure we focus these tools where they can drive the most value, as opposed to worsening conditions through bad practice.

Bluesky Lossy Timeline / Ranking

It is interesting not only to see Bluesky’s meteoric rise, but also their ability to serve to the increasing massive scale in innovative ways that leverage clear learnings from existing players - in this case it is interesting to see their appoach with “lossy timelines”: Lossy Timelines basically was their approach to scaling to the growing number of users, which basically accepts controlled imperfection by probabilistically dropping some timeline updates for users with extremely high follow counts to prevent system overload (hot shards) and reduce latency. They introduce a loss factor which allows them to calibrate this accordingly to ensure efficient ranking algorithms, which shows really interesting improvements in performance in latency and fanout times. These type of innovations are what really showcase the value of elegant solutions to already tested AI systems (i.e. search, ranking, etc).

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #322

Sun, 16 Feb 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Microsoft on GenAI Making Us Dumb
Text to SQL: The Ultimate Guide
Perplexity Deep Research
Anthropic Economic Index
Build Your Own
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Microsoft on GenAI Making Us Dumb

GenAI is making us dumber, and the Microsoft Research team has a survey that dives into it: Whilst generative AI tools can significantly reduce the cognitive effort required for routine tasks, they simultaneously reduce critical thinking demands to verifying and integrating AI outputs. This research initiatives from MSFT survey shows that knowledge workers with high confidence in AI tend to engage less critically, which results in over-reliance on AI and hence a decline in independent problem-solving skills. This also has some interesting lessons for all of us as production ML practitioners, as it means we have to think about the way we design GenAI systems to ensure they do not only automate processes, but also incorporate features like transparent feedback and query steering to encourage embedding of human-in-the-loop.

Text to SQL: The Ultimate Guide

Text-to-SQL GenAI tools are helping organisations accelerate analytics productivity, however the ecosystem is still nascent and hence understanding the best tools and best practice can help speed up adoption rate: There are 4 key considerations when diving into text-to-SQL GenAI tooling, namely: 1) directly prompting large language models with full schema context, 2) leveraging retrieval-augmented generation (RAG) to filter and use only relevant data, 3) deploying multi-agent systems for improved error recovery and query refinement, and 4) fine-tuning customized contextual LLMs for high accuracy and enterprise-grade performance. There is a great article on this which evaluates every method based on cost, latency, accuracy, and data security, which offers us as production ML practitioners some clear guidance.

Perplexity Deep Research

As OpenAI, Gemini and the rest have released a Deep Research Agent, Perplexity follows suit launching their own AI-driven tool that automates in-depth research by iteratively searching, reading, and reasoning through vast amounts of data to produce comprehensive reports: It is interesting to see that the foundational LLM model space is innovating at increasing pace, however the path is quite linear; when one player releases something, the rest follow with the same release, this latest one with the exact same name! Either way, the domain of deep research certainly has potential opportunity for innovation, and it will be interesting to see how products like this will change the current state of science across the board.

Anthropic Economic Index

Anthropic has released an Economic Index based on millions of anonymized Claude LLM conversations mapped to specific work tasks, providing insights into the impact driven by these tools: Anthropic has published their analysis using the O*NET framework via its Clio tool to quantify AI’s impact on the labor market for their tooling. This study finds that AI usage is concentrated in technical fields such as engineering, and is mainly employed to augment rather than automate tasks. Approximately 36% of occupations use AI for at least 25% of their tasks, though only 4% see AI in 75% or more, with mid-to-high wage roles like programmers and data scientists leading the trend.

Build Your Own

What better way to learn something that by building it from scratch! This open source repo has compiled every possible combination of “Build your own ” in every language available: This is a fantastic source of knowledge to expand our hands-on skills through hands-on tutorials that walk you through building various technologies from scratch! This list extends across concepts like building databases from scratch, to building neural networks or visual recognition systems from scratch!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - Oct 8-9 @ Austin

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #321

Sun, 09 Feb 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

The RLHF Book
Music AI Generation & Lawsuits
Opinions from a Decade in Software
Programming Changing as We Know It
New FC Foundation Model from China
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The RLHF Book

Really happy to see a brand new free online book on RLHF! It is not every day we can find comprehensive information about an emerging topic like Reinforcement Learning from Human Feedback: This little online free-book on RLHF offers a concise overview of all-things Reinforcement Learning from Human Feedback (RLHF) for production ML use-cases on its role in post-training language models. It provides quite a comprehensive overview across the multi-step process of training a base model to follow instructions, collecting human preference data to build a reward model, and applying reinforcement learning techniques to fine-tune the model’s behavior. The RLHF book contains a few chunky chapters on 1) framing the problem space, 2) optimization of LLMs (policy gradients, rejection sampling, instruction training, etc), 3) Advanced topics, and 4) Open questions. Check it out!

Music AI Generation & Lawsuits

This weekend I was playing with GenAI Music Generation models, and I was blown away that I was able to create songs that are not only relalistic but that were actually not terrible! In the background of this, there has been some mind-blowing developments in this space, not only with the technology but also with the respective lawsuits. It is worth doing a revisit of the cutting‐edge research that enables music generation such as Google’s MusicLM, Meta’s MusicGen, and many others highlighted in this research survey. Similarly it is interesting to see the copyright lawsuits that are appearing towards music generation services such as Riffussion, Suno and Udio. This is a really exciting area of research, but at the same time it’s a critical phase where industry is fast evolving to fit into the new-world of generative AI systems with copyright and IP, but also with creativity and innovation at high stakes.

Programming Changing as We Know It

The one and only Tim O’Reilly weights in on how software engineering will disappear as we know it - putting into words what we know already: software engineering won’t go anywhere, but it will evolve as it has throughout the last few decades: This is quite a nice reflective piece which revisits a decade of evolving software development practices, and how we can extrapolate towards the impact of LLMs in software development, and more importantly how productionisation of ML will also drive a lot of the changes due to the new challenges that arise different to traditional code. Especially for practitioners like all of us it will be important to stay ahead to future-proof our careers as we are going through what seems to be a major shift and evolution into the field.

New FC Foundation Model from China

China strikes again with another Foundation model that shakes the world with a surprise! This time it is a time-series forecasting Foundation model which challenges the tech giants and their previous large model releases. Often in foundation models for time-series forecasting, the approach taken on their tokenization approach tends to have a significant impact on the performance (for better or for worse), and this model introduces a native continuous patch tokenization and a novel TimeFlow Loss based on flow-matching to generate flexible, probabilistic forecasts without relying on discrete tokenization, which is really interesting compared to other competitors. Of course, we also would not be able to talk about a foundation model, without a massive dataset, and this model carries a 1 trillion+ time-series points of training data, which enables for pretty impressive zero-shot performance on both point and probabilistic forecasting benchmarks vs other models. This is an exciting space to keep an eye for!

Opinions from a Decade in Software

This reflective piece revisits a decade of evolving software development practices. A lot of the lessons really resonate with me, so it is definitely a worth read as it tackles quite a lot of observations that reflect the state of development as of today. Some of the key points challenge the allure of over-engineered solutions and trendy tools, advocates for good-old typed languages, and reminds us that soft skills are as important as hard skills in the programming world. It is quite a short read but definitely worth skimming through, as these are at the very least good reminders of some useful lessons from years of developmnet.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #320

Sun, 02 Feb 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

International AI Safety Report
The DeepSeek Illustrated Guide
AI Engineering Pitfalls with Chip Huyen
CMU ML in Production Course
No-Hype DeepSeek Reading List
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

International AI Safety Report

The first International AI Safety Report was chaired by Yoshua Bengio and developed by a diverse group of 96 AI and an Expert Advisory Panel: Although there are no major surprises in the report, this is a great overview of the concepts, taxonomies and best practices in AI Safety, standardising risks associated with deploying general-purpose AI models. Some of the categorisation of malicious use-cases encompass deepfakes, cyberattacks, and dual-use applications (e.g., bioweapon design), as well as malfunctions such as hallucinations, bias, and loss of control. This is a fantastic resource, and it’s really great to see the steps forward in this space - check it out.

The DeepSeek Illustrated Guide

As the hype for the DeepSeek-R1 continues to grow, we have been able to see few deep dives that go beyond the hype; this is a great visual guide to understand the foundations of this new model. What better way to understand an emerging context than with an illustrated guide which covers DeepSeek-R1’s model architecture, multi-stage process, pre-trained base, fine-tuning approach on 600,000 chain-of-thought examples, and its reinforcement learning approaach R1-Zero.

AI Engineering Pitfalls w Chip Huyen

Chip Huyen dives into a practical roadmap for modern AI engineering on the TWIML AI Podcast! This is a great overview for production ML practitioners with an interest to blend classical ML with generative models. The topics discussed cover relevant areas of AI engineering such as prompt engineering, agent design, open-source models, synthetic data generation, and evolving compute challenges. This is certainly a while podcast to check out to get up to speed with the foundations of AI Engineering.

CMU ML In Production Course

Carnegie Mellon University has just dropped a no-nonsense in-depth course on production machine learning systems, and I have to say it’s one of the most comprehensive resources I’ve seen: It is great to see CMU covering a course syllabus on production ML that covers some of the key pressing topics in the MLOps ecosystem, really diving into even the foundations of both real time and batch ML. The course content and slides really go broad covering best practices designing systems that manage prediction errors, safety, fairness, and scalability, to deploying, testing, and monitoring models using tools like Docker, Apache Kafka, Jenkins, and Kubernetes. It is unfortunate at least on some of the slides it suggests that the course is not recorded, but hopefully the content for this or a similar resource is published - for the time being, the PDF slides and repo do seem comprehensive enough!

No-Hype DeepSeek Reading List

If you want to go beyond the hype of the DeepSeek-R1 new model, this is great reading list that covers some really interesting papers and resources for ML practitioners: This is really quite a great curated list on DeepSeek’s R1 model, covering some of the fundamental research underpinning DeepSeek-R1 including Transformer architectures, Chain-of-Thought reasoning, Mixture of Experts, and Reinforcement Learning in the context of LLMs (RLAIF vs. RLHF, Self-Rewarding LMs, Thinking LLMs, DPO).

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #319

Sun, 26 Jan 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

DeepSeek AI Taking by Storm
Add RecSys Engine to Your Apps
7 Most Influential CompSci Papers
LLMOps in Prod Case Studies
OpenAI’s New Browser “Operator”
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

DeepSeek AI Taking by Storm

DeepSeek AI’s model has taken the world by storm after beating OpenAI’s O1 model with a pretty interesting Reinforcement Learning approach which they cover in their latest paper and it’s quite interesting: The DeepSeek team published an in-depth overview of the innovations that are pushing the state-of-the-art in LLM reasoning models, which primarily introduces an approach that leverages large-scale reinforcement learning. DeepSeek-R1 leverages a multi-stage pipeline mixing RL to optimse for chain-of-thought reasoning that addresses cold-star issues and incorporates additional supervised training for writing and role-play task. This is quite a mindblowing open release, this really makes it clear that open source AI is really bringing the full potential to take over the global ML race.

Add RecSys Engine to Your Apps

The recent innovations in AI are also allowing day-to-day developers to make full use of state-of-the-art capabilities, and this is a great opportunity for anyone to bring these innovations to their existing applications: This is a great deep dive into how you can develop your own state-of-the-art recommender system by integrating vector search into existing database applications using good-old Postgres and the pgvector extension. This does not require re-engineering your entire data stack as you just need to introduce a table for embeddings and building a vector index. Definitely recommend exploring whether your existing applications can benefit from bringing some of these innovations into the mix!

7 Most Influential CompSci Papers

In computer science there has been monumental shifts in knowledge, and there are some research papers that have spearheaded these innovations - these are 7 papers thatlaid the foundation for modern computing: We can’t get enough of the best papers in computer science, as these are also serving as the foundation on many innovations in machine learning - 7 of the papers worth checking out are: 1) Turing’s 1936 work defined what machines can compute, 2) Shannon’s 1948 paper formalized how information travels, 3) Codd’s relational model organized data storage, 4) Cook’s NP-completeness clarified problem complexity, 5) Vint Cerf and Kahn’s TCP/IP enabled global networking, 6) Tim Berners-Lee’s WWW proposal opened the internet to the masses, and 7) Brin/Page’s PageRank revolutionized information retrieval. This is certainly not an exhaustive list, but certainly some papers I had not checked out so definitely worth sharing!

LLMOps in Prod Case Studies

457 real-world examlpes of LLM in production across industry -the ZenML team have done it again! It is great to see a applications and best practice of LLMOps in industry, and the team behind ZenML have brought an extensive piece that cove LLMops use-cases with deep dives across themes such as RAG to ground models in enterprise data, sophisticated prompt engineering and orchestration, and stringent evaluation pipelines as well as human-in-the-loop validation. With all the hype in LLMs it’s good to see a list of use-cases that actually hit production - looking forward to seeing a list that shows the use-cases that are hitting bringing clear return on investments into the businesses!

OpenAI’s New Browser “Operator”

OpenAI brings another innovation that they hope is going to drive similar hype as ChatGPT under “Operator”, which aims to control your browser to perform action-based automations: OpenAI’s new operator framework is a new browser automation agent powered by their new Computer-Using Agent model which combines GPT-4’s vision capabilities with reinforcement learning to interact with websites just as a human would. This basically means an OpenAI agent clicking, typing, and scrolling in your browser - it is interesting to think what are the considerations from a security, data, privacy perspective that will be opened up as users start enabling full access for AI models to perform actions across their workspaces.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #318

Sun, 19 Jan 2025 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

AI Engineer 2025 Reading List
Causal Inference meets Deep Learning
Jensen Huang NVIDIA Keynote
OpenAI’s Economic Blueprint
Papers Every Dev Must Read
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

AI Engineer 2025 Reading List

What better way to start 2025 than with an AI Engineering curated list of all-the-best research papers! This is a great resource bringing together around 50 resources on AI Engineering that span across frontier LLMs, benchmarks/evals, prompting, retrieval-augmented generation, agents, code generation, vision, voice, image/video diffusion, and more. This is quite a comprehensive resource which should keep us busy for a while - if there’s any papers missing do make sure to contribute them upstream!

Causal Inference meets Deep Learning

The intersection between Causal Inference and Deep Learning is A fascinating area of research - this research paper provides a really comprehensive overview on the current state of the ecosystem: This paper provides a great introduction to the foundational concepts in Causal Inference, and then links to literature that explores deep learning applications (e.g., adversarial methods, contrastive learning, reinforcement learning, and diffusion models). This intersection of research has quite a breadth of applications - in industry there’s quite a broad use-case across natural language processing, marketing intelligence, graph representation, and computer vision. There’s also clearly several really interesting challenges that remain to be solved, such as limited interventional data, difficulties in confounder identification, and the need for benchmarks. This is certainly a space to keep a close eye as it develops!

Jensen Huang NVIDIA Keynote

NVIDIA CEO Jensen Huang delivered quite an insightful talk at the GTC 2025, which provided a great overview of the past and the future: This keynote provided a surprisingly great retrospective into the history and breakthroughs of GPUs in graphical applications (first), and then AI workloads. It is interesting to see how ubiquitous the the rise of agentic AI has become, and the push for this field to be the next gold-mine. And of course there was a lot of hype towards robots in general as well, as nvidia delves further into the hardware / robotics space.

OpenAI’s Economic Blueprint

OpenAI proposed an Economic Blueprint as a gold-standard for AI at a national level - it is interesting to see organisations pushing national agendas at this stale: OpenAI pitches for a centralized, federal-led approach to AI regulation and infrastructure, which in USA-terms would replace state-by-state rules on AI, which is proposed to spur innovation and reinforce national security. It is interesting to see that the areas proposed are quite intuitive, such ascoordinated investments in chips, energy, data, and workforce development, alongside clearer “rules of the road” for safe, responsible AI deployments - however with a clear bias towards the technologies that OpenAI depends on.

Papers Every Dev Must Read

Yet another article on “Papers Every Developer Should Read”, however we can’t get enough of these for 2025: This is another great article which brings classic computer science papers together. As always these cover key areas across system design, distributed computing, data storage, metrics, infrastructure, and performance - and more. If you haven’t had a chance to catch up on some of these, now is your chance!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

WeAreDevelopers 2025 - 9th July @ Berlin

Other upcoming MLOps conferences in 2025:

ODSC East - May 13 @ Boston
Data & AI Summit - 9th June @ San Francisco
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI Europe - 08 Oct @ Amsterdam
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #317

Sun, 12 Jan 2025 00:00:00 GMT

How time flies!! We continue to celebrate 6 years since starting this weekly MLE newsletter 🎉🎉🎉 What started with just one commit, today now has almost 70,000 subscribers 🚀 And not a single Sunday missed 🤯 Thank you to everyone for your continued support - in today’s newsletter we share a special edition celebrating our achievements throughout 2023!

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Chip Huyen on Agentic AI
10 Year Review of Anomaly Detection
Modular SVGs with GenAI Models
Efficient Coding with LLMs
Extracting AI models from Apps
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Chip Huyen on Agentic AI

Chip Huyen shares an exerpt from her latest book on AI Engineering, diving specifically into the concept of AI agents: Agentic systems extend the capabilities of an ML model by providing structured tools such as a code executor or search engine, and a planning mechanism to decide which tools to invoke and in what order. The mechanisms to design the system, including the “tools” accessible is quite important; too few tools limit capabilities; too many complicate usage. AI Agents are still emerging in the field, which means that best practice and guidance is being developed - this is a great resource to get the latest literature digested into a single place.

10y Review of Anomaly Detection

Time-series anomaly detection has advanced significantly over the last decade, and these are some key methods, benchmarks and concepts: When looking towards anomaly detection techniques no single method excels universally; current benchmarks and labeling often lack realism or consistency, and evaluation metrics vary widely - there has been some really interesting efforts recently which aim to unify datasets to tackle this such as TSB-UAD and TSB-AD. There has also been a known gap to assess the performance of different methods, for which various measures have been presented such as AUC-based measures, Range-AUC, VUS, etc. There has been continued progress on more robust benchmarks, methods to handle noise and lag, and new techniques that address high-dimensional or irregular time-series data through ensembling, AutoML, and beyond. Check out this survey from various universities which provides a survey of the past decade of time-series anomaly detection.

Modular SVGs with GenAI Models

Creating SVGs is a pain, but as expected a new AI model is making this process extremely simple: NeuralSVG is a text-to-SVG framework that generates clean, layered, and easily editable SVGs from text using a compact MLP. The architecture is quite simple and builds on top of Score Distillation Sampling (SDS) from modern diffusion models, but instead of producing dense, pixel-like vector shapes, it enforces a layered design with a small number of semantically meaningful elements. It is quite interesting to see GenAI reaching further applications and tackling more and more opportunities in the creative space to support domain experts with productivity tooling.

Efficient Coding with LLMs

One of the main opportunities in AI is to support developers as a co-pilot - Tailscale co-founder (+ former Google StaffEng) walks through his learnings adopting LLMs in day-do-day coding: It is interesting to see practitioners adopting LLMs for development in real-world workflows - in this case the primary usecases are 1) autocomplete, 2) search-like tasks, and 3) interactive chat-based code generation. This is still an area which is quite nascent and hence there’s still a lot of new opportunities that will emerge changing the way that we think about it; certainly a space to keep a close eye for developers.

Extracting AI models from Apps

Did you know it’s possible to extract AI models shipped with Android Apps? This is a really interesting article that goes through extracting a TFLite model from an APK: As broader adoption towards on-device AI models increase, security considerations will need to evolve in order to protect these resources, particularly given that often on-device processing is done to protect privacy and security of users. It’s quite surprising how easily it is possible to extracted ML Models from Android apps even when encrypted - here it basically just requires hooking into the model-loading process with dynamic instrumentation tools like “Frida”. It is interesting to see how fast the field of ML Security is evolving, this is clearly becoming one of the most important areas of attention as AI hits production in larger and broader magnitudes.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #316

Sun, 05 Jan 2025 00:00:00 GMT

How time flies!! We are celebrating 6 years since starting this weekly MLE newsletter 🎉🎉🎉 What started with just one commit, today now has almost 70,000 subscribers 🚀 And not a single Sunday missed 🤯 Thank you to everyone for your continued support - in today’s newsletter we share a special edition celebrating our achievements throughout 2023!

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Databases Year in Review 2024
Crunching Data Anywhere with DuckDB
Learnings from LLMs in 2024
Evolution of SRE at Google
The Quiet Chinese AI Giant
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Databases Year in Review 2024

2024 Database year in review: Snowflake vs Databricks! Open Source vs Commercial! DuckDB vs the world! Who is going to win?? The database ecosystem had a lot of interesting developments throughout 2024. Some of the key highlights: 1) disruptive license changes (e.g., Redis and Elasticsearch), 2) the escalating Databricks–Snowflake rivalry, 3) LLMs and all-things RAG, and 4) DuckDB’s rise as a lightweight analytical engine embedded in other systems. This is a great introspective year-in-review for 2024 from CMU, check it out!

Crunching Data Anywhere with DuckDB

DuckDB is the database that even data scientsits want to love: crunching data anywhere, from laptops to servers: DuckDB is an OSS in-process SQL database for intuitive data-workdlows which can be embedded directly into your Python, R, or other jupyter environments. This is a fantastic video from GOTO Conf that deeps dive on some of the designs that are fueling DuckDBs rise to popularity: columnar storage, vectorized execution, and built-in support for common data formats (CSV, Parquet, JSON). DuckDB runs inside your application process which means that it’s as easy to leverage as Pandas for data processing, but enabling for terabyte-size processing. If you haven’t yet heard of DuckDB this is a great resource to dive into it and get an intuition on the potential opportunity in this space!

Learnings from LLMs in 2024

Django Co-creator on things we learned about LLMs in 2024: As always really insightful snippets from Simon Willison, this time doing a 2024 year-in-review on LLMs. Key lessons: 1) The “GPT-4 barrier” was shattered by multiple labs; 2) Multimodal capabilities; 3) Limitations on “agents” with actual autonomy; 4) Rigorous automated evaluations; 5) Apple Intelligence’s disappointments; 5) Concerns on environmental impact; 6) + more.

Evolution of SRE at Google

As organisations grow their Machine Learning capabilities, there are key learnings that we can take from the SRE space: this is a great write-up on how SRE has evolved at Google beyond traditional reliability tools (like SLOs, error budgets, and postmortems) to address the complexity of modern systems, especially AI-driven ones where certain failures (e.g., privacy breaches) cannot be allowed. This actually has quite an interesting overview on adopting “STAMP” (System-Theoretic Accident Model and Processes), which then Google renamed these as control problems which shift the focus to the broader interactions among system components. This is quite an interesting approach which ensure prevention of catastrophic failures - certainly worth taking note for MLOps and ML Engineering practitioners.

The Quiet Chinese AI Giant

Deepseek AI took the world by surprise with a foundation model from China that surpassed all models - this is quite an interesting fire-side chat with the founder Liang Wenfeng (translated from chinese): Deepseek AI is a Chinese AI startup backed by a massive eastern hedge fund, which has quietly gathered what seems to be pretty massive compute resources, and most interestingly non-NVIDIA GPUs, wich possible 50k “Hopper” GPUs. As part of their release they also introduced some innovative approaches to their foundation model, with multi-head latent attention and sparse MoE, which ended up beating OpenAI’s o1 on multiple benchmarks. What is most interesting is that this has now triggered a nationwide price war with ultra-cheap inference costs to enter the global AI race. This is certainly an interesting space as other nations enter the race of foundation models, which are driving further innovation and breakthroughs in the field - certainly a space to continue keeping an eye to.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #315

Sun, 29 Dec 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

AI Agents to Control Computers
Best Software Engineering Papers
Alibaba’s Reasoning LLM Model
Maximum Likelihood ELI5
AMD GPU Inference Optimization
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

AI Agents to Control Computers

Salesforce Research has released an AI vision foundation model that can control user interfaces across mobile, desktop and web 🤖 This is quite an interesting research breakthrough as this new model is able to locating elements in applications and take actions through reasoning workflows. This is quite an interesting two-stage training pipeline which separates the learning of element detection, and then optimize for the multi-step action planning on top of the applications without relying on specialized HTML inputs.

Best Software Engineering Papers

Here are the all-time-best research papers in software engineering for your end of year reading list: This is a great collection of research papers spanning across quite a range of topics, including programming paradigms, distributed systems, networking, cryptography, databases, and more. This list includes some of the early classics from Turing, Dijkstra, and Knuth, which are some of the historical foundations of programming that serve as the backbone of our systems today. Check it out!

Alibaba’s Reasoning LLM Model

Alibaba’s QWEN model has been making the rounds with their surprisingly advanced reasoning foundation model: It is quite interesting to see the fast development of open source / open models, this time with the QwQ model, which brings together state-of-the-art innovations resulting in impressive results across mathematics and coding benchmarks. It seems this race to the top is only accelerating, so it is certainly an exciting space to keep an eye as further more capable open models are released and benchmarked across broader tasks.

Maximum Likelihood ELI5

It is often easy to dismiss some foundational concepts in machine learning; this is a great resource to brush up on fundamentals to build a strong understanding on core concepts such as the intuition behind Maximum Likelihood Estimation. This article does a great job of framing model training as “finding parameters that maximize the probability of observed data” - this makes it intuitive that one sees that minimizing MSE follows from assuming outputs come from a Gaussian distribution, and minimizing Cross Entropy stems from assuming outputs follow a Bernoulli distribution.

An interesting observation in the article is that MLE is equivalent to minimizing the KL divergence between the true data distribution and the model’s distribution. For machine learning practitioners, understanding these foundational concepts can help you make informed choices about which loss functions to use and interpret how modeling assumptions align with real-world data.

AMD GPU Inference Optimization

NVIDIA CUDA has led the way on ML inference by quite a margin, however slowly other alternatives are reaching similar performance in real-world AI applications: This is a great practical optimization benchmark which explores how AMD compares and competes with NVIDIA GPUs on Llama2 inference. This is an interesting approach leveraging GPU compulation to automatically generates optimized GPU kernels for multiple backends by leveraging the Vulkan SDK, which is an abstraction layer that supports 1000s+ of GPUs beyond NVIDIA.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #314

Sun, 22 Dec 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Raschka’s Top LLM Papers in 2024
DeepMind on Agent Cooperation
Google’s Globally Distributed DB
Gentle Intro to Graph Neural Networks
An Evolved Transformer Memory
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Raschka’s Top LLM Papers 2024

Sebastian Raschka’s “best of 2024” on LLM research papers has some great resources: There’s quite a broad selection of research themes, many quite interesting emerging areas such as parameter-efficient fine-tuning (e.g., LoRA, MoE), extending context windows, retrieval-augmented generation (RAG), multi-modal expansions that integrate visual or structured data, refined alignment strategies (like DPO and RLHF) to steer model outputs, knowledge editing for domain-specific modifications, and advanced compression (quantization, pruning). There’s quite a lot of variety in this list, with also quite a lot of papers that delve into techniques for improved inference efficiency, dynamic architectures (state space models, mixture-of-experts), and novel prompting or training paradigms. If anyone is looking for a Christmas read, this is a great resource to catch up on 2024 LLM research over the winter holidays!

DeepMind on Agent Cooperation

DeepMind published an interesting paper exploring how LLM agents might learn to cooperate over multiple “generations” through an indirect reciprocity scenario: They tested three different models (Claude 3.5 Sonnet, Gemini 1.5 Flash, and GPT‑4o), some interesting insights such as showing that only Claude consistently evolved higher levels of cooperation across generations. Surprisingly, GPT‑4o tended to revert to “mutual defection”, and Gemini 1.5 showed only “weak gains”. It seems like there is a growing number of similar studies around LLMs for simulations of specific environments, which provide interesting sandboxes to test human-like interactions between actors at scale.

Google’s Globally Distributed DB

An excellent paper for MLOps and Machine Learning Engineering practitioners is the classic Google’s Spanner paper: Google proposed an interesting globally distributed database architecture which offers strong external consistency at the cost of slight latency increase. The paper dives into how it integrates a two-phase commit with Paxos replication to automatically shard data and ensure high availability with lock-free snapshot reads. This commit wait is what introduces the design trade-off of slightly increased write latency (e.g., “commit wait”), but it is also what delivers synchronous replication and globally consistent reads, which is particularly useful for example in large-scale machine learning and MLOps systems requiring reproducible data snapshots, as well as the ability to evolve data schemas and indexes without downtime.

Gentle Intro to Graph DL

This is one of the best visual/intuitive introductions to Graph Neural Networks out there! As a reminder on GNNs, these graph neural networks are able to generalize deep learning by handling entities as nodes and their relationships as edges. This enables “passing messages” between connected nodes to learn context-aware representations that can power node-, edge-, or graph-level predictions. There are some nuances when working with GNNs vs traditional NNs, such as key design choices on how to represent nodes/edges, how to aggregate local or global information (e.g., via sum, mean, or max pooling), and how many layers to stack for broader context. GNNs are certainly worth learning, as a lot of real-world applications we see these relationships, and we have shown success in varied domains such as social networks, molecule property prediction, recommendation systems, etc.

An Evolved Transformer Memory

OpenAI O1’s impressive performance at high compute ($3k est. per task) highlights that performance is the next tech race - Japanese startup Sakana AI brings a new approach to reduce memory by 75% with improved inference: Sakana AI releases this new “universal transformer memory” technique which uses small neural “attention memory” modules (NAMMs) to dynamically decide which tokens to keep or discard within an LLM’s context window. This sounds like quite a simple technique, but it promises to potentially cut memory usage by up to 75%. This is ceratinly going to be an interesting space to watch throughout 2025 and beyond!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #313

Sun, 15 Dec 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Ilya Sutskever NeurIPS Keynote
Google Releasing Gemini 2.0
Tiktok’s Recommendation System
META FAIR Major Releases
OpenAI Releases Text-to-Video
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Ilya Sutskever NeurIPS Keynote

Ilya Sutskever’s NeurIPS keynote is up - an interesting reflection on the decade since the seminal “Sequence to Sequence” work that helped ignite the modern era of large-scale neural NLP: This is a comprehensive session where Ilya recounts how the original Seq2Seq approach established a template for present-day AI: big models plus big data equals breakthroughs. Over time, this scaling principle was validated far beyond translation, culminating in today’s GPT-style models. However, it seems we are exhausting the “fossil fuel” of internet-scale data, and future progress will hinge on new techniques - eg. agents interacting with their environments, generating synthetic data, and improved reasoning capabilities.

Google Releasing Gemini 2.0

Google drops the mic last week releasing Gemini 2.0 with a bunch of new features on their AI studio, doubling down towards “agentic” AI with multimodal input/output: As expected Google is building on the initial Gemini 1.x foundation, extending long-context and multimodality capabilities, improves latency and performance, and introducing features like native image and audio generation. Something that comes across as novel is the integration into Google’s products and ecosystem, however the race continues to move at breakneck speed so we can only expect similar pace from the tech ecosystem.

Tiktok’s Recommendation System

Tiktok has published the architecture behind their recommendation system called “Monolith”, a real-time massive-scale recommendation system designed specifically to address production challenges such as large-scale, sparse, and dynamic feature spaces: The paper provides interesting insights such as collisionless embedding table based on Cuckoo hashing, which enables dynamic inclusion and eviction of new model features. Monolith tightly integrates training and serving which they highlight as one of the reasons they can allow for fast online updates so that the model can adapt to changing user behavior within minutes.

META FAIR Major Releases

Meta Fundamental AI Research (FAIR) has quietly shared a huge release last week with new open-source AI systems across CLIP, Motivo and Seal: Meta released Motivo, a foundation model that enables embodied humanoid agents to efficiently solve complex tasks without additional training. Meta also released Video Seal, a robust watermarking solution for videos that remains intact through common transformations. They’ve also introduced Flow Matching, a hierarchical byte-level tokenizer-free approach (Dynamic Byte Latent Transformer) for generative modeling. Additionally, they released a new version of Meta CLIP, which improves on previous versions for vision-language alignment. Quite surprising and exciting to see so much movement from META’s research arm furthering the research ecosystem across quite a few of these interesting areas.

OpenAI Releases Text-to-Video

OpenAI has finally released their Text-to-Video SORA model as a public offering! As per the usual naming convention, this comes with Sora Turbo, focusing on fast generation of higher fidelity videos across multiple aspect ratios, and up to 20-second. These services continue to surprise us with the quality of the video generation, certainly still with quite some limitations (such as many posts showing the limits when rendering scenes from gymnastics, etc). The model is still imperfect, struggling with complex sequences and realistic physics - however it is great to see finally OpenAI is releasing to encourage community input and iteration norm-setting, and responsible use.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #312

Sun, 08 Dec 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

The State of Responsible AI
Production LLM Usecase List
Google’s 5-Day GenAI Course
DeepMind World Foundation AI
MSFT Quantifying Bad Days
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

The State of Responsible AI

What are the most pressing issues on ML Governance? What about the most important concepts of Responsible AI? Our keynote on Responsible AI is now live at the Technical University Munich’s Institute for Ethics in AI page 🚀 In this session we navigated the current landscape of responsible AI, focusing on the industrial, organizational, and technical aspects crucial for successful AI deployment, including governance challenges, accountability, security concerns, infrastructure complexities, risk mitigation, and building scalable, reliable AI systems that drive innovation while adhering to responsible practices. Check out the full video and slides!

Production LLM Usecase List

ZenML dropped a massive new database of LLMOps usecases with 300+ curated generative AI and LLM implementations in the real world: It is important for practitioners to go beyond the hype when it comes to LLMs, and concrete / practical examples in industry can provide an actionable guidance - this includes architectural choices, tooling stacks, evaluation methodologies, and operational best practices. This provides quite an interesting perspective on choices for handling RAG, monitoring, frameworks choice (e.g. LangChain), etc.

Google’s 5-Day GenAI Course

Google has released a self-paced 5-day intensive course on GenAI foundations with a structured approach to modern generative AI workflows: This is quite a comprehensive practical resource covering foundational models and prompt engineering, embeddings and vector databases, generative agents, domain-specific LLMs, and finally MLOps for generative AI. This is a good opportunity for practitioners to gain hands-on experience through whitepapers, code labs, and expert-led discussions on Kaggle and YouTube.

DeepMind World Foundation AI

DeepMind announces an exciting large-scale “GenAI world model” for creating and simulating interactive 3D environments from a single prompt image. In practice, this means ML practitioners can quickly create an endless variety of interactive 3D settings to train and stress-test agents in simulated environments like games. This model operates as an autoregressive latent diffusion system trained on extensive video data, enabling it to produce coherent, action-responsive virtual worlds that can be explored with standard keyboard and mouse controls. This is quite an exciting area of research, particularly as we have seen papers also exploring the interactions of agents at scale across virtual worlds.

MSFT Quantifying Bad Days

Microsoft demystifies the concept of “Bad Days” in development through a quantitative and qualitative study of main drivers and broader impact in productivity and beyond: Microsoft is known for publishing interesting research in developer productivity (e.g. SPACE framework, etc), and this time they provide insightful results breaking down main drivers of “bad days” including slow builds, long pull request delays, excessive meetings, unclear processes, poor documentation, and difficult team dynamics. These “bad days” lead to stress, reduced morale, and even career dissatisfaction - by correlating self-reported “bad day” factors with system-level metrics like build and PR times, the study provides evidence-based validation of developer concerns. For ML practitioners in production environments, this work can be quite interesting to identify the specific areas where investments can be made to improve overall productivity and developer happiness.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #311

Sun, 01 Dec 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Number of ML Models in Prod
Building a GenAI Platform
Open Source Agentic Workflows
Over 1000 Python Videos
Advent of Code 2024
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Number of ML Models in Prod

Almost 70% of teams reported to have less than 100 models in production; however most of them are looking to double the number of models in production within the next 12 months. This provides an interesting snapshot of the production ML ecosystem, as although many teams are operating at smaller relative sclaes, the rate of growth is only increasing; we can see this with 9-15% of teams operating already over 1000 models in production. We are uncovering important insights as part of our survey on The State of Production ML in 2024; please contribute to this valuable investigation on machine learning tools and platforms used in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀

Building a GenAI Platform

Chip Huyen has put together a fantastic resource on production architectural patterns and best practices for building GenAI platforms: This is a great guide for production machine learning practitioners on building a robust generative AI platform, starting from a basic model deployment and progressively adding essential components such as enhanced context with retrieval-augmented generation (RAG), safety guardrails to prevent data leakage and manage outputs, model routing and gateways for scalability and control, caching strategies for latency and cost optimization, and complex logic with write actions for advanced capabilities. This is a very much needed resource, as it emphasises the importance of observability and orchestration to monitor, debug, and manage complex AI pipelines effectively.

Open Source Agentic Workflows

Our Production ML Github List has reached over 17,000+ stars 🚀 this provides a snapshot of open source tools in the ecosystem and we just added “Agentic Frameworks” to the list! We have put together this list to help machine learning practitioners to deploy, monitor, version, scale, and secure their production ML systems. The new “Agentic Workflow” section captures the tools available for building AI agents and multi-agent systems, including libraries like AgentScope, AutoGen, Chidori, LangGraph, between many others. If you know of any open source framework that is not listed please do give us a heads up or feel free to open up a PR!

Over 1000 Python Videos

This is an absolutely fantastic to browse over 1000 python videos; more importantly for machine learning practitioners you can find a curated collection of machine learning talks from various conferences like PyCon, PyData, and EuroPython: This is quite a great resource for people that are looking to develop continuously, particularly in the ML space, there are a broad range of talks which cover topics such as scalable machine learning pipelines, MLOps best practices, model deployment strategies, deep learning advancements, and practical applications in industries like finance, healthcare, and technology.

Advent of Code 2024

The time has arrived to brush up our skills and jump into the advent of code. There will be one programming challenge released every day to take your skills to the test and have some fun, this is a great time to also pick up a new programming language if you’ve been wanting to explore one for a while. It’s also interesting that different to previous years, this time there is a big disclaimer discouraging the use of LLMs for submissions due to the increasing use of these; it is interesting to see how these become more pervasive, and perhaps also how some of these type of challenges will also have to adapt through time to make it such that they can still be providing a challenge despite AI-supported development.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #310

Sun, 24 Nov 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

MLOps Prod Best Practice
GenAI Simulating 1k Real People
Salesforce’s Time Series ML
The Data Engineer Handbook
Anthropic’s Responsible AI
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

MLOps Prod Best Practice

In the MLOps ecosystem only about 30% leverage CI/CD or Dev/Staging/Prod environments; only 16% leverage A/B tests, and just about 10% leverage canary deployments. It is important to consider that the Machine Learning Development Lifecycle is still in the early stages of maturity in industry, particularly in production where best practices in software operations at scale are only slowly being adopted into the ML ecosystem. We are uncovering important insights as part of our survey on The State of Production ML in 2024; please contribute to this valuable investigation on machine learning tools and platforms used in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀

GenAI Simulating 1k Real People

Stanford leveraging GenAI to replicate 1,052 real individuals in a simulation of agents which aims to provide new mechanisms for testing large-scale policymaking and social science. This is quite an interesting area of GenAI exploring how agents can replicate the attitudes and behaviors of the individuals that they represent, and simulate their interactions in specific test scenarios. These agents were evaluated against their real-world counterparts with professional surveys, performing with 85% accuracy compared to original human responses. This of course opens up important ethical considerations as world-simulations can be created with game-like NPCs replicating real-people behaviour and their interactions - certainly a space to keep a close eye on.

Salesforce’s Time Series ML

Salesforce research enters the Foundation Model race with a time-series forecasting large model, which comes with interesting innovations in model size and optimization: Salesforce introduces Moirai-MoE, a state-of-the-art time series foundation model for universal forecasting which outperforms some of the previously released foundation models. It’s great to see development in the forecasting space particularly after the releases of other foundation models from the likes of Google, Amazon, Nixtla, etc - looking forward to seeing the benchmarks as this field continues to evolve.

The Data Engineer Handbook

A great resource for beginner and experienced data engineers - the Data Engineering Handbook: This is a great compilation of resources in the data engineering space with a clear roadmap for entering the field + a YouTube bootcamp - certainly worth checking out the wealth of blogs, whitepapers, podcasts, newsletters, glossaries, design patterns, courses, and certifications.

Anthropic’s Responsible AI

Anthropic has detailed its approach to responsible AI through risk assessment and mitigation practices across their end to end foundation model lifecycle. Anthropic presents their Responsible Scaling Policy, which outlines: 1) safeguards with model capabilities; 2) conducting rigorous risk assessments; 3) ensuring robust security and privacy measures; 4) contributing to global technical standards; 5) fostering societal impact by partnering with organizations; and 6) addressing trust and safety concerns. It is quite interesting to see organisations publishing their committments to responsible AI; it will now be more interesting to follow how these are implemented in practice.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #309

Sun, 17 Nov 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

MLOps Organisational Setups
National Regulation on AI
Netflix’s Time Series Infrastructure
GenAI Increasing Tech Debt
What is an AI Engineer?
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

MLOps Organisational Setups

Central functions support machine learning ambitions across organisations; only ~30% of organisations have a central Data / ML platform, and less than 10% have central AI Risk Governance: Establishing central functions to support teams is a growing trend in machine learning maturity across organisations, an we see some interesting trends, with less than 10% or organisations with a central “AI Inventory”, or an “AI Risk & Governance Function”; less than 15% with a central dev productivity function, and; slightly over 30% having a central Data Platform or central ML Platform 💻 We are uncovering great insights as part of our survey on The State of Production ML in 2024; please contribute to this valuable investigation on machine learning tools and platforms used in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀

National Regulation on AI

We have overhauled our repository of AI Regulation, Principles & Guidelines, which now contains a listing of major developments of national AI strategies, regulations and guidelines: The latest iteration of this github list provides an overview of global AI regulation, highlighting key resources from economic areas such as like the European Union’s AI Act, the United Statesxecutive Order on AI, China’s Interim Measures for Generative AI Services, and the UK’s pro-innovation regulatory approach - between many others. As always it’s an open source initiative so if there are any developments that are not included please do feel free to contribute with a PR!

Netflix’s Time Series Infrastructure

The scale that Netflix has to deal with is massive, and was reflected in last week’s global streaming issues - they share one of their approaches to scaling, including their TimeSeries Data Abstraction Layer to efficiently store and query massive volumes of immutable temporal event data: This is quite an in-depth walkthrough into Netflix’s architecture to handle up to 10 million writes per second and petabytes of data with low millisecond latency by using temporal partitioning and event bucketing strategies as part of their TimeSeries Abstraction Layer. They have been able to build flexible storage backends like Cassandra and Elasticsearch and tunable configurations for scalability and cost efficiency which is critical for internal use-cases such as production machine learning at massive scale.

GenAI Increasing Tech Debt

Contrary to the belief that AI improves dev quality, we are starting to see growing resources showcasing how it actually amplifies tech debt costs: This is an insightful resource that shows how GenAI can widen the productivity gap between “low tech-debt” and “high tech-debt” codebases. Generative AI tools tend to be useful with simple, modular, and ubiquituously developed systems; however, they struggle with complex or legacy codebases that contain tech debt, often making it hard to leverage AI effectively.

What is an AI Engineer?

The rise of generative AI and Large Language Models has transformed the role of the AI Engineer into a high-demand position that blends deep technical expertise with strategic business insight; but what does it actually consist of? This is a great resource from Gradient Flow that dives into providing some definition on the role that companies are seeking as AI engineers; this often consist technical practitioners who can develop and fine-tune LLMs for domain-specific applications, integrate them into scalable production systems using skills in Python, AI frameworks like PyTorch and TensorFlow. However this also extends to cloud platforms and MLOps practices - from a more personal perspective this seems to me like the role itself is still nascent and this would have to be broken down into sub-roles, such as data engineering, ML engineering, MLOps engineering and domain expertise, instead of fitting all into a single role.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #308

Sun, 10 Nov 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Tech University Munich AI Talk
Most Popular ML Training Tools
McKinsey on AI Power Capacity
Superlinked on Vector Indexes
OpenAI Doubling Down Search
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Tech University Munich AI Talk

What are the most pressing issues on ML Governance? What about the most important concepts of Responsible AI? Join us next week to find out, as we’ll be joining the Technical University Munich’s Institute for Ethics in AI to dive into the topic of “Responsible AI in 2024”: In this session we will explore the current landscape of responsible AI, focusing on the industrial, organizational, and technical aspects crucial for successful AI deployment, including governance challenges, accountability, security concerns, infrastructure complexities, risk mitigation, and building scalable, reliable AI systems that drive innovation while adhering to responsible practices.

Most Popular ML Training Tools

Which do you think are the most popular frameworks in production ML? Tensorflow is a tool of the past with only 8%! Sklearn reaches the top with 35% closely followed by Pytorch with 32% - other contenders are XGBoost with 7% and Catboost with 6% 💻 We are uncovering great insights as part of our survey on The State of Production ML in 2024; please contribute to this valuable investigation on machine learning tools and platforms used in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀

McKinsey AI Power Capacity

McKinsey dives into the explosive growth of generative AI has been leading to a significant capacity shortfall due to the growing demand of AI data centers requiring global capacity to triple by 2030: An interesting analysis diving into how this surge has been driven primarily by hyperscalers hosting advanced AI workloads, and which limit computational resources for production machine learning practitioners, which requires the need to adapt strategies for efficient resource utilization.

Superlinked on Vector Indexes

Vector indexing techniques are making strides on bridging the gap between traditional RDBMS and Vector Databases: It is interesting to see how we’re revisiting the foundations of databases with recent innovations on efficiency of similarity search in high-dimensional data for production machine learning applications which is becoming critical for Retrieval Augmented Generation. Superlinked dives into various approaches for indexing methods to address foundational challenges, such as Inverted File Indexing where clusters data points using K-means clustering. It is quite refreshing to see simple coding examples illustrating complex topics, particularly in this case showcasing how similar indexing techniques can help optimize search performance.

OpenAI Doubling Down Search

Recently we have seen OpenAI exploring other avenues for innovation, most recently taking the search Giants by positioning ChatGPT powered products to tackle web search requirements: Quite interesting to see innovations in the space of web search, particularly in context of fine-tuning LLMs (aka GPT-4o) for conversational answers enriched with up-to-date information and direct links to relevant web sources. Only time will tell whether this avenue will be able to properly challenge the search giants in their own game, or whether this will fall under an interesting but limited set of features for information retrieval.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #307

Sun, 03 Nov 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Most Popular ML Training Tools
Billion Scale Time-Series Model
Vector DBs as Wrong Abstraction
Embeddings are Underrated
GPT-4o Data Poisoning
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Most Popular ML Training Tools

Almost 30% use a custom built in-house tool for ML model training, with Databricks being the 2nd most popular choice with 20%, closely followed by AWS SageMaker. Insightful results from our survey on the State of Prod ML in 2024; it seems that although the area of ML model training is one of the most consolidated areas in regards to tooling, there is still quite a significant percentage of organisations that do not use an off-the-shelf framework and build custom in-house tools for their ML Training. Not surprisingly it does seem like the main choices are the cloud providers, however with quite a large gap between Azure/Google vs AWS/Databricks. If you have a chance we would be grateful if you could spend a few minutes on the survey, as you’ll contribute valuable information about the machine learning tools and platforms you use in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀

Billion Scale Time-Series Model

This is an exciting time for time series machine learning, with a new Billion-parameter Mixture-of-Experts Forecasting model entering the arena released by Princeton, Squirrel AI and Griffith University, together with the largest OSS TS dataset: Following the steps of Google, Amazon, Nixtla and others, Time-MoE is the latest time series foundation model which brings new innovative approaches with a Mixture-of-Experts Transformer architecture aiming to handle universal zero-shot forecasting. They have been able to scale this model to 2.4 billion parameters by pre-training on “Time-300B”, the largest open-access time series dataset comprising over 300 billion data points from more than nine domains. It is an exciting time for research in this space, not only new models and architectures being released, but also huge datasets to support further benchmarking and research.

Vector DBs as Wrong Abstraction

TimescaleDB comes with a controversial take on Vector Databases - “they are the wrong abstraction”: In production AI applications, managing vector embeddings tends to bring complexities due to vector databases treating embeddings as standalone data disconnected from their source - this is raised as an abstraction that should be addressed as it would otherwise lead to synchronization issues and stale data. TimescaleDB proposes treating embeddings as derived data similar to database indexes, which is interesting given recent extensions from DBs like planetscale to integrate embeddings natively into indexes, similarly through a “native vectorizer” abstraction. In this case however they still leverage the OSS pgai Vectorizer for PostgreSQL which helps automating the synchronization of embeddings with their source data within the database, however this does provide an insight on some of the open challenges that are yet to be addressed, and perhaps also one of the reasons why we still see lack of standardisation on VectorDBs in the State of Prod ML 2024 Survey.

Embeddings are Underrated

Although many may claim that LLMs and GenAI are driving the current AI revolution, there is a strong reason to believe that one of the biggest breakthroughs and drivers are the innovations within embeddings, implemented beyond just text: Vector embeddings have grown to become a hugely powerful tool in ML, not only powering the GenAI / LLM products that have been blowing minds recently, but also powering things like product recommendations, similarity search and beyond. This is a great intuitive overview of embeddings with a few practical examples that can provide a high level conceptual understanding for individuals that may not have come across some of these internals - there are other interesting applications, such as how AirBNB used embeddings for similar listing recommendations all the way back in 2018.

GPT-4o Data Poisoning

One of the most pressing challenges when deploying GenAI products is ensuring robust guardrails are in place - companies are investing heavily on increasing the safety of their products, but data poisoning and jailbreaking attacks keep advancing at a faster pace, and here is a great example: Even advanced and robust GenAI powered products like GPT-4o are suceptible to security vulnerabilities - this is a great example showcasing data poisoning and a potent new attack called “jailbreak-tuning,” where attackers inject harmful behaviors by fine-tuning models on poisoned datasets, even when advanced moderation systems are in place. This is a great deep dive into three threat models, including 1) malicious fine-tuning, 2) imperfect data curation, and 3) intentional data contamination, which really make it clear that that larger models become more susceptible to these attacks as they scale. This also makes it clear that it only continues to become more critical to ensure robust evaluation metrics, fine-tuning safeguards, and ongoing stress-testing to prevent exploitation of these vulnerabilities in production environments - this means that organisations also have to invest into their existing security / red-teaming capabilities to support this.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #306

Sun, 27 Oct 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Top 3 Challenges in Prod ML
Google DeepMind Weather Forecast
PlanetScale Embeddings SQL
Google Shopping 10m Embedding
META Llama LLM Optimisation
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Top 3 Challenges in Prod ML

The Top 3 Challenges in Production Machine Learning include 1) ML Monitoring, 2) Data, and 3) Showcasing business value: The challenges highlighted in production machine learning seem to resonate quite a lot with what we see in practice; the top 3 challenges in are: 1) Monitoring - Establishing standardised and robust monitoring for ML systems; 2) Data - Access to relevant data for training (which is aligned data in production inference), and; 3) Impact - Showcasing business impact and business value on usecases. Further challenges highlighted include: 4) Inconsistency of training and experimentation environments; 5) Building production-grade ML pipelines; 6) Gaps in tooling and support for model productionisation, and 7) Governance and Domain Risks. These are really important insights from 2024 Survey on The State of Production ML - if you have a chance we would be grateful if you could spend a few minutes on the survey, as you’ll contribute valuable information about the machine learning tools and platforms you use in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀

Google DeepMind Weather Forecast

Google DeepMind has released an exciting foundation model for weather forecasting - this is a hybrid approach that brings together traditional physics-simulations and deep learning to surpass the current best models for weather and climate forecasting: The Google team provides a great deep dive into how they implemented their new NeuralGCM model using JAX to overcome the limitations of pure physics or machine learning approaches. This approach is showcased as achieving higher accuracy than state-of-the-art models whilst being faster, more cost-effective, producing more detailed forecasts, and doing everything with a simplified codebase (vs 1m+ line FORTRAN codebase). This is actually published together with the full code available, as well as the pre-trained models, as well as a fully-fledged benchmarking suite - this is certainly an exciting time for this space.

PlanetScale Embeddings SQL

Vector embeddings have been exploding in popularity, and similarly we have seen a similar growth on vector databases - this week we see an interesting new challenger with PlanetScale introducing native support for AI embedding vectors in its MySQL-compatible database; this is quite exciting as it comes as an integrated index that supports all relational database features, eliminating the need for a separate vector database. The vector search functionality is built upon Microsoft Research’s SPANN and SPFresh algorithms, which in themselves deserve their own deep dive, as they have enabled integration into MySQL’s default storage engine (InnoDB); this means that inserts, updates, and deletes of vector data are immediately reflected in the vector index as part of SQL transactions, ensuring ACID compliance. This is quite an interesting advancement, as we are able to see the foundational DB field and emerging technologies converge to ensure standardisation and best practice to simplify architecture and operations while leveraging familiar SQL features.

Google Shopping 10m Embedding

Vector Embedding Platform Marco has released a Google Shopping Dataset with 10m products for benchmarking of multi-modal ranking evaluation, together with a new approach to improve multi-modal retrieval and ranking. This is quite an interesting deep dive into their “Generalized Contrastive Learning” framework for optimization of retrieval and ranking by encoding multiple data types (like text and images) and incorporating fine-grained relevance directly into embeddings. From the research they share, it seems this framework extends CLIP-style models to improve multi-modal retrieval and ranking whilst addressing limitations of existing approaches through unified representations of documents composed of multiple fields, enhancing intra-modal understanding, and optimizing embeddings for efficient storage in vector databases. It is great to see initiatives that bring together novel approaches together with benchmarking frameworks that encourage further breakthroughs.

META Llama LLM Optimisation

Efficiency in LLMs is becoming growingly important; Meta has now released quantized versions of its Llama 3.2 1B and 3B models optimized for mobile devices which they suggest up to 4× speedup + 56% reduction in model size + 41% less memory usage compared to the original models: This is quite an interesting approach using QLoRA + post-training quantization method which doesn’t require the original training data. It is quite insightful to see that the race to larger accurate models is almost marching the race to smaller accurate models!

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #305

Sun, 20 Oct 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

State of Prod ML 2024 Survey
AirStreetCapital State of AI Report
Real Time Data Infra at Uber
Microsoft’s CPU Inference ML Lib
Meta’s Open AI Data & Models
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

State of Prod ML 2024 Survey

Almost 60% of real-time machine learning is still powered by FastAPI/Flask or Custom Wrappers - it seems the MLOps ecosystem is not yet consolidated, showing huge opportunity in this space: These are really important insights from 2024 Survey on The State of Production ML; we have designed the questions to provide meaningful insights on the current landscape of production ML in 2024 - if you have a chance we would be grateful if you could spend a few minutes on the survey, as you’ll contribute valuable information about the machine learning tools and platforms you use in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀 We are also working on an interactive visualisation for everyone to be able to slice and dice across the data to derive meaningful insights on the production ML ecosystem!

AirStreetCapital State of AI Report

The State of AI Report 2024 from AirStreet Capital is out! Really interesting insights from 2024, including highlights on the performance gap between proprietary models like GPT-4 and open-source alternatives has narrowed significantly, with advancements in planning, reasoning, and multimodal capabilities extending AI’s applications into fields like mathematics, biology, and neuroscience. One interesting observation is that despite U.S. sanctions, Chinese labs continue to produce competitive AI models through alternative means. Similarly, the economic impact of global AI has surged, with public companies reaching a combined enterprise value of $9 trillion, although questions about long-term sustainability and viable business models remain. Check out the full insights on the report!

Real Time Data Infra at Uber

Uber has built a robust real-time data infrastructure to process Petabyte-scale data per day, using open-source technologies like Apache Kafka, Flink and Apache Pinot - these support production machine learning applications that require processing massive data volumes with low latency. Uber enables scalable stream processing and low-latency analytics critical for ML workflows like real-time prediction monitoring by enhancing these tools through leveraging frameworks such as FlinkSQL for easier streaming job creation with SQL and adding upsert capabilities to Pinot for real-time data updates. This is quite an interesting deep dive from Uber into their challenges and lessons learned throughout their large-scale data journey.

Microsoft’s CPU Inference ML Lib

Microsoft has released an exciting project to bring GPU-native LLMs into CPUs with Bitnet.cpp; this is their official inference framework for 1-bit Large Language Models: This C++ library is inspired from Llama.cpp and provides optimized kernels for fast and energy-efficient inference on CPUs, with future support planned for NPUs and GPUs. This framework achieves significant speedups of 6×+ and energy reductions up to 82.2% on both ARM and x86 architectures, which is quite exciting for even 100B-parameter models.

Meta’s Open AI Data & Models

Meta’s FAIR team has released several new AI research artifacts to continue supporting the advancement of science across the community: These open source releases include: SAM 2.1 dataset on image and video segmentation; Meta Spirit LM as multimodal language model integrating speech and text; Layer Skip to accelerate LLM performance; SALSA to validate security for post-quantum cryptography standards; Meta Lingua for large-scale language model training; Meta Open Materials 2024 to accelerate AI-assisted inorganic materials discovery, and more.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #304

Sun, 13 Oct 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

If you are a Machine Learning Practitioner looking for an interesting opportunity, I’m currently hiring for an Applied Science Manager for Forecasting & Causal Inference for a Senior Applied Scientist for Forecasting - do check it out and do feel free to share broadly!

This week in Machine Learning:

Building Virtual Worlds with ML
State of Prod ML 2024 Survey
Repo of ML Monitoring Metrics
Salesforce Red Teams for AI/LLMs
Cognitive Load in Developers
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

State of Prod ML 2024 Survey

Dire results on Diversity from the 2024 Survey on the State of Production Machine Learning; only under 5% of respondents identifying as female - there is a lot of work for all of us in the Prod ML ecosystem! These have been really important insights from 2024 Survey on The State of Production ML; we have designed the questions to provide meaningful insights on the current landscape of production ML in 2024 - if you have a chance we would be grateful if you could spend a few minutes on the survey, as you’ll contribute valuable information about the machine learning tools and platforms you use in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀 We are also working on an interactive visualisation for everyone to be able to slice and dice across the data to derive meaningful insights on the production ML ecosystem!

Building Virtual Worlds with ML

This is mind blowing: Who would’ve thought that training Image Diffussion Models on Videogame visuals+player inputs would result in fully ML-generated virtual worlds 🤯 Researchers from Microsoft and Geneva/Edinburgh University have released DIAMOND (DIffusion As a Model Of eNvironment Dreams) - a reinforcement learning agent that leverages diffusion models for world modeling in Atari games, eliminating the need for discretization and reducing mode collapse issues inherent in token-based approaches. DIAMOND enhances the modeling of visual details, and achieves a mean human-normalized score of 1.46 on the Atari 100k benchmark (a new state-of-the-art for agents trained entirely within a world model), and demonstrates that diffusion models can serve as effective drop-in replacements for real environments in reinforcement learning. And the code is open source and available in github - this is an exciting new domain I had not come across, certainly an area to keep an eye into.

Repo of ML Monitoring Metrics

The Little Book of ML Metrics is an open-source handbook designed for production machine learning practitioners, and it is aiming to build a comprehensive reference to all-things-metrics in ML monitoring. This is a great ambitious project aiming to consolidate a comprehensive list of metrics in ML monitoring across the domains of regression, classification, clustering, ranking, computer vision, NLP, generative AI, probabilistic models, bias, fairness, and business metrics.

Salesforce Red Teams for AI/LLMs

Salesforce’s Responsible AI team presents a framework for reproducible red teaming that tackles the challenges of AI product testing. It is great to see organisations drive forward the discourse on red teaming for AI, Salesforce doubling down on four key areas: 1) High-quality, use case-specific data that is properly stored and maintained for reproducibility; 2) Programmatic access to products via APIs or clients to automate and scale testing efficiently; 3) Clear taxonomies for evaluating outputs to ensure stakeholder alignment and consistent assessments; and 4) Comprehensive test plans to manage expectations and scope technical work effectively.

Cognitive Load in Developers

One of the biggest challengers to productivity in development is cognitive load, and this is a great resource that dives into key areas to reduce the mental effort needed to understand code. Extraneous cognitive load, caused by overly complex conditionals, excessive small modules or microservices, and unnecessary abstractions, can be reduced by simplifying code, favoring deep modules with simple interfaces, and using language features sparingly. Properly applying principles like Domain-Driven Design (DDD) and avoiding unnecessary complexity ensures that code remains understandable and maintainable, improving productivity and collaboration across teams.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #303

Sun, 06 Oct 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Meta’s MovieGen Model
State of Prod ML 2024 Survey
Ngrok’s Central Data Platform
Google’s NotebookLLM Podcasts
LLMs Large-Scale Annotation
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Meta’s MovieGen Model

META has released a mind blowing high-def AI text-to-video model that has been making waves in the AI ecosystem: This is quite an interesting release as this model encompasses also features for video editing through text prompts, personalized video creation by conditioning on user-provided images, and audio generation for sound effects and soundtracks. The space of text-to-video models is evolving quite fast - quite an exciting area to keep an eye on as we’ll most likely see counter-releases from the usual suspect tech giants following this release.

State of Prod ML 2024 Survey

Over 50% do not have any Machine Learning monitoring; important insights from 2024 Survey on The State of Production ML! We have designed the questions to provide meaningful insights on the current landscape of production ML in 2024 - if you have a chance we would be grateful if you could spend a few minutes on the survey, as you’ll contribute valuable information about the machine learning tools and platforms you use in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀 We are also working on an interactive visualisation for everyone to be able to slice and dice across the data to derive meaningful insights on the production ML ecosystem!

Ngrok’s Central Data Platform

A deep dive into ngrok’s journey building their internal data platform with a small engineering team: An interesting case study building an internal data platform with lessons integrating data engineering into their broader software development practices, focusing on open-source tools and collaborative workflows within a Go monorepo. Ngrok transitioned from AWS services to a Kubernetes-based stack using tools like Dagster, Airbyte, Apache Flink, and dbt with a lot of interesting lessons learned along the way.

Google’s NotebookLLM Podcasts

Google’s introduced an Audio Overview feature to their NotebookML product which has been used to generate custom podcasts from any content provided - people have been using instruction manuals, personal notes, newsletters and anything they get their hands on. This is quite an interesting use-case of Google Gemini integrated into a Google experimental product to generate audio content for what promises to be quite a lot of high potential user-level applications.

LLMs Large-Scale Annotation

A recent research paper from Zalando presenting a framework to leverage multimodal large language models to efficiently evaluate large-scale product retrieval systems by automating the relevance assessment of query-product pairs using both textual and visual product information. This is quite an interesting approach to reduce the time and cost associated with human annotations whilst keeping quality. This research initiative evaluates on datasets with 20,000 query-product pairs in English and German showing that MLLM-generated annotations align closely with human judgments, making it suitable for continuous, scalable, and multilingual evaluations in production environments, such as e-commerce search engines.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #302

Sun, 29 Sep 2024 00:00:00 GMT

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

State of Prod ML 2024 Survey
ML Optimization Gone Wrong
Google Measuring Dev Goals
XKCD 10y: Hard vs Impossible
GPU Puzzles For Fun & Profit
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

State of Prod ML 2024 Survey

34% of organisations take between 1-3 months to productionise a machine learning model, and over 20% take even longer up to 6 months - fantastic insights only a few weeks from launching the Prod ML 2024 Survey! We have designed the questions to provide meaningful insights on the current landscape of production ML in 2024 - if you have a chance we would be grateful if you could spend a few minutes on the survey, as you’ll contribute valuable information about the machine learning tools and platforms you use in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀 We are also working on an interactive visualisation for everyone to be able to slice and dice across the data to derive meaningful insights on the production ML ecosystem!

ML Optimization Gone Wrong

When a measure becomes a target, it ceases to be a good measure; Goodhart’s law showcasing when too much optimization is deterimenal - great deep dive from Anthropic (+ former Google Brain) researcher. Increased efficiency can paradoxically worsen outcomes, which is a phenomenon termed by the strong version of Goodhart’s Law, which is compared to the concept of overfitting in machine learning. Overfitting occurs when an ML model over-optimizes a specific dataset instead of the generalised distribution expected to be seen in the real world. It is possible to mitigate these isuses by better aligning proxy objectives with true goals, introducing regularization penalties, injecting noise, applying early stopping, and adjusting system capacities.

Google Measuring Dev Goals

Google on Measuring Machine Learning Productivity Goals across organisations: A great paper from Google’s Developer Productivity team on the importance of understanding and measuring overarching developer goals to enhance productivity and experience, especially in complex, iterative workflows common in production machine learning. Google developed a concise list of 30 durable and observable developer goals spanning the software development lifecycle by combining attitudinal data from surveys with behavioral data from usage logs.

XKCD 10y: Hard vs Impossible

Thrilled to celebrate 10 years of XKCD, which now for over a decade have brought humorous and insightful comics reflecting the challenges and ironies in computer science. Today they share an ironic and comical post on foundation ML models in the context of how difficult it is to distinguish “easy” tasks from “hard” tasks in software development, but indeed in this case in context of LLMs. Here is to many more years of insightful and inspiring XKCD comics!

GPU Puzzles For Fun & Profit

This is the time to learn General-Processing GPU compute programming, and GPU-Puzzles are a fantastic way to get started: This new interactive notebook tutorial is a great intro to GPGPU designed for research & production machine learning practitioners to learn GPU programming fundamentals using Python’s NUMBA, which compiles Python code into CUDA kernels. This resource is quite comprehensive as it teaches essential concepts like thread and block management, shared memory usage, and efficient computation of core deep learning algorithms such as pooling, convolution, and matrix multiplication (of course).

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #301

Sun, 22 Sep 2024 00:00:00 GMT

Even if it’s already been a week, we continue celebrate together our 300th Issue 🚀🚀🚀 This is a HUGE milestone we want to celebrate with YOU! As part of this we are launching a survey on The State of Production ML, and your contribution would make a significant difference to the whole ML ecosystem ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

State of Prod ML: 2024 Survey
Learnings from 100k A/B Tests
Pragmatic Machine Learning Eng
Stanford’s ML System Seminars
PolaRS GPU Accel DataFrames
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

State of Prod ML 2024 Survey

In Machine Learning, Spreadsheets just keep coming back - this time as one of the top choices for experiment tracking and model registry tools - only a week since we launched our survey on the State of Production ML we are already seeing great insights! We have designed the questions to provide meaningful insights on the current landscape of production ML in 2024 - if you have a chance we would be grateful if you could spend a few minutes on the survey, as you’ll contribute valuable information about the machine learning tools and platforms you use in your production ML development. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀

Learnings from 100k A/B Tests

What do you learn from running 100,000 A/B tests? Despite the disproportionately larger A/B-testing culture at billion-user scale, companies like Amazon and Meta still face significant challenges in large-scale experimentation, and this is resource captures some of these challenges. Some of the key issues that these type of organisations face include information overload from numerous concurrent experiments, diminishing returns on the approaches to extract causal insights, infrastructural limitations affecting scalability, and cultural differences in experimentation practices. This is a great insight for organisations that are building their experimentation practices, although we have to remember that only a small subset of companies have the scale of some of these tech giants.

Pragmatic Machine Learning Eng

Free Online Book: The Pragmatic Programmer for Machine Learning Engineering, A Practical Deep Dive. This is a great resource on the role of software engineering in developing robust, efficient, and maintainable machine learning systems for production. The book bridges the gap between machine learning and software engineering by offering best practices for designing, coding, deploying, documenting, and testing machine learning pipelines, and provides key lessons on recognizing that poor software practices can lead to technical debt, reproducibility issues, and costly failures.

Stanford’s ML System Seminars

Stanford has been publishing seminar videos on the frontier of machine learning systems, covering key concepts around challenges and solutions in AI research and industry in conversation with thought leaders in the space. This is quite a great video series as it features expert speakers from academia and industry, the series covers a wide range of topics including programming ML systems with frameworks like JAX, data labeling with tools like Snorkel, deploying robust models, hyperparameter optimization, and end-to-end ML pipelines.

PolaRS GPU Accel DataFrames

Exciting to continue seeing developments in GPGPU with PolaRS (aka Pandas in Rust) which has integrated GPU acceleration into its Python library. It’s quite interesting to see the initial release leveraging NVIDIA RAPIDS’ cuDF which provides production machine learning practitioners up to 13x speed improvements on compute-intensive data processing tasks like joins and GROUP BYs. It will certainly be an exciting space to keep an eye on - indeed it is exciting for the opportunity for leveraging Vulkan-based GPGPU backends for support across 1000s of GPU cards, such as with our GPU acceleration framework Vulkan Kompute.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #300

Sun, 15 Sep 2024 00:00:00 GMT

Today we celebrate together our 300th Issue 🚀🚀🚀 This is a HUGE milestone we want to celebrate with YOU! As part of this we are launching a survey on The State of Production ML, and your contribution would make a significant difference to the whole ML ecosystem ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in Machine Learning:

Survey: The State of Prod ML 2024
OpenAI Reasoning with LLMs
Copilot is making Devs Worse
The ML Engineering Open Book
Rise of OSS Time Series DBs
Upcoming events on Production ML
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Survey: State of Prod ML 2024

As we celebrate our 300th newsletter we invite everyone to contribute to an exciting survey on the State of Production Machine Learning! We have designed the questions to provide meaningful insights on the current landscape of production ML in 2024. By participating, you’ll contribute valuable information about the machine learning tools and platforms you use and your ML ecosystem. Your input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community 🚀

OpenAI Reasoning with LLMs

OpenAI has introduced O1 - a new large language model trained with reinforcement learning to perform complex reasoning using a hidden chain of thought to improve problem-solving abilities. This new release seems to introduce a different approach over previous models like GPT-4o, which is now specialising on benchmarks for mathematics and science expertise, and programming competence. In practice there has been a broad range of mixed responses, but there seems to be a strong claim for this approach to have some strong potential for improving safety and alignment by allowing the model to internally reason about safety rules.

Copilot is making Devs Worse

Copilot is Making Programmers Worse at Programming: AI code-generation tools like GitHub Copilot have potential to improve productivity, however they can also be double-edged sword which may worsen programmers’ fundamental skills by introducing over-reliance on auto-generated code. This reliance can then lead into hindering development on core programming competencies, such as reducing problem-solving abilities, and introducing a lack of ownership over code quality. Someone is going to have to maintain all the code that is written by AI, so we need to be mindful that the age-old “clean-code” principles are not going anywhere even with these intelligent tools rising.

The ML Engineering Open Book

A great resource diving into Machine Learning Engineering, providing resources specialised for large language models (LLMs) and multi-modal models (VLMs): Great repo that provides practical methodologies, tools, and step-by-step instructions covering critical aspects across the production ML lifecycle such as hardware considerations (compute, storage, network), orchestration with SLURM, training and inference strategies, debugging techniques, and performance optimization.

Rise of OSS Time Series DBs

Open source time series databases like Prometheus and InfluxDB have become growingly popular for production machine learning practitioners to efficiently store and query large volumes of time-stamped data needed for monitoring models, detecting anomalies, and forecasting resources: Traditional relational databases may struggle with the scale and performance demands of time series data if not optimized accordingly - this has led to the rise of open source solutions that offer better stability, efficiency, and scalability for these use-cases. This is an interesting deep dive from VictoriaMetrics, which covers some of the shortcomings from existing solutions as well as their approach to addtessing these.

Upcoming MLOps Events

Upcoming conferences where we’re speaking:

TUM AI@WORK 10th October @ Germany

Other upcoming MLOps conferences in 2024:

ODSC West - 29th October @ USA
Data & AI Summit - 10th June @ USA
AI Summit London - 12th June @ UK
World Summit AI - 9th October @ Neatherlands
MLOps World - 8th November @ USA

Open Source MLOps Tools

Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
ACM’s Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the “Awesome AI Guidelines” list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

The ML Engineer — Issue #299

Sun, 08 Sep 2024 00:00:00 GMT

Canva's 25B Events per Day, Uber's Kafka Tiered Storage, DB Lessons to Know for Devs, Effects of Gen AI on High Skill, Explicit is Better than Implicit + more 🚀

The ML Engineer — Issue #298

Sun, 01 Sep 2024 00:00:00 GMT

Raschka's 3hr Coding Workshop, Lessons on Four Years of MLOps, 150% Productive with Cursor, Time Management Techniques, Convincing PMs to Tech Debt + more 🚀

The ML Engineer — Issue #297

Sun, 25 Aug 2024 00:00:00 GMT

On Being a Senior Engineer, Postgres as a Search Engine, What's Going on in ML, How Google Search Works, Good vs Bad Code Refactoring + more 🚀

The ML Engineer — Issue #296

Sun, 18 Aug 2024 00:00:00 GMT

The State of Prod ML in 2024, PyCon US Videos are Out, Pop Culture in the Age of AI, PapersWeLove in CompSci, Seeing Theory: Probability & Stats + more 🚀

The ML Engineer — Issue #295

Sun, 11 Aug 2024 00:00:00 GMT

Salesforce's New Foundation AI, Building a Custom ML Platform, 25 CompSci Papers to Read, Lessons on AI Training, Introducing Apple Foundation AI + more 🚀

The ML Engineer — Issue #294

Sun, 04 Aug 2024 00:00:00 GMT

Building A Generative AI Platform, Amazon’s Exa-Migration to Ray, 10 Years of TPUs at Google, A Visual Guide to Quantisation, Lessons of 35 Years in Software + more 🚀

The ML Engineer — Issue #293

Sun, 28 Jul 2024 00:00:00 GMT

Stackoverflow 2024 Dev Survey, Netflix's Workflow Orchestrator, META Releases LLAMA 3, Lessons from a Year of LLMs, Google's Climate Foundation AI + more 🚀

The ML Engineer — Issue #292

Sun, 21 Jul 2024 00:00:00 GMT

AI & Machine Learning Security, Goldman on GenAI Value Gap, Yoshua Bengio on AI Safety, The Future of AI in Engineering + more 🚀

The ML Engineer — Issue #291

Sun, 14 Jul 2024 00:00:00 GMT

Building Notion's Data Lake, RouteLLM for Effective GenAI, Goldman on GenAI Value Gap, How to Interview ML Engineers + more 🚀

The ML Engineer — Issue #290

Sun, 07 Jul 2024 00:00:00 GMT

META's Text-to-3D-Asset Model, Sequoia on AI’s $600B Question, Uber Modernising Batch & AI Infra, Machine Learning Operations, Runway's Text-to-Video Gen v3 + more 🚀

The ML Engineer — Issue #289

Sun, 30 Jun 2024 00:00:00 GMT

Google Releases Gemma 2, Learnings from 900 OSS AI Tools, LLMs for Compiler Optimisation, From Baremetal to 70b Model, Lessons from 15 Years of Coding + more 🚀

The ML Engineer — Issue #288

Sun, 23 Jun 2024 00:00:00 GMT

Building a Multi-Petabyte Data Platform, Chip Huyen on AI Engineering, META Multi-Modal Architecture, Google Audio Gen Model, Distributed Systems Fun & Profit + more 🚀

The ML Engineer — Issue #287

Sun, 16 Jun 2024 00:00:00 GMT

Chip Huyen on AI Engineering, How META Trains LLMs at Scale, AI Search The Bitter Lesson, Uncensoring LLM Research, Apple AI Private Cloud Compute + more 🚀

The ML Engineer — Issue #286

Sun, 09 Jun 2024 00:00:00 GMT

AI in Engineering at Google, The LLM Fine Tuning Index, Free Bayesian Data Analysis Book, Alibaba's Latest LLM Qwen2, Microsoft's Climate Foundation Model + more 🚀

The ML Engineer — Issue #285

Sun, 02 Jun 2024 00:00:00 GMT

Lessons from a year of LLM Apps, Andrew Ng on Real-World GenAI, McKinsey State of AI Report, Tech Managers Anti-Patterns, Japan's Push for Open Research + more 🚀

The ML Engineer — Issue #284

Sun, 26 May 2024 00:00:00 GMT

Stanford Foundation Model Index, Diffusion Models Compendium, Continuous Delivery of AI Systems, Automated Vehicles Act UK, GenAI Red Teaming Report 2024 + more 🚀

The ML Engineer — Issue #283

Sun, 19 May 2024 00:00:00 GMT

Ways to Deploy an ML Model, xLSTM Innovation in ML, Llama3 Implemented in NumPy, META Scaling Law for RecSys, GPUs Go Brrr + more 🚀

The ML Engineer — Issue #282

Sun, 12 May 2024 00:00:00 GMT

Stanford on Machine Unlearning, StackOverflow and OpenAI, Open Time Series Large Models, Scraping with AI (Prompts), The Illustrated Word2Vec + more 🚀

The ML Engineer — Issue #281

Sun, 05 May 2024 00:00:00 GMT

AI Generating Comics from Text, Stable Text-to-Motion Framework, Tesla Autopilot Training 5m+ Cars, Reading Papers for Career, Measuring Personal Growth + more 🚀

The ML Engineer — Issue #280

Sun, 28 Apr 2024 00:00:00 GMT

Production ML Ecosystem 16k ⭐️, A Guide to Visual Transformers, Primer on Hacking LLM Models, Snowflake's Largest LLM, Foundational Tips for Engineers + more 🚀

The ML Engineer — Issue #279

Sun, 21 Apr 2024 00:00:00 GMT

Stanford State of AI Report, Stripe Feature Store Airbnb OSS, NSA Security Framework for AI, LLMs as Effective Regressors, Llama 3 Latest META's Release + more 🚀

The ML Engineer — Issue #278

Sun, 14 Apr 2024 00:00:00 GMT

Airbnb's New OSS Feature Store, ML Books for Engineers, 300 ML Systems Design Usecases, Half Billion GPT Token Lessons, The Lifecycle of an AI Copilot + more 🚀

The ML Engineer — Issue #277

Sun, 07 Apr 2024 00:00:00 GMT

Data Quality View of ML, 3Blue1Brown on AI Transformers, NLP Fundamentals Deep Dive, Cognitive Load in Software, How to Think of Eng Quality + more 🚀

The ML Engineer — Issue #276

Sun, 31 Mar 2024 00:00:00 GMT

Scaling ML Infrastructure at Uber, Largest OSS LLM from Databricks, Google AI Forecasting Floods, Big Data Skills through XKCD, AI Accountability Report from NTIA + more 🚀

The ML Engineer — Issue #275

Sun, 24 Mar 2024 00:00:00 GMT

Twitter/X Releases OSS Grok, Mamba State Space Architecture, Microsoft GenAI Online Lessons, An Intro to SQL for Scientists, Forecasting Principles & Practice + more 🚀

The ML Engineer — Issue #274

Sun, 17 Mar 2024 00:00:00 GMT

META's GenAI Infrastructure, MIT Course on Foundation Model, Amazon's FC Foundation Model, ML Engineering & Sys Design, High Quality Local AI Images + more 🚀

The ML Engineer — Issue #273

Sun, 10 Mar 2024 00:00:00 GMT

ML Competitions in 2023, How Discord Stores Trillion Msgs, 5 Lessons in 6y the Hard Way, Neural Networks Zero to Hero, 70B Param Model at Home + more 🚀

The ML Engineer — Issue #272

Sun, 03 Mar 2024 00:00:00 GMT

UK AI Policy Adops Proposals, Alibaba's Realistic AI Video, Don't Mock ML (in Unit Tests), Amazon Billion Param TTS ML, Karpathy Tutorial on GPT Tokens + more 🚀

The ML Engineer — Issue #271

Sun, 25 Feb 2024 00:00:00 GMT

Salesforce FC Foundation Model, Google DeepMind Gemma.cpp, Stanford Privacy in the AI Era, Graph Neural Nets at Linkedin, GPT in 60 Lines of NumPy + more 🚀

The ML Engineer — Issue #270

Sun, 18 Feb 2024 00:00:00 GMT

META's V-JEPA vs OpenAI Sora, The MLOps Bookshelf, Google DeepMind Gemini 1.5, Unit Tests using LLMs at Meta, Mistral-7B on an Acid Trip + more 🚀

The ML Engineer — Issue #269

Sun, 11 Feb 2024 00:00:00 GMT

Search Engine in 80 Lines of Python, One Trillion Row Challenge, Comparing LLMs to Lawyers, Infra Decisions Endrose or Regret, AI Generated Calls Now Illegal + more 🚀

The ML Engineer — Issue #268

Sun, 04 Feb 2024 00:00:00 GMT

Democratising LLM Inference, Google Forecast Foundation Model, Cyber-resilience Act and OSS, Stanford's Modern Algorithms, LLaVA 1.16 OCR & Reasoning + more 🚀

The ML Engineer — Issue #267

Sun, 28 Jan 2024 00:00:00 GMT

MLL Sampling in Large Lang Models, UK Govt's GenAI Framework, Why Machine Learning is Hard, Hand-picked AI Fundamentals, Building a Brag-List for Success + more 🚀

The ML Engineer — Issue #266

Sun, 21 Jan 2024 00:00:00 GMT

MLL Forecasting & Causal Inference, Value of Open Source Software, TextToSpeech Inverting Whisper, ISO Global Standards on AI, Meta Large Scale Infrastructure + more 🚀

The ML Engineer — Issue #265

Sun, 14 Jan 2024 00:00:00 GMT

MLL Survey of Vector Databases, Microsoft & LF on Recommenders, Stanford on LLM Hallucination, DeepMind vs Doctors Turing Test, AI in RecSys Ranking + more 🚀

The ML Engineer — Issue #264

Sat, 06 Jan 2024 00:00:00 GMT

MLL Celebrating 5 years towards 2024, Databases 2023 Year in Review, Meta's Audio2Animation AI Model, Efficient Multimodal OSS LLMs, Deep Learning on Relational DBs + more 🚀

The ML Engineer — Issue #263

Sun, 31 Dec 2023 00:00:00 GMT

MLL Celebrating 5 years towards 2024, Real Time AI Image Generation, Deep Learning 33 years vs now, Cohere LLM Foundation Course, Habits of Great Engineers + more 🚀

The ML Engineer — Issue #262

Sat, 23 Dec 2023 00:00:00 GMT

Machine Learning at Stanford, Don't Build a Vector Database, LLM 3D Visualisation Demo, Scaling Pinterest to Millions Users, Optimising LLMs from Datasets, Open Sour ce MLOps Tools + more 🚀

The ML Engineer — Issue #261

Sun, 17 Dec 2023 00:00:00 GMT

Billion Scale Vector Search, Mozilla's MemoryCache Local AI, Deep Learning Foundations Book, Google's Machine Learning for ML, Probabilistic Data Structures + more 🚀

The ML Engineer — Issue #260

Sun, 10 Dec 2023 00:00:00 GMT

Practical Deep Learning FastAI, AI Generating Triangle Meshes, Extracting Data from ChatGPT, Stable Difussion Visual Anagrams, Designing Distributed SQL Engine + more 🚀

The ML Engineer — Issue #259

Sun, 03 Dec 2023 00:00:00 GMT

The ML Engineer this week, Instacart's Prod ML Platform, AI Self-Operating Computer, Statistical vs DL Forecasting, Google DeepMind Music Model, Code is Run more than Read + more 🚀

The ML Engineer — Issue #258

Sun, 26 Nov 2023 00:00:00 GMT

The ML Engineer this week, AI Policy, Principles & Guidelines, Neural Networks: Zero to Hero, Stable Difussion Video Release, Hidden Tech Debt in ML Systems, Hacking Google Bard Vulnerability + more 🚀

The ML Engineer — Issue #257

Sun, 19 Nov 2023 00:00:00 GMT

The ML Engineer this week, TimeGPT: Magical Auto Forecast, tldraw; When UX and AI Intersect, Deep Learning Weather Forecast, Geneva Uni Neural Nets Course, Humans vs GPT-4 on Reasoning + more 🚀

The ML Engineer — Issue #256

Sun, 12 Nov 2023 00:00:00 GMT

The ML Engineer this week, Building AI Products Differently, Vector Databases Applications, Misconceptions in Engineering, Takeaways On AI Executive Order, OECD New Definition on AI + more 🚀

The ML Engineer — Issue #255

Sun, 05 Nov 2023 00:00:00 GMT

The ML Engineer this week, ACM TPC & State of Tech Policy, Python 17,000x Faster Data, Google's ML Weather Forecast, Most Valuable Traits of Top Devs, Animated AI Neural Architectures + more 🚀

The ML Engineer — Issue #254

Sun, 29 Oct 2023 00:00:00 GMT

The ML Engineer this week, Text Embeddings Visual Overview, Lessons from 20 Years at Scale, Advanced Python Mastery Course, Linkedin Managed Search Infra, Bare Min Knowledge on Unicode + more 🚀

The ML Engineer — Issue #253

Sun, 22 Oct 2023 00:00:00 GMT

The ML Engineer this week, META/FB Decoding Brain Images, Every Dev Should Know on GPUs, Reflections on AI Summit 2023, Multimodal Architecture Agents, The Startup CTO Handbook + more 🚀

The ML Engineer — Issue #252

Sun, 15 Oct 2023 00:00:00 GMT

The ML Engineer this week, Multimodality and LLMs Overview, State of AI Report for 2023, Stable Difussion from Scratch, Inverted Transformer Forecasting, Introduction to Modern Statistics + more 🚀

The ML Engineer — Issue #251

Sun, 08 Oct 2023 00:00:00 GMT

The ML Engineer this week, The State of Production ML, How Linkedin Uses Embeddings, Large Language Models in 2023, StableDiffusion XL in 298mb RAM, Picking a Vector DB in 2023 + more 🚀

The ML Engineer — Issue #250

Sun, 01 Oct 2023 00:00:00 GMT

The ML Engineer this week, GPT-4V(ision) First Impressions, Causality for Machine Learning, Hardest Part of Building Software, Habits of Effective Engineers, CMU Deep Learning Systems + more 🚀

The ML Engineer — Issue #249

Sun, 24 Sep 2023 00:00:00 GMT

MIT on Efficient Deep Learning, DALL-E 3 Text-to-Image Release, InfoQ AI, ML & Data Eng Trends, Guide to Contributing to OSS, Key Knowledge: Idempotency + more 🚀

The ML Engineer — Issue #248

Sun, 17 Sep 2023 00:00:00 GMT

StabilityAI's Text-to-Song Model, Dataset-Driven LLM Optimiztion, Google's MLP for Forecasting, Hallucination Detection for LLMs, Death by 1000 Microservices + more 🚀

The ML Engineer — Issue #247

Sun, 10 Sep 2023 00:00:00 GMT

Andrew Ng: Opportunities in AI, 10y of Forecast Reconciliation, Glassdoor’s ML Platform Registry, Advanced NLP Course w SpaCy, Privacy Nightmare on Wheels + more 🚀

The ML Engineer — Issue #246

Sun, 03 Sep 2023 00:00:00 GMT

Generative Agents & Forums, LangChain Cheat Sheet, Measuring Developer Productivity, Retrieval-Augmented Generation, Intro to Fourier Transforms + more 🚀

The ML Engineer — Issue #245

Sun, 27 Aug 2023 00:00:00 GMT

Patterns for LLM-based Systems, Meta's CodeLlama Released, An Elegant Puzzle Book Notes, Hands on Train & Deploy ML, Advanced Probabilistic ML Book + more 🚀

The ML Engineer — Issue #244

Sun, 20 Aug 2023 00:00:00 GMT

Stanford's New NLU Course, Out-of-distribution Detection in DL, Vector Databases 4-Part Series, How is LLaMa.cpp Possible, CAP Theorem Plain English Intro + more 🚀

The ML Engineer — Issue #243

Sun, 13 Aug 2023 00:00:00 GMT

Stanford Course on Intro to ML, ML Models Learn vs Generalise, Building Llama from Scratch, MLFlow Tracking and MinIO, MLSecOps Kubeflow Exploration + more 🚀

The ML Engineer — Issue #242

Sun, 06 Aug 2023 00:00:00 GMT

MadeWithML MLOps Course, Instruction Tuning Llama 2, Transparency LLMs Beyond OSS, AI Music Generation from META, How Google Manages Tech Debt + more 🚀

The ML Engineer — Issue #241

Sun, 30 Jul 2023 00:00:00 GMT

Building a ChatGPT Terminal UI, HuggingFace Audio ML Course, Migrations as the Fix to Tech Debt, Vision & Language to Action ML, Tutorial Estimating Causal Effects + more 🚀

The ML Engineer — Issue #240

Sun, 23 Jul 2023 00:00:00 GMT

Principles for Responsible Gen AI, ML Feature Store at DeliveryHero, Generative AI Video Editing, Meta Public Release of Llama 2, Learn Self-Attention from Scratch + more 🚀

The ML Engineer — Issue #239

Sun, 16 Jul 2023 00:00:00 GMT

Awesome Prod ML 5y & 14k ⭐, Stackoverflow on ML Ecosystem, Poisoning LLMs to Spread Misinfo, Survey on LLM Evaluations, Keras 3.0 on JAX, PyTorch & TF + more 🚀

The ML Engineer — Issue #238

Sun, 09 Jul 2023 00:00:00 GMT

UK Government AI Regulation, Andrew Ng Generative AI Course, MLOps at Reasonable Scale, Foundation Model Transparency, Train Serve Skew in ML Models + more 🚀

The ML Engineer — Issue #237

Sun, 02 Jul 2023 00:00:00 GMT

ML System Design Case Studies, Future of Custom Tuned LLMs, LLM App to Prod Databaricks, Facebook Ranking Explained, Python at Netflix in Production + more 🚀

The ML Engineer — Issue #236

Sun, 25 Jun 2023 00:00:00 GMT

The State of Production ML 2023, Economic Potential of Gen AI, Playing with Steamlit & LLMs, MITRE ML Security Mitigations, A/B Testing Problems & Learnings + more 🚀

The ML Engineer — Issue #235

Sun, 18 Jun 2023 00:00:00 GMT

Machine Learning Security at NeurIPS, Google's Secure AI Framework, AI Fairness Across Linkedin, META's AI Music Generation, How Google Manages Tech Debt + more 🚀

The ML Engineer — Issue #234

Fri, 09 Jun 2023 00:00:00 GMT

OWASP Top 10 for Generative AI, Architectural Tech Debt Costs, Harvard's Intro to AI with Python, MLOps at Reasonable Scale, Securing AI Systems Series + more 🚀

The ML Engineer — Issue #233

Sun, 04 Jun 2023 00:00:00 GMT

Deep Learning GenAI Courses, Industral Transformer Forecasting, Ride Sharing App From Scratch, Writing LLM-Powered Applications, Programmer Interrupted Cost + more 🚀

The ML Engineer — Issue #232

Sun, 28 May 2023 00:00:00 GMT

Water Footprint of Deep Learning, Karpathy on the State of GPT, 2023 State of AI from Databricks, Lessons Creating a GPT4 Plugin, Andrew Ng Next on 10 Years of AI + more 🚀

The ML Engineer — Issue #231

Sun, 21 May 2023 00:00:00 GMT

Interactive AI GAN Image Editing, O'Reilly Trends to Watch 2023, U.S AI Policy Turning Point, Developer Experience Framework, Battle of Large Language Models + more 🚀

The ML Engineer — Issue #230

Sun, 14 May 2023 00:00:00 GMT

The MLOps Fundamentals Guide, AI Prompt Injection Explained, Binance Real Time ML for Fraud, Scaling Prime Video Infrastructure, CMU Database Group Youtube + more 🚀

The ML Engineer — Issue #229

Sun, 07 May 2023 00:00:00 GMT

LLM Applications for Production, AI Generating Music from Text, The Little Book of Deep Learning, Google Leaked Memo on OSS AI, New AI Programming Language + more 🚀

The ML Engineer — Issue #228

Sun, 30 Apr 2023 00:00:00 GMT

Call to Protect OSS AI in EU, Large Language Models Survey, Growth of AI Through Cloud Lens, Free LangChain Course Online, Neural Nets from Scratch in Zig + more 🚀

The ML Engineer — Issue #227

Sun, 23 Apr 2023 00:00:00 GMT

MIT Course Foundation Models, The MLOps Bookshelf Collection, Curated Set of Key LLM Papers, AI Practical Codebase Analysis, RocksDB Under The Hood + more 🚀

The ML Engineer — Issue #226

Sun, 16 Apr 2023 00:00:00 GMT

The Actually Open AI Chat-GPT, Building the Future with LLMs, Evolution of RecSys Architectures, Google's Responsible AI Agenda, ChatGPT Productivity Hacks + more 🚀

The ML Engineer — Issue #225

Sun, 09 Apr 2023 00:00:00 GMT

Stanford's AI Index Report 2023, FastAI New Advanced ML Course, Top ML Papers to Read in 2023, OpenAI Approach to AI Safety, Facebook's MLOps Ecosystem + more 🚀

The ML Engineer — Issue #224

Sun, 02 Apr 2023 00:00:00 GMT

Facebook's MLOps Ecosystem, Twitter OSS Recommender Algo, Large Language Models MLOps, GPT-4-ALL Access with LLaMa, Finance GenAI w BloombergGPT + more 🚀

The ML Engineer — Issue #223

Sun, 26 Mar 2023 00:00:00 GMT

ChatGPT Internals and Interface, Training ML Across 1000 GPUs, Early Experiments with GPT4, The AI-Powered Dev Experience, Google ML Crashcourse w TF + more 🚀

The ML Engineer — Issue #222

Sun, 19 Mar 2023 00:00:00 GMT

Language Models ArXiv Paper, Discord's Trillions of Messages, LLM Prompt Engineering Guide, Fully OSS Alternative to ChatGPT, Airflow and Great Expectations + more 🚀

The ML Engineer — Issue #221

Sun, 12 Mar 2023 00:00:00 GMT

LLM Stable Diffussion Moment, The Utility DataFlow Computing, Postgres Architecture Explained, Cost of Architectural Complexity, Migrating from Flask to FastAPI + more 🚀

The ML Engineer — Issue #220

Sun, 05 Mar 2023 00:00:00 GMT

Large Language & Image Models, Real World ML Systems Survey, Coinbase High-Perf Data Series, NVIDIA OSS RecSys Ecosystem, 20 Lessons in 20 Years of Dev + more 🚀

The ML Engineer — Issue #219

Sun, 26 Feb 2023 00:00:00 GMT

NeurIPS 2022 Production ML Videos, MIT Intro to Data-Centric ML, ML Innovation at Spotify with Ray + more 🚀

The ML Engineer — Issue #218

Sun, 19 Feb 2023 00:00:00 GMT

Incredible PyTorch Ecosystem, Large Scale RecSys Architectures, Technology Trends 2023 February, Flask to FastAPI Migration Series, The ArXiv of ChatGPT Failures + more 🚀

The ML Engineer — Issue #217

Sun, 12 Feb 2023 00:00:00 GMT

Awesome Production ML OSS, Tech Behind Github Code Search, Preparing for the Europe AI Act, Rise of Enterprise AI Leadership, React.js The Documentary + more 🚀

The ML Engineer — Issue #216

Sun, 05 Feb 2023 00:00:00 GMT

GitHub CEO on Europe & AI Act, Google's AI Text-Video Editing, Data Version Control AI Course, Andrew Ng on Data Centric AI, Monitoring at Scale Prom Tutorial + more 🚀

The ML Engineer — Issue #215

Sun, 29 Jan 2023 00:00:00 GMT

AI-Assisted Programming Costs, Text-to-basically-any-sound-effect, The Illustrated Stable Diffussion, Transformer Models Introduction, SQLAlchemy 2.0 Release Post + more 🚀

The ML Engineer — Issue #214

Sun, 22 Jan 2023 00:00:00 GMT

Karpathy Building 0->1 ChatGPT, Raschka on Model Eval/Selection, Deep Learning Tuning Playbook, Hidden Tech Debt in Prod ML, Working with Golang and SQL + more 🚀

The ML Engineer — Issue #213

Sun, 15 Jan 2023 00:00:00 GMT

Doordash from Heuristic to ML, Big Data w P(X) Data Structures, ML Papers Explained, Myths and Legends in HPC, Talking AI with AI from Greylock + more 🚀

The ML Engineer — Issue #212

Sat, 07 Jan 2023 00:00:00 GMT

Production MLOps at Ubisoft, Measuring an Engineering Org, 2022-2023 Reviews & Predictions, Reinforcement Learning Summary, FastAPI Framework Best Practice + more 🚀

The ML Engineer — Issue #211

Sun, 01 Jan 2023 00:00:00 GMT

Check out our new Awesome 2022-2023 Year-In-Review & Tech Predictions List 🚀, 2022-2023 Reviews & Predictions, Algos & Data Structures to Try, Day-1 Decisions Make or Break, Math for Computer Science & ML, Which AI Explanation to Choose + more 🚀

The ML Engineer — Issue #210

Sun, 25 Dec 2022 00:00:00 GMT

The History of AI & Deep Learning, Trends and Insights from Github, AI Generation Text to 3D Models, ForwardForward Algorithm Insight, The T-Shaped MLOps Engineer + more 🚀

The ML Engineer — Issue #209

Sun, 18 Dec 2022 00:00:00 GMT

OSS Insights at Scale with TiDB, O'Reilly Trends December 2022, Stanford ML Systems Design, Linux Foundation 2022 Report, Statistical vs DL Forecasting + more 🚀

The ML Engineer — Issue #208

Sat, 10 Dec 2022 00:00:00 GMT

NeurIPS Workshop Keynote Vid, How Production ML Breaks, The Illustrated Stable Diffusion, The McKinsey State of AI Report, Anomaly Detection Benchmark + more 🚀

The ML Engineer — Issue #207

Sun, 04 Dec 2022 00:00:00 GMT

NeurIPS on Prod ML Challenges, ChatGPT Continues to Surprise, Advent of Code Skill Polishing, Binance MLOps Architecture, PyTorch 2.0 Has Been Released + more 🚀

The ML Engineer — Issue #206

Sun, 27 Nov 2022 00:00:00 GMT

PyData Global 2022 Coming Up, Algorithmic Responsibility Panel, Securing ML Algorithms in EU, Large Language Models Review, SICP Key CompSci Foundation + more 🚀

The ML Engineer — Issue #205

Sun, 20 Nov 2022 00:00:00 GMT

The Complete Guide to NLP, Galactica The Scientific LLM, Thoughts on ML Engineering, Monzo on Argo Rollouts at Scale, NeurIPS on Prod ML Challenges + more 🚀

The ML Engineer — Issue #204

Sun, 13 Nov 2022 00:00:00 GMT

Metadata In End To End MLOps, NeurIPS on Prod ML Challenges, The Modern Startup Architecture, TikTok SotA Recommender Sys, The Stream Processing Index + more 🚀

The ML Engineer — Issue #203

Sun, 06 Nov 2022 00:00:00 GMT

Kubernetes AI Keynote & Videos, Metadata for End to End MLOps, Tech Trends to Watch Nov 2022, Andrej Karpathy NN Zero to Hero, Maths for CompSci & ML Book + more 🚀

The ML Engineer — Issue #202

Sun, 30 Oct 2022 00:00:00 GMT

ACM US & Europe AI Principles, Prometheus The Documentary, Healthcare Causal Inference ML, Bias Bounty on Algorithmic Bias, Designing Data Product Canvas + more 🚀

The ML Engineer — Issue #201

Sun, 23 Oct 2022 00:00:00 GMT

Kubernetes AI Opening Keynote, Data Mesh Architecture Definition, NVIDIA Optimizing Data Collect, Growing Metadata Mgmt Systems, Data Product Management + more 🚀

The ML Engineer — Issue #200

Sun, 16 Oct 2022 00:00:00 GMT

The State of AI Report is Out, Kubernetes AI Day North America, Bert, LSTMs & Toxic Detection, Stripe with Migrations at Scale, OpenAI Introducing Whisper + more 🚀

The ML Engineer — Issue #199

Sun, 09 Oct 2022 00:00:00 GMT

AI Infrastructure Landscape, RecSys Recap & Best Papers, DoorDash Recommendations, Google Cloud Architecture Centre, AlphaTensor Matrix Multiplication + more 🚀

The ML Engineer — Issue #198

Sun, 02 Oct 2022 00:00:00 GMT

Pro-Innovation AI Regulation, Full Stack Deep Learning Course, Netflix on Large Scale RecSys, Meta New Text-to-Video Model, The Vector Database Index + more 🚀

The ML Engineer — Issue #197

Sun, 25 Sep 2022 00:00:00 GMT

Instacart Real Time ML Journey, Operationalising ML Survey, Automate with Python in K8s, Building NoSQL from Scratch, Green Programming Languages + more 🚀

The ML Engineer — Issue #196

Sun, 18 Sep 2022 00:00:00 GMT

PyTorch Joins Linux Foundation, Deploy & Monitor ML at NeurIPS, Trends to Watch September, ML Algorithms from Scratch, Curated List of Awful AI Cases + more 🚀

The ML Engineer — Issue #195

Sun, 11 Sep 2022 00:00:00 GMT

Best Practices for ML Engineering, NCSC Principles for ML Security, Practical OCR with PaddleOCR, Large Scale RecSys at Alibaba, PapersWithCode Annual Trends + more 🚀

The ML Engineer — Issue #194

Sun, 04 Sep 2022 00:00:00 GMT

Secure Machine Learning Talk, Stable Diffusion Public Release, Popular NLP Projects 2022, CompSci Papers We Love, Code Performance Speed Limits + more 🚀

The ML Engineer — Issue #193

Sun, 28 Aug 2022 00:00:00 GMT

Deploying ML at NeurIPS 2022, Scaling AI Education at Uber, Scale System Design Knowledge, Open Source Ecosystem at Wolt, Confidential Computing in ML + more 🚀

The ML Engineer — Issue #192

Sat, 20 Aug 2022 00:00:00 GMT

Data Quality Mgmt at Linkedin, Design Patterns in ML Systems, Debt in ML Maintenance at Scale, Data-Oriented Design in Software, Data Visualisation in Python + more 🚀

The ML Engineer — Issue #191

Sat, 13 Aug 2022 00:00:00 GMT

O'Reilly Radar Trends to Watch, Advanced ML Model Monitoring, Building Prod MLOps Pipelines, Building a Two-Layered RecSys, SciPy 2022 Conference Videos + more 🚀

The ML Engineer — Issue #190

Sun, 07 Aug 2022 00:00:00 GMT

FastAI Practical ML Course, Golang Based ML Framework, JuliaCon Recordings Released, MLSecOps Top 10 Vulnerabilities, Building Containers From Scratch + more 🚀

The ML Engineer — Issue #189

Sun, 31 Jul 2022 00:00:00 GMT

MLSecOps Top 10 Vulnerabilities, Uber's Data Workflows at Scale, Implementing Research Papers, DeepMind AlphaFold Universe, Statistical ML Summer School + more 🚀

The ML Engineer — Issue #188

Sun, 24 Jul 2022 00:00:00 GMT

Uber's OLAP Infra at Scale, Netflix Scaling with GraphQL, Scaling Microservices at Lyft, Data Annotation & Synthetic Gen, NVIDIA on Practical Art with AI + more 🚀

The ML Engineer — Issue #187

Sun, 17 Jul 2022 00:00:00 GMT

MLOps Meetup Driverless & E2e, Stitchfix and their MLOps Platform, AI Infra Ecosystem Report 2022, Stanford Transformers Course, Practical Guide to A/B Testing + more 🚀

The ML Engineer — Issue #186

Sun, 10 Jul 2022 00:00:00 GMT

MLOps Definition & Architecture, When Language Models Too Big, No Language Left Behind LLM, Software Dev Lifecycle Evolution, Lessons From Chaos Monkey + more 🚀

The ML Engineer — Issue #185

Sun, 03 Jul 2022 00:00:00 GMT

Free Open Source MLOps Course, MLOps Taxonomy & Methodology, ML Tracking & Experiment Tools, Shipping To Production Principles, Reverse Interview Candidate Tips + more 🚀

The ML Engineer — Issue #184

Sun, 26 Jun 2022 00:00:00 GMT

Andrew Ng New ML Course, MLOps Open Source Course, Deep Learning RecSys Survey, Effective Software Testing Guide, Stackoverflow Developer Survey + more 🚀

The ML Engineer — Issue #183

Sun, 19 Jun 2022 00:00:00 GMT

Reproducible Deep Learning, Trends to Watch in June 2022, Embedding Search at Faceboook, Fallacies of Distributed Systems, Github Malware Store Database + more 🚀

The ML Engineer — Issue #182

Sun, 12 Jun 2022 00:00:00 GMT

High Performance ML at Scale, Orchestrating ML Applications, Kafka Streaming Patterns for ML, The Go Programming Language, The SPACE of Dev Productivity + more 🚀

The ML Engineer — Issue #181

Sun, 05 Jun 2022 00:00:00 GMT

ML ∪ Security ∪ DevOps, Machine Learning Trends to Know, Observability Trends & AIOps, Featurestore Types for ML & Data, Data Science and Storytelling + more 🚀

The ML Engineer — Issue #180

Sun, 29 May 2022 00:00:00 GMT

Netflix end-to-end MLOps System, Pick&Choose MyMLOps Stack, How to Avoid Pitfalls of MLOps, Google Brain's Imagen Model, Airbnb Recommender Systems + more 🚀

The ML Engineer — Issue #179

Sun, 22 May 2022 00:00:00 GMT

PyCon Machine Learning Security, Approaching (Almost) all ML Book, Airbnb's Microservice Architecture, Image Outlier Detection Tutorial, The HackerNews Algo Internals + more 🚀

The ML Engineer — Issue #178

Sun, 15 May 2022 00:00:00 GMT

Enterprise Declarative ML System, Alibaba Scale Recommenders, DeepMind's Generalist Agent, Product Success DS Guide, Transformers from Scratch + more 🚀

The ML Engineer — Issue #177

Sun, 08 May 2022 00:00:00 GMT

MLOps London Meetup May, End-to-end Open Source MLOps, Machine Learning at Discord, Faceboook's 175B Model Release, Running Kubernetes in Production + more 🚀

The ML Engineer — Issue #176

Sun, 01 May 2022 00:00:00 GMT

Monzo's Machine Learning Stack, Real World RecSys that Scales, MLOps London Meetup May, Human-Centric MLOps in K8s, Rethink interviews for Great Devs + more 🚀

The ML Engineer — Issue #175

Sun, 24 Apr 2022 00:00:00 GMT

Production ML and Security: MLSecOps, MLOps in Food Delivery at Wolt, OpenAI Davinci Building Games, AI in 2021 GitLab DevOps Survey, C is Not a Language Anymore + more 🚀

The ML Engineer — Issue #174

Sun, 17 Apr 2022 00:00:00 GMT

Spotify Semantic AI for Podcasts, Microsoft on Data Drift at Scale, ZenML E2E Deployment Seldon, Borg, Omega and Kubernetes, The Annual 2022 AI Index + more 🚀

The ML Engineer — Issue #173

Sun, 10 Apr 2022 00:00:00 GMT

Linkedin's Explainable AI RecSys, Google's AI Autogen Summary, OpenAI's Text to Image Model, Wisdom from 50+ Years of Code, Continuous Intelligence at Scale + more 🚀

The ML Engineer — Issue #172

Sun, 03 Apr 2022 00:00:00 GMT

Secure ML with MLSecOps, The State of Data Engineering, Doordash Declarative Fabricator, Google Brain Distributed Flow ML, Hashicorp from CEO to Tech IC + more 🚀

The ML Engineer — Issue #171

Sun, 27 Mar 2022 00:00:00 GMT

Machine Learning Youtube List, Data Management Trends 2022, Ultimate Guide to Text Similarity, OpenTelemetry and Python, Lessons Learned from 10y OSS + more 🚀

The ML Engineer — Issue #170

Sun, 20 Mar 2022 00:00:00 GMT

MLOps Virtual & Online Talks, Karpathy's Deep Learning Retro, Stanford 2022 AI Index Report, Microsoft ML Microservice Scale, Kubernetes the Hard Way Course + more 🚀

The ML Engineer — Issue #169

Sun, 13 Mar 2022 00:00:00 GMT

The Future of AI Infrastructure, Radar Trends to Watch March, Cloud Resources for DL Training, Google NLP Model for Everything, Cloud Security Themes for 2022 + more 🚀

The ML Engineer — Issue #168

Sun, 06 Mar 2022 00:00:00 GMT

Kompute Promoted to Incubation, Modern C++ Computer Vision, Another Road to MLOps Mastery, Extending Kubernetes for MLOps, Airflow Unbundling ETL at Scale + more 🚀

The ML Engineer — Issue #167

Sun, 27 Feb 2022 00:00:00 GMT

ML Monitoring with Outliers & Drift, Netflix Infra Systems Design, Building Infrastructure Platforms, Uber Orbit Bayesian ML Platform, The State of ML in JuliaLang + more 🚀

The ML Engineer — Issue #166

Sun, 20 Feb 2022 00:00:00 GMT

Data Scientists and Kubernetes, Cross Vendor GPU Acceleration, Open Python ML Inference Server, Building ML Infra at Netflix, Interpretable Machine Learning + more 🚀

The ML Engineer — Issue #165

Sun, 13 Feb 2022 00:00:00 GMT

Evolution of AI Ethics Meetup, Distribution Shifts and Monitoring, O'Reilly Radar Trends Feb 2022, Overview of MLOps Definition, Millions of Messages with Kafka + more 🚀

The ML Engineer — Issue #164

Sun, 06 Feb 2022 00:00:00 GMT

DeepMind AI Dev AlphaCode, Databases 2022 Year in Review, Evolving Notebook Infra at Twitter, Doordash Feature Eng System, O'Reilly on Causal Inference + more 🚀

The ML Engineer — Issue #163

Sun, 30 Jan 2022 00:00:00 GMT

CI/CD for Prod ML at Scale Event, Machine Learning at Discord, Kubernetes the Documentary Pt.2, Open Source MLOps E2e Stacks, ML Algorithms Cheat Sheet + more 🚀

The ML Engineer — Issue #162

Sun, 23 Jan 2022 00:00:00 GMT

MLOps Meetup Online this Week, Architecture Series for MLOps, Testing Approach in Data Science, A Gentle Intro to Shapley Values, Kubernetes the Documentary + more 🚀

The ML Engineer — Issue #161

Sun, 16 Jan 2022 00:00:00 GMT

Principles for Responsible AI, Google Research ML Themes, Introduction to Explainable ML, ML Architectures from Industry, Validation & Testing of ML Models + more 🚀

The ML Engineer — Issue #160

Sun, 09 Jan 2022 00:00:00 GMT

MLOps Free University Course, AI & Data Trends for 2022, How to Evaluate MLOps Tools, The State of Graph ML in 2022, Deep Learning Interview Prep + more 🚀

The ML Engineer — Issue #159

Sun, 02 Jan 2022 00:00:00 GMT

We wish a HAPPY NEW YEAR to all MLE Newsletter subscribers!! 🎉🎇🎁🎊🎈⛄❄🥳, A Year Full of Amazing AI Papers, Model Monitoring Areas Overview, Data Science for Infrastructure, Master Dataclasses in Python, Graph Neural Networks Overview + more 🚀

The ML Engineer — Issue #158

Sun, 26 Dec 2021 00:00:00 GMT

We wish happy holidays to all our MLE Newsletter subscribers!! 🎉🎇🎁🎊🎈⛄❄🥳, Prod ML Monitoring Deep Dive, Data & AI Platforms at Shopify, Evolution of the Canonical Stack, Must Read Books in MLOps 2022, 2022 AI Predictions from Experts + more 🚀

The ML Engineer — Issue #157

Sun, 19 Dec 2021 00:00:00 GMT

Call to Build Models like OSS, Cross-Vendor GPGPU at CppCon, The 18 Highest Paying Dev Roles, Making Language Models Smart, Scaling Teams Parallel Systems + more 🚀

The ML Engineer — Issue #156

Sun, 12 Dec 2021 00:00:00 GMT

Video for NeurIPS 2021 is Out!, Twitter Processing Billion Events, Avoiding Data Disasters Example, NLP and AI in Financial Services, 26-Week Data Science Course + more 🚀

The ML Engineer — Issue #155

Sun, 05 Dec 2021 00:00:00 GMT

NeurIPS LXAI Opening Keynote, Machine Learning Advent of Code, Open MLOps End to End Tech, Kubernetes Introduction Course, O'Reilly Radar Trends December + more 🚀

The ML Engineer — Issue #154

Sun, 28 Nov 2021 00:00:00 GMT

NeurIPS LXAI Opening Keynote, Reddit Production ML Architecture, State of Privacy Preserving ML, Organisational Data Challenges, Dynamic AI Learning Mind Map + more 🚀

The ML Engineer — Issue #153

Sun, 21 Nov 2021 00:00:00 GMT

O'Reilly Radar Trends to Watch, Thoughtworks MLOps Platforms, MLOps Anti-Paterns & Lessons, Containers from the Bottom Up + more 🚀

The ML Engineer — Issue #152

Sun, 14 Nov 2021 00:00:00 GMT

MLOps LDN November Meetup, Scalable Explainable NLP Search, KubeCon North America 2021, AI begins with Data Quality, Reflections 10k hrs Programming + more 🚀

The ML Engineer — Issue #151

Sun, 07 Nov 2021 00:00:00 GMT

NeurIPS LX Workshop Keynotes, HuggingFace Online NLP Course, Neural Network for Chess Engine, SpaCy vs NLTK. Normalization, The State of AI-Generated Code + more 🚀

The ML Engineer — Issue #150

Sun, 31 Oct 2021 00:00:00 GMT

MLOps and Devops around Data, The Guide to GitOps Architecture, Case Study for Ethics in AI/ML, Large Scale ML Multi-Modal Data, ML Monitoring at EuroPython + more 🚀

The ML Engineer — Issue #149

Sun, 24 Oct 2021 00:00:00 GMT

NeurIPS Practical Responsible AI, Github on Deploying ML Safely, The State of AI Report 2021, Shopify on Building ML Models, 20 Lessons Across 20 Years + more 🚀

The ML Engineer — Issue #148

Sun, 17 Oct 2021 00:00:00 GMT

Linkedin's Approach to XAI, Prod ML Resources Curated List, Top Places to Work for ML, ML, AI & Data Landscape (MAD), Training System for Industry Scale + more 🚀

The ML Engineer — Issue #147

Sun, 10 Oct 2021 00:00:00 GMT

Real Time Fraud Detection at ING, Practical AI Ethics from MLOps, Federated Learning Research, The Road to Intelligent RPA, Technical Debt Opinion Survey + more 🚀

The ML Engineer — Issue #146

Sun, 03 Oct 2021 00:00:00 GMT

GPT2 with ONNX, Triton & Tempo, 2021 Data/AI Salary Survey, An Intuitive Deep Dive on Drift, A Gentle Introduction to MLOps, Julia Multi-Dispatch Effectiveness + more 🚀

The ML Engineer — Issue #145

Sun, 26 Sep 2021 00:00:00 GMT

MLOps LDN Online & In-Person, First GPU Acceleration Kompute, ICML 2021 Videos & Slides, Graph Database Ecosystem, Kubernetes Operators Made Easy + more 🚀

The ML Engineer — Issue #144

Sat, 18 Sep 2021 00:00:00 GMT

Kompute v0.8.0 Now Released, ML and High Interest Tech Debt, Gentle Intro to Graph Neural Nets, Educational ML with EpyNN, Adoption of GraphQL at Paypal + more 🚀

The ML Engineer — Issue #143

Sun, 12 Sep 2021 00:00:00 GMT

Machine Learning EngSci Book, Graph Deep Learning Overview, Massively Scaling ML Training, FAANG Interview Prep Repository, Awesome Kubernetes Lists + more 🚀

The ML Engineer — Issue #142

Sun, 05 Sep 2021 00:00:00 GMT

Principles for Digital Citizenship, Stories from ML Practitioners, Drift Without Labelled Data, Software Design & Data Scientists, Data Quality Unpacked + more 🚀

The ML Engineer — Issue #141

Sat, 28 Aug 2021 00:00:00 GMT

Kompute Joins Linux Foundation, MLOps London Meetup Kickoff, Feature Stores Demystified, Data Science Efficient Python, Auditing ML for Compliance Risk + more 🚀

The ML Engineer — Issue #140

Sun, 22 Aug 2021 00:00:00 GMT

CompSci Favourite Papers, AI & Data Trends to Watch, Language Model Learnings, Trending Open Source MLOps, TorchServe Model Optimization + more 🚀

The ML Engineer — Issue #139

Sun, 15 Aug 2021 00:00:00 GMT

StackOverflow 2021 Dev Survey, Alibi for ML Explainability, Data Science Role Evolution, CPU Transformer Optimization, Open End-to-end MLOps Platform + more 🚀

The ML Engineer — Issue #138

Sun, 08 Aug 2021 00:00:00 GMT

EU AI Regulation Consultation, Declarative ML Systems, MLOps Event with Andrew Ng, MLOps Feature Stores Review, An Intro to Statistical Learning + more 🚀

The ML Engineer — Issue #137

Sat, 31 Jul 2021 00:00:00 GMT

Uber Distributed Computing AI, Building Robust ML Workflows, Bjarne on Future of Programming, Papers with Code Highlights, A Base ML Project Starter + more 🚀

The ML Engineer — Issue #136

Sun, 25 Jul 2021 00:00:00 GMT

A Time Series DB from Scratch, Best of Speed & Simpl. w Julia, Towards Next Gen Dataflow, The Future of Deep Learning, Privacy & Cybersecurity Merging + more 🚀

The ML Engineer — Issue #135

Sun, 18 Jul 2021 00:00:00 GMT

Drift Detection: An Introduction, The HashiCorp OpenCore Story, Guide to Onboarding Developers, Neural Net in Julia from Scratch, Exploiting Security in ML Binaries + more 🚀

The ML Engineer — Issue #134

Sun, 11 Jul 2021 00:00:00 GMT

From ML Model to Microservice, HuggingFace Founder on MLOps, How to NOT Measure Latency, MIT Deep Learning Intro Course, Data Mesh Design & Architecture + more 🚀

The ML Engineer — Issue #133

Sun, 04 Jul 2021 00:00:00 GMT

Pitfalls of Causal Inference in XAI, 7 Layers of MLOps Security, Growing Large Language Models, Automated Data Wrangling, Clever vs Insightful Code + more 🚀

The ML Engineer — Issue #132

Sun, 27 Jun 2021 00:00:00 GMT

Machine Learning Interview Prep, ML Data Structures with Pydantic, Applied NLP Thinking Solutions, Disasters in a Microservice World, A Day in the Life of a Senior Dev + more 🚀

The ML Engineer — Issue #131

Sun, 20 Jun 2021 00:00:00 GMT

HuggingFace Online Course 🤗, TWIML’s AI Solutions Guide, The Top Trends in Tech Report, Going Full Circle with Kafka, Introduction to Vector Databases + more 🚀

The ML Engineer — Issue #130

Sun, 13 Jun 2021 00:00:00 GMT

All-Things-Python at Netflix, Monitoring ML Systems Course, AI Risk and Liability in Industry, Kubernetes Learning from Scratch, What is Your ML Model Hiding + more 🚀

The ML Engineer — Issue #129

Sun, 06 Jun 2021 00:00:00 GMT

GitOps Demystified in Practice, Robust ML Monitoring Overview, DeepCheapFakes and Impact, Detecting & Mitigating Ethical Risk, Biases in AI Systems Taxonomy + more 🚀

The ML Engineer — Issue #128

Sun, 30 May 2021 00:00:00 GMT

Key AI Research Labs In Europe, Responsible AI at Linkedin, Lessons from Netflix, Spotify, etc., Twitter on Elastic + Neural Nets, Three ML Roles in Organisations + more 🚀

The ML Engineer — Issue #127

Sun, 23 May 2021 00:00:00 GMT

Get started with MLOps, Building Drift Monitoring System, Better Abstractions in Prod ML, Start Guide to Anomaly Detection, Revisiting Dev Design Patterns + more 🚀

The ML Engineer — Issue #126

Sat, 15 May 2021 00:00:00 GMT

Kubernetes AI & ML Day Talks, ML Cards for MLOps Governance, MLOps & Differences to DevOps, AI Usecases Beyond Automation, Andrew Ng Courses on MLOps + more 🚀

The ML Engineer — Issue #125

Sat, 08 May 2021 00:00:00 GMT

AI Adoption in Enterprise 2021, State of Data Engineering in 2021, Choosing Best E2E MLOps Tools, Python Beg. to Adv. Resources, Importance of Reliable Metadata + more 🚀

The ML Engineer — Issue #124

Sun, 02 May 2021 00:00:00 GMT

Automated ML Eval at Scale, Tech Capabilities for MLOps, Selecting ML Feature Eng Method, Eng Best Practices in Data Gov, The NLP Index with 3000+ Repos + more 🚀

The ML Engineer — Issue #123

Sun, 25 Apr 2021 00:00:00 GMT

Real time Data Infra at Uber, CERN on Scaling 600 Clusters, OpenAI Powered Linux Shell, DevSecOps for Machine Learning, Python 3.10 Feature Highlights + more 🚀

The ML Engineer — Issue #122

Sat, 17 Apr 2021 00:00:00 GMT

Exploiting Security ML Pickles, Free AutoML Online Course, Neural Networks in Minecraft, ML Deployment Online Course, Defining DataOps and MLOps + more 🚀

The ML Engineer — Issue #121

Sun, 11 Apr 2021 00:00:00 GMT

MIT Deep Learning Life Sciences, Build Your Voice Cloning App, Data Quality in AI Products, Deep Learning for Cancer Dataset, New Sklearn Features Overview + more 🚀

The ML Engineer — Issue #120

Sun, 04 Apr 2021 00:00:00 GMT

Machine Learning in Healthcare, CPU Architecture Golden Age, Growing Importance of Metadata, Data Quality at Airbnb + more 🚀

The ML Engineer — Issue #119

Sun, 28 Mar 2021 00:00:00 GMT

Andrew Ng deep dive on MLOps, ML & DL Online Courses, Playbook for Model Monitoring, Building a Database from Scratch, Continuous Retraining Strategy + more 🚀

The ML Engineer — Issue #118

Sun, 21 Mar 2021 00:00:00 GMT

Visualizing Timeliness at Airbnb, Video Calls on Steroids w NVIDIA, Break a Prod Model in 20 Days, The Annual 2021 AI Index, The Next Generation of AI + more 🚀

The ML Engineer — Issue #117

Sun, 14 Mar 2021 00:00:00 GMT

Starcraft AI Programming Tutorial, The Netflix Cosmos Platform, Scaling ETL Workers with K8s, Production ML at Booking.com, Kaggle Tricks & Best Practices + more 🚀

The ML Engineer — Issue #116

Sun, 07 Mar 2021 00:00:00 GMT

Scaling Linkedin Experiments, The Rise of Metadata Systems, Data Version Control 2.0 Release, The Graph Neural Net Repository, The Python SpeedSheet Docs + more 🚀

The ML Engineer — Issue #115

Sun, 28 Feb 2021 00:00:00 GMT

Towards Simple Trustworthy AI, Python Developers Survey 2020, SpaCy 3.0 Launch Highlights, Neural Massive Online Multiplayer, Teaching Cars to See at Scale + more 🚀

The ML Engineer — Issue #114

Sun, 21 Feb 2021 00:00:00 GMT

Gen Z and Artificial Intelligence, Security in Prod ML Systems, How To Do All in Computer Vision, Medicine's ML Problem, Top 50 Matplotlib Visualisations + more 🚀

The ML Engineer — Issue #113

Sun, 14 Feb 2021 00:00:00 GMT

Linux Foundation AI Principles, Beyond CUDA Python in GPU, Building a Chess AI Engine, Data in Healthcare & Life Science, ML Reliability Engineering + more 🚀

The ML Engineer — Issue #112

Sun, 07 Feb 2021 00:00:00 GMT

Misinformation: Tech & Solutions, Metadata Management Systems, Papers With Code Datasets, AI & ML Platforms in 2021, AI Regulatory Proposals + more 🚀

The ML Engineer — Issue #111

Sun, 31 Jan 2021 00:00:00 GMT

Top 2020 ML Notebooks, Massive Scale Kubernetes for AI, Tech in 2020 Data Science, Architecture of ML Systems, State of AI Ethics Report 2021 + more 🚀

The ML Engineer — Issue #110

Sun, 24 Jan 2021 00:00:00 GMT

Entire CS Curriculum in Youtube, From MLOps to MLOops, 2020's Top ML Papers, Python Data Science Startups, The "Simplest" NumPy Course + more 🚀

The ML Engineer — Issue #109

Sun, 17 Jan 2021 00:00:00 GMT

Production ML Monitoring at SF, 100+ Free (& OSS) Dev Books, The geopolitics of AI & ML, Building Decent Software Startup, Uber’s Real-Time Push Platform + more 🚀

The ML Engineer — Issue #108

Sun, 10 Jan 2021 00:00:00 GMT

Harvard Introduction to CompSci, ML & DL Compendium 2017-2020, NYU Deep Learning with PyTorch, Python & Jupyter in Excel (Yep), Learning NLP the Practical Way + more 🚀

The ML Engineer — Issue #107

Sun, 03 Jan 2021 00:00:00 GMT

Year Review Papers with Code, Machine Learning going Realtime, An eXplainability toolbox for ML, Simplifying Outlier Detection, Public Engineering Career Ladder + more 🚀

The ML Engineer — Issue #106

Sun, 27 Dec 2020 00:00:00 GMT

We wish happy holidays to all our MLE Newsletter subscribers!!, Top 10 Python Libraries of 2020, 2020 Curated List of AI Research, NumPy Deep Dive Illustrated, Metadata Journey at PayPal, Interactive C++ for Data Science + more 🚀

The ML Engineer — Issue #105

Sun, 20 Dec 2020 00:00:00 GMT

Navigating Resposible AI, The NLP Pytorch Tutorial, Research at Microsoft in 2020, Gentle Intro to Concept Drift, Uber on Data Workflows at Scale + more 🚀

The ML Engineer — Issue #104

Sun, 13 Dec 2020 00:00:00 GMT

End-to-end Production ML Monitoring, Metadata Architectures Explained, Applied ML in Production, State of AI Ethics Panel, FOSDEM 2021 CFP HPC & ML + more 🚀

The ML Engineer — Issue #103

Sun, 06 Dec 2020 00:00:00 GMT

UK Data Strategy Consultation, Applying the MLOps Lifecycle, Break into NLP with Andrew NG, Uber on Scale Data Queries, NLP Applications Podcast + more 🚀

The ML Engineer — Issue #102

Sun, 29 Nov 2020 00:00:00 GMT

AI Diversity of Dev & Apps, High Performance NLP, Facebook on Data Discovery, Netflix on Real Time Batch, ML Street Talk Podcast + more 🚀

The ML Engineer — Issue #101

Sun, 22 Nov 2020 00:00:00 GMT

E2E Production ML Monitoring, Feature Stores Demystified, ML in Compiler Optimization, Explainable AI in Drug Discovery, Challenges in Deploying ML + more 🚀

The ML Engineer — Issue #100

Sun, 15 Nov 2020 00:00:00 GMT

Cross-vendor GPU Python ML, ML Metadata Management Tools, Practical Guide To Responsible AI, Navigating ML Deployment, Stanford MLSys Seminars + more 🚀

The ML Engineer — Issue #99

Sun, 08 Nov 2020 00:00:00 GMT

Definitive AI Monitoring Guide, Computational Limits of DL, Safely Rolling out ML to Prod, Audio ML Infrastructure at Spotify, Stop using k8s for ML (use k8s) + more 🚀

The ML Engineer — Issue #98

Mon, 02 Nov 2020 00:00:00 GMT

Andrew Ng on Production AI, Netflix's Distributed Tracing Infra, Importance of Data in MLOps, Image Outlier Detection in ML, The State of AI Ethics Report + more 🚀

The ML Engineer — Issue #97

Sun, 25 Oct 2020 00:00:00 GMT

Accelerating GPU Workloads, Software Eng for Deep Learning, The Rise of MLOps, 200 Best ML & Python Tutorials, Running 1M+ Batch Jobs in K8s + more 🚀

The ML Engineer — Issue #96

Sun, 18 Oct 2020 00:00:00 GMT

Android Apps with ML on GPU, The Canonical ML Stack, Modern Data Infra Architectures, FB Eng Lead discusses Fairness, AI Ethics - Whose Ethics? Event + more 🚀

The ML Engineer — Issue #95

Sun, 11 Oct 2020 00:00:00 GMT

Real Time ML at Scale, Awful AI Listing Scary Usecases, Scientific Computing with Python, Deep Learning Models Repo, AI for Software Development + more 🚀

The ML Engineer — Issue #94

Sun, 04 Oct 2020 00:00:00 GMT

Distributed Training of ML Models, State of AI Report 2020, Embracing Logging in ML, A Brief History of ML Platforms, Whose Ethics? Eastern + Western + more 🚀

The ML Engineer — Issue #93

Sun, 27 Sep 2020 00:00:00 GMT

GPU Accelerated ML in GameDev, Modular vs E2E ML Platforms, Dagster Data Orchestration, ML for MedTech Monitoring, Whose Ethics? Eastern + Western + more 🚀

The ML Engineer — Issue #92

Sun, 20 Sep 2020 00:00:00 GMT

Data Version Control with Dmitry, Differential Privacy Series, Which GPUs for Deep Learning, Reinforcement Learning Pathmind, Explainable AI Monitoring + more 🚀

The ML Engineer — Issue #91

Sun, 13 Sep 2020 00:00:00 GMT

Machine learning in cross-vendor & mobile GPUs made simple, AI, Data & Ethics Research, Detecting Shifts in Policy with ML, AI Enabled Code Compilers, AI O'Reilly 2020 Trends to Watch + more 🚀

The ML Engineer — Issue #90

Sun, 06 Sep 2020 00:00:00 GMT

AI, Data & Ethics Research, AI Summarize Human Feedback, LipSync DeepFake AI Improved, Getting started with ML resource, AI Monitoring & Assurance + more 🚀

The ML Engineer — Issue #89

Sun, 30 Aug 2020 00:00:00 GMT

FastAI Practical Data Ethics, Building Conversational AI Apps, Computational Learning Intro, Topic Modelling with Gensim, Optical Character Recog for All + more 🚀

The ML Engineer — Issue #88

Sun, 30 Aug 2020 00:00:00 GMT

FastAI New Course, Libs & Book, Massive Scaling Google Meets, DeepFakes Threat Report, AI Developer Tools Landscape, Beginner to Prof. Dev with Python + more 🚀

The ML Engineer — Issue #87

Sun, 16 Aug 2020 00:00:00 GMT

Prod Model Server Features, Computational Causal Inference, Practical AI's 100th Episode, Realistic Tennis AI w Vid2Player, New Jupyter Book Launch + more 🚀

The ML Engineer — Issue #86

Sun, 09 Aug 2020 00:00:00 GMT

Innovation, Regulation and AI, The Role of AI Product Managers, Data Observability in Production, Graph Algorithms in Industry, Open RL Benchmark 0.3.0 + more 🚀

The ML Engineer — Issue #85

Sun, 02 Aug 2020 00:00:00 GMT

Experts Recommend ML Books, Why NLP Beyond English, Kubeflow ML Prod Workflow, Configuring Cross-Validation, Philosophers on GPT-3 (feat. AI) + more 🚀

The ML Engineer — Issue #84

Sun, 26 Jul 2020 00:00:00 GMT

Netflix On Cutting Data Costs, Awesome Github GPT-3 List, Lessons from ML Prod Monitoring, Airflow Summit Videos are Out, Uber on Editing Massive GeoData + more 🚀

The ML Engineer — Issue #83

Sun, 19 Jul 2020 00:00:00 GMT

Building an Enterprise DL Stack, 5 Key Features for ML Platforms, AI Dungeon Open World w GPT3, The State of Apache Airflow, D2IQ KUDO for Kubeflow + more 🚀

The ML Engineer — Issue #82

Mon, 13 Jul 2020 00:00:00 GMT

Full Stack Deep Learning Course, Software Engineers in ML, Web Services vs Streaming in ML, Continuous ML (CML) CI/CD, Papers With Code Methods + more 🚀

The ML Engineer — Issue #81

Mon, 06 Jul 2020 00:00:00 GMT

Getting ML into Production, Top Books on ML Feature Eng, MSFT Adversarial ML in Industry, Google on Neural Nets for Tables, Getting into a Causal Flow + more 🚀

The ML Engineer — Issue #80

Sun, 28 Jun 2020 00:00:00 GMT

MLflow Joins Linux Foundation, DVC 1.0 features for MLOps, Designing Industrial Scale ML, Machine Learning Operations + more 🚀

The ML Engineer — Issue #79

Sun, 21 Jun 2020 00:00:00 GMT

Model Serving Ecosystem, GitHub Actions for MLOps, Building OSS Tools for NLP Devs, Reinforcement Learning Apps, NLP Transfer Learning at Scale + more 🚀

The ML Engineer — Issue #78

Sun, 14 Jun 2020 00:00:00 GMT

Outlier & Anomaly Detection ML, The State of ML in Python 2020, Applied Homomorphic Encryption, OpenAI NLP API Beta Launch, Continuous Delivery Podcast + more 🚀

The ML Engineer — Issue #77

Sun, 07 Jun 2020 00:00:00 GMT

Made with ML Platform, Identifying & Mitigating AI Risks, ACM ByteCast with Donald Knuth, Microsoft NLP Bias Research, Feature Selection with Cont. Data + more 🚀

The ML Engineer — Issue #76

Sun, 31 May 2020 00:00:00 GMT

Highlights on EuroPython & ACM, ML in Prod Deployment Guide, Frameworks used by ML Startups, GPT-3 Deep Dive Explanation, Scaling Data with Outliers for ML + more 🚀

The ML Engineer — Issue #75

Sun, 24 May 2020 00:00:00 GMT

Microsoft Programming AI, ML Infra for Model Building, Discourse Rethinking Public Data, Advanced NLP Video Course, What to Do When AI Fails + more 🚀

The ML Engineer — Issue #74

Sun, 17 May 2020 00:00:00 GMT

Coding Habits for Data Scientists, Enterprise AI Adoption 2020, Natural Language Processing 101, AI Scalability & Performance, MLOps is Not Enough + more 🚀

The ML Engineer — Issue #73

Sun, 10 May 2020 00:00:00 GMT

Real Time ML Stream Processing, Statement on Contact Tracing, ICLR 2020 Videos Released, Why TinyML will be Huge, PapersWithCode: A home for ML + more 🚀

The ML Engineer — Issue #72

Sun, 03 May 2020 00:00:00 GMT

Monitoring ML Models in Prod, 65 Free Springer ML Books, Neural Network Music Generator, Open Source Deep Learning, AI, COVID19 & Contact Tracing + more 🚀

The ML Engineer — Issue #71

Sun, 26 Apr 2020 00:00:00 GMT

A Practical Intro to Responsible AI, Simulating Real World in Python, Advanced NLP with SpaCy, 500 Free CompSci Courses, Modelling & Simulating Epidemics + more 🚀

The ML Engineer — Issue #70

Sun, 19 Apr 2020 00:00:00 GMT

Privacy Preserving AI Lecture, Backpropagation 101 from Thinc, Harvard Offering Free Courses, GPT2 AI Dungeon Game Update + more 🚀

The ML Engineer — Issue #69

Sun, 12 Apr 2020 00:00:00 GMT

Insights for Remote ML Teams, Human-in-the-loop in Prod ML, Netflix & Druid for Real Time Data, The Importance of Data Prep + more 🚀

The ML Engineer — Issue #68

Sun, 05 Apr 2020 00:00:00 GMT

GitLab Data Lessons Learned, Data Discovery at Spotify, Exploratory Data Analysis Dive, Tokenisers & How Machines Read, Intel Demystifying the AI Stack + more 🚀

The ML Engineer — Issue #67

Sun, 29 Mar 2020 00:00:00 GMT

COVID-19 AI solutions at scale, Democratising Deep Fakes 😬, Shopify on Scaling AI, Industry Reinforcement Learning, Transfer Learning in NLP + more 🚀

The ML Engineer — Issue #66

Sun, 22 Mar 2020 00:00:00 GMT

PyTorch ML from Scratch, The MLOps References List, Deep Learning & Info Retrieval, Integrating SHAP Explainability, AI meets operations with OReilly + more 🚀

The ML Engineer — Issue #65

Sun, 15 Mar 2020 00:00:00 GMT

Explainability, Security & MLOps, Python Machine Learning Books, DevOps in Machine Learning, A Tour on E2E ML Platforms, Adversarial ML Reading List + more 🚀

The ML Engineer — Issue #64

Sun, 08 Mar 2020 00:00:00 GMT

Kubernetes ML for Everyone, Production-Ready ML Systems, Explaining Long Term ML Impact, Quantifying Reproducibility of ML, Adversarial Examples Resource + more 🚀

The ML Engineer — Issue #63

Sun, 01 Mar 2020 00:00:00 GMT

Building Blocks of Interpretability, Ethics in AI and Big Data Course, Reviewing Emotional Expressions, Model Explainability for Business, Microsoft's Data Science Process + more 🚀

The ML Engineer — Issue #62

Sun, 23 Feb 2020 00:00:00 GMT

Jurgen's Retrospective AI 2010s, Hyperconnected Missinformation, MLOps: The End of End-to-End, How to Interpret an ML Model, Empirical Quality Metrics for DL + more 🚀

The ML Engineer — Issue #61

Sun, 16 Feb 2020 00:00:00 GMT

Microsoft's NLP Recipes, Messaging & Data Ingestion++, Why Imbalanced ML is so hard, AI for Data Cleaning at Scale, Training Models with 1b+ Params + more 🚀

The ML Engineer — Issue #60

Mon, 10 Feb 2020 00:00:00 GMT

Hands on MLOps for AI at Scale, Why ML Degrades in Production, Kaggle Kernel on Interpretability, Building Domain Specific NLP, Bayesian Product Raking Wayfair + more 🚀

The ML Engineer — Issue #59

Mon, 03 Feb 2020 00:00:00 GMT

Table Detection & NLP with DL, Towards general conv. agent, State of privacy preserving ML, Distributed Delayed Job Queueing, Applying confidence models + more 🚀

The ML Engineer — Issue #58

Mon, 27 Jan 2020 00:00:00 GMT

Feature Stores for ML, Key AI & Data Trends for 2020, LF AI 2019 Year in Review, From local to global XAI, Sampling methods for imbalances + more 🚀

The ML Engineer — Issue #57

Sun, 19 Jan 2020 00:00:00 GMT

Google Research 2019 + Beyond, Facebook OSS Year in Review, AI Lessons Learned with Rakuten, Move fast and break things w AI, Intro to Ethics in AI + more 🚀

The ML Engineer — Issue #56

Sun, 12 Jan 2020 00:00:00 GMT

Evolution of ML Infrastructure, 30 Woman Advancing AI, Calculating the Value of Data, Intro to Ethics in AI, A Guide to File Formats in ML + more 🚀

The ML Engineer — Issue #55

Mon, 06 Jan 2020 00:00:00 GMT

Machine Learning System Design, Machine Learning Interviews, A Deep Dive into Online Learning, Unsupervised NLU via GPT-2, Open Source Business Models + more 🚀

The ML Engineer — Issue #54

Sun, 29 Dec 2019 00:00:00 GMT

Yoshua Bengio; Towards system 2, AI Index 2019 Report, Microsoft's NLP Best Practices, The day that changed Netflix tech, Attention and Augmented RNNs + more 🚀

The ML Engineer — Issue #53

Sun, 22 Dec 2019 00:00:00 GMT

Real time computer vision at scale, Ray for the curious, Evolution of Zulily’s Airflow, Key trends in ML for 2020, A gentle intro to imbalanced ML + more 🚀

The ML Engineer — Issue #52

Sun, 15 Dec 2019 00:00:00 GMT

Top Python ML Libraries in 2019, NeurIPS 2019 Videos are Out, Modern NLP with SpaCy Podcast, Testing Guide for Software, Spotify on Better ML Infrastructure + more 🚀

The ML Engineer — Issue #51

Sun, 08 Dec 2019 00:00:00 GMT

Play Endless Game Built by AI, Gentle Intro to Model Selection, Code Reviews for Jupyter NBs, Netflix Releases Metaflow, Adversarial Detection Hands On + more 🚀

The ML Engineer — Issue #50

Sun, 01 Dec 2019 00:00:00 GMT

Data Science Best Practices, A Contract for the Web, Deep Learning Indaba 2019, Uncertainty Quantification in DL, Google XAI Whitepaper + more 🚀

The ML Engineer — Issue #49

Sun, 24 Nov 2019 00:00:00 GMT

Outlier & Adversarial Detector, The AI Governance Dilemma, Contextually Keyed Word Vectors, Time Series Anomaly Detection, Pyro 1.0 Released + more 🚀

The ML Engineer — Issue #48

Sun, 17 Nov 2019 00:00:00 GMT

ONNX Joins the Linux Foundation, The Nuances in DevOps for ML, Continuous Delivery for ML, Learnings reaching 2% in Kaggle, The New Data Exchange + more 🚀

The ML Engineer — Issue #47

Mon, 11 Nov 2019 00:00:00 GMT

E2E ML with MLFlow and Seldon, Reconstructing thoughts with ML, Scalable AutoML with Ray, Tensorflow World Videos, 14 types of learning in ML + more 🚀

The ML Engineer — Issue #46

Mon, 04 Nov 2019 00:00:00 GMT

6 lessons learned at Booking.com, EurNLP 2019 videos released, Consistency of AI Summarization, Linux Foundation Trusted AI, AI Ethics - Whose Ethics? + more 🚀

The ML Engineer — Issue #45

Mon, 28 Oct 2019 00:00:00 GMT

Deep fake detection challenge, Human knowledge to improve AI, Neural text search data flow, The Causal Inference Book, Netflix Open Sources Polynote + more 🚀

The ML Engineer — Issue #44

Sun, 20 Oct 2019 00:00:00 GMT

Awesome AI Guidelines List, MLFlow simplifying model mgmt, Choosing intuitive visualisations, ML Explainability at AI O'Reilly, Machine learning in 6 steps + more 🚀

The ML Engineer — Issue #43

Sun, 13 Oct 2019 00:00:00 GMT

IEML joins the Linux Foundation, Case studies with NumFocus, ML for business & ops intelligence, The open FairML Book, AI Ethics - whose ethics? + more 🚀

The ML Engineer — Issue #42

Sun, 06 Oct 2019 00:00:00 GMT

Serverless for ML in Kubernetes, When a model is too big for prod, Optimising Prod ML at Apple, Turn your ML into interactive apps, Modern Applications at AWS + more 🚀

The ML Engineer — Issue #41

Mon, 30 Sep 2019 00:00:00 GMT

One data engine to rule them all, The ImageNet for Code, Tackling data processing at scale, Wisdom from debugging at scale, Netflix reimagining experiments + more 🚀

The ML Engineer — Issue #40

Mon, 23 Sep 2019 00:00:00 GMT

Tricking ML Classifiers, ML deployment paradigm, Five must-know graph algorithms, Survey fairness and bias in ML, Google's OSS differential privacy + more 🚀

The ML Engineer — Issue #39

Sun, 15 Sep 2019 00:00:00 GMT

Management for Data Science, Selection vs Detection of outliers, Rules for sharing notebooks, AutoML and AI at Google, 5 sampling algos for everyone + more 🚀

The ML Engineer — Issue #38

Sun, 08 Sep 2019 00:00:00 GMT

Continuous Delivery for ML, AIOps and why you should care, Lang models as knowledge bases, AWS data security best practices, A smooth approach to prod ML + more 🚀

The ML Engineer — Issue #37

Sun, 01 Sep 2019 00:00:00 GMT

Real Time NLP: Spacy and Kafka, Becoming an ML practitioner, Cracking the black box (XAI), Notebook innovation at Netflix, How AI solves scale complexities + more 🚀

The ML Engineer — Issue #36

Sun, 25 Aug 2019 00:00:00 GMT

The state of Federated Learning, Data Science Best Practices, Progression of a Data Scientist, Observability 3 year retrospective, Intro to transformer architecture + more 🚀

The ML Engineer — Issue #35

Sun, 18 Aug 2019 00:00:00 GMT

The future of Data Engineering, From self study to ML Engineering, 12 NLP Researchers to Follow, N-Shot Learning with Small Data, Causal Inference: Counterfactuals + more 🚀

The ML Engineer — Issue #34

Sun, 11 Aug 2019 00:00:00 GMT

A survey on the state of AutoML, Got speech? Voice Applications, Cloud native semantic text search, Learning from adversaries, Python-compatible spreadsheets + more 🚀

The ML Engineer — Issue #33

Sun, 04 Aug 2019 00:00:00 GMT

Brooklin for data streaming, Tensorflow AI Interpretability, LIDAR and its smart applications, All hail the (AI) algorithm, Machines (and AI) Gone Wrong + more 🚀

The ML Engineer — Issue #32

Sun, 28 Jul 2019 00:00:00 GMT

E2e ML Pipelines in Enterprise, Code-free deep learning Ludwig, ML Reidentification and Privacy, OSS + AI will take us to the Moon, Large Scale Distributed Systems + more 🚀

The ML Engineer — Issue #31

Sun, 21 Jul 2019 00:00:00 GMT

End-to-end XAI in production, Causal inference to improve UX, Managing ML in enterprise, Intro to Adversarial Examples, The GAN Story so far + more 🚀

The ML Engineer — Issue #30

Sun, 14 Jul 2019 00:00:00 GMT

Production-level ML Explainers, AI Explanations w Counterfactuals, Privacy & Cybersecurity Merging, Hightlights of AI O'Reilly Beijing, 18 Impressive GANs Applications + more 🚀

The ML Engineer — Issue #29

Sun, 07 Jul 2019 00:00:00 GMT

Major trends in AI & Data, Production-level AI Explanations, 85% of Big Data Projects Fail, ML Mastery on building GANs, Question-answering AI in K8s + more 🚀

The ML Engineer — Issue #28

Mon, 01 Jul 2019 00:00:00 GMT

The state of AI in 2019, Production ML in 2019, Model governance and ops, The best of modern NLP, Adversarial examples with FGSM + more 🚀

The ML Engineer — Issue #27

Sun, 23 Jun 2019 00:00:00 GMT

Distributed AI made easy w Ray, Model Interpretation with Alibi, Principled Machine Learning, Comparing Time Series Models, The quest for high-quality data + more 🚀

The ML Engineer — Issue #26

Sun, 16 Jun 2019 00:00:00 GMT

PyTorch Hub + Reproducible ML, Privacy-preserving AI free course, MLFlow for pipeline management, E2E NLP Pipelines with Kubeflow, The Brains behind SpaCy + more 🚀

The ML Engineer — Issue #25

Mon, 03 Jun 2019 00:00:00 GMT

Google Research on MLOps, The Book on AutoML, Deep Learning for face detection, Maintainable ETL Pipelines, Counterfactuals for Explainable AI, The Semi-Supervised Revolution + more 🚀

The ML Engineer — Issue #24

Sun, 26 May 2019 00:00:00 GMT

Standford's Deep NLP Course, The Data Orchestration Layer, The Illustrated Transformer, People plus AI Guidebook, Build a reproducible ML Pipeline, GANs in Action Book + more 🚀

The ML Engineer — Issue #23

Sun, 19 May 2019 00:00:00 GMT

Best practices for ML Engineering, The Journal of Open Source, Visualising Attention in Deep NLP, Pruning API in Tensorflow, ML Explanations with VIBI, Two missing links in serverless + more 🚀

The ML Engineer — Issue #22

Sun, 12 May 2019 00:00:00 GMT

A conversation on practical NLP, "I don't like notebooks" @ ICLR19, A Berkeley view on serverless, Face detection in Python OpenCV, Human-Centric ML Infrastructure, A tutorial on Convolutional NNs + more 🚀

The ML Engineer — Issue #21

Mon, 06 May 2019 00:00:00 GMT

Alibi for black box explanations, Karpathy's tips on training NNs, Nando on learning to learning, A gentle intro to ImageNet, SparkML Kafka Environment, An introduction to computer vision + more 🚀

The ML Engineer — Issue #20

Mon, 29 Apr 2019 00:00:00 GMT

An R book for programmers, Real time ML with Kafka & Spark, Deltalake DB & data lake layer, Amunsen data discovery engine, DAWN Machine Learning Tools, Serverless operator reuse + more 🚀

The ML Engineer — Issue #19

Mon, 22 Apr 2019 00:00:00 GMT

Advanced NLP with SpaCy, Grid search across scikit models, A chat on ML version control, Beyond black holes with the code, Scientific python in the browser, XAI v0.0.5 released into the wild + more 🚀

The ML Engineer — Issue #18

Sun, 14 Apr 2019 00:00:00 GMT

Reducing Bias in Bios, Data Augmentation for Images, Common statistical tests, AutoML with code generation, Strata Data San Fran Highlights, Stackoverflow Developer Survey + more 🚀

The ML Engineer — Issue #17

Sun, 07 Apr 2019 00:00:00 GMT

The SpaCy universe of resources, Preprocessing images with Keras, Calling out statistical significance, ✨ Introducing Plotly Express ✨, Time series with TF Probability, Massive Multi-Task Learning + more 🚀

The ML Engineer — Issue #16

Sun, 31 Mar 2019 00:00:00 GMT

🥳🎉We've reached 1000🎈🎆, 1. Real time ML big data streams, 2. Test Driven Development in AI, 3. Machine Learning Landscape, 4. Detecting ouliers & anomalies, 5. Checklist to debug neural nets, 6. Evaluating model performance + more 🚀

The ML Engineer — Issue #15

Sun, 24 Mar 2019 00:00:00 GMT

AI debiasing doesn't debias bias, The NLP of human noises, Feature visualisation via activation, The GAN stroke of genious, Essential NLP Tools, Code & Tips, AI comedy generated by humans + more 🚀

The ML Engineer — Issue #14

Sun, 17 Mar 2019 00:00:00 GMT

Domain knowledge in DL, Federated learning in Tensorflow, Live ggplots for your twitter rants, Evolving Jupyter in the Lab, 8 Books for Computer Vision, Qrash Course on RL, Introducing data labelling section + more 🚀

The ML Engineer — Issue #13

Sun, 10 Mar 2019 00:00:00 GMT

A deep dive on ML versioning, How to become an ML engineer, Developing competence in DL, 🚀 beyond Jupyter with Jupytext, Tensorflow 2.0 Alpha is out, Adversarial drawing with GANs + more 🚀

The ML Engineer — Issue #12

Sun, 03 Mar 2019 00:00:00 GMT

Bias and Explainability in ML, Federated Learning with PyTorch, Data Visualisation Deep Dive, Practical recommenders tutorial, Learning Curves in ML evaluation, Machine Learning Cybersecurity + more 🚀

The ML Engineer — Issue #11

Sun, 24 Feb 2019 00:00:00 GMT

Top Sources for ML Datasets, Major RL Papers in 2018, Get Better Deep Learning Results, A beginners intro to deep NLP, This AirBnB does not exist, Feature flags in Machine Learning + more 🚀

The ML Engineer — Issue #10

Sun, 17 Feb 2019 00:00:00 GMT

Delayed impact of FAIR ML, Getting started with Google Collab, Data Science Salaries in Europe, Building tensorflow from scratch, Better language models, AI where? In the blockchain 💰 + more 🚀

The ML Engineer — Issue #9

Sun, 10 Feb 2019 00:00:00 GMT

Tools to scale your production ML, Outlier Detection by ML for ML, How Facebook scales their ML, Transfer Learning to improve DL, Towards Federated Learning, Quantum ML Online Course + more 🚀

The ML Engineer — Issue #8

Sat, 02 Feb 2019 00:00:00 GMT

Papers with code update, NVIDIA's AI generated graphics, Serverless and machine learning, Tensorflow 2.0 APIs, 16k research paper analysis, Google Brain Research in 2018 + more 🚀

The ML Engineer — Issue #7

Sun, 27 Jan 2019 00:00:00 GMT

AI vs Human - Starcraft II Edition, From zero to distributed, Tensorflow 2.0, I heard you like machine learning, Impact of Learning Rate, Monitoring Parking with R-CNN + more 🚀

The ML Engineer — Issue #6

Sun, 20 Jan 2019 00:00:00 GMT

Machine Learning Michaelangelo, The new role of data engineers, Why data scientists love MLOps, Seeing theory: Probability & stats, Data Science vs Engineering, Accelerate ML with Batch Norm + more 🚀

The ML Engineer — Issue #5

Sun, 13 Jan 2019 00:00:00 GMT

XAI - eXplainability library for AI, Deep neural inspection, AI against alzheimer's disease, Differential privacy for tensorflow, The role of ML in databases + more 🚀

The ML Engineer — Issue #4

Sun, 06 Jan 2019 00:00:00 GMT

Andrew Ng and Reproducibility, 2018 machine learning nostalgia, Magic of Feature Engineering, Explainability/bias with tensorflow, Ensembles and more ensembles, CI/CD for Machine Learning + more 🚀

The ML Engineer — Issue #3

Sun, 30 Dec 2018 00:00:00 GMT

Extreme ML with Kafka, Programming Explainable ML, Computer Vision Everywhere, Beyond Accuracy with ROCs, Super-SlowMo Generators, EC on Trustworthy AI + more 🚀

The ML Engineer — Issue #2

Sun, 23 Dec 2018 00:00:00 GMT

Towards pandas 1.0, Top 50 matplotlib visualisations, Papers with code, An introduction to kubeflow, Style-based Face Generators, Adversarial debugging for NLP + more 🚀

The ML Engineer — Issue #1

Sun, 16 Dec 2018 00:00:00 GMT

Combat Imbalanced Classes, Compliant machine learning, Data science career transitions, Curiousity driven data science, Automate boring tasks, Facebook open sources PyText + more 🚀

The ML Engineer — Newsletter archive

The ML Engineer — Issue #399

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #398

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #397

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #396

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #395

This week in ML Engineering:

Model’s Pareto with Raschka and Databricks

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #394

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #393

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #392

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #391

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

About us

The ML Engineer — Issue #390

This week in ML Engineering:

Upcoming MLOps Events

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below: