Thank you for being part of over 70,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter /newsletter/ ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Mistral on Voxtral 3 Voice Agents

Europe scores another win with Mistral's latest Voice Foundation Model release Voxtral 3 🇪🇺🚀🇫🇷 This is their family of speech models + a voice agent SDK!

Mistral published a score of 4.1% error rate across 45 languages vs 16.0% error for Whisper large-v3. The voice agent API costs $0.04 per minute with Mistral Small 4 + self-hosting.

They report that their agent pipeline keeps each stage as a separate model, with "Mini Transcribe Realtime 3" as a listener, Mistral Small 4 for reasoning, and Voxtral TTS for speech - super interesting to see the ecosystem coming together.

It is great to see a European lab shipping the whole voice agent stack! As an active OpenAI Whisper (OSS) user, this is fantastic to hear, and a great trend direction.

OpenAI on Containing Frontier Training

Following the OpenAI's recent security incident and now a new incident during training, they have now published how they locked down their frontier training environments, and how they red-teamed them with their own agents.

As a reminder, back on September 20 OpenAI had an internal research model which exploited a misconfigured DNS resolver to get limited access to the live internet during a training run resulting in various exploits reported.

As part of this, they paused the training of their most capable models until October 1, and frontier training now runs with no internet access at all, using an indexed offline copy of the web for browsing tasks.

It is interesting to see how frontier models need to evolve their approach to testing. They ran teams of frontier-model agents against copies of their own infrastructure, handed them the source code and config, and burned through hundreds of billions of tokens.

The agents found microVM escapes that led to upstream fixes in Kata Containers, gVisor and Moby, plus a few kernel CVEs. They also found paths to the internet through almost every narrow channel left open (metrics, logging, auth), and covert channels between microVMs, including a CPU cache-timing channel where the agents agreed on an encoding by themselves!

OpenAI says the findings dropped significantly as the campaign went on, but they expect future models to find holes they don't know about yet. For production ML practitioners this is a good reminder that agent sandboxes now need to be red-teamed by agents too, as the attack surface grows with the capability of the model you run.

Agents testing agents all around!

Uber built an agent to automate end to end workflows across their Legal teams - Build vs Buy is getting a whole new definition! They now have agentic capabilities to review counterparty edits on contracts and make recommendtions.

Their first version was plain RAG over the legal playbooks, and it got both the clause matching and the tone wrong (too defensive in some cases, too positive in others).

They fixed it with a feedback loop that stores every lawyer decision together with the original text, the redline and the final wording. At runtime a similarity search narrows thousands of past negotiations down to about 20 examples, and a second LLM checks that the intent actually matches; there is no fine-tuning involved.

Non-negotiable / high risk cases go through a deterministic rules engine that overrides the model, and older feedback is down-weighted with a 365-day half-life to deal with drift.

Uber claims a 20%+ cut in contract review time and 91% decision accuracy, although the post doesn't say how accuracy was measured. I have to say the most practical lesson is that the lawyers write the prompt content and the engineers own its structure, which applies well beyond legal.

OWASP GenAI Red Teaming Manual

The OWASP Foundation released 144 pages on how to actually red-team an AI agent. Agentic security now has a proper methodology! Here's the OWASP GenAI Security Project Red Teaming Manual Version 1.0:

This is one of the most comprehensive resources on agentic red teaming, covering from planning / reconnaissance all the way through to exploitation, persistence and reporting. Here you can see the mapping of every test to the OWASP Agentic Top 10.

The agentic section is the most useful part for practitioners, with concrete steps for MCP tool poisoning and "rug pull" definition drift, malicious Agent Skills, permission aggregation across subagents, and memory drift after context compaction.

There is also a lot of solid guidance on measurement. They ask teams to report attack success rate together with Success@N and the attacker budget, and to record a missing trace as a finding and never as a defensive pass.

Every confirmed exploit should produce a paired red and blue artifact, so the attack ends up as a detection rule and a regression test in CI. It also comes with a companion lab repo of deliberately vulnerable sandboxes, definitely worth checking out if you are running agents in production!

NVIDIA Brings CUDA to Rust

NVIDIA is bringing native Rust to GPU kernels. They announced CUDA Rust + two open source tracks where they plan to bring compile-time memory safety into GPU kernels.

The cuda-oxide track follows the classic SIMT model through a custom rustc backend that compiles down to PTX. The cutile-rs track works on tiles, and it is already on crates.io and used by Hugging Face's Grout and mistral.rs.

I found super interesting the safety side! The borrow checker catches aliasing between kernel inputs and outputs at compile time, and launches are checked against the limits each kernel declares.

NVIDIA is clear that both tracks are early and not production-ready, and there are no performance numbers yet - but this is the right time to try it out and give feedback!

Upcoming MLOps Events

The MLOps ecosystem continues to grow at break-neck speeds, making it ever harder for us as practitioners to stay up to date with relevant developments. A fantsatic way to keep on-top of relevant resources is through the great community and events that the MLOps and Production ML ecosystem offers. This is the reason why we have started curating a list of upcoming events in the space, which are outlined below.

You can also find our upcoming events and past talk recordings on the talks and events page.

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

Check out the fast-growing ecosystem of production ML tools & frameworks at the github repository which has reached over 20,000 ⭐ github stars. We are currently looking for more libraries to add - if you know of any that are not listed, please let us know or feel free to add a PR. Here's a few featured open source libraries that we maintain:

  • SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
  • KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
  • Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
  • Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
  • AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
  • Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin