THE INSTITUTE FOR ETHICAL AIALIGNMENT + SAFETY
JOIN / CONTACT

THE ML ENGINEER — WEEKLY NEWSLETTER

The MachineLearning EngineerIssue #399

Join 70k+ AI professionals receiving weekly curatedarticles, tutorials and blog posts onproduction machine learning.

#398
Issue #39909/08/26llmsai-agentsml-security
Issue #399 🤖 - The New Institute for AI, Multi-Tenant Multi-Tier Memory for AI Agents, LLMs Reward Expertise, Qwen3.8-Max and Open Weights, Harness Design for Long-Running Apps, Mistral's Shieldstral Safety Classifier + more 🚀

The New Institute

We are excited to announce the new face of the Institute for Ethical AI Alignment & Safety!

We have rebuilt the Institute website from the ground up, together with some of our key initiatives.

Since our founding in 2017, we have built a track record of contributions across public and private institutions.

Our work runs from individual practice to national regulation; by principle, by process, by standards, by regulation.

Several of our recommendations have been adopted across the EU and UK policy. Our mission has not changed since 2017.

We look forward to continue contributing to a future where frontier AI is safe, aligned and accountable to people and society.

This week in ML Engineering:

Multi-Tenant Multi-Tier Memory for AI Agents

Our 4-part series on agent memory has been published & featured in the front-page of HackerNoon homepage as a top story! LLMs are stateless by design, so without a memory layer every session starts from zero, and the number of dedicated memory tools has been growing almost daily. This came out of my recent work extending the Kubernetes Agent Orchestration System (KAOS) to support multi-tiered memory persistence (aka short-, medium- and long-term memory). Along the way I hit most of the same issues that anyone would when building or integrating multi-tiered memory into an agentic system, so I thought it would be useful to compile all the learnings, design choices and examples. Check it out, together with the rest of the series!

LLMs Reward Expertise

Do language models still reward expertise, or has that ship sailed? Sean Goedecke argues for the first; the distinction is between getting something usable out of a model and extracting the maximum value from it. A non-expert can get sort-of-okay Python, but only someone who knows what a good answer looks like can evaluate the output critically. His main evidence is Terence Tao’s published ChatGPT conversation on the Jacobian Conjecture, where Tao’s messages are short and to the point, and the model answers in a talking-to-mathematicians register rather than an explaining-to-amateurs one. Tao pushes back and he almost never takes the model’s advice. He is careful with the caveat that non-experts still get real value, and that OpenAI had a team of expert mathematicians filtering the suggestions, a step you cannot currently skip. For production ML practitioners the takeaway is that knowledge is still more improtant than ever, and now is even becoming the bottleneck; does this mean we need to accelerate our learning?

Qwen3.8-Max and Open Weights

Qwen3.8-Max has been released! And for the first time they say a Qwen-Max-class model will get open weights: the model scales to 2.4 trillion parameters with 95B active, is built on the Qwen 3.5 architecture, and is available through QwenCloud now with the weights promised next week. The more interesting part is actually the long-horizon runs, as they report a ten day autonomous coding run building the oh-my-cli project, which after roughly 16 days of fully autonomous operation had accumulated 265 commits, 127 PRs and 151 issues! Whether that is good or bad work is yet to be seen… Open weights at this scale would be quite something; let’s see what actually lands next week.

Harness Design for Long-Running Apps

Anthropic has written up the harness they use to have Claude build entire applications over multi-hour runs, and more interestingly what they deleted from it as the model improved: The architecture is three agents framed as a separation of generator and judge, with a planner that expands a one to four sentence prompt into a full product spec, a generator that implements against it, and an evaluator that drives the running app through Playwright MCP like a real user, checking UI, API endpoints and database state. It is also quite a well timed piece as it comes next to their piece on how they contain Claude across products, where filesystem and egress boundaries are what holds once the model-layer defences fail.

Mistral’s Shieldstral Safety Classifier

Europe strikes again! Mistral has released Shieldstral, an open-weights multimodal safety classifier under Apache 2.0: Shieldstral is a 3B parameter model covering text and images, it runs on a single 16GB NVIDIA GPU, and the weights are on Hugging Face. It is interesting how they are proposing to reframe moderation as binary question answering rather than fixed label classification. Basically a plain-language yes/no question and the document being judged as the three inputs; the output is a calibrated probability read off a single token, so a policy change means rewriting the question instead of retraining. They claim it outperforms models up to 7x its size and claim a new state of the art on multimodal safety, although the post does not publish per-benchmark numbers to check that against. One classifier instead of a guardrail model per deployment is a good direction, and great to see it shipped under a permissive licence!

Upcoming MLOps Events

The MLOps ecosystem continues to grow at break-neck speeds, making it ever harder for us as practitioners to stay up to date with relevant developments. A fantsatic way to keep on-top of relevant resources is through the great community and events that the MLOps and Production ML ecosystem offers. This is the reason why we have started curating a list of upcoming events in the space, which are outlined below.

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

Check out the fast-growing ecosystem of production ML tools & frameworks at the github repository which has reached over 20,000 ⭐ github stars. We are currently looking for more libraries to add - if you know of any that are not listed, please let us know or feel free to add a PR. Here’s a few featured open source libraries that we maintain:

  • SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
  • KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
  • Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
  • Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
  • AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
  • Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

Enjoy The ML Engineer? Subscribe to receive the next issue.