THE INSTITUTE FOR ETHICAL AIALIGNMENT + SAFETY
JOIN / CONTACT

THE ML ENGINEER — WEEKLY NEWSLETTER

The MachineLearning EngineerIssue #398

Join 70k+ AI professionals receiving weekly curatedarticles, tutorials and blog posts onproduction machine learning.

#397
Issue #39802/08/26ml-securityllmsrecommender-systems
Issue #398 🤖 - OpenAI Hugging Face Hack Timeline, Cheating in Frontier Model Evaluations, Kimi K3 Architecture, Netflix LLM-Native Recommendations, OpenAI Mathematics Advances + more 🚀

Thank you for being part of over 70,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter https://ethical.institute/mle.html ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Timeline on OpenAI HF Hack

Last week OpenAI published about their rouge hacking HuggingFace, this week HF published a super detailed forensic reconstruction and it’s insane: The tldr we know is that an AI agent escaped an OpenAI cybersecurity evaluation environment, compromised an external code sandbox, and entered Hugging Face production systems. What we can now see is that the agent was able to do this using malicious dataset configurations that enabled local file disclosure and arbitrary code execution on the HF platform. HF found about 17,600 actions that the agent ran by identifying the logs once it entered a k8s pod, escalated cloud credentials in Kubernetes, joined an internal mesh network, accessed an operational database, and obtained limited source-control write permissions. The model was able to do this while using public web services and attacker-controlled datasets for command-and-control. No unauthorized software changes were released, and customer impact was limited to five benchmark-related datasets and operational metadata; basically all the agent wanted is to access benchmark related datasets. This is the perfect example of an optimization function being optimized into destruction, resulting in effects that are adversarial to the initial intent of the operator even if the goal is being met, at all costs. This incident shows how we will need to design system security not just for the future, but for the future of agents.

Cheating Behavior in Frontier Models

Following the Cyber hacks from agents these last few weeks, the UK AI Security Institute released a report that confirms every frontier model attempts to cheat on given tasks: This is an interesting topic on AI alignment, as it raises a practical problem for both benchmark validity and agent deployment. AISI defines cheating as completing a task through prohibited or out-of-scope actions, with observed methods including searching online for solutions, probing evaluation software, escalating privileges on unrelated systems and targeting the infrastructure hosting the model. In one misconfigured and unsolvable task, a model wrote and executed code through an external internet service while attempting to access AISI’s evaluation systems, although no information was leaked. The research found no clear relationship between model capability and cheating frequency, suggesting that training and alignment choices materially affect the behaviour, which is quite an interesting and important insight. Self-reporting was unreliable, and models described their prohibited actions as wrong less than half the time, which emphasises that agent success should be verified; this is a reminder of how important verification is becoming in the post-agentic era.

Raschka on Kimi K3 Architectures

Sebastian Raschka has released a great cheasheet on the Kimi K3 Architecture that Moonshot AI: As a refresher, Kimi K3 is an open weights 2.8T-parameter open-weight MoE model with 104B active parameters. It’s been doing the rounds due to its capability but also the fact that it has native vision and a 1M-token context window. From the architecture overview, Kimi K3’s 93-block architecture extends Kimi Linear with a 3:1 mix of Kimi Delta Attention and gated multi-head latent attention, Stable LatentMoE layers that execute experts in a compressed representation, cross-layer Attention Residuals, and NoPE throughout instead of RoPE. The design targets long-context inference efficiency, although serving it remains is still challenging as KDA introduces recurrent state, full-attention layers still require KV caches, 896 routed experts create communication pressure, and Attention Residuals add cross-layer memory traffic. vLLM support therefore includes hybrid prefix caching, fused KDA and residual kernels, FP4 MoE execution, expert parallelism and separate NVIDIA and AMD paths. It is interesting to see how closely the model and serving engine were developed together.

Netflix on the LLM-Native RecSys

Netflix has developed an LLM-native Recommender System, and they share some of the learnings they gathered along the journey: They launched GenRec, which is a ranking model that converts user histories, item metadata, and request context into natural-language inputs rather than relying on thousands of manually engineered features. The system starts from a Netflix-adapted foundation model, then applies more frequent ranking-specific post-training using catalog classification, language-model objectives, and reward-weighted examples aligned with long-term member outcomes and content-balancing requirements. They use a catalog-aware scoring head to restrict the outputs to available titles, and they use prefill-only inference on vLLM scores candidate sets without autoregressive generation. It is impressive to see that they claim this architecture improved Mean Reciprocal Rank by about 1.6% while using roughly 40 times fewer Phase-2 labelled examples than the existing production ranker. This means that it is possible to use some of these foundation models to extract signal in domains that can benefit, which would otherwise require highly custom models. They also mention that their four-week A/B test also reported statistically significant improvements across short-term and long-term metrics. This is certainly an interesting area of research and practice, like many other industries we will likely see a lot of changes in the status quo.

OpenAI 10 Advances in Maths

OpenAI (self) reports that an internal version of its forthcoming Astra model generated solutions to ten long-standing problems across geometry, coding theory, group theory, circuit complexity, quantum complexity, lattice problems and extremal combinatorics - whether hype, marketing or fact, it is encouraging to think about scientific research progressing with support from these models. The reported advances span highly specialised mathematical domains that are beyond my knowledge, but one I found it interesting that the workflows that they used combined large-scale model inference, human manuscript preparation and formal verification. OpenAI estimates that the discovery process would have cost roughly 2k USD at Sol API rates, which likely is their marketing/sales pitch, but if that is the case it’s indeed much more affordable, even if still out from hobby usage. Let’s see when that Astra model comes out, with all this talk about AGI that model better be good! But let’s indeed see when we actually get it in Europe.

Upcoming MLOps Events

The MLOps ecosystem continues to grow at break-neck speeds, making it ever harder for us as practitioners to stay up to date with relevant developments. A fantsatic way to keep on-top of relevant resources is through the great community and events that the MLOps and Production ML ecosystem offers. This is the reason why we have started curating a list of upcoming events in the space, which are outlined below.

Events we are speaking at this year:

Other relevant events:

In case you missed our talks, check our recordings below:

Open Source MLOps Tools

Check out the fast-growing ecosystem of production ML tools & frameworks at the github repository which has reached over 20,000 ⭐ github stars. We are currently looking for more libraries to add - if you know of any that are not listed, please let us know or feel free to add a PR. Here’s a few featured open source libraries that we maintain:

  • SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
  • KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
  • Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
  • Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
  • AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
  • Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

✉️ Email, 🐦 Twitter, 💼 Linkedin

Enjoy The ML Engineer? Subscribe to receive the next issue.