QWEN-3.5 Challenging Sonnet

The race to build reliable multimodal agents is ON, but is the future open? QWEN-3.5 gives us hope that it is... Sonnet-like performance in your local machine: Qwen3.5 was released a few weeks ago, however the community is now starting to report impressive performance comparable to some of the more popular proprietary models - this is really exciting. To dive into the internals briefly, Qwen3.5-397B-A17B basically combines sparse MoE and hybrid attention so only 17B parameters are active per token, but still able to show strong performance across reasoning, coding, tool use, vision, document understanding, video, and multilingual tasks. For ML practitioners the key takeaways are less about absolute benchmark wins and more about the system design: early text-vision fusion, extensive RL-based post-training for agent behavior, support for 201 languages and dialects, as well as infrastructure advances such as FP8 training, heterogeneous multimodal training, and asynchronous RL that aim to improve throughput, stability, and cost efficiency. It is really hard to keep up, but open source (open weight) releases like these really set the example and raise the bar for organisations trying to hold the moat with proprietary closed models.

Simon Willison on Agentic Patterns

Agentic engineering is quickly becoming one of the most important shifts in how production software and ML systems get built, but how do we build these right? Simon Willison shares a really great write-up on Agentic Engineering Patterns, where he is documenting live as a guide for devs using coding agents to accelerate real software work - note, not vibe coding but agentic engineering. This has quite a lot of sound (+ sometimes obvious) advise, such as ensuring supervision, secure code execution, and iterative testing. For production ML practitioners, the main point is that as code generation becomes cheap, the real challenge shifts to building workflows that preserve reliability, review quality, and team judgment - the team part is the hardest. This is definitely one of the many resources that will be important to keep a close eye on!

The Q, K, V Matrices

Attention has really been all you need so far... understanding the internals is a great way to boost your skills, and the core foundation is the Q, K, V Matrices: The core implementation of transformer based architectures relies on the attention mechanism which is formulated by the Q, K, V Matrices, and this is a great deep dive to refresh or learn about it. This basically introduces the matrices end to end with a hands-on NumPy example, including the nuanced multiplications, as well as connecting them to dot-product attention, where query-key similarity produces attention weights that are applied to values. For production ML practitioners, the key value is in demystifying self-attention at the tensor level while also explaining why separate projections matter and how choices like per-head dimension affect the usual trade-offs between model capacity, memory use, and compute.

What is Entropy?

Understanding entropy is foundational to how we reason about uncertainty, information flow, and system behavior - this becomes even more important with the growing popularity of probabilistic systems with multi-agent architectures: This is one of the best explainers / introductions to the concept of entropy, which is best framed as a measure of uncertainty over possible underlying states rather than as vague "disorder" - also one of the best visual explainers in the topic. Shannon entropy is also a good start to explain the expected surprise, then showing the same idea carries into physics, where a macrostate can correspond to many microstates and entropy grows with the number and probability of those compatible configurations. For production ML practitioners, the useful takeaway is that entropy is not mystical; it is a practical concept for reasoning about uncertainty, compression, latent-state ambiguity, and how modeling choices about state abstraction can shape both system behavior and interpretation.

META AMD GPU Workloads

In production ML, efficiency increasingly determines success, and Meta published their approach to large-scale training in AMD swarms: Meta released RCCLX as an open-source framework to tackle one of the biggest practical bottlenecks in large-scale training and inference with GPU communication overhead. For production ML teams, the key point is that Meta is adding a new backend for shipping communication optimizations for AMD platforms through Torchcomms, which now uses Direct Data Access collectives to cut intra-node latency, which is particularly key for decode-heavy LLM inference. Meta reports meaningful internal gains on MI300-class hardware, including faster decode and prefill performance, lower time-to-incremental-token, reduced latency, and higher throughput, while keeping numerical accuracy within acceptable bounds for its workloads. It does seem like the sub-field of GPU engineering will only continue to explode in popularity, so if you are a practitioner that wants to stay relevant amid the rise of AI, this would definitely be a relevant domain!

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

✉️ Email, 🐦 Twitter, 💼 Linkedin