01 / EMPOWERING INDIVIDUALS
By Principle.
Best practices.Applied principles.Personal and professional commitments.
REACH OUT · SUBSCRIBE · APPLY TO JOIN
The Institutefor Ethical AIAlignment & Safety
We are an independent research institute with a mission toensure that frontier AI is safe, aligned and accountable topeople and society.
09RESPONSIBLE AI PRINCIPLES
393NEWSLETTER ISSUES PUBLISHED
70k+ML PRACTITIONERS REACHED
20k+GITHUB STARS, OSS ECOSYSTEM
EST.2017 — FOUNDED
MEMBERSHIPS, ADVISORY ROLES & STANDARDS COLLABORATIONS
01 — THE FOUR-PHASE RESPONSIBLE AI STRATEGY
From individual practice to national regulation.
02 / EMPOWERING ORGANISATIONS
By Process.
Practical industry frameworks.Applied guides.Principles translated into implementation.
03 / EMPOWERING INDUSTRIES
By Standards.
Technical and industry standards.Standards bodies engagement.Cross-industry initiatives.
04 / EMPOWERING NATIONS
By Regulation.
Policy and regulatory work.Public frameworks and requirements.International governance.
02 — PRINCIPLES FOR ALIGNED & SAFE AI
Nine Principles for AI Alignment and Safety.
Each principle specifies a commitment, the failure modes it addresses and the controls that implement it. We invite both individuals and organisations to pledge towards these principles.
PRINCIPLE 01 — COMMITMENT
Human Augmentation by Oversight
We commit to assess the consequences of incorrect outputs and automated actions and to design systems with human oversight to ensure aligned and safe outcomes, enabling Human Augmentation by Oversight.
FAILURE MODES
- Automation of decisions the AI Act treats as high risk or prohibits
- Agents acting beyond their mandate
- Review in name only
PRACTICAL CONTROLS
- Impact assessment before automation
- Approval gates for consequential actions
- Bounded autonomy with escalation paths
PRINCIPLE 02 — COMMITMENT
Fairness by Calibration
We commit to introduce processes proportionate to risk that mitigate unintended discrimination across groups in our production AI systems, enabling Fairness by Calibration.
FAILURE MODES
- Inherited model bias propagated across the system
- Application data and prompts amplifying skew
- Fine-tuning shifting behaviour unnoticed
PRACTICAL CONTROLS
- Disaggregated behavioural evaluations
- Review of provider model documentation
- Re-evaluation after fine-tuning and updates
PRINCIPLE 03 — COMMITMENT
Explainability by Justification
We commit to develop tools and processes to enhance the transparency and explainability of AI systems where reasonable, enabling Explainability by Justification.
FAILURE MODES
- Unnecessarily convoluted agentic graphs
- Stated reasoning diverging from actual behaviour
- Lack of deterministic flows where systematic execution required
PRACTICAL CONTROLS
- Interpretability and attribution tooling
- Transparency artifacts such as model and system cards
- Infrastructure for semantic root cause analysis
PRINCIPLE 04 — COMMITMENT
Provenance by Reproducibility
We commit to develop the infrastructure that captures the lineage of our AI systems' operations, so historical steps can be reconstructed and reproduced, enabling Provenance by Reproducibility.
FAILURE MODES
- Unpinned model versions
- Gaps in data governance throughout system
- Historical steps that cannot be reconstructed
PRACTICAL CONTROLS
- Version pinning for models, prompts and configuration
- Determinism in atomic steps through seed/input metadata
- Data provenance and lineage across operations
PRINCIPLE 05 — COMMITMENT
Reskilling by Design
We commit to identify, document and systematically support the reskilling of our domain experts and mitigate the impact on workers affected by automation, enabling Reskilling by Design.
FAILURE MODES
- Lack of systematic plan for workforce support
- No retraining and reskilling paths
- Undocumented process change
PRACTICAL CONTROLS
- Workforce impact assessment
- Business change planning
- Process documentation
PRINCIPLE 06 — COMMITMENT
Evaluation by Observability
We commit to evaluate our AI systems against the domain-specific consequences of their behaviour, and to ensure robust monitoring capabilities are in place for as long as the system operates, enabling Evaluation by Observability.
FAILURE MODES
- Benchmark contamination and overfitting
- Lack of telemetry, tracing, logging and general observability
- Drift that goes undetected after deployment
PRACTICAL CONTROLS
- Domain-grounded capability and safety evals
- Cost-weighted metrics with production monitoring
- Incident feedback into the evaluation suite
PRINCIPLE 07 — COMMITMENT
Trust by Privacy
We commit to build and communicate privacy processes that protect the data of stakeholders interacting with our AI systems directly and indirectly, enabling Trust by Privacy.
FAILURE MODES
- Personal data extractable from a model by prompting
- Sensitive data flowing to third parties the user never agreed to
- One user's data surfacing in another user's session
PRACTICAL CONTROLS
- Privacy impact assessment before connecting data to a model
- Memory segregation across users with no shared session context
- End-to-end encryption of personal data in transit and at rest
PRINCIPLE 08 — COMMITMENT
Safety by Security
We commit to develop the processes and infrastructure required to secure our AI systems across their lifecycle, and to treat the safety of their actions as part of that boundary, enabling Safety by Security.
FAILURE MODES
- Agents granted standing permissions beyond their task
- Models, data and tools adopted without vetting or integrity checks
- Agent credentials shared across users, tasks and environments
PRACTICAL CONTROLS
- Threat modelling against the MLSecOps Top 10 and OWASP agentic attack vectors
- Trust boundaries separating model instructions from external content
- Red-teaming, layered safeguards and monitoring
PRINCIPLE 09 — COMMITMENT
Human Alignment by Intent
We commit to evaluate whether our AI systems pursue what was intended rather than what was measured, and to test for misgeneralisation, deception and misuse where reasonable, ensuring Human Alignment by Intent.
FAILURE MODES
- No definition of what the system must never pursue
- High risk systems that cannot be interrupted once running
- Objectives never revisited as the deployment context changes
PRACTICAL CONTROLS
- Periodic objective reviews as deployment conditions change
- Post-deployment audits of behaviour against the stated intent
- Monitoring for divergence between the metric and intended outcome
03 — POLICY & STANDARDS
Policy written by practitioners.
- 20+recommendations adopted across the EU GPAI Code of Practice and the UK framework
- 11/11major European digital instruments with technical responses
- 30+published policy products since 2020
6ACTIVE MANDATES
~10ORGANISATIONAL PRINCIPLES
5REGIONAL POLICY AREAS
04 — OPEN SOURCE & INITIATIVES
Open Foundation to Support North-Star Principles.
The Institute builds and maintains open-source infrastructure, research and applied frameworks that put the principles into working practice. This section spotlights a few flagships, from cross-vendor GPU compute and an agent control plane to the annual practitioner survey, alongside the initiatives that grew around them.
FIG 01LINUX FOUNDATION AI & DATA
Cross-vendor GPU computing
A general-purpose GPU compute framework built on Vulkan for cross-vendor graphics cards. It supports asynchronous, mobile-enabled GPU processing.
The Institute donated Kompute to the Linux Foundation in 2021.
2.5KSTARS
197FORKS
42CONTRIBUTORS
FIG 04
K8s Agent OS (KAOS)
An open-source control plane for running autonomous agents under operational constraints. It provides scoped credentials, sandboxed execution, budget and time-horizon limits, human approval gates and a replayable audit trail. These controls implement Principle 09 in the infrastructure.
Scoped credentialsPer-agent service accounts, short-lived tokens
Sandboxed executionNamespaced pods, egress allow-lists
Budget limitsToken, compute and wall-clock ceilings
Time horizonMax unattended task length per agent class
Human approvalGates on irreversible or high-impact actions
Audit trailReplayable traces, lineage, evaluation hooks
268 STARS18 FORKS5 CONTRIBUTORSAPACHE-2.0 LICENSE
SELECTEDMonitoring & observability
SHARE42%
YOY VS 2024-3 pts
The State of Production ML Survey
Two annual practitioner surveys covering the tools, practices and challenges of production machine learning, compared year on year.
FIG 02 — XAI PIPELINE
Data analysisDISTRIBUTIONS · PROXIES
Model evaluationATTRIBUTION · SUBGROUPS
Production monitoringDRIFT · OUTLIERS
XAI Explainability Framework
Explainability and bias mitigation across three stages, combining technical tools with the organisational processes around them.
1.3K STARS185 FORKS3 CONTRIBUTORSMIT LICENSE
CATEGORYCATEGORIES
Serving & inference
Orchestration & pipelines
Monitoring & observability
Feature & vector stores
Explainability & fairness
ML security
GPU & acceleration
Production ML open-source list
The community-maintained catalogue of production ML and MLOps tooling, curated weekly.
20.8K STARS2.6K FORKS211 CONTRIBUTORS550+ LIBRARIES24 CATEGORIES
From requirements to supplier evidence
01Define governance requirementsRFP
02Set evaluation criteriaSCORE
03Request supplier evidenceEVIDENCE
04Compare and document responsesREVIEW
05Carry obligations into deliveryCONTROL
Governance
ML Maturity Model
Governance becomes measurable when organisations assess capability and request supplier evidence. The ML Maturity Model provides a four-level framework, while AI-RFX translates requirements into supplier questions, evaluation criteria and evidence requests.
Broken access controlUnrestricted model endpoints
Cryptographic failuresAccess to model artifacts
InjectionArtifact exploit injection
Insecure designInsecure ML pipeline design
Security misconfigurationML infrastructure misconfiguration
Vulnerable componentsSupply-chain vulnerabilities in ML code
Logging & monitoring failuresObservability and lineage failures
Security
Agentic & ML Security
The Institute's security programme spans the MLSecOps Top 10, partnership work with OpenSSF, and active review of OWASP guidance for agentic systems.
Related work includes: the Agentic & ML Security framework.
05 — NETWORK & NEWSLETTER
Contact us, Subscribe or Apply.
Responsible technology requires changes to models and to the systems around them. Membership includes the Ethical AI Network and the Machine Learning Engineer newsletter.
1,034NETWORK MEMBERS
70,412NEWSLETTER SUBSCRIBERS
393ISSUES PUBLISHED