THE INSTITUTE FOR ETHICAL AIALIGNMENT + SAFETY
JOIN / CONTACT

STATE OF PRODUCTION ML · 2025

The State of Production ML in 2025

What 135 practitioners report from production: LLM and time series work surged, monitoring remains the top challenge at 42%, PyTorch edged ahead of scikit-learn, and AI risk and governance functions doubled year on year. Every answer compares against the 2024 baseline as you move through the chapters.

TOP CHALLENGE0142%

selected monitoring model performance as a production challenge

LEADING LIBRARY0235%

primarily use PyTorch, Lightning or Fast.ai

BIG MOVER03+21 pts

year-on-year growth in AI risk and governance teams

SURVEY RESPONSES04135

submitted rows, with question bases varying by nonblank answers

01 / ML CONTEXT

ML context

What teams build, where they run it and what slows the route from experiment to production.
FINDING 01

PyTorch leads, but the library field remains split.

PyTorch, Lightning and Fast.ai account for 35% of answers. Scikit-learn follows at 33%, two points higher than in 2024.

  1. PyTorch/Lightning/Fast.ai35%−1
  2. scikit-learn33%+2
  3. XGBoost12%+2
  4. TensorFlow6%−2
  5. CatBoost4%−2
  6. LightGBM4%−0
  7. All1%
  8. Darts1%
View all 15 options
  1. Evidently1%
  2. JAX1%
  3. langgraph1%
  4. PerpetualBooster1%
  5. sktime1%
  6. spaCy1%
  7. synapse1%
FINDING 02

LLM work and time series both moved sharply upward.

LLMs reached 59%, up 12 points year on year. Time series rose even faster to 58%, a 16-point gain.

Multiple selections allowed
  1. LLMs59%+12
  2. Time Series / Forecasting58%+16
  3. Tabular44%+6
  4. Recommender Systems36%+5
  5. Text / NLP (Non-LLM)32%−6
  6. Image / Computer Vision30%+6
  7. Causal Inference22%+6
  8. Search20%+3
View all 13 options
  1. All1%
  2. Classification1%
  3. Physics-informed ML1%
  4. Simulations1%
  5. Speech to text1%
FINDING 03

Demand forecasting became the dominant reported use case.

It was selected by 52% of respondents, up 20 points from 2024. Recommenders follow at 38%, close to last year's share.

Multiple selections allowed
  1. Demand Forecasting52%+20
  2. Recommender systems38%−1
  3. Marketing Intelligence31%+8
  4. Risk31%+4
  5. Pricing27%+6
  6. Search27%+0
  7. Fraud24%−4
  8. AI Products7%
View all 28 options
  1. Embedded Systems2%
  2. Information Extraction2%
  3. Anomaly detection1%+1
  4. Data Extraction1%
  5. Embbeded systems1%
  6. Automation1%
  7. Chatbot1%+0
  8. Classification1%−0
  9. Customer support1%
  10. electrical net design1%
  11. Intelligence Video Analytics1%
  12. Logistics1%
  13. Patterb Matching1%
  14. Predictive ML1%
  15. Product1%
  16. Safety Testing1%
  17. Security1%
  18. Simulations1%
  19. time estimation1%
  20. Translation1%
FINDING 04

Production cycles shifted toward the three-month mark.

Forty-one percent report taking one to three months, ten points more than in 2024. The three-to-six-month group fell by 12 points.

  1. Less than 3 months41%+10
  2. Less than a month23%−1
  3. Less than 6 months14%−12
  4. Less than a week11%+4
  5. Less than a year6%+1
  6. More than a year5%−1
FINDING 05

AWS still leads, while Azure and GCP gained ground.

AWS remains first at 32%, but its share is 22 points below 2024. Azure also has 32%, while GCP reaches 26%.

  1. Amazon Web Services32%−22
  2. Azure32%+11
  3. Google Cloud Platform26%+9
  4. Self Hosting3%
  5. Databricks2%+1
  6. Cloudera1%
  7. Hetzner1%
  8. IBM1%+0
View all 11 options
  1. IONOS1%
  2. Snowflake1%
  3. SSPCloud1%
FINDING 06

Monitoring is the most common production challenge.

Forty-two percent selected monitoring model performance. Access to training data and robust data pipelines follow at 37% and 34%.

Multiple selections allowed
  1. Machine learning monitoring and observability42%−3
  2. Access to relevant data for training37%+4
  3. Building production-grade machine learning and data pipelines34%+3
  4. Inconsistency of training and experimentation environments34%+3
  5. Gaps in tooling and support for model productionisation22%−11
  6. Showcasing business impact and business value22%−6
  7. Governance and domain risks21%+5
  8. Lack of specialised engineers19%−8
View all 17 options
  1. Machine learning security14%+7
  2. Lack of specialised data scientists11%−0
  3. Feasibility1%
  4. incorporating the Machine Learning Model in the right place in the business process1%
  5. Infrastructure/policies constraints1%
  6. It/ot integration1%
  7. Lack of data1%
  8. lack of general engineers1%
  9. Legacy SDLC constraints1%

02 / PLATFORMS & TOOLS

Platforms & tools

The systems practitioners choose across tracking, data, training, serving, monitoring and foundation models.
FINDING 07

MLflow has consolidated its lead in experiment tracking.

Two thirds of respondents who use a tracking or registry tool chose MLflow. Its share is 18 points higher than in 2024.

  1. MLflow66%+18
  2. Weights & Biases12%−0
  3. Custom Built In-house tool9%−7
  4. Data Version Control (DVC)5%+2
  5. Spreadsheets3%−6
  6. Hopsworks Model Registry2%
  7. Azureml1%
  8. Evidently1%
View all 10 options
  1. Optuna1%+0
  2. Snowflake1%+0
FINDING 08

Most feature-store users still rely on in-house systems.

In-house tooling leads at 55%, though seven points below 2024. FEAST doubled its share to 20%.

  1. Custom Built In-house tool55%−7
  2. FEAST20%+10
  3. Databricks11%+4
  4. Hopsworks6%−0
  5. azure ml's2%
  6. Custom Implementation2%
  7. databricks feature store2%
  8. Fennel2%+0
View all 9 options
  1. Snowflake2%−1
FINDING 09

Vector database choices remain notably fragmented.

Pinecone leads at 21%, followed by in-house systems at 18%. Both gained roughly seven to eight points year on year.

  1. Pinecone21%+7
  2. Custom Built In-house tool18%+8
  3. Azure AI11%
  4. Milvus8%−4
  5. Weaviate8%+1
  6. Elasticsearch7%+2
  7. Databricks4%−1
  8. pgvector4%−1
View all 20 options
  1. Hopsworks3%+2
  2. Qdrant3%+0
  3. azure search1%
  4. Chroma1%+0
  5. ChromaDB1%+0
  6. Elastic1%
  7. LanceDB1%+0
  8. OpenSearch1%−2
  9. PG-Vector1%
  10. Quadrnt1%
  11. Turbopuffer1%
  12. Vespa1%
FINDING 10

Airflow remains the default orchestrator.

Airflow leads at 47%, six points above 2024. Databricks is a distant second at 16%, but gained 12 points.

  1. Airflow47%+6
  2. Databricks16%+12
  3. Custom Built In-house tool13%−4
  4. Argo Workflows4%−7
  5. Amazon Glue3%+1
  6. Azure Data Factory3%
  7. Dagster3%+2
  8. Celery2%+0
View all 19 options
  1. Agilab1%
  2. AWS Step Functions1%+0
  3. Azure ML Pipelines1%+0
  4. Flyte1%
  5. GCP Vertex AI pipelines1%
  6. GitHub actions1%
  7. Hopsworks1%
  8. Kubeflow1%−0
  9. Oozie1%
  10. Temporal1%
  11. ZenML1%
FINDING 11

Databricks moved to the front of training platforms.

Databricks leads at 28%, with SageMaker and Azure ML Studio both at 16%. Azure ML Studio doubled its 2024 share.

  1. Databricks28%+6
  2. Amazon SageMaker16%−1
  3. Azure ML Studio16%+8
  4. Google Cloud Vertex AI15%+5
  5. Custom Built In-house tool13%−21
  6. Hopsworks2%+1
  7. Agilab1%
  8. Cloudera1%
View all 17 options
  1. Cloudera CML1%
  2. Custom1%
  3. Domino Datalab1%
  4. Evidently1%
  5. Kedro1%
  6. Lightning AI1%
  7. Metaflow1%+0
  8. MLflow1%
  9. Perpetual ML Suite1%
FINDING 12

Serving choices spread beyond Python web frameworks.

FastAPI and Flask still lead at 38%, but fell ten points. Databricks rose to 18%, a gain of 12 points.

  1. FastAPI/Flask Wrapper38%−10
  2. Databricks18%+11
  3. Custom Built In-house tool12%−5
  4. Sagemaker8%−4
  5. KServe7%+4
  6. BentoML4%+2
  7. Triton2%+1
  8. Agilab1%
View all 19 options
  1. AWS1%
  2. AWS Lambda1%+0
  3. Azure Managed Online Endpoints1%+0
  4. Azure ML1%+0
  5. Django1%
  6. Docker / Azure Functions1%
  7. LitServe1%
  8. Nvidia triton1%
  9. Seldon Core1%−1
  10. Tesseract Core1%
  11. Triton Inference Server1%+0
FINDING 13

In-house monitoring leads, but packaged tools are gaining.

In-house systems account for 41%, 11 points below 2024. Evidently rose eight points to 27%.

  1. Custom Built In-house tool41%−11
  2. Evidently AI27%+8
  3. Neptune AI9%+3
  4. Arize AI6%+1
  5. Databricks5%
  6. Agilab1%
  7. AWS Clarify1%
  8. Databricks lakehouse monitoring1%
View all 14 options
  1. Grafana1%+0
  2. Hopsworks1%+0
  3. NannyML1%−2
  4. NannyML plus custom library1%
  5. Perpetual1%
  6. Tensorboard1%
FINDING 14

The central data platform market has no runaway leader.

Delta Lake leads at 20%, closely followed by Azure Data Lake at 19% and Snowflake at 17%. Azure gained 11 points.

  1. Deltalake20%−2
  2. Azure DataLake19%+11
  3. Snowflake17%+0
  4. Custom Built In-house tool13%−3
  5. GCP / BigLake13%−1
  6. AWS / Lakeformation9%−10
  7. Databricks2%
  8. Agilab1%
View all 14 options
  1. Cloudera CDP1%
  2. Cloudera/HDFS/Hive1%
  3. Databricks on Azure1%
  4. Dremio1%+0
  5. Perpetual ML Suite1%
  6. StarRocks1%
FINDING 15

OpenAI leads a rapidly diversifying service market.

OpenAI accounts for 30%, down ten points. Azure AI follows at 22%, while Gemini tripled its share to 19%.

  1. OpenAI30%−10
  2. Azure AI22%+1
  3. Google Gemini19%+13
  4. Amazon Bedrock12%−0
  5. Anthropic5%−1
  6. Custom Built In-house tool5%−4
  7. All of the above2%
  8. Cursor1%
View all 13 options
  1. Databricks1%
  2. DeepSeek V31%
  3. Fuel iX1%
  4. Nebius1%
  5. Ollama1%

03 / ORGANISATION & OPERATIONS

Organisation & operations

How production ML is organised, scaled and deployed across teams and infrastructure.
FINDING 16

Technology and finance anchor the respondent base.

Technology represents 21% and financial services 15%. The overall industry mix remains broad, with no single sector near a majority.

  1. Technology21%−0
  2. Financial services15%−1
  3. Retail9%−0
  4. Energy5%+1
  5. Telecommunications5%+1
  6. Transportation & warehousing5%+0
  7. Aerospace & defense4%+3
  8. Food4%+1
View all 29 options
  1. Healthcare4%−4
  2. Media & Entertainment4%−3
  3. Utilities4%+3
  4. Insurance3%−2
  5. Consulting2%+1
  6. Hospitality2%+2
  7. Consultancy2%−0
  8. Electronics2%+1
  9. Adtech, gaming1%
  10. AlcoBev1%
  11. Automobile1%
  12. Consultant1%
  13. Edu1%
  14. Engineering design1%
  15. Government1%+0
  16. government, official statistics1%
  17. Industry1%
  18. insurtech1%
  19. IT and Software Consultation1%
  20. Manufacturing1%+0
  21. Risk Management1%
FINDING 17

Respondents span both enterprises and small teams.

Organisations with 5,000 to 50,000 employees are the largest group at 22%. Those with 1,000 to 5,000 follow at 21%.

  1. 5,000-50,000 employees22%+5
  2. 1,000-5,000 employees21%+4
  3. 10-50 employees18%+8
  4. 50,000+ employees11%+1
  5. Less than 10 employees10%+2
  6. 250-1,000 employees9%−9
  7. 50-250 employees9%−11
FINDING 18

Most fleets are measured in tens, not thousands.

Two to five models and ten to twenty models are the two largest groups, each near 24%. Only 8% report more than 1,000.

  1. 2-524%+0
  2. 10-2023%+3
  3. 21-10018%−1
  4. 5-916%+4
  5. 100-100013%−1
  6. 12%−1
  7. 1000+2%−4
  8. 01%−1
View all 9 options
  1. Varies, we consult1%
FINDING 19

Teams expect their production fleets to keep expanding.

The largest plan is ten to twenty models at 29%, up 11 points. Plans for two to five models also rose nine points.

  1. 10-2028%+11
  2. 21-10024%−1
  3. 2-517%+9
  4. 100-100015%+1
  5. 5-912%−8
  6. 1000+2%−10
  7. 01%−0
FINDING 20

Governance is becoming an operating function.

AI risk and governance teams reached 41%, up 21 points. Central ML platforms remain most common at 66%.

Multiple selections allowed
  1. Central Machine Learning Platform / Team66%+0
  2. Data Platform / Data Engineering Organisation63%−2
  3. AI Risk & Governance Function41%+21
  4. A Developer Productivity Team which also covers machine learning36%+10
  5. AI Inventory (Keeping track of all AI usecases and models)21%+3
FINDING 21

Batch remains the dominant inference mode.

Forty-three percent say fewer than one in ten models run real-time inference, nine points more than in 2024.

  1. Less than 10%43%+9
  2. Between 10% and 30%26%+2
  3. Between 50% and 90%19%−1
  4. More than 90%12%−10
FINDING 22

CI/CD is now close to universal among respondents.

Eighty-four percent report CI/CD support, up 15 points. Separate development, staging and production environments reached 66%.

Multiple selections allowed
  1. CI/CD for continuous deployment84%+15
  2. Development-Staging-Production Environments66%+9
  3. A/B Tests for Models49%+7
  4. Canary Deployments31%+5
  5. Progressive Rollouts27%−3
FINDING 23

ML engineers remain the largest practitioner group.

ML engineers account for 42% of answers. MLOps engineers follow at 22%, with data scientists close behind at 21%.

  1. Machine Learning Engineer42%−2
  2. MLOps Engineer22%+3
  3. Data Scientist21%−2
  4. Business / Domain Practitioner7%+4
  5. Software Engineer5%−1
  6. Data Engineer2%+0
  7. Product Manager2%−2

04 / RESPONDENT PROFILE

Respondent profile

The role levels, ages, locations and identities represented in this survey.
FINDING 24

Individual contributors form the centre of the sample.

Junior to senior individual contributors make up 43%, five points above 2024. Staff-level and above represent 23%.

  1. Individual Contributor (Junior to Senior)43%+5
  2. Individual Contributor (Staff+)23%−7
  3. Manager13%−5
  4. Director/VP12%+2
  5. C-Suite8%+5
FINDING 25

The age profile broadened beyond the early thirties.

Ages 35 to 39 are the largest group at 24%. Ages 30 to 34 fell by 13 points, while ages 40 to 44 gained seven.

  1. 35-3924%+2
  2. 30-3423%−13
  3. 25-2917%+3
  4. 40-4417%+7
  5. 22-246%+4
  6. 45-496%−3
  7. 50-544%+1
  8. 55-592%+0
View all 10 options
  1. 60+2%
  2. 18-211%+0
FINDING 26

Responses reflect a widely distributed community.

Germany and India are the largest consistently labelled groups at 11% each. Free-text country naming means small variants remain visible separately.

  1. Germany11%+4
  2. India11%+4
  3. Netherlands8%−1
  4. UK5%+4
  5. Spain4%−1
  6. USA4%−14
  7. South Africa3%
  8. Brazil3%+0
View all 51 options
  1. Australia2%+1
  2. Canada2%−1
  3. Denmark2%+1
  4. Finland2%+1
  5. France2%−4
  6. germany2%+1
  7. india2%+0
  8. INDIA2%
  9. Israel2%−1
  10. Italy2%+1
  11. Japan2%−1
  12. Pakistan2%
  13. Sweden2%
  14. The Netherlands2%+1
  15. United Kingdom2%−7
  16. Austria1%+0
  17. Bangalore, INDIA1%
  18. Belgium1%
  19. Chile1%
  20. Danmark1%
  21. España1%
  22. france1%
  23. Indian1%
  24. Ireland1%
  25. Lebanon1%
  26. Luxembourg1%+0
  27. Mexico1%
  28. N/A1%
  29. Nigeria1%
  30. nl1%+0
  31. Norway1%+0
  32. Poland1%
  33. Portugal1%−1
  34. Romania1%+0
  35. Rwanda1%
  36. Swiss1%
  37. the Netherlands1%
  38. Türkiye1%
  39. uk1%
  40. Ukraine1%
  41. united states1%
  42. United States1%−1
  43. Usa1%
FINDING 27

The gender imbalance remains pronounced.

Ninety percent self-identify as male and 9% as female. The female share is two points higher than in 2024.

  1. Male90%−3
  2. Female9%+2
  3. Prefer not to disclose1%
  4. Prefer to not disclose1%

05 / METHODOLOGY

Methodology

This edition compares the 2025 answers with the same questions and answer labels from 2024. Question-level response bases are summarised here so every percentage can be read in context.
RESPONSES135

Submitted rows in the 2025 source. Per-question nonblank respondent bases range from 64 to 135.

COLLECTION2025

Responses were collected through the State of Production ML survey. The report aggregates the supplied CSV at build time and publishes no raw-response table.

READ THE 2024 BASELINE