MLOps is the operational discipline of training, deploying, monitoring, and retraining machine learning models reliably.
MLOps applies the lessons of DevOps to machine learning systems, with the added complexity that the artifact you're deploying (a model) is a function of data that itself changes. A model that was excellent last quarter can silently degrade this quarter because the input distribution shifted, a feature pipeline broke, or a labeling process changed upstream. MLOps establishes the observability, reproducibility, and automation needed to catch and correct these issues before they cost you customers or money. It is less about specific tools (SageMaker, Vertex AI, MLflow, Weights & Biases) and more about disciplined workflows any of those tools can support.
(1) Data pipeline — ingesting raw data, transforming it into features, versioning both. (2) Training pipeline — reproducibly training a model from versioned data and code, producing a versioned model artifact with metrics. (3) Deployment pipeline — promoting a trained model to production with the same rigor as a service deploy (staging, canary, rollback). (4) Monitoring — tracking input distributions, prediction distributions, and downstream business metrics after deployment. Most teams build 1 and 2 well and neglect 3 and 4, which is why so many models decay unnoticed. Aim for parity across all four before scaling to more models.
Every trained model should be reproducible from: the code (git SHA), the data (dataset version / snapshot ID), the hyperparameters (config file), and the environment (container image or lockfile). MLflow, Weights & Biases, and Neptune all handle this well; the important thing is that the choice is standardized across the team. Metadata to log per run: metrics (train/val/test), hyperparameters, dataset ID, code SHA, hardware, wall clock. The rule: if you can't reproduce a model's training in one command from these artifacts, you don't really own it — you can't diagnose regressions or roll back with confidence.
(1) Batch inference — model runs on a schedule, writes predictions to a table. Simplest; appropriate when predictions don't need to be real-time (churn scores, next-day recommendations). (2) Online inference behind an API — model wrapped in a service, called synchronously. Requires latency and reliability engineering; monitor P95 latency and error rate as you would any service. (3) Embedded — model shipped inside the client (mobile, browser). Painful to update; use only when latency or privacy demands it. For online inference, adopt canary rollout and shadow mode: run the new model in parallel, log its predictions without serving them, compare distributions, then promote.
Accuracy monitoring is delayed — you often don't know if a prediction was right for days or weeks. Real-time signals: (a) input drift — has the distribution of features shifted materially from training? Population Stability Index (PSI) or KL divergence per feature, alerted when they cross thresholds. (b) prediction drift — has the distribution of model outputs changed? Often faster to detect than accuracy loss. (c) downstream metric — the business KPI the model influences (conversion, click, retention). (d) infrastructure — latency, throughput, error rate. When accuracy ground truth arrives, retro-actively join it to predictions and update dashboards. Alert on drift, not on accuracy loss — by the time accuracy drops, damage is done.
Small orgs (1-3 ML people): one team owns end-to-end from data to deployment. MLOps is 20-30% of everyone's job. Medium (4-15 ML people): split into an ML platform team (owns feature store, training infra, deployment tooling) and applied ML teams (own models for specific product areas). Large (15+): full MLOps platform team with PMs, plus specialized ML infra roles (feature engineering, serving infra, monitoring). Regardless of scale, avoid the anti-pattern of data scientists throwing notebooks over the wall to a separate 'ML engineering' team — the handoff kills iteration speed. Data scientists should ship models to production themselves, on a platform that makes it safe to do so.
Investor directory · Fundraising library · Articles A–Z · Company funding database