Feature Stores: Bridging the Gap Between Data Engineering and ML
Feature stores solve the training-serving skew problem and give data teams a clean interface to ML. Here's when you need one and how to choose.
The ML deployment problem
Machine learning models have a deployment problem. Data scientists train models in notebooks using carefully curated features. But getting those models — and the features they depend on — into production is where most ML initiatives stall.
The core issue is feature duplication and inconsistency. A data scientist builds a feature like 'customer 30-day rolling average spend' in a notebook. An engineer builds the same feature differently in a production pipeline. The model trained on the notebook version behaves differently when served with the pipeline version — leading to silent degradation that's extremely difficult to diagnose.
This training-serving skew is one of the most common reasons ML projects fail to deliver value in production. Feature stores were created to solve it.
What is a feature store?
A feature store is a centralised system for defining, storing and serving ML features. It provides:
- Feature definitions — a single source of truth for how each feature is computed, with versioning and lineage
- Offline store — historical feature values for training, accessible via batch queries at scale
- Online store — low-latency feature serving for real-time inference
- Point-in-time correctness — ensures training data uses only features that were available at the time of the prediction, preventing data leakage
How feature stores work
When a data scientist defines a feature in the feature store, it's computed once and stored in both the offline and online stores. Training pipelines read from the offline store; inference services read from the online store. Both use the same definition, computed the same way — eliminating training-serving skew.
The feature registry acts as a catalogue. Teams can discover existing features, understand their definitions and reuse them across projects. This reduces duplication of effort and ensures consistency across models.
Key benefits
Consistency between training and serving
Because features are computed once and served from the same store for both training and inference, the model sees identical feature values in both contexts. This eliminates one of the most common sources of model degradation in production.
Feature reuse
When features are centralised, teams across the organisation can discover and reuse them. A 'customer churn risk score' built by one team can be leveraged by another team building a retention model — without rebuilding the feature pipeline.
Point-in-time correctness
Feature stores handle the temporal logic of joining features to events. When training a model to predict churn, you need features as they were at the time the customer churned — not as they are now. Feature stores handle this automatically, preventing the data leakage that silently inflates training metrics.
Feature store technologies
- Databricks Feature Store — integrated with Databricks and Unity Catalog, providing governance and lineage for features alongside tables and models
- Feast — an open-source feature store that works with multiple storage backends and ML frameworks
- Azure ML Feature Store — Azure's managed feature store offering, integrated with Azure ML workspaces
Getting started
Start with your most critical ML model. Identify the features it depends on, define them in a feature store and ensure both training and serving pipelines read from the same definitions. The initial setup takes effort, but the payoff — consistent, reusable, governed features — compounds across every future model you build.
Feature stores bridge the gap between data engineering and ML by treating features as first-class data products — defined once, governed centrally and served consistently across training and inference.
