Five Things That Break When You Move a Model From Notebook to Production

Your model works in a notebook. The accuracy is good, the plots look right, your colleagues are impressed. You decide to deploy it.

Then things break — not dramatically, but in quiet, frustrating ways. The predictions are wrong but not obviously wrong. The service crashes under load. The model works on your machine but not in the container. These are the most common failure modes, and they're all avoidable.

1. Dependency mismatches

The notebook runs on your machine with whatever packages you've installed over the past year. The production environment is different — different Python version, different library versions, different OS.

The classic symptom: the model loads fine but produces different predictions. Or it throws a cryptic deserialization error.

ModuleNotFoundError: No module named 'sklearn.ensemble._forest'

This happens when you train with scikit-learn 1.3 and try to load the pickle with scikit-learn 1.2. The internal module paths changed between versions, and pickle serializes the full path.

The fix

Pin every dependency with exact versions. Not scikit-learn>=1.0, not scikit-learn~=1.3. Use scikit-learn==1.3.2.

pip freeze > requirements.txt

Better yet, use the same Docker image for training and serving. If the model was trained in python:3.11-slim with scikit-learn==1.3.2, serve it from the same image. This eliminates version mismatches entirely.

FROM python:3.11-slim
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY serve.py model/
CMD ["python", "serve.py"]

MLflow helps here too — it logs the conda.yaml or requirements.txt alongside the model artifact, so you can always reconstruct the exact environment that produced a model.

2. Missing preprocessing steps

This one is insidious. Your notebook has a cell that normalizes the data:

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model.fit(X_train_scaled, y_train)

The model is trained on scaled data. It expects scaled inputs. But the serving code receives raw features and passes them directly to model.predict(). The predictions are garbage, but the model doesn't throw an error — it just returns wrong answers confidently.

Variations of this problem:

  • Feature encoding trained in the notebook but missing in production. Categorical variables that were one-hot encoded during training arrive as raw strings at serving time.
  • Feature engineering done in a notebook cell that nobody copied to the serving code. The model expects a price_per_sqft feature that only exists in the notebook.
  • Missing imputation. The notebook filled NaN values with the column median. The serving code passes NaN values through, and the model does something unpredictable.

The fix

Bundle preprocessing with the model. Use a scikit-learn Pipeline:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier

pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("classifier", RandomForestClassifier(n_estimators=100)),
])

pipeline.fit(X_train, y_train)

Now pipeline.predict(X_raw) handles scaling internally. The serving code doesn't need to know about preprocessing — it just calls predict() on whatever comes in.

For more complex preprocessing (text tokenization, image resizing, feature engineering), the same principle applies: serialize the entire transformation chain, not just the model. If preprocessing can't be part of the model object, make it a separate artifact that gets deployed alongside the model with an explicit contract between the two.

3. No input validation

In the notebook, you control the input. You loaded the dataset, cleaned it, and shaped it correctly. In production, input comes from users, APIs, and upstream services — and it will be wrong in ways you didn't anticipate.

Common failures:

  • Wrong number of features. The model expects 10 features, the request sends 9. NumPy reshapes silently and the prediction is meaningless.
  • Wrong data types. A feature that should be float arrives as a string. Or an integer arrives as a float with a decimal point.
  • Out-of-range values. The model was trained on ages between 18 and 90. A request arrives with age = -1 or age = 500.
  • Missing fields. An optional field in the API is omitted and arrives as None, which propagates through the model as NaN.

The fix

Validate at the API boundary. Define the expected schema and reject bad input before it reaches the model.

from pydantic import BaseModel, validator
from typing import List

class PredictionRequest(BaseModel):
    features: List[float]

    @validator("features")
    def check_feature_count(cls, v):
        if len(v) != 4:
            raise ValueError(f"Expected 4 features, got {len(v)}")
        return v

from fastapi import FastAPI, HTTPException

app = FastAPI()

@app.post("/predict")
def predict(request: PredictionRequest):
    try:
        result = model.predict([request.features])
        return {"prediction": int(result[0])}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

This returns a clear 422 error when input is wrong, instead of returning a wrong prediction that's indistinguishable from a correct one.

Validation isn't glamorous, but it's the difference between "the model is broken" (it isn't — the input is wrong) and "bad request, here's what you need to fix."

4. No logging or monitoring

In the notebook, you see the predictions right there in the output cell. In production, the model runs inside a service, receives requests, and returns responses — and nobody is watching.

Without logging, you can't answer basic questions:

  • Is the model being used? How many predictions per day? Is traffic increasing or decreasing?
  • What is it predicting? If the model starts predicting the same class for every input, that's a problem — but you won't know without logging.
  • How fast is it? If prediction latency spikes from 10ms to 500ms, users notice before you do.
  • Is the input distribution changing? If the average feature values shift significantly from training data, the model's predictions become unreliable. This is data drift.

The fix

Log predictions, inputs (or summaries of inputs), and latency from day one.

import time
import logging

logger = logging.getLogger("model-serving")

@app.post("/predict")
def predict(request: PredictionRequest):
    start = time.time()

    result = model.predict([request.features])
    prediction = int(result[0])

    latency = time.time() - start
    logger.info(
        "prediction",
        extra={
            "prediction": prediction,
            "feature_count": len(request.features),
            "latency_ms": round(latency * 1000, 2),
        },
    )

    return {"prediction": prediction}

For metrics, expose a Prometheus endpoint:

from prometheus_client import Histogram, Counter

PREDICTION_LATENCY = Histogram(
    "prediction_latency_seconds",
    "Time spent processing prediction",
)
PREDICTION_COUNT = Counter(
    "predictions_total",
    "Total predictions",
    ["predicted_class"],
)

@app.post("/predict")
def predict(request: PredictionRequest):
    with PREDICTION_LATENCY.time():
        result = model.predict([request.features])
        prediction = int(result[0])

    PREDICTION_COUNT.labels(predicted_class=str(prediction)).inc()
    return {"prediction": prediction}

You don't need a full observability platform on day one. Structured logs and a few Prometheus metrics get you surprisingly far. The key is having them from the start — retrofitting monitoring after a production incident is stressful.

5. "Works on my machine" deployment

The model works locally. You scp the model file and the serving script to a VM, run python serve.py, and it works. Three weeks later the VM is rebooted, Python is updated by a system package upgrade, and the service doesn't come back up.

Or: you hand the model to the platform team and say "deploy this." They ask: "How? What Python version? What dependencies? Does it need a GPU? How much memory? What port? What's the health check endpoint?" You don't know the answers to half of these.

The fix

Containerize the entire serving stack. A Dockerfile is a deployment specification — it captures everything needed to run the model.

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY serve.py .
COPY model/ model/

EXPOSE 8000
HEALTHCHECK CMD curl -f http://localhost:8000/health || exit 1

CMD ["uvicorn", "serve:app", "--host", "0.0.0.0", "--port", "8000"]

This answers every question the platform team had:

  • Python version: 3.11
  • Dependencies: pinned in requirements.txt
  • GPU: not needed (if it were, use nvidia/cuda base image)
  • Port: 8000
  • Health check: GET /health

The container runs the same way everywhere — on the developer's laptop, in CI, in staging, in production. The "works on my machine" problem is eliminated.

Add a health check endpoint in the serving code:

@app.get("/health")
def health():
    return {"status": "ok"}

Kubernetes, ECS, or whatever orchestrator you use can hit this endpoint to know if the service is alive and ready.

The pattern

All five of these problems have the same root cause: the notebook environment is forgiving in ways that production isn't. In a notebook, you're the only user, you control the input, you can see the output, and the environment is whatever happens to be installed.

Production is the opposite. Multiple users, arbitrary input, invisible output, and a fixed environment that must be explicitly defined.

The fixes aren't complicated. Pin dependencies, bundle preprocessing, validate input, add logging, and containerize. None of these require advanced tools or infrastructure — they're habits that pay off immediately.

Moving models from notebook to production is exactly what deploying.ai helps teams do. If you're hitting these problems (or want to avoid them), let's talk.