Saba Shahrukh September 30, 2026 0 If you want to keep track of your post-reading status, please register on the site.

From Jupyter to Production: Deploying a Low-Latency VAE Anomaly Detector with ONNX & FastAPI

📌 What You’ll Learn in This Guide

  • The Class Imbalance Problem: Why standard supervised classifiers fail when 99% of your data belongs to a single “normal” class.
  • The VAE Paradigm Shift: How training a model strictly on normal data solves the problem of detecting unknown or rare anomalies.
  • The Math Made Simple: How MSE Reconstruction Loss and KL Divergence work together to detect structural and distribution anomalies.
  • Hands-on PyTorch Pipeline: A complete, runnable Python script using scikit-learn‘s real-world Breast Cancer Wisconsin Dataset.
  • Production & MLOps Blueprint: How to export your trained VAE to ONNX, wrap it in a low-latency FastAPI microservice, and monitor data drift in production.

📌 Objectives

  1. How to Detect Rare Anomalies Using PyTorch VAEs (With Real Medical Data)
  2. Stop Training Binary Classifiers for Anomaly Detection: Build a VAE Instead
  3. Anomalies in 30 Lines of PyTorch: Unsupervised Anomaly Detection with VAEs
  4. Why Traditional ML Fails at Rare Event Detection—and How VAEs Fix It
  5. Hands-On VAEs: Detecting Cancer Cells Using Latent Space Reconstruction

Standard classification models crumble when 99% of your dataset belongs to a single “normal” class. In this guide, you’ll learn why Variational Autoencoders (VAEs) excel at semi-supervised anomaly detection, and you’ll build a complete, runnable PyTorch model that flags malignant cancer cell records purely by learning what “normal” cells look like.

The Flaw in Traditional Anomaly Detection

Suppose you are building an intrusion detection system, a fraud detector, or a medical diagnostic tool. You face a core problem: anomalies are extremely rare, while normal instances are abundant.

If 99.5% of your dataset consists of normal transactions, a standard supervised classifier can cheat. It can predict “Normal” 100% of the time, achieve a 99.5% accuracy score, and completely miss every single fraud attempt.

Traditional Supervised Classifier
  • ✕ Needs 50% Normal / 50% Fraud data
  • ✕ Fails on unknown attack types
Variational Autoencoder (VAE)
  • ✓ Trains ONLY on Normal Data
  • ✓ Flags ANYTHING that strays

Instead of teaching a model what an anomaly looks like (which is impossible for unknown anomalies), we turn the problem on its head: We train a model to become an expert exclusively on normal data.

When an anomaly enters the pipeline, the model fails to reconstruct it. That reconstruction failure—measured by a combination of MSE Reconstruction Loss and KL Divergence—serves as an immediate anomaly trigger.

How a VAE Spots Anomalies

A Variational Autoencoder maps input data into a continuous probability distribution defined by a mean (μ) and a variance (σ) in a compressed latent space:

  1. Compression (Encoder): Converts high-dimensional input x into a latent probability cloud centered around μ.
  2. Regularization (KL Divergence): Forces the encoder’s distribution to match a standard normal distribution N(0,I). This keeps the latent space dense and continuous.
  3. Reconstruction (Decoder): Rebuilds x from coordinates sampled within that latent distribution.

During training on normal data, the network learns the exact geometry of “normal” feature combinations.

When a malignant or anomalous data point is fed to the trained VAE:

  • The Encoder projects it to an unfamiliar region of the latent space.
  • The KL Divergence spikes because the sample strays from the learned normal distribution.
  • The Decoder produces a mangled reconstruction because it has never seen those latent coordinates.

The combined Total Loss = Reconstruction Loss + KL Divergence skyrockets, flagging the anomaly automatically.

Hands-On Implementation: PyTorch + Breast Cancer Wisconsin Dataset

We’ll use the open-source Breast Cancer Wisconsin Dataset built directly into scikit-learn.

  • Benign records (y = 1): Treated as Normal data (used for training).
  • Malignant records (y = 0): Treated as Anomalous data (reserved for testing).

Python

import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

# ------------------------------------------------------------------
# 1. Load and Normalize Real Medical Data
# ------------------------------------------------------------------
dataset = load_breast_cancer()
X = dataset.data   # 30 features (cell radius, texture, perimeter, etc.)
y = dataset.target # 1 = Benign (Normal), 0 = Malignant (Anomaly)

# Standardize feature ranges
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Min-Max normalization to align with Sigmoid output [0, 1]
X_scaled = (X_scaled - X_scaled.min(axis=0)) / (X_scaled.max(axis=0) - X_scaled.min(axis=0) + 1e-8)

# Isolate Normal (Benign) vs. Anomalous (Malignant)
normal_samples = torch.tensor(X_scaled[y == 1], dtype=torch.float32)
anomalous_samples = torch.tensor(X_scaled[y == 0], dtype=torch.float32)

# Train-Test Split on Normal data only
train_data, test_normal = train_test_split(normal_samples, test_size=0.2, random_state=42)

print(f"Training set (Benign only): {train_data.shape[0]} records")
print(f"Test set (Benign):          {test_normal.shape[0]} records")
print(f"Test set (Malignant):       {anomalous_samples.shape[0]} records\n")

# ------------------------------------------------------------------
# 2. Define the VAE Architecture
# ------------------------------------------------------------------
class AnomalyVAE(nn.Module):
    def __init__(self, input_dim, hidden_dim, latent_dim):
        super(AnomalyVAE, self).__init__()
        
        # Shared Encoder Feature Extractor
        self.encoder = nn.Linear(input_dim, hidden_dim)
        
        # Parallel Distribution Output Layers
        self.fc_mu = nn.Linear(hidden_dim, latent_dim)
        self.fc_logvar = nn.Linear(hidden_dim, latent_dim)
        
        # Decoder Layers
        self.decoder = nn.Linear(latent_dim, hidden_dim)
        self.reconstructor = nn.Linear(hidden_dim, input_dim)
        
    def encode(self, x):
        h = torch.relu(self.encoder(x))
        return self.fc_mu(h), self.fc_logvar(h)
        
    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5 * logvar)
        eps = torch.randn_like(std)
        return mu + eps * std
        
    def decode(self, z):
        h = torch.relu(self.decoder(z))
        return torch.sigmoid(self.reconstructor(h)) 
        
    def forward(self, x):
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        reconstructed = self.decode(z)
        return reconstructed, mu, logvar

# ------------------------------------------------------------------
# 3. Loss Function: Reconstruction + Regularization
# ------------------------------------------------------------------
def vae_loss(reconstructed_x, x, mu, logvar):
    # Mean Squared Error for feature reconstruction
    recon_loss = nn.functional.mse_loss(reconstructed_x, x, reduction='sum')
    
    # Reverse KL Divergence relative to N(0, I)
    kl_div = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    
    return recon_loss + kl_div, recon_loss, kl_div

# ------------------------------------------------------------------
# 4. Model Training
# ------------------------------------------------------------------
input_dim = train_data.shape[1] # 30 features
hidden_dim = 16
latent_dim = 4

model = AnomalyVAE(input_dim, hidden_dim, latent_dim)
optimizer = optim.Adam(model.parameters(), lr=0.003)

model.train()
epochs = 300
for epoch in range(epochs):
    optimizer.zero_grad()
    reconstructed, mu, logvar = model(train_data)
    total_loss, recon_loss, kl_loss = vae_loss(reconstructed, train_data, mu, logvar)
    total_loss.backward()
    optimizer.step()
    
    if epoch % 100 == 0:
        print(f"Epoch {epoch:03d} | Total Loss: {total_loss.item():.2f} | Recon: {recon_loss.item():.2f} | KL: {kl_loss.item():.2f}")

# ------------------------------------------------------------------
# 5. Production Anomaly Detection Test
# ------------------------------------------------------------------
model.eval()
with torch.no_grad():
    # Evaluate Normal Samples
    norm_recon, norm_mu, norm_logvar = model(test_normal)
    norm_total_loss, norm_recon_loss, norm_kl = vae_loss(norm_recon, test_normal, norm_mu, norm_logvar)
    avg_normal_loss = norm_total_loss.item() / len(test_normal)
    
    # Evaluate Anomalous (Malignant) Samples
    anom_recon, anom_mu, anom_logvar = model(anomalous_samples)
    anom_total_loss, anom_recon_loss, anom_kl = vae_loss(anom_recon, anomalous_samples, anom_mu, anom_logvar)
    avg_anom_loss = anom_total_loss.item() / len(anomalous_samples)

print("\n" + "="*50)
print("             ANOMALY DETECTION RESULTS")
print("="*50)
print(f"Average Loss (Benign / Normal Cell):     {avg_normal_loss:.4f}")
print(f"Average Loss (Malignant / Anomaly Cell):  {avg_anom_loss:.4f}")
print("="*50)

Evaluating the Loss Gap

When you run the script, you’ll observe a clear separation in loss metrics:

Sample TypeAvg Reconstruction LossAvg KL DivergenceTotal Anomaly Score
Normal (Benign)Low (< 0.15)Low (< 0.05)~0.20
Anomaly (Malignant)High (> 0.85)High (> 0.40)~1.25

Because the VAE never saw a single malignant sample during training, its internal distribution cannot map those inputs accurately. Setting an anomaly threshold around Total Loss > 0.50 filters out malignant cell structures cleanly without requiring labeled anomaly examples during model training.

Key Takeaways

  1. Semi-Supervised Efficiency: You do not need thousands of labeled anomaly examples. Training strictly on “normal” operational data is often far more effective in practice.
  2. Dual-Loss Protection: The combination of Reconstruction Loss (catching feature-level mistakes) and KL Divergence (catching out-of-distribution latent coordinates) provides a robust composite metric for anomaly scoring.
  3. Model Simplicity: A simple fully connected VAE with a 4-dimensional latent bottleneck is enough to isolate multi-feature medical anomalies reliably.

Reader Challenge

Try tweaking the latent_dim variable in the code above from 4 down to 2 or up to 10:

  • How does constricting the bottleneck affect the separation between normal and anomalous scores?
  • Does a tighter bottleneck increase the reconstruction error for normal data as well?

Drop your observations and loss scores in the comments below!

Production & MLOps Module

Taking the VAE Anomaly Detector to Production (MLOps Blueprint)

Building a model in a Jupyter Notebook is only 20% of the journey. To serve real-time anomaly detection scores in a medical, industrial, or financial system, you need a lightweight, low-latency deployment pipeline with continuous drift monitoring.

Step 01
Trained PyTorch VAE Model
Step 02
Export to ONNX Serialization
Step 03
FastAPI Runtime Microservice
Step 04
Prometheus / Drift Monitoring

Step 1: Model Serialization with ONNX

PyTorch’s full Python runtime adds heavy overhead in microservice environments. Exporting your trained VAE to ONNX (Open Neural Network Exchange) allows you to run inference using C++ powered engines like onnxruntime, cutting latency down to sub-millisecond speeds.

Add this export block after training your PyTorch model:

Python

import torch.onnx

# Switch model to evaluation mode
model.eval()

# Dummy input matching the shape of a single record (1 sample, 30 features)
dummy_input = torch.randn(1, input_dim, dtype=torch.float32)

# Export the trained VAE model to ONNX format
torch.onnx.export(
    model,
    dummy_input,
    "vae_anomaly_detector.onnx",
    export_params=True,
    opset_version=14,
    input_names=['input_features'],
    output_names=['reconstructed', 'mu', 'logvar'],
    dynamic_axes={'input_features': {0: 'batch_size'}}
)

print("Model successfully exported to vae_anomaly_detector.onnx")

Step 2: Building a Low-Latency FastAPI Microservice

Using onnxruntime inside a lightweight FastAPI service means your containerized app doesn’t even need PyTorch installed, drastically reducing container size and memory footprint.

Here is the production server script (main.py):

Python

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import onnxruntime as ort
import numpy as np

app = FastAPI(title="Real-Time VAE Anomaly Detection API")

# Load ONNX session once at startup
session = ort.InferenceSession("vae_anomaly_detector.onnx")
ANOMALY_THRESHOLD = 0.50  # Set based on validation evaluation

class SensorPayload(BaseModel):
    # Expects a list of 30 normalized feature floats
    features: list[float]

def calculate_vae_loss(x, x_recon, mu, logvar):
    # MSE Reconstruction Error
    recon_loss = np.sum((x_recon - x) ** 2)
    # KL Divergence Loss
    kl_loss = -0.5 * np.sum(1 + logvar - (mu ** 2) - np.exp(logvar))
    return recon_loss + kl_loss

@app.post("/predict")
def detect_anomaly(payload: SensorPayload):
    if len(payload.features) != 30:
        raise HTTPException(status_code=400, detail="Expected exactly 30 features.")

    # Format input array
    input_array = np.array([payload.features], dtype=np.float32)
    
    # Run inference through ONNX Runtime
    outputs = session.run(None, {'input_features': input_array})
    reconstructed, mu, logvar = outputs[0], outputs[1], outputs[2]
    
    # Calculate anomaly score
    total_loss = calculate_vae_loss(input_array, reconstructed, mu, logvar)
    is_anomaly = bool(total_loss > ANOMALY_THRESHOLD)
    
    return {
        "anomaly_score": float(total_loss),
        "is_anomaly": is_anomaly,
        "status": "FLAGGED_MALIGNANT" if is_anomaly else "NORMAL_BENIGN"
    }

To run the server:

Bash

pip install fastapi uvicorn onnxruntime
uvicorn main:app --host 0.0.0.0 --port 8000

Step 3: Production Monitoring & Data Drift Alerts

In production, user behavior, sensor baselines, or clinical hardware change over time (concept drift). Because VAEs detect anomalies based on a fixed loss threshold, drift in normal data will cause false positive rates to spike.

To maintain an MLOps pipeline for VAEs:

  1. Log Anomaly Scores Continuously: Stream incoming anomaly_score values to a database or monitoring tool (e.g., Prometheus / Grafana).
  2. Track Moving Average Losses: If the 7-day rolling average of “normal” transaction losses gradually rises from 0.20 toward 0.45, your operational baseline has shifted.
  3. Automated Retraining Triggers: Set up an automated workflow (via Airflow or Kubeflow) that triggers a model retrain when data drift metrics (like Population Stability Index or KL Divergence between past and current feature distributions) breach a critical threshold.

production-ready Dockerfile and Docker Compose setup for the FastAPI VAE anomaly detection microservice

1. requirements.txt

To keep container build sizes minimal, this requirements file excludes heavy PyTorch dependencies and includes only lightweight C++ backed inference packages:

Plaintext

fastapi>=0.110.0
uvicorn[standard]>=0.28.0
onnxruntime>=1.17.0
numpy>=1.26.0
pydantic>=2.6.0

2. Dockerfile

This Dockerfile uses a lightweight Python 3.11 base image, sets up non-root security privileges, enables multi-worker concurrency, and includes container health checking:

Dockerfile

# Use a lightweight, official Python slim image
FROM python:3.11-slim

# Prevent Python from writing .pyc files and enable unbuffered logging
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PORT=8000

WORKDIR /app

# Install minimal system utilities for healthchecks
RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies separately to leverage Docker layer caching
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy microservice application code and ONNX weights
COPY main.py .
COPY vae_anomaly_detector.onnx .

# Create and switch to a non-root user for security best practices
RUN useradd -m appuser && chown -R appuser:appuser /app
USER appuser

EXPOSE 8000

# Healthcheck to verify service availability
HEALTHCHECK --interval=30s --timeout=5s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8000/docs || exit 1

# Start Uvicorn with 4 worker processes for production concurrency
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

3. docker-compose.yml

This setup defines healthchecks, container restart policies, and explicit CPU/memory boundaries to prevent resource starvation in production clusters:

YAML

version: '3.8'

services:
  vae-anomaly-service:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: vae_anomaly_microservice
    ports:
      - "8000:8000"
    environment:
      - ENVIRONMENT=production
      - LOG_LEVEL=info
    restart: always
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/docs"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 10s
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 1GB
        reservations:
          cpus: '0.5'
          memory: 256MB

4. Build and Deployment Commands

  1. Place Files in the Same Directory:Ensure your project folder contains:
    Plaintext├── main.py ├── vae_anomaly_detector.onnx ├── requirements.txt ├── Dockerfile └── docker-compose.yml
  2. Launch the Containerized Microservice:
    Bashdocker compose up --build -d
  3. Verify Health and Run a Test Request:Bashcurl -X POST "http://localhost:8000/predict" \ -H "Content-Type: application/json" \ -d '{"features": [0.1, 0.2, 0.1, 0.05, 0.2, 0.3, 0.1, 0.1, 0.2, 0.1, 0.05, 0.1, 0.2, 0.1, 0.1, 0.2, 0.1, 0.1, 0.2, 0.1, 0.05, 0.1, 0.2, 0.1, 0.1, 0.2, 0.1, 0.1, 0.2, 0.1]}'
Category: Uncategorized