22 Aug 2026 · 7 min read · project: rocketml
What it takes to make a small model production-shaped
RocketML wraps a TF-IDF and LogisticRegression sentiment classifier, deliberately small and deliberately unremarkable, in CI, a hand-written Helm chart, and real Prometheus and Grafana monitoring. The model isn't the point: getting any small model from a training script to a monitored, Kubernetes-deployed API is. That includes a README number that sat wrong until someone actually checked it against the live API.
How a push becomes a running container
Every push and every pull request runs the same two gates: lint, then test. Only a push to main goes further, and it only goes further once the test job passes:
The last three stages exist because the serving image can't be assembled from anything sitting in the repo: the artifact is gitignored, so CI has to produce a fresh one before it can bake it in. Every image on main carries a model that was just trained, not one checked in months ago.
Key decision
train.py logs metrics and registers every run. But serving never talks to it. Training also writes a plain joblib artifact, the image bakes that in, and a config-driven loader reads it straight off disk, lazily and cached, which is also why /health doesn't depend on the model being loaded at all. docker run the container by itself, with no MLflow service anywhere nearby, and it still predicts, because there's nothing left at runtime for it to depend on. The tradeoff: promoting a model means re-running training and rebuilding the image rather than pointing serving at a new registry version. At this scale that's simpler than adding a live registry client to the request path.The alternative that was actually running first was heavier than it needed to be, and the obvious-looking fix for that turned out not to work:
| Approach | Runtime deps | Image size | Outcome |
|---|---|---|---|
| Full mlflow client | Flask, SQLAlchemy, pandas, mlflow | 1.23 GB | Works, but the registry client is most of the weight |
| mlflow-skinny | -- | n/a | Rejected: metapackage in MLflow 3.x, ships no importable module |
| Joblib, baked in | scikit-learn only | 561 MB | Chosen |
Getting it onto Kubernetes
The chart in deploy/helm/rocketml isn't helm create output. Writing it by hand, a Deployment and a ClusterIP Service with health probes and resource requests and limits, was the point of that phase: this was my first real Kubernetes and Helm work, and the fastest way to actually understand what each piece of a chart does was to write it rather than generate it. In-cluster, RocketML gets scraped through a ServiceMonitor instead of a static Prometheus config, so the Prometheus Operator that ships with kube-prometheus-stack discovers it on its own.
None of it worked on the first try. A ServiceMonitor without the release: monitoring label gets silently ignored by the Operator's Prometheus, no error, nothing scraped. A helm upgrade that requests more than it limits gets rejected outright at admission, which is a safe failure but not an obvious one the first time it happens. And a local image built with docker build is invisible to a kind cluster until it's explicitly kind loaded, since kind keeps its own image store separate from Docker's.
The number that was wrong
The README's curl example claimed a score of 0.93, predict_proba's confidence for one specific negative review, not an accuracy metric. It was wrong.
What actually happened
Known limitations
Stated plainly rather than left for someone else to find:
- MLflow, Prometheus, and Grafana are real, but nothing about them is public. They come up through docker compose or a local kind cluster, not anywhere hosted. There's no public dashboard to link to for any of the three, only the local stack this repo can bring up on demand.
- CI retrains the model on every push to main. About 1-2 minutes each time, downloading IMDB and refitting. Deterministic, so it's not wrong, but it's wasteful. A later version would train once, version the artifact, and decouple the build from training.
- The Helm chart still has rough edges. No startupProbe yet, so the readiness probe can flap on a cold start. The Grafana dashboard hardcodes a datasource UID instead of templating it. The ServiceMonitor lives in deploy/k8s/ rather than the chart itself.
- The model's own accuracy number is intentionally unremarkable. About 0.86 accuracy and 0.87 F1 on a 5,000-review IMDB subset. Nobody should be impressed by it. That's not what this project is trying to demonstrate.
Building RocketML wasn't really about the sentiment model. It was about everything a model needs around it before someone else can depend on it: a container that doesn't need a live connection to a registry to answer a request, CI that won't build an image until the tests pass, and a chart I wrote by hand so I'd understand what a rolling deploy actually does instead of trusting a template I didn't write. The roadmap has a few more of these left: Terraform for the cluster itself, ArgoCD for GitOps-style deploys, drift monitoring on the predictions this already logs, and rate limiting on the public endpoint before it actually needs it.
Python · FastAPI · scikit-learn · Docker · GitHub Actions · MLflow · Prometheus · Grafana · Kubernetes · Helm