forglemmigej

The problem

A flashcard app has to decide every day which cards to show. Show a card too early and I waste time; too late and I’ve forgotten the word. Behind that decision is a prediction: how likely am I to remember this card right now? So the scheduler is really a machine-learning model whose output drives a real decision, and the ground truth arrives immediately, because every review is a label.

I’m moving to Denmark and learning Danish with proper courses, like DanishClass101 and Dansk Sprogskole, but forglemmigej is the daily vocabulary practice next to them, and it was a good excuse for an MLOps project. I use it myself every morning, so a version that only looked good in a demo would have been useless to me.

What I built

A phone-first web app that shows Danish words in short, human-written example sentences. A content pipeline builds the deck from a frequency list, Wiktionary and Tatoeba. It keeps a sentence only when its English translation confirms the word’s meaning, since a learner can’t spot a wrong card.

The scheduler is FSRS, the algorithm built into Anki. Alongside it, challenger models predict every review in shadow mode: a LightGBM model pre-trained on public Anki review logs, and FSRS refitted to my own reviews once there’s enough data. Every prediction is logged, so all models are scored on exactly the same reviews.

Why it’s an MLOps project

Training a model is the easy part. The harder question is when a new one is allowed to take over. A nightly job evaluates every model on reviews it hasn’t seen. A challenger gets promoted only if it predicts better in at least 95% of bootstrap resamples over cards, on at least 300 reviews, and isn’t worse calibrated, because the scheduler acts on the probability itself.

So far the gate hasn’t promoted anything, which is what it should do: at a few dozen useful reviews a day, the evidence takes weeks to build up. On public data the LightGBM model clearly beats default FSRS (log loss 0.334 vs 0.405). The gate is there to find out whether that holds on my own reviews.

The rules come from DevOps habits: an append-only review log enforced by the database, point-in-time features shared by training and serving, a model registry with exact parameters, and a rollback that rebuilds every card’s state from the log.

Infrastructure

Everything runs on a small HP mini PC at home with no GPU: Debian provisioned with Ansible, single-node k3s, PostgreSQL, MLflow and Grafana. Access goes through Tailscale only, and nothing is exposed to the internet. GitHub Actions lints, tests and builds every change, with actions pinned to commit SHAs. Deployment runs from my laptop, because hosted runners can’t reach the homelab and I didn’t want to open a way in.

I switch the machine off in the evening and when I travel, so the nightly job runs before shutdown and Kubernetes stops PostgreSQL cleanly. The model report is pushed out as static data, so it stays online while the server is off.