CrickPal AI Summary
Real-Time LLM Commentary Engine
Live ball-by-ball commentary in, readable match summaries out, updated every few deliveries by a self-hosted LLM.
AI headline
Hawks recover from 42 for 3 with a 90-run stand
AI summary
Early wickets left the Hawks wobbling at 42 for 3 inside the powerplay. A patient fourth-wicket partnership turned the innings around, rotating strike against spin and cashing in on short deliveries late. They reach the final four overs with wickets in hand and a total above par in sight.
- After over 6
- After over 12
- After over 16latest
The problem
CrickPal has live ball-by-ball commentary for every match. Readers who arrive mid-match want two things fast: what’s happened so far, and the headline.
Constraints
- Live and continuous, with several matches running at once.
- Cost must stay flat as matches get longer.
- After a crash, no summary can be missed or duplicated.
How it works
- Live commentarymongodb
- Poller checks active matches
- Batch every few deliveries
- Previous summary + new deliveries only
- Self-hosted LLMollama
- Narrative + headline
- Saved togethersql server · one transaction
- Summary APIfastapi
- Admin consoleangular · jwt
A poller checks each active match in turn and batches new deliveries every few balls. The self-hosted model gets its previous summary plus only those new deliveries, writes the detailed narrative, then a one-line headline from it. Both are saved together in one transaction and served through FastAPI, with an admin console for per-match settings.
Key decisions
- Incremental, not full re-summarization. The model gets its own previous summary plus only the new deliveries. The prompt stays roughly the same size from the first over to the last.
- Exactly-once processing on a single instance. A timestamp watermark, a carried-over ball count and a dedup table keyed on each delivery’s ID. A failed batch is never marked done, so it’s retried on the next cycle.
- Self-hosted model, protected. Ollama means no per-token bill. A circuit breaker and backoff stop retries from hammering a struggling model, and JSON output mode keeps responses parseable.
- Settings in the database, not the code. Prompt, model, temperature and batch size can be overridden per match from the admin console, with system defaults as the fallback. No redeploys to experiment.
Hard problems
- Junk output. Models sometimes echo the prompt or return placeholder text. An output guard rejects it before it’s saved.
- Matches that go quiet. After an idle timeout, leftover deliveries are flushed into a final summary and processing stops by itself.
Results
- Narrative and headline summaries for live matches, updated every few deliveries.
- No per-token cost, and prompt size that doesn’t grow with the match.
- Prompt changes tested live, per match, with no redeploy.
What I’d do next
- Exclusive match ownership between servers, so several instances can share the load without ever processing the same match twice.
- An evaluation set to score summary accuracy against the actual commentary.