DocuPilot
Enterprise RAG Platform
An AI assistant that answers staff questions from hundreds of pages of procedures, and shows the exact page every answer came from.
- 600+Evidence 1 · 600+ staffProduction user count: staff with access to DocuPilot at the time of writing.Go to Resultsstaff
- 14Evidence 2 · 14 document categoriesDocument categories configured in production.Go to Resultsdocument categories
- 12,000+Evidence 3 · 12,000+ chunksChunks indexed across the production document set.Go to Resultschunks
- ~40%Evidence 4 · ~40% fewer manual queriesAn estimate: the drop in procedure questions reaching supervisors after rollout.Go to Resultsfewer manual queries
- sub-100 msEvidence 5 · sub-100 ms retrievalMedian vector-search time, taken from production query logs.Go to Resultsretrieval
What do I do if the cash drawer count is short at closing?
Recount the drawer with a second staff member present, then record the difference on the closing sheet1. Tell your supervisor before you leave; differences over the limit need a short written note2.
1 Cash Handling SOP, page 142 Closing Checklist, page 3
The problem
A US youth-sports organization runs on detailed procedures: SOPs, training guides, checklists. Staff kept asking supervisors questions that were answered somewhere in those documents. They needed instant answers, but a wrong answer here means someone follows the wrong procedure.
Constraints
- Documents are PDFs and Word files full of tables, checklists and scanned pages.
- Not every staff member is allowed to see every document.
- It has to run on the organization’s own Windows server with a GPU.
- Every answer must be checkable by the person reading it.
How it works
- Upload
- Duplicate checksha-256
- Parse structuredocling · ocr · tables
- Structure-aware chunkspage + position kept
- Embedtext-embedding-3-small
- One index per documentfaiss
- Loginjwt
- Allowed documents for this role
- Search each allowed documentfaiss
- Write the answergpt-4o-mini
- Answer-grounded citation selection
- Answer with page highlights
Documents are parsed for structure, not just text, so a table stays a table. Each chunk remembers its page and position. At question time, only documents the user’s role allows are searched. After the answer is written, it’s compared back against the passages to decide which ones really support it, and only those are cited.
Key decisions
- Parse structure, not just text. Plain PDF text extraction mangles tables and checklists. Docling keeps the document’s real structure, so chunks follow sections and every citation can highlight the exact region on the page.
- One index per document, with access by intersection. A request can narrow the set of documents searched, never widen it. If someone asks for a document they aren’t allowed to see, the system says it doesn’t exist rather than “forbidden”, so it doesn’t even confirm the document is there.
- Answer-grounded citation selection. Retrieval scores say what looked relevant before the answer existed. Blending in each passage’s similarity to the final answer shows what the answer actually relied on.
- A two-stage citation gate. A minimum score stops greetings and small talk from getting fake citations. A ratio to the best match keeps only passages that genuinely support the answer.
Hard problems
- Large scanned documents (around 200 pages) are slow to parse. Parsing runs as a background task with a cached converter, GPU acceleration and a timeout guard, so uploads never block the app.
- Citations on small talk. Without a minimum score, even a reply to “Hello!” would get sources attached. The absolute score floor prevents that.
Results
- ~40% fewer manual questions reaching supervisors.
- 600+ staff across 14 document categories, with 12,000+ chunks indexed.
- Sub-100 ms median retrieval.
- Every query logged with latency, citations and user feedback, so problems are visible.
What I’d do next
- An automated evaluation set (faithfulness and citation accuracy, e.g. with RAGAS), so quality is measured on every change, not only through user feedback.
- Hybrid search (keyword + vector) for exact codes and form numbers that embeddings handle poorly.