Concierge Chatbot
Long-Context Visitor Assistant
A visitor assistant that started as a full RAG pipeline, which I replaced with one cached prompt once it was clear the content was only about 50 FAQs.
Anywhere good to eat near the fields?
Can you book us a table for tonight?
I can't make bookings, but the event help desk can point you to places that take reservations.
Answers from about 50 FAQs
They all fit in one cached prompt, so there's no search step that can miss.
Suggests what to ask next
Only from the real FAQ list, never invented.
Clear limits
Out-of-scope requests get a fixed reply, so it never promises anything.
The problem
Families visiting a US youth-sports organization’s events ask about dining, lodging and attractions nearby. The organization wanted an assistant that answers those questions and never makes promises on its behalf.
Constraints
- Answers must stay accurate as the information changes.
- Out-of-scope questions must be declined cleanly.
- It should be cheap and simple to run, since it’s a small, low-traffic service.
How it works
- Website scrape
- Q&A pairs
- Change detectioncontent hash
- Versioned indexfaiss
- Daily scheduler
- Scope classifier
- Answer
- All ~50 FAQs in one cached prompt
- One model call: route + answer + follow-up
- Fixed replies for out-of-scope questions
The first version was a full RAG pipeline: scraping, change detection, a versioned index and a daily scheduler. Now all ~50 FAQs sit in one cached system prompt, one model call returns the route, the answer and a follow-up suggestion, and out-of-scope replies are fixed, pre-written text.
Key decisions
- Know when not to use RAG. The concierge content turned out to be only about 50 FAQs, which fits in one prompt. Removing retrieval meant fewer moving parts and no retrieval misses, and prompt caching keeps it cheap.
- One call per message. A single model call decides the route, writes the answer and picks a follow-up suggestion, instead of a chain of calls.
- Constrain the model where it matters. Follow-up suggestions can only come from the real FAQ list, and out-of-scope replies are fixed text, so the bot never invents a promise on the organization’s behalf.
Results
- A simpler concierge: one model call per message and no index to maintain.
- No retrieval step that can miss the right answer.
What I’d do next
- Build an evaluation set and compare the RAG and long-context designs on accuracy and cost, with numbers.