Why not one prompt
The guard, the classifier and the three specialists could all have been one system prompt with a long list of instructions. They are separate because their failure modes are separate: a model asked to refuse off-topic questions and take an order will do both less reliably than two models asked to do one each, and when it goes wrong there is no way to tell which instruction lost.
Splitting them also makes the refusal cheap. The guard runs first and returns immediately on "not allowed", so a question about the weather never reaches retrieval or the order agent at all.
The routing chain
AgentController runs a fixed pipeline:
- Guard agent — decides
allowedornot allowedagainst an explicit list of what the shop will answer. Questions about staff, or how a menu item is made, are refused by name. - Classification agent — picks one of
details_agent,order_taking_agentorrecommendation_agent. - The chosen specialist — produces the reply.
Each agent returns the same envelope: a role, a content, and a memory object naming which
agent ran and what it decided. That envelope is the state mechanism — the conversation carries its
own history forward, so the next turn's agent can read what the last one concluded without a
session store.
Only the last three messages are sent to the guard and the classifier. Routing does not need the whole transcript, and sending it would grow the prompt without improving the decision.
Grounding the details agent
Questions about the shop go through retrieval, not the model's memory. The user's message is embedded through a separate embedding endpoint, queried against a Pinecone index, and the top matches are pasted into the prompt as explicit context before the question.
The index is built in build_vector_database.ipynb from the shop's own material — the product
catalogue and an "about us" document. That is what stops the assistant inventing an opening time.
Taking an order
The order-taking agent holds the menu and its prices in its prompt and validates every item against it. An item that is not on the menu is named back to the user and dropped rather than silently accepted, and the remaining valid order is repeated. The order accumulates in the agent's own output object across turns, and the agent chains into the recommendation agent once the order is complete.
Recommending from data, not from the model
The recommendation agent does not ask the model what to suggest. It has two sources:
- Apriori association rules, mined in
recommendation_engine_training.ipynbfrom a real transaction dataset of around 150,000 sales receipts. Results are sorted by confidence, and capped at two per product category so a single popular category cannot fill the whole list. - Popularity, as a fallback when no association rule applies.
The model's role is to phrase the recommendation, not to choose it.
Making the model's JSON trustworthy
Every structured agent runs its output through double_check_json_output, a second model call
that repairs malformed JSON before it is parsed. A model asked for JSON returns almost JSON often
enough that parsing it directly is a crash waiting for the demo.
Serving and the client
The Python side is a RunPod serverless handler — runpod.serverless.start with the controller as
the handler — packaged in a Docker image, so GPU capacity is only paid for while a request is in
flight. The model is reached through the OpenAI client pointed at a RunPod base URL, which keeps
the code portable across any OpenAI-compatible endpoint.
The client is React Native with Expo, using file-based routing, NativeWind for styling, and Firestore for the product catalogue. It carries a chat room, a menu, product detail pages, and a cart with a delivery toggle.