Dead Simple Two Brain System for Reliable AI Chatbots

Recently I had to build a chatbot that works on factual data instead of long blocks of text. Like most people, I started with a basic RAG approach, convert everything to text, store it in Chroma, retrieve it when needed. It looked promising at first, until it wasn’t.

Why did it fall short?

Because limiting the number of retrieved results limits everything else.

With cost constraints, you can’t pull a huge number of chunks. For simple questions that only need a small amount of context, RAG behaves well. But once a query requires a wider view, comparing stats across seasons, listing every player in a match, assembling several related details, things begin to break down.

I even considered pre writing answers for every possible question that could come from the data, but the combinations were endless and completely unmanageable.

The solution: a two brain system

I ended up moving to a simple two step pipeline inspired by agent style reasoning, but without the unnecessary complexity.

Brain 1: Reformulate the query and identify needed indexes

This first step receives the chat history (trimmed to keep token usage practical) and rewrites the user’s latest message into a clean standalone query that includes all needed context.

Then it figures out which data sources are required. To make this work, your data should be organized into clear chunks:

interface Chunk {
  id: string;
  description: string;
  data: array | json | string | toon | databaseTable;
}

Examples could be:

• generalInformation
• season1Matches
• season1Players
• season2FanRatings

Brain 1 only needs the id and description, and returns structured output like:

interface Answer {
  reformulatedQuery: string;
  indexes: string[];
}

This gives you flexible context and targeted data selection without relying on blind text retrieval.

Brain 2: Fetch the data and generate the final answer

Using the reformulated query and selected indexes, you retrieve the relevant data and feed it to a stronger LLM.

Its job is simple: read real data and produce a clear, factual answer.
No hallucination-prone guessing. No missing context. No unpredictable search results.

Token usage becomes more stable, and answers become more reliable.

Try the two brain system

Pick a question. Yakku shows what basic RAG and the two brain pipeline do with it. The data is made up, the flow is real.

Indexes generalInformation season1Matches season1Players season2Players season2FanRatings

Top scorer across both seasons? Who played in the opening match? Follow up: "and compared to the first?"

Possible failure points

Every system has limits, and this one is no different:

• If you have too many indexes, Brain 1 may become expensive and harder for weaker models to handle.
• Poorly designed indexes lead to poor data retrieval.
• If the conversation requires very long memory, the trimmed history might miss important intent.

Ways to improve the design

You can take this approach even further:

• Split Brain 1 into parallel workers. Each works on a subset of indexes, and their outputs are combined, similar to map reduce.
• For raw or unstructured information (like live news), you can still use RAG as an optional support system.
• Instead of passing full chat history every time, use a conversation summary. This saves tokens and keeps reasoning clean.
• For structured data, use database tables directly as indexes and have the model generate SQL. This avoids loading large datasets unnecessarily.

Final thoughts

This two brain system is a simple and effective way to build faster, more accurate chatbots that rely on real data instead of scattered text retrieval. It provides clearer reasoning, flexible context, and far better control than classic RAG.