# Support Knowledge Base RAG Assistant

> Answer support questions from your own documentation, with citations

| | |
|---|---|
| **Category** | Customer Support |
| **Type** | Agentic Workflow |
| **Difficulty** | Advanced |
| **Automation** | Mostly automated |
| **Runtime** | ~15 min |
| **Est. cost** | $0.35 |
| **Version** | 1.0.0 |

## Purpose

A support assistant that only answers from your documentation. Questions are embedded, the relevant passages retrieved and re-ranked, and the answer composed strictly from what was retrieved — with citations the user can click.

The important behaviour is the refusal: when the corpus does not contain the answer, it says so and escalates rather than inventing one.

## Problems solved

- Documentation exists but nobody can find the right page
- Generic chatbots answer from training data, not your docs
- No audit trail for what the assistant told a customer
- Documentation drift goes unnoticed until someone complains

## Benefits

- Every answer cites the document it came from
- Refuses rather than inventing when the corpus is silent
- Re-indexes as documentation changes — no retraining
- Retrieval respects the asking user's permissions

## Limitations

- Answer quality is capped by retrieval quality — bad chunks, bad answers
- Chunking strategy needs tuning per corpus
- Multi-hop questions spanning several documents remain hard
- Keeping the index in sync with the source of truth is ongoing work

## Success metrics

- Deflection rate
- Groundedness (answers fully supported by citations)
- Escalation rate and its accuracy
- Documentation gaps surfaced per week

## Prerequisites

- **Api key** — LLM API key
- **Account** — Vector database
- **Software** — Document corpus
- **Permission** — Access-control mapping

## AI tools used

- **ChatGPT**
- **Claude**

## Steps

### 1. Ingest and chunk the corpus

Split documents into passages that each stand alone, preserving headings and document identity.

- _Time:_ 4 min

### 2. Embed and index

Generate embeddings and write them to the vector store with permission metadata attached.

- _Tool:_ Pinecone
- _Time:_ 3 min

### 3. Retrieve for the question

Embed the incoming question and fetch the nearest passages, filtered by what this user is allowed to see.

- _Time:_ 1 min

> ⚠️ Filter by permission at retrieval time — never retrieve everything and rely on the model to decline

### 4. Re-rank

Re-score candidates for actual relevance to the question, not just embedding proximity.

- _Time:_ 1 min

### 5. Compose a grounded answer

Answer strictly from the retrieved passages, citing each claim — or state that the answer is not in the documentation.

- _Model:_ Claude Sonnet
- _Time:_ 2 min

```text
Answer the question using ONLY the passages provided.

Question: {{question}}

Passages:
{{passages}}

Rules:
- Every factual claim must cite the passage it came from, as [1], [2]…
- If the passages do not contain the answer, reply exactly: NOT_IN_DOCS
- Do not use knowledge from outside these passages, even if you are confident
- Be direct. No preamble.
```

> ⚠️ The NOT_IN_DOCS path must be wired to escalation — a question that silently dead-ends is worse than no assistant

### 6. Escalate or resolve

Cited answers go to the user; NOT_IN_DOCS routes to a human and logs a documentation gap.

- _Time:_ 2 min

### 7. Review gaps weekly

The logged NOT_IN_DOCS questions become the documentation backlog.

- _Time:_ 2 min

## FAQ

**Does RAG eliminate hallucination?**

It reduces it substantially but does not eliminate it — the model can still misread a retrieved passage. Clickable citations are what let a user verify, which is why they are not optional here.

**Why is re-ranking a separate step?**

Nearest-neighbour search returns passages that are semantically close, which is not the same as passages that answer the question. Re-ranking is usually the biggest single quality jump available.

**How do we keep the index current?**

Re-index on document change rather than on a schedule. A weekly cron leaves up to a week of wrong answers after a policy update.

---

_Exported from AIToolsay — https://invitationbuddy.com/workflows/support-knowledge-base-rag-assistant_