How to Build a RAG Chatbot From Your PDFs Step by Step

The goal is a script that takes a question, finds the few passages in your PDFs that matter, and answers only from them, with a page number. When the PDFs do not contain the answer, it says so instead of guessing.

That is all a RAG chatbot is: retrieval first, generation second. The model never "knows" your documents. It sees a handful of pasted passages with each question.

What you need

• Python 3.10 or newer and pip install pypdf openai numpy
• One or more text-based PDFs (not scans)
• An OpenAI API key in the OPENAI_API_KEY environment variable
• A chat model name of your choice, set in CHAT_MODEL below

The embedding and chat calls here use the OpenAI Python client. Any provider with an embeddings endpoint works the same way; only the two API calls change.

First, extract text page by page

Keep the page number with the text. You need it later for citations, and it is hard to recover after chunking.

from pypdf import PdfReader

def load_pages(path):
    reader = PdfReader(path)
    return [(i + 1, p.extract_text() or "") for i, p in enumerate(reader.pages)]

The pypdf docs on text extraction are blunt about the limits: a scanned page is an image, pypdf does not do OCR, and extraction may return little or no text. Print the character count per page. A page with zero characters is a scan, and the bot will never find anything on it. Tables and multi-column layouts may also come out in a jumbled reading order, so look at a sample page before you trust the rest.

Second, split into chunks

Whole pages are too coarse and single sentences lose context. A common middle ground is a few hundred words with some overlap.

def chunk(pages, size=1200, overlap=200):
    out = []
    for page, text in pages:
        text = " ".join(text.split())
        start = 0
        while start < len(text):
            out.append({"page": page, "text": text[start:start + size]})
            start += size - overlap
    return out

Sizes are in characters, and the default values in the function are a starting point to tune on your documents, not a recommendation backed by any benchmark. The overlap exists so a sentence cut at a boundary still appears whole in one chunk. This version never joins text across pages, which is a deliberate simplification: a fact split over a page break can be missed. If your documents have long running sections, merge pages before splitting and store the page range instead.

Third, embed the chunks

import numpy as np
from openai import OpenAI

client = OpenAI()

def embed(texts):
    resp = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return np.array([d.embedding for d in resp.data])

According to the OpenAI embeddings guide, text-embedding-3-small returns 1536 dimensions by default and accepts up to 8192 tokens of input. Embed chunks in batches of a hundred or so rather than one huge request, and save the vectors to disk (np.save) so you do not pay for them again on every run.

Fourth, retrieve the closest chunks

The same guide recommends cosine similarity. With normalized vectors that is a dot product.

def normalize(m):
    return m / np.linalg.norm(m, axis=1, keepdims=True)

def top_k(question, chunks, vectors, k=4):
    q = normalize(embed([question]))[0]
    scores = normalize(vectors) @ q
    best = np.argsort(scores)[::-1][:k]
    return [(chunks[i], float(scores[i])) for i in best]

For a few hundred chunks, plain numpy is enough. A vector database earns its place when the collection stops fitting comfortably in memory, not before.

Fifth, answer from the passages only

CHAT_MODEL = "your-chat-model-name"

def answer(question, chunks, vectors):
    hits = top_k(question, chunks, vectors)
    context = "\n\n".join(f"[page {c['page']}]\n{c['text']}" for c, _ in hits)
    system = (
        "Answer using only the passages below. Cite the page like [page 3]. "
        "If the passages do not contain the answer, reply exactly: "
        "I can't find that in the documents."
    )
    resp = client.chat.completions.create(
        model=CHAT_MODEL,
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": f"{context}\n\nQuestion: {question}"},
        ],
    )
    return resp.choices[0].message.content

The fixed refusal sentence matters. It gives you a string to check for in tests, and it gives the model an easy way out other than inventing something.

Last, test with three questions

Do not judge the bot on questions you already know it answers. Use three kinds, and write down the expected result before you run each one.

1. Answer in one place. Pick a fact stated once in a single paragraph. Expect the right answer and the right page.
2. No answer in the documents. Ask about something your PDFs never mention. Expect the exact refusal sentence. If you get a confident answer, the model is using its own training data, so tighten the system prompt and lower k.
3. Answer spread over two chunks. Pick a question whose parts sit in different places, for example a limit in one section and its exception in another. Expect both pages cited. If only one appears, raise k or the chunk overlap and rerun all three tests, since changes for one often break another.

Nothing here guarantees correct answers. The tests show whether retrieval and refusal behave on your documents, and that is the most a script like this can promise.

Where it goes next

Once the three tests pass, the practical upgrades are re-ranking the retrieved chunks, keeping the chunk text and page in a small database, and putting the bot behind a web form. If you plan to put it on a site, the older note on embedding a chatbot covers the iframe side, and the chatbot glossary defines the terms used above.