Ask Your Business Documents Questions Offline with Ollama and Python

Every small business has a folder of documents nobody reads. The supplier price list from March. The SOP for handling returns. The rental contract. The BPJS letter. The 40-page manual for the espresso machine. When a question comes up (“what’s the notice period on the shop lease?”, “what’s the grosir price for 10 cartons?”) somebody either digs through PDFs for ten minutes or asks the owner on WhatsApp. The owner gets that question five times a week.

In this guide you’ll build a small tool that answers those questions from your own files, and shows which file and which page each answer came from, so you can check it in seconds. It runs entirely on your own computer with Ollama, a free tool for running open AI models locally. Nothing gets uploaded, there’s no API key, and there’s no per-question fee.

The tool:

• reads every PDF, .txt and .md file in a docs/ folder
• splits them into chunks and turns each chunk into an embedding (a list of numbers that captures meaning) with the multilingual bge-m3 model, so Indonesian and English documents both work
• only re-processes files that changed since the last run
• finds the most relevant chunks for your question and has Qwen3 8B answer from those chunks only, with source numbers
• prints the file name, page and match score for every source

Software cost: $0. Build time: about 45 minutes, most of it waiting for model downloads.

Before you build: do you even need this?

Be honest about the alternative. Google NotebookLM (notebooklm.google.com) does almost exactly this for free with a Google account: upload PDFs, ask questions, get cited answers. It needs no code and its answers will usually be better than an 8B model on a laptop.

Build the local version when at least one of these is true:

• the documents are things you shouldn’t upload to a third party: employee data, customer lists, contracts, financials. Indonesia’s Personal Data Protection Law (UU PDP, No. 27/2022) applies to personal data in those files
• you want it to work offline, or on a shop PC that staff use without your Google account
• you want to plug it into something else later (a Telegram bot, an internal web page)

If none of those apply, use NotebookLM and skip the rest of this article.

What you need

The two models, pulled through Ollama:

• bge-m3: embedding model from BAAI, about 1.2 GB, MIT licence, trained on 100+ languages including Indonesian. This is the important choice. Many tutorials use nomic-embed-text, which is mainly English and gives noticeably worse matches on Indonesian documents.
• qwen3:8b: chat model from Alibaba, about 5.2 GB download, Apache 2.0 licence, handles Indonesian and English well. On an 8 GB machine use qwen3:4b (about 2.5 GB) instead.

Running cost: electricity. A laptop drawing ~45 W for an extra hour a day is about 1.4 kWh a month, roughly Rp 2,000 at PLN’s ~Rp 1,445/kWh household rate. For comparison, sending the same questions to a cloud API would also cost cents, so the real reason to go local is privacy, not price.

How fast is it?

Speed depends almost entirely on your hardware, so measure before you promise anyone anything. Ollama prints the real numbers if you run ollama run qwen3:8b --verbose and ask something: look at the eval rate line (tokens per second).

As a rough guide for an 8B model at Ollama’s default 4-bit quantisation:

• Recent laptop, CPU only: single digits to low teens of tokens per second. A 150-word answer takes about 20–40 seconds.
• Apple Silicon Mac (M1 or newer, 16 GB): usually fast enough to feel like a chat.
• NVIDIA GPU with 8 GB+ VRAM: a few seconds per answer.

Indexing is a one-off cost: embedding a few hundred pages on a CPU takes a few minutes. After that, only new or changed files are processed.

Step 1: Install Ollama and pull the models

On Linux:

curl -fsSL https://ollama.com/install.sh | sh

On Windows or macOS, download the installer from ollama.com/download and run it. Ollama runs in the background and listens on http://localhost:11434.

Then pull the models and test the chat model once:

ollama pull bge-m3
ollama pull qwen3:8b          # or qwen3:4b on 8 GB RAM
ollama run qwen3:8b "Jelaskan dalam satu kalimat apa itu invoice."

If you get a sensible sentence back, the hard part is done.

Step 2: Set up the project

mkdir docqa && cd docqa
python3 -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install ollama pypdf numpy
mkdir docs

Copy your documents into docs/. Subfolders are fine. Start small: 10–20 files you actually look things up in.

Check that your PDFs contain real text and are not just photos of paper. Open one and try to select a sentence with the mouse. If you can’t, it’s a scan, and pypdf will return nothing for it (see Reality check).

Step 3: The script

Save this as docqa.py:

import hashlib, json, sys
from pathlib import Path

import numpy as np
import ollama
from pypdf import PdfReader

DOCS, INDEX = Path("docs"), Path("index.json")
EMBED_MODEL = "bge-m3"
CHAT_MODEL = "qwen3:8b"       # qwen3:4b on 8 GB RAM
CHUNK_CHARS, OVERLAP, TOP_K = 1200, 200, 5

SYSTEM = """You answer questions using ONLY the numbered sources provided.
Rules:
- After every fact, cite the source number like [2].
- If the sources do not contain the answer, say exactly: "Not found in the documents."
- Copy numbers, prices and dates exactly as written in the source.
- Answer in the same language as the question. Be brief."""

def read_pages(path):
    if path.suffix.lower() == ".pdf":
        reader = PdfReader(path)
        return [(i + 1, p.extract_text() or "") for i, p in enumerate(reader.pages)]
    return [(1, path.read_text(encoding="utf-8", errors="ignore"))]

def split(text):
    text = " ".join(text.split())
    step = CHUNK_CHARS - OVERLAP
    return [text[i:i + CHUNK_CHARS] for i in range(0, len(text), step) if text[i:i + CHUNK_CHARS].strip()]

def file_hash(path):
    return hashlib.sha256(path.read_bytes()).hexdigest()

def build_index():
    old = json.loads(INDEX.read_text()) if INDEX.exists() else {"files": {}, "chunks": []}
    files = [p for p in DOCS.rglob("*") if p.suffix.lower() in (".pdf", ".txt", ".md")]
    current = {str(p): file_hash(p) for p in files}

    # keep chunks of unchanged files, drop deleted/changed ones
    chunks = [c for c in old["chunks"] if old["files"].get(c["file"]) == current.get(c["file"])]
    for p in files:
        key = str(p)
        if old["files"].get(key) == current[key]:
            continue
        new = [{"file": key, "page": page, "text": t}
               for page, text in read_pages(p) for t in split(text)]
        if not new:
            print(f"  ! no text found in {key} (scanned PDF?)")
            continue
        for i in range(0, len(new), 32):
            batch = new[i:i + 32]
            vecs = ollama.embed(model=EMBED_MODEL, input=[c["text"] for c in batch])["embeddings"]
            for c, v in zip(batch, vecs):
                c["vec"] = v
        chunks += new
        print(f"  indexed {key}: {len(new)} chunks")

    INDEX.write_text(json.dumps({"files": current, "chunks": chunks}))
    print(f"Index: {len(current)} files, {len(chunks)} chunks")

def ask(question):
    chunks = json.loads(INDEX.read_text())["chunks"] if INDEX.exists() else []
    if not chunks:
        sys.exit("Index is empty. Put documents in docs/ and run: python docqa.py index")
    mat = np.array([c["vec"] for c in chunks], dtype=np.float32)
    mat /= np.linalg.norm(mat, axis=1, keepdims=True)
    q = np.array(ollama.embed(model=EMBED_MODEL, input=[question])["embeddings"][0], dtype=np.float32)
    q /= np.linalg.norm(q)
    scores = mat @ q
    top = np.argsort(-scores)[:TOP_K]

    context = "\n\n".join(
        f"[{n}] ({chunks[i]['file']}, page {chunks[i]['page']})\n{chunks[i]['text']}"
        for n, i in enumerate(top, 1))
    resp = ollama.chat(
        model=CHAT_MODEL,
        messages=[{"role": "system", "content": SYSTEM},
                  {"role": "user", "content": f"Sources:\n{context}\n\nQuestion: {question}"}],
        options={"num_ctx": 8192, "temperature": 0.1},
        think=False,
    )
    print("\n" + resp["message"]["content"].strip() + "\n")
    print("Sources:")
    for n, i in enumerate(top, 1):
        print(f"  [{n}] {chunks[i]['file']} p.{chunks[i]['page']}  (match {scores[i]:.2f})")

if __name__ == "__main__":
    if len(sys.argv) > 1 and sys.argv[1] == "index":
        build_index()
    elif len(sys.argv) > 1:
        ask(" ".join(sys.argv[1:]))
    else:
        print('Usage: python docqa.py index | python docqa.py "your question"')

A few choices worth explaining:

• Chunks of 1,200 characters with 200 overlap. Small enough that the right paragraph isn’t buried in noise, big enough to keep a table row with its header most of the time. The overlap stops an answer from being cut in half at a chunk boundary.
• Page numbers are stored with every chunk. That’s what makes the answers checkable. An answer you can’t verify is worse than no answer.
• num_ctx: 8192. Ollama’s default context window is small (4,096 tokens on any machine with less than 24 GB of VRAM, which is most office PCs), and five chunks plus the prompt can overflow it. When that happens Ollama silently drops the start of the prompt, which is where the rules are. Setting it explicitly avoids that.
• think=False. Qwen3 “thinks” out loud before answering by default. For lookups that just adds 30+ seconds on a CPU. The think parameter needs a recent Ollama (0.9+) and ollama Python library (0.5+). If you get an error, run pip install -U ollama and update Ollama.
• Plain JSON and NumPy instead of a vector database. For a few thousand chunks, comparing against every one takes milliseconds. You don’t need Chroma, Qdrant or LangChain for a folder of business documents.

Step 4: Index and ask

python docqa.py index

  indexed docs/supplier/pricelist-sinar-jaya-2026.pdf: 14 chunks
  indexed docs/sop/retur-barang.md: 3 chunks
  indexed docs/legal/sewa-ruko.pdf: 22 chunks
Index: 3 files, 39 chunks

Then ask in plain language:

python docqa.py "Berapa lama pemberitahuan kalau mau berhenti sewa ruko?"

Pemberitahuan harus diberikan paling lambat 3 (tiga) bulan sebelum
masa sewa berakhir, secara tertulis [1].

Sources:
  [1] docs/legal/sewa-ruko.pdf p.4  (match 0.71)
  [2] docs/legal/sewa-ruko.pdf p.5  (match 0.58)
  ...

Open page 4 and check. That habit is the whole point of the tool: the AI finds the page, and you read the clause.

Whenever you add or edit documents, run python docqa.py index again. Unchanged files are skipped, so it’s quick.

Step 5: Make it usable for the team

A command line is fine for you. For staff, two simple options:

A desktop shortcut on Windows. Save as tanya.bat next to docqa.py:

@echo off
cd /d %~dp0
call .venv\Scripts\activate
:loop
set "Q="
set /p Q=Pertanyaan: 
if not defined Q goto loop
python docqa.py "%Q%"
goto loop

Auto re-index every night on Linux/macOS with cron (crontab -e):

0 2 * * * cd /home/you/docqa && .venv/bin/python docqa.py index >> index.log 2>&1

If you want staff to ask from their phones, the Telegram bot guide shows the bot plumbing; replace the reply logic with a call to ask() (and change it to return the text instead of printing it). Keep the bot private. Anyone who can message it can read your documents.

What it saves, realistically

Time it rather than guessing. For one week, write down each time someone looks something up in a document and roughly how long it took. A typical small shop might log 10–20 lookups a week at 3–10 minutes each, which is about 30 minutes to 3 hours a week. With the tool, a lookup is the answer time plus 30 seconds to check the cited page.

The bigger win is often not the minutes. It’s that the owner stops being the only person who knows what the contract says.

If you want to earn from this: setting it up for other small businesses (an accountant’s office, a clinic, a property manager) is a real one-off job: install Ollama on their PC, load their documents, test 20 of their real questions, train one person to re-index. The value you deliver is the testing and the honest “this works for these questions and not these”, not the 90 lines of code. Price it as a few hours of IT work in your local market, and don’t promise accuracy you haven’t measured on their documents.

Reality check

Scanned PDFs return nothing. pypdf reads the text layer. A photo or scan of paper has no text layer, and the script prints no text found. You need OCR first: ocrmypdf (free, ocrmypdf input.pdf output.pdf, add -l ind+eng with the Indonesian Tesseract language pack installed) adds a text layer you can then index.

Tables come out messy. PDF text extraction flattens tables into a stream of words and numbers. A price list with 6 columns may lose which number belongs to which column. Test your price lists specifically. If they’re mangled, export the source spreadsheet to CSV and save it as .txt, or ask the supplier for the Excel file.

An 8B model still makes mistakes. It can mix up two similar products, or merge a price from one chunk with a product from another. The citations and “copy numbers exactly” rule reduce this. They don’t eliminate it. Never quote a price or a legal deadline to someone without opening the cited page.

“Not found” is a feature. If the answer is spread across many pages (“summarise every clause about penalties”), retrieval of 5 chunks won’t catch all of it. This tool is for lookups, not for analysing a whole contract. For that, read the contract, or use a larger model with the full document.

Retrieval misses happen with different wording. If the document says “pengembalian barang” and you ask about “refund”, the match score may be low. When the top match is below about 0.4, treat the answer with suspicion and try rephrasing. The exact threshold depends on your documents, so look at the scores for a few known questions first.

Old versions mislead. If docs/ holds the 2025 and 2026 price lists, the model may quote the old one. Delete or move outdated files out of docs/. The index drops chunks from files that are no longer there on the next run.

Speed on old hardware is painful. On an older laptop with 8 GB RAM, even qwen3:4b can take a minute per answer. If staff give up waiting, they’ll go back to asking you on WhatsApp. Test on the actual machine before you roll it out.

Model names change. Ollama’s library adds and renames models often. If ollama pull fails, check ollama.com/library for the current tag and update CHAT_MODEL or EMBED_MODEL. If you change EMBED_MODEL, delete index.json and re-index. Vectors from different embedding models can’t be compared.

Wrapping up

This is a boring tool, and that’s why it works. No subscription, no upload, no prompt engineering course. A folder of files, two free models and a script that tells you which page to read. Start with the five documents people ask you about most, run the index, ask ten questions you already know the answers to, and check every citation. If it gets 8 or 9 right and says “Not found” for the rest, it’s ready for the team.