Architecture
The system is split into an offline ingestion path and an online query path, sharing the same embedding model so both sides land in the same vector space.
| Stage | What happens |
|---|---|
| Ingestion | PDF → per-page text extraction → fixed-size chunking with overlap → embedding → Qdrant upsert |
| Query | Question → embedding → top-K retrieval → numbered context block → Claude generation → citation parsing |
Design choices
Chunking — fixed-size with overlap, per page. Each PDF page is split into 1000-character windows with 200 characters of overlap, so a sentence that falls on a chunk boundary still appears whole in at least one chunk. A recursive, paragraph-aware splitter would be the natural refinement — left as a documented next step rather than implemented, since this baseline already produces accurate citations for a corpus this size.
Citations — numbered context, not model self-report. Every retrieved chunk is labeled ([1], [2], ...) directly in the prompt, and Claude is instructed to reuse those same numbers inline. The app then parses which numbers actually appear in the answer with a regex, rather than trusting any claim the model makes about what it read — the citation is checked, not assumed.
Vector store — Qdrant, embedded/local mode. QdrantClient(path=...) persists the HNSW index to a local folder with no server process and no Docker — the simplest way to run the project immediately after cloning it.
Example interactions
Both examples below are copied verbatim from a real run of the app — no answer or citation has been edited or paraphrased.
Question: “What are the technical components needed to build a RAG system, from embedding to indexing?”
Technical Components of a RAG System: From Embedding to Indexing
Retrieval & Indexing
- RAG uses Maximum Inner Product Search (MIPS) via FAISS for fast document retrieval, run on CPU; storing document index vectors requires ~100 GB of CPU memory for all of Wikipedia [3].
- FAISS's compression tools can reduce that requirement to 36 GB [3].
Compute & Training Infrastructure
- Training is distributed across 8 × 32GB NVIDIA V100 GPUs, though training and inference can also run on a single GPU [3].
Notable Design Choices
- RAG does not require intermediate retrieval supervision during training [2], nor a re-ranker or extractive reader to achieve state-of-the-art performance [4].
⚠ The sources provide limited detail on the embedding generation process itself.
| # | Source | Similarity | Cited |
|---|---|---|---|
| 1 | rag_lewis_2020.pdf, p.17 | 0.541 | not cited |
| 2 | rag_lewis_2020.pdf, p.6 | 0.522 | ✅ cited |
| 3 | rag_lewis_2020.pdf, p.17 | 0.514 | ✅ cited |
| 4 | rag_lewis_2020.pdf, p.5 | 0.482 | ✅ cited |
| 5 | rag_lewis_2020.pdf, p.10 | 0.464 | not cited |
Question: “What is the computational cost of training GPT-5?”
The provided sources do not contain any information about GPT-5 or its computational training costs. The sources only discuss the RAG model training setup [1][4] and the HNSW algorithm for approximate nearest neighbor search [2][3][5]. I cannot answer this question based solely on the available sources.
No fabricated numbers, no hedged guess — the system correctly refuses to answer outside the scope of the indexed documents, which is exactly the behavior the system prompt is designed to enforce.
An honest finding: retrieval stayed single-paper
The first example above was phrased broadly across the whole RAG pipeline, but all five retrieved chunks came from a single paper rather than pulling from the SBERT or HNSW papers as well — a real retrieval-quality limitation worth naming rather than hiding.
top_k=10 (double the default) still returned exclusively RAG-paper chunks — no SBERT or HNSW content entered the context even once. This rules out “too few chunks retrieved” as the cause and points instead at chunk granularity or embedding-space clustering: the embeddings of RAG's own infrastructure section are apparently closer to this query than any chunk from the other two papers, at every rank up to 10. Recursive/semantic chunking, or a different embedding model, are the next things to test before concluding this is a fundamental limitation rather than a tuning issue.