# Sources — Build Your Own Vector Database from a Repository of Files

Published: 2026-07-20

## Primary video

- Title: **Build Vector Database From Scratch**
- Creator/channel: **Rahul Pandey**
- Video ID: `4Tj6kv5-6W0`
- URL: https://www.youtube.com/watch?v=4Tj6kv5-6W0
- Duration: 34:39
- Video upload date reported by YouTube metadata: 2026-03-14
- Transcript fetched: 2026-07-20 06:12:45 CDT
- Transcript segments: 945

Local source artifacts used to create this guide:

- `.hermes/guide-publishing/sources/4Tj6kv5-6W0.metadata.json`
- `.hermes/guide-publishing/sources/4Tj6kv5-6W0.transcript.json`
- `.hermes/guide-publishing/sources/4Tj6kv5-6W0.transcript.txt`

## Timestamp-derived section map

| Time | Video section | Guide use |
|---|---|---|
| 00:00–01:30 | Vector database dilemma and intuition-first scope | Scope, architecture-selection warning |
| 01:30–03:02 | Four-stage progression; Ollama and `nomic-embed-text` | Architecture overview and prerequisites |
| 03:02–04:13 | Embeddings and 768-dimensional example | Embedding explanation |
| 04:13–06:41 | Cosine similarity | Similarity formula and normalization |
| 06:41–11:00 | SQLite/NumPy `VectorDB`, schema, add/search | SQLite source-of-truth design |
| 11:00–15:53 | Batch insertion, top-K search, metadata | Repository chunks, batch embedding, filtering |
| 15:53–19:46 | Persistence and O(N) scaling | Persistence, recovery, evaluation |
| 19:46–22:57 | Standalone FAISS | Index-versus-database distinction |
| 22:57–27:57 | FAISS plus SQLite | Hybrid architecture used by the guide |
| 27:57–29:17 | ChromaDB and HNSW | Upgrade paths and ANN tradeoffs |
| 29:17–33:41 | Four-way benchmarking and scaling | Evaluation framework and benchmark caveats |
| 33:41–34:39 | Conclusions | Algorithm/architecture takeaway |

## Companion repository

- Repository: https://github.com/iRahulPandey/builld-vector-db-from-scratch
- Inspected commit: `14fbff29acf4e877fa2cd7804098bea854bf25be`
- Commit timestamp: 2026-03-14T09:41:42+01:00
- Files inspected: `README.md`, `pyproject.toml`, `test.py`, and the extracted `walkthrough.ipynb`

The notebook contains the video's instructional `VectorDB`, standalone FAISS, `FaissVectorDB`, ChromaDB, and benchmark code. Its hybrid example maps zero-based FAISS positions to one-based SQLite IDs. The downloadable guide implementation instead uses `faiss.IndexIDMap2` and explicit SQLite chunk IDs so paths remain stable through repository updates and deletions.

## Official and primary technical references checked

The following URLs returned HTTP 200 during guide preparation on 2026-07-20:

- Ollama embedding API: https://docs.ollama.com/api/embed
- FAISS getting started: https://github.com/facebookresearch/faiss/wiki/Getting-started
- Chroma introduction: https://docs.trychroma.com/docs/overview/introduction
- Sentence-BERT paper: https://arxiv.org/abs/1908.10084
- HNSW paper: https://arxiv.org/abs/1603.09320
- ANN Benchmarks: https://ann-benchmarks.com

## What is transcript-derived versus guide-authored

**Transcript/repository-derived:**

- embeddings as numerical semantic representations;
- 768-dimensional `nomic-embed-text` example;
- cosine similarity;
- SQLite text/metadata/vector storage;
- brute-force O(N) search;
- standalone FAISS;
- FAISS plus SQLite architecture;
- ChromaDB/HNSW comparison;
- source-video benchmark narrative and measured examples.

**Guide-authored practical extensions:**

- walking an arbitrary repository of files;
- text, code, PDF, DOCX, and PPTX extraction;
- overlap-aware chunking with approximate line ranges;
- SHA-256 change detection;
- stable numeric IDs with `IndexIDMap2`;
- incremental embedding of only new/changed files;
- deletion handling;
- SQLite-stored recovery vectors and `rebuild-index`;
- CLI commands, health invariants, path-prefix filtering;
- RAG citation pattern, security guidance, and retrieval evaluation.

## Benchmark caveat

The guide describes the pattern reported by the video but does not present its numbers as universal performance claims. Timing depends on CPU, operating system, Python/NumPy/FAISS/Chroma versions, index parameters, warm-up, vector distribution, and whether embedding time is included. The video times search separately using precomputed embeddings for its main comparisons.

## Companion program verification

Published companion files:

- `repo_vector_db.py.txt` (served as text; save locally as `repo_vector_db.py`)
- `repo_vector_db.requirements.txt`

The Python program was syntax-checked and executed in an isolated `uv` environment with NumPy and `faiss-cpu`. Its deterministic self-test exercised:

- initial repository synchronization;
- FAISS index creation;
- semantic-like test search;
- unchanged-file detection;
- changed-file replacement;
- deleted-file removal;
- FAISS rebuild from SQLite;
- equality of SQLite chunk count and FAISS vector count.

Observed test result: `PASS: sync, unchanged detection, update, deletion, FAISS rebuild, and search`.