Why Vector Search?
Retrieval-Augmented Generation (RAG) uses vector databases to perform semantic search, fetching relevant context for LLMs to generate accurate responses. Choosing the right vector index is critical for maintaining low search latency as document counts scale.
Case Study: Slow Helpdesk QA Searches
A customer support bot was taking 10-15 seconds to answer questions. Profiling showed that query embedding search inside a database containing 2 million support articles took 8 seconds per request.
The Bug: Brute-Force Flat Indexing
The vector database was configured to use a flat index (exact search). While highly accurate, exact search requires comparing the query vector against every single vector in the database, resulting in a time complexity of O(N).
The Fix: HNSW (Hierarchical Navigable Small World) Indexing
We rebuilt the vector database index using HNSW. HNSW organizes vectors into a multi-layered graph, enabling approximate nearest neighbor (ANN) search with logarithmic time complexity O(log N):
# Creating HNSW index in pgvector
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64);This index optimization reduced vector search times from 8 seconds to 22 milliseconds, restoring normal chatbot response latency.