Intermediate · Systems

Indexing

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Organizing data so that lookups, searches, or similarity queries run quickly.

Technical Definition

Organizing data so that lookups, searches, or similarity queries run quickly.

How it works

Vector indexes (HNSW, IVF, ScaNN) make billion-vector similarity search practical. Keyword indexes (inverted files) power full-text search. Hybrid indexes combine both. The index choice trades build cost, query latency, recall, and memory.

Related Concepts

• Embedding — A dense vector representation that captures semantic meaning, mapping discrete items like words into continuous mathematical space.
• Vector Database — A specialized database optimized for storing, indexing, and querying high-dimensional embedding vectors using similarity search.
• Similarity Search — Finding the items in a dataset most similar to a given query, usually by vector distance.