pgvector Index Management Embedding Pipeline Optimization

Tuning IVFFlat Lists for High-Throughput Similarity Search

This page shows how to size and tune the IVFFlat lists parameter — together with its query-time partner ivfflat.probes — so a pgvector index sustains high query throughput at a fixed recall target. It scopes the problem narrowly: given a populated embedding table, how to compute a starting lists, build the index on real data, sweep lists × probes against measured recall, confirm the planner actually uses the index, and rebuild safely once cluster quality drifts.

Up: Optimizing m and ef_construction Parameters

The Inverted File with Flat storage (ivfflat) index partitions the embedding space into lists Voronoi cells with K-means, stores each vector uncompressed inside exactly one cell, and at query time scans only the probes cells nearest the query vector. That design trades a little recall for a hard memory ceiling and fast rebuilds — the opposite balance from the RAM-resident graph you tune in optimizing m and ef_construction parameters. Whether that trade is right for your workload at all is a prior decision, made with the HNSW vs IVFFlat algorithm selection framework; everything below assumes you have already committed to IVFFlat and now need lists and probes dialed in. The one structural fact that drives the whole procedure: lists is frozen into the centroids the moment CREATE INDEX finishes, so changing it means a full rebuild, while probes is a session-level knob you retune live — exactly the ef_search relationship, one level down.

Prerequisites

  • PostgreSQL 15+ with the pgvector extension 0.5+ installed (CREATE EXTENSION vector;). ivfflat has existed since 0.4, but 0.5+ gives you the stable ivfflat.probes GUC and EXPLAIN integration used below.
  • A table already populated with representative vectors. Unlike HNSW, IVFFlat trains K-means centroids at build time, so the data must be loaded before the index is created — building on an empty or tiny table produces degenerate lists that never recover without a rebuild.
  • numpy >= 1.24 and psycopg 3.1+ (or asyncpg 0.28+) to drive the recall sweep against exact ground truth.
  • maintenance_work_mem sized to hold the K-means working set during the build (commonly 2–4 GB), and a decided distance metric — the operator class you index must match the query operator, as laid out in cosine vs L2 distance metrics.
  • A fixed probe set of query vectors with exact top-k neighbors precomputed, so every configuration is scored against the same ground truth.
The IVFFlat lists and probes tuning loop Load representative vectors, compute a starting lists value from the row count (rows over 1000, or square root of rows), CREATE INDEX on the populated table, then SET probes to about the square root of lists and ANALYZE. A decision checks whether the query plan uses the IVFFlat scan; if not, fix the operator class and re-ANALYZE, then retry. Once the plan is correct, measure recall at k against exact ground truth. A second decision checks whether recall clears the target: if not, raise probes live with no rebuild and re-measure, or — when probes already scans well past ten percent of lists and recall still plateaus — rebuild with fewer, larger lists. When recall clears the target, lock the smallest passing probes value. Load representative vectors Compute lists (rows/1000 · √rows) CREATE INDEX on populated table SET probes = √lists · ANALYZE Plan usesIVFFlat scan? Yes No Fix opclass,re-ANALYZE Measure recall@k vs exact truth recall ≥target? Yes No Raise probes(live, no rebuild) plateau Rebuild:fewer lists Lock smallest passing probes build / query step decision rebuild (frozen lists)
The tuning loop: build once on populated data, verify the plan really uses the index, then sweep recall — raising probes live for as long as it helps and rebuilding with fewer lists only when recall plateaus below target.

Step-by-step tuning procedure

1. Compute a starting lists from the loaded row count

pgvector’s own guidance is the right baseline: use rows / 1000 for tables up to ~1M rows, and sqrt(rows) above that. The rows / 1000 rule keeps roughly a thousand vectors per cell — enough that a single probes scan evaluates a statistically meaningful candidate set without dragging in the whole table. Derive it directly from the live count rather than guessing:

SQL
SELECT
    n AS row_count,
    CASE
        WHEN n <= 1000000 THEN GREATEST(n / 1000, 1)
        ELSE round(sqrt(n))::int
    END AS suggested_lists
FROM (SELECT count(*) AS n FROM document_chunks) c;

For high-dimensional embeddings (≥768 dims) bias downward from this number: K-means cluster coherence degrades as dimensionality rises (the curse of dimensionality flattens the distance spread), so fewer, larger cells stay more meaningful than many sparse ones. Combine the resulting index size with the per-row payload from the pgvector storage overhead analysis before you commit, so you know the resting footprint a rebuild will cost.

2. Build the index on the populated table

Raise maintenance_work_mem so K-means trains in RAM, then create the index with an explicit lists and an operator class that matches the query metric. A vector_cosine_ops index cannot serve an L2 (<->) query — the planner will silently fall back to a sequential scan.

SQL
SET maintenance_work_mem = '4GB';

CREATE INDEX CONCURRENTLY idx_chunks_ivf
    ON document_chunks
    USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 1000);

The IVFFlat build is dominated by serial K-means assignment and benefits little from max_parallel_maintenance_workers (that knob accelerates HNSW graph builds, not IVFFlat centroid training). Because CREATE INDEX is single-threaded here, stage large builds off the live write path — the non-blocking machinery is covered in asynchronous index build strategies, and stalls are triaged in resolving pgvector index build timeout errors.

3. Set probes and refresh statistics

ivfflat.probes defaults to 1 — one cell scanned, which is fast but usually far below target recall. A sound starting point is sqrt(lists). Set it at the session or role level for the querying service, and re-ANALYZE so the planner has fresh row estimates.

SQL
SET ivfflat.probes = 32;   -- ~sqrt(1000); tune against measured recall
ANALYZE document_chunks;

4. Sweep lists × probes against exact ground truth

Parameter tuning without a recall measurement is guessing. Drive the whole matrix from Python so each configuration is built, measured, and torn down under identical conditions, scoring recall@k against exhaustive (exact) neighbors.

PYTHON
import psycopg
from pgvector.psycopg import register_vector


def benchmark_ivfflat(conn, probe_vectors, ground_truth,
                      lists_vals, probes_vals, k=10):
    """Build each lists config, sweep probes, measure recall@k vs exact truth."""
    register_vector(conn)
    results = []
    for lists in lists_vals:
        with conn.cursor() as cur:
            cur.execute("DROP INDEX IF EXISTS bench_ivf")
            cur.execute(
                "CREATE INDEX bench_ivf ON embeddings "
                "USING ivfflat (vec vector_cosine_ops) WITH (lists = %s)",
                (lists,),
            )
            for probes in probes_vals:
                cur.execute("SET ivfflat.probes = %s", (probes,))
                hits = 0
                for q, truth in zip(probe_vectors, ground_truth):
                    cur.execute(
                        "SELECT id FROM embeddings ORDER BY vec <=> %s LIMIT %s",
                        (q, k),
                    )
                    approx = {row[0] for row in cur.fetchall()}
                    hits += len(approx & set(truth[:k]))
                recall = hits / (len(probe_vectors) * k)
                results.append({"lists": lists, "probes": probes,
                                "recall": round(recall, 4)})
    return results

Read the surface the way you would an ef_search curve: for a fixed lists, recall climbs with probes and plateaus once you scan the cells that actually hold the true neighbors. Aim for the smallest probes (typically evaluating 1–5% of total lists) that clears target — every extra probe is linear query cost. If recall plateaus below target no matter how high probes goes, the partition granularity itself is wrong: rebuild with fewer lists so each cell covers more of the space.

Parameter reference

Parameter Type Default Production recommendation Notes
lists build-time (int) none (required) rows / 1000 up to 1M rows; sqrt(rows) above. Bias lower for ≥768 dims Number of K-means cells, frozen at build. Changing it requires a full REINDEX.
ivfflat.probes query-time (int) 1 Start at sqrt(lists); sweep so 1–5% of lists is scanned The only live knob. Set per-session/role; never rely on the default of 1.
maintenance_work_mem build-time (mem) 64MB 2–4 GB, sized to the K-means working set Too low forces on-disk passes and slows the build sharply.
Operator class index DDL — vector_cosine_ops / vector_l2_ops / vector_ip_ops Must match the query operator (<=> / <-> / <#>) or the planner reverts to a seq scan.
max_parallel_maintenance_workers build-time (int) 2 Leave default IVFFlat’s serial K-means ignores it; it only parallelizes HNSW builds.

Verification

First, confirm the index is actually used — the single most common silent failure is the planner choosing a sequential scan, which makes every recall number you collect measure exact search, not your index.

SQL
EXPLAIN (ANALYZE, BUFFERS)
SELECT id
FROM   embeddings
ORDER  BY vec <=> :probe_vector
LIMIT  10;

Look for an Index Scan using ... ivfflat node and a low Buffers: shared hit count relative to the table size. A Seq Scan means an operator/opclass mismatch or a stale ANALYZE — fix that before trusting any metric. Then confirm the approximate results actually match exact neighbors on a fixed probe set:

PYTHON
def recall_gauge(conn, probes, ground_truth, k=10, ivf_probes=32):
    """Return recall@k for a fixed probe set — export to Prometheus / gate in CI."""
    with conn.cursor() as cur:
        cur.execute("SET ivfflat.probes = %s", (ivf_probes,))
        hits = 0
        for q, truth in zip(probes, ground_truth):
            cur.execute(
                "SELECT id FROM embeddings ORDER BY vec <=> %s LIMIT %s", (q, k)
            )
            approx = {row[0] for row in cur.fetchall()}
            hits += len(approx & set(truth[:k]))
    return hits / (len(probes) * k)

A recall number at or above your floor (e.g. recall@10 ≥ 0.92) with the EXPLAIN plan showing the IVFFlat scan means the configuration is sound. Emit the gauge as a metric so slow drift shows up as an alert, not a user complaint.

Troubleshooting

  • Recall collapses and every query looks like a full scan. The index was almost certainly built on an empty or nearly-empty table, so K-means produced one degenerate list. Confirm with SELECT count(*) FROM document_chunks; against the build timestamp, then REINDEX INDEX CONCURRENTLY idx_chunks_ivf after the data is loaded. IVFFlat must always be built after ingestion.
  • EXPLAIN shows a Seq Scan despite a valid index. Operator/opclass mismatch (cosine index, <-> query), a rebuild left invalid, or a stale ANALYZE. Match the query operator to the opclass, run ANALYZE document_chunks;, and re-check the plan — queries still return correct rows, so nothing errors on its own.
  • Recall short of target at high QPS. Raise ivfflat.probes before touching lists, because probes is a live knob and lists needs a rebuild. Only when a high probes value (scanning well past ~10% of lists) still misses target is the partition count itself wrong — rebuild with fewer lists.
  • CPU spikes on a subset of queries. Skewed or near-duplicate embeddings collapse K-means centroids into dense regions, so some probes scans hit oversized cells. Check cell balance and consider de-duplicating before ingestion; at very high dimensionality (>1536) also weigh product quantization or dimensionality reduction, since IVFFlat’s pruning advantage erodes as distances converge.
  • probes leaks across tenants. In a pooled connection setup, a SET ivfflat.probes on a reused backend persists into the next borrower’s session. Pin it with SET LOCAL inside the transaction, or set it per-role, so multi-tenant traffic cannot inherit another workload’s probe budget.