pgvector Index Management Embedding Pipeline Optimization

A Grafana Dashboard for Vector Search Latency

This page shows how to build a Grafana dashboard that tells you, at a glance, whether your pgvector search tier is healthy: query latency percentiles, index-scan throughput, cache-hit ratio, and measured recall. It scopes the problem narrowly: given Prometheus already holds an application-side latency Histogram plus the exported pg_stat index metrics, which PromQL expression drives each panel, what the dashboard JSON looks like, and where to set alert thresholds.

Up: Prometheus Metrics for pgvector

Latency for approximate nearest-neighbour search is a tail problem: the mean hides the p99 requests that stall because the HNSW traversal fell out of cache or ef_search was set too high. A useful dashboard therefore leads with percentiles, not averages, and pairs them with the two variables that move the tail — index-scan rate and cache-hit ratio — plus a recall gauge so you never trade latency for silently degraded results. Every server-side series here comes from the exporter set up in configuring postgres_exporter for pgvector metrics; the latency percentiles require one extra piece of application instrumentation covered in step 1.

Prerequisites

  • Prometheus 2.40+ scraping postgres_exporter, so pgvector_index_* series exist.
  • Grafana 10+ with the Prometheus data source configured.
  • An application-side histogram of search latency exported to Prometheus. Without it there is no percentile to plot — the database does not measure end-to-end query time.
  • A recall value published as a gauge by your CI or a periodic job. Recall is not observable from PostgreSQL; it must be measured against ground truth and pushed.
  • Familiarity with histogram_quantile and rate windowing; every panel below is a variation on those.

Step-by-step

1. Instrument the application with a latency histogram

The database cannot report end-to-end search latency, so wrap the query in a Prometheus Histogram. Bucket boundaries must bracket your real distribution — for ANN search that is single-digit to low-tens of milliseconds, so seed buckets there rather than accepting the client default.

PYTHON
from prometheus_client import Histogram

SEARCH_LATENCY = Histogram(
    "vector_search_latency_seconds",
    "End-to-end pgvector similarity search latency",
    buckets=(0.002, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0),
    labelnames=("index_method",),
)

async def search(conn, probe, k=10):
    with SEARCH_LATENCY.labels(index_method="hnsw").time():
        return await conn.fetch(
            "SELECT doc_id FROM doc_chunks ORDER BY embedding <#> $1 LIMIT $2",
            probe, k,
        )

2. Panel: p95 / p99 query latency

histogram_quantile reconstructs a percentile from the bucket counters. Take the per-bucket rate over 5 minutes, sum by the le (less-than-or-equal) label, and ask for the 0.95 and 0.99 quantiles.

PROMQL
# p99
histogram_quantile(0.99,
  sum by (le) (rate(vector_search_latency_seconds_bucket[5m])))
# p95
histogram_quantile(0.95,
  sum by (le) (rate(vector_search_latency_seconds_bucket[5m])))

3. Panel: index-scan rate

How hard the vector index is working, in scans per second, straight from the exported counter. A drop to zero while traffic continues means the planner stopped using the index.

PROMQL
sum by (index_name) (rate(pgvector_index_idx_scan[5m]))

4. Panel: cache-hit ratio

The single best leading indicator of the latency tail. When the index no longer fits in shared_buffers, this sags below ~0.99 and p99 climbs.

PROMQL
sum(rate(pgvector_index_idx_blks_hit[5m]))
  / (sum(rate(pgvector_index_idx_blks_hit[5m]))
     + sum(rate(pgvector_index_idx_blks_read[5m])))

5. Panel: recall gauge

Recall is pushed by your regression job, not scraped from PostgreSQL. Display the latest value as a stat panel with thresholds so a regression turns the tile red. Wiring that measurement job is covered in recall regression testing in CI.

PROMQL
max(vector_search_recall_at_10)

6. Assemble the dashboard JSON

Grafana dashboards are JSON. This minimal skeleton wires the p99 panel and the cache-hit panel; import it and add the remaining targets. Keep datasource as a variable so the same JSON works across environments.

JSON
{
  "title": "pgvector Search Health",
  "schemaVersion": 39,
  "panels": [
    {
      "type": "timeseries",
      "title": "Query latency p95 / p99",
      "gridPos": { "h": 8, "w": 12, "x": 0, "y": 0 },
      "fieldConfig": { "defaults": { "unit": "s" } },
      "targets": [
        { "refId": "A", "expr": "histogram_quantile(0.99, sum by (le) (rate(vector_search_latency_seconds_bucket[5m])))", "legendFormat": "p99" },
        { "refId": "B", "expr": "histogram_quantile(0.95, sum by (le) (rate(vector_search_latency_seconds_bucket[5m])))", "legendFormat": "p95" }
      ]
    },
    {
      "type": "stat",
      "title": "Index cache-hit ratio",
      "gridPos": { "h": 8, "w": 6, "x": 12, "y": 0 },
      "fieldConfig": { "defaults": { "unit": "percentunit",
        "thresholds": { "mode": "absolute", "steps": [
          { "color": "red", "value": null },
          { "color": "yellow", "value": 0.97 },
          { "color": "green", "value": 0.99 }
        ] } } },
      "targets": [
        { "refId": "A", "expr": "sum(rate(pgvector_index_idx_blks_hit[5m])) / (sum(rate(pgvector_index_idx_blks_hit[5m])) + sum(rate(pgvector_index_idx_blks_read[5m])))" }
      ]
    }
  ]
}

7. Add alert rules

Turn the two tail drivers into alerts. Fire on sustained p99 breach and on a cache-hit ratio that falls below the point where latency degrades.

YAML
groups:
  - name: pgvector-search
    rules:
      - alert: VectorSearchP99High
        expr: histogram_quantile(0.99, sum by (le) (rate(vector_search_latency_seconds_bucket[5m]))) > 0.1
        for: 10m
        labels: { severity: warning }
        annotations: { summary: "pgvector p99 latency above 100ms for 10m" }
      - alert: VectorIndexCacheColdRatio
        expr: sum(rate(pgvector_index_idx_blks_hit[5m])) / (sum(rate(pgvector_index_idx_blks_hit[5m])) + sum(rate(pgvector_index_idx_blks_read[5m]))) < 0.95
        for: 15m
        labels: { severity: warning }
        annotations: { summary: "Vector index cache-hit ratio below 95%" }

Parameter reference

Panel Type PromQL core Alert threshold Notes
Query latency p95/p99 timeseries (s) histogram_quantile(0.99, sum by (le) (rate(..._bucket[5m]))) p99 > 100 ms for 10m Always sum by le before the quantile; buckets must bracket real latency.
Index-scan rate timeseries (ops) sum by (index_name) (rate(pgvector_index_idx_scan[5m])) == 0 while traffic > 0 Flat zero under load means the planner skipped the index.
Cache-hit ratio stat (percentunit) hit / (hit + read) over rate < 0.95 for 15m Earliest leading indicator of a growing latency tail.
Recall@k stat (gauge) max(vector_search_recall_at_10) < target (e.g. 0.95) Pushed by CI, not scraped; without it latency wins can hide quality loss.
Rows per scan timeseries rate(pgvector_index_idx_tup_read[5m]) / rate(pgvector_index_idx_scan[5m]) trend-based Rising value can signal ef_search set too high.

Verification

Confirm each target returns data before trusting the panel. In the Grafana panel editor, run the p99 query and check the value is non-NaN; a NaN almost always means the histogram has no samples yet.

PROMQL
# should be > 0 once traffic has flowed for one scrape window
sum(rate(vector_search_latency_seconds_count[5m]))

Cross-check the dashboard percentile against the database’s own view by comparing the p99 panel to pg_stat_statements mean time for the search statement; they should track each other, with the panel slightly higher because it includes network and driver time. If the exported index series are missing entirely, revisit the exporter setup rather than the dashboard. The in-database counterpart to the scan-rate panel is detailed in building a pg_stat_user_indexes dashboard.

Troubleshooting

  • histogram_quantile returns NaN or a flat line. No samples in the _bucket series for the window, or you forgot to sum by (le) first. Verify rate(..._count[5m]) > 0, and never apply the quantile to already-aggregated per-instance buckets without the le grouping.
  • p99 is pinned exactly at a bucket boundary. Your buckets are too coarse and the real value falls in the top bucket. Add finer boundaries around the observed range (step 1); histogram_quantile can only interpolate within a bucket, so a value above the last finite le reports as that boundary.
  • Latency panel and cache-hit panel disagree about health. Expected and useful: rising p99 with a still-high cache-hit ratio points at CPU-bound traversal (ef_search too high) rather than I/O. Add the rows-per-scan panel to disambiguate.
  • Recall gauge is stale or empty. The CI job that publishes vector_search_recall_at_10 has not run since the last index rebuild. Treat a missing recall series as a page-worthy gap, not a green state; a rebuilt index can pass latency alerts while its recall silently regressed.
  • Series count in Grafana balloons and queries slow. A high-cardinality label (per-doc_id, per-queryid) leaked into the histogram or exporter. Drop it at the instrumentation layer; a latency histogram should carry only low-cardinality labels such as index_method.