Configuring postgres_exporter for pgvector Metrics

This page shows how to stand up postgres_exporter so Prometheus can scrape per-index usage, size, and cache-hit metrics for your pgvector tables. It scopes the problem narrowly: deploy the exporter under a least-privilege monitoring role, write a custom queries.yaml that turns pg_stat_user_indexes and pg_statio_user_indexes into labelled time series, enable pg_stat_statements, and point a Prometheus scrape job at the result.

Up: Prometheus Metrics for pgvector

pgvector exposes no metrics of its own — an HNSW or IVFFlat index is just another relation, so everything you can observe comes from PostgreSQL’s cumulative statistics views. postgres_exporter reads those views on each scrape and renders them as Prometheus counters and gauges. The default exporter build ships a handful of database-wide series, but nothing at the granularity you need to reason about a specific vector index: whether it is actually being scanned, how large it has grown relative to the base table, and whether its blocks live in shared_buffers or get read from disk on every probe. A custom queries.yaml closes that gap, and it is the data source the sibling Grafana dashboard for vector search latency renders.

Prerequisites #

• PostgreSQL 15+ with the pgvector extension installed and at least one HNSW or IVFFlat index built. The exporter reads statistics, not vectors, so any pgvector version works.
• postgres_exporter 0.15+ (the prometheuscommunity/postgres_exporter image or the release binary).
• A Prometheus server (2.40+) that can reach the exporter’s :9187 endpoint.
• shared_preload_libraries = 'pg_stat_statements' set, requiring one PostgreSQL restart. This is the only change that cannot be applied with a reload.
• Authority to CREATE ROLE and GRANT pg_monitor. Keep this credential distinct from the application role used by ingestion; the separation mirrors the boundary work in securing pgvector tables with row-level security.

Step-by-step #

1. Create a least-privilege monitoring role #

Never point the exporter at a superuser or the application role. pg_monitor is the purpose-built built-in role: it bundles pg_read_all_settings, pg_read_all_stats, and pg_stat_scan_tables, which is exactly what the statistics views require and nothing more. The CONNECTION LIMIT caps blast radius if the exporter gets stuck reconnecting.

CREATE ROLE exporter WITH LOGIN PASSWORD 'REPLACE_ME'
    CONNECTION LIMIT 5;
GRANT pg_monitor TO exporter;

-- pg_stat_statements lives in whatever schema it was created in;
-- grant usage so the exporter can read its view
GRANT USAGE ON SCHEMA public TO exporter;

2. Enable pg_stat_statements #

Query-level latency and call counts come from pg_stat_statements, which must be preloaded. Set it, restart once, then create the extension.

-- postgresql.conf (requires restart):
--   shared_preload_libraries = 'pg_stat_statements'
--   pg_stat_statements.max = 10000
--   pg_stat_statements.track = 'top'

CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

3. Write queries.yaml for per-index vector metrics #

This is the heart of the setup. The exporter loads a YAML file where each top-level key is a namespace, query is the SQL it runs each scrape, and every column is tagged as either a label or a usage type (COUNTER, GAUGE). Join pg_stat_user_indexes (scan counts) to pg_statio_user_indexes (block I/O) on indexrelid, and filter to pgvector indexes by access method so you do not emit a series for every B-tree in the database.

pgvector_index:
  query: |
    SELECT
      s.schemaname,
      s.relname       AS table_name,
      s.indexrelname  AS index_name,
      am.amname       AS index_method,
      s.idx_scan      AS idx_scan,
      s.idx_tup_read  AS idx_tup_read,
      pg_relation_size(s.indexrelid) AS index_size_bytes,
      io.idx_blks_hit  AS idx_blks_hit,
      io.idx_blks_read AS idx_blks_read
    FROM pg_stat_user_indexes  s
    JOIN pg_statio_user_indexes io ON io.indexrelid = s.indexrelid
    JOIN pg_class     c  ON c.oid = s.indexrelid
    JOIN pg_am        am ON am.oid = c.relam
    WHERE am.amname IN ('hnsw', 'ivfflat')
  metrics:
    - schemaname:   { usage: "LABEL",   description: "Schema" }
    - table_name:   { usage: "LABEL",   description: "Base table" }
    - index_name:   { usage: "LABEL",   description: "Vector index" }
    - index_method: { usage: "LABEL",   description: "hnsw or ivfflat" }
    - idx_scan:       { usage: "COUNTER", description: "Index scans initiated" }
    - idx_tup_read:   { usage: "COUNTER", description: "Index entries returned" }
    - index_size_bytes: { usage: "GAUGE", description: "On-disk index size" }
    - idx_blks_hit:   { usage: "COUNTER", description: "Index blocks from cache" }
    - idx_blks_read:  { usage: "COUNTER", description: "Index blocks from disk" }

Filtering on am.amname IN ('hnsw','ivfflat') is what keeps label cardinality bounded — you emit one series set per vector index, not per relation. Understanding which access method each index uses is the province of HNSW vs IVFFlat algorithm selection; the index_method label lets you slice dashboards by it.

4. Run the exporter with the custom queries file #

Point the exporter at the role from step 1 via DATA_SOURCE_NAME and mount the YAML with --collector.custom_query.hr_directory (hr = high-resolution collector). Disable the noisy default collectors you do not need so the scrape stays cheap.

docker run -d --name pgvector-exporter -p 9187:9187 \
  -e DATA_SOURCE_NAME="postgresql://exporter:REPLACE_ME@db-host:5432/appdb?sslmode=require" \
  -v "$(pwd)/queries.yaml:/etc/queries.yaml:ro" \
  quay.io/prometheuscommunity/postgres_exporter:v0.15.0 \
  --collector.custom_query.hr_directory=/etc/ \
  --no-collector.stat_bgwriter \
  --auto-discover-databases

5. Add the Prometheus scrape job #

Finally, tell Prometheus to pull the endpoint. A 15s interval matches the default statistics refresh and keeps rate() windows honest.

scrape_configs:
  - job_name: pgvector
    scrape_interval: 15s
    static_configs:
      - targets: ['exporter-host:9187']
        labels:
          cluster: prod-vectors

Parameter reference #

Verification #

Curl the endpoint directly and grep for the namespaced series. Every pgvector index should appear with its labels populated.

curl -s http://exporter-host:9187/metrics | grep 'pgvector_index_'
# pgvector_index_idx_scan{index_method="hnsw",index_name="doc_chunks_embedding_idx",...} 41822
# pgvector_index_index_size_bytes{index_method="hnsw",...} 8.05306368e+08
# pgvector_index_idx_blks_hit{index_method="hnsw",...} 1.9932e+06

Confirm Prometheus is actually scraping by checking target health, then evaluate a cache-hit-ratio expression in the Prometheus expression browser:

# should be close to 1.0 for a warm, well-sized index
rate(pgvector_index_idx_blks_hit[5m])
  / (rate(pgvector_index_idx_blks_hit[5m]) + rate(pgvector_index_idx_blks_read[5m]))

A steady ratio near 1.0 means the index lives in shared_buffers; a sagging ratio is your earliest signal that the index no longer fits in RAM. If idx_scan for a vector index stays flat at zero while queries run, the planner is not using it — a validation concern examined in index validation and error categorization.

Troubleshooting #

• pg_stat_statements metrics are all zero or the namespace is missing. The library was not preloaded. SHOW shared_preload_libraries; must list it; if it does not, edit postgresql.conf and restart — a reload is insufficient because preloaded libraries load at startup only.
• Custom series never appear, only default pg_* metrics do. The --collector.custom_query.hr_directory path is wrong or the container cannot read the mounted file. Check exporter logs for a YAML parse error, and confirm the mount is :ro but world-readable inside the container.
• Series count explodes and Prometheus memory climbs. A label is high-cardinality — usually you removed the amname filter and are emitting one series per B-tree, or a queryid label from pg_stat_statements is unbounded. Keep labels to schema/table/index/method; never label by raw query text or doc_id.
• idx_scan appears to reset to a small number. The statistics were reset (pg_stat_reset(), a crash, or a major-version upgrade). Counters are cumulative since the last reset; always chart them through rate()/increase(), which tolerate resets, rather than the raw counter.
• Permission denied reading a statistics view. The role lost pg_monitor, or you are reading pg_stat_statements in a schema the role lacks USAGE on. Re-GRANT pg_monitor TO exporter; and grant schema usage; the exporter needs no table-level SELECT.

Related #

• Grafana dashboard for vector search latency — turn these scraped series into panels and alerts
• Building a pg_stat_user_indexes dashboard — the in-database view behind the exported metrics
• Detecting HNSW index bloat with pgstattuple — going deeper than pg_relation_size on index growth
• Index validation and error categorization — diagnosing an index that never gets scanned
• Up: Prometheus Metrics for pgvector