What is RAG with citations?

RAG with citations is retrieval-augmented generation under a checkable contract: every statement in an answer names, by identifier, a passage retrieved for that question, and the system declines when no passage supports an answer. What separates it from plain RAG is enforcement. Code verifies the answer against the retrieved set; the prompt only asks.

Lewis et al. (2020) paired a generator with a dense vector index read by a retriever, and called provenance for a model’s decisions an open problem. Citations make provenance checkable. Multi-tenant RAG adds a second promise, that users retrieve only their own tenant’s passages, kept by the database.

The cite-or-decline contract at a glance

Guarantee Where it is enforced How to test it
Every claim traces to a retrieved passage A citation check after generation, against this request’s passage IDs An answer citing an unretrieved ID, or with an uncited sentence: rejected
No supporting passage means a refusal A reranker threshold before generation, the citation check after it Ask what no document answers: a typed refusal, no model call below the threshold
One tenant never sees another’s rows Row-Level Security with FORCE; a policy on a transaction-scoped setting; a role without BYPASSRLS One query as two tenants: no shared row. No tenant set: an error
A filtered search still returns enough results Iterative index scans, a tenant_id index or per-tenant partitions Several tenant sizes, via the production path, against exact search: compare the sets
A settings change cannot silently break the contract A golden-set gate in CI Change chunking on a branch: the gate names the cases that broke

Where the citation check lives, and what it rejects

The citation rule runs in code after generation, because a prompt can be talked around. Each retrieved passage carries an identifier that survives re-indexing; I would use the document ID plus a hash of the chunk text, so a citation points at the exact words the model saw. The prompt asks for them after each sentence.

The check rejects an identifier not retrieved for this request, a sentence without one, and an answer with no valid citation. A rejected answer is not repaired; it becomes a typed refusal with the reason and the scope searched, so products show one message and a way to add the document. A cheaper gate runs first: if no passage clears the reranker threshold, the model is never called.

The check proves traceability, not truth. A sentence can cite a passage that does not say what it says, and a poisoned document, a risk in OWASP’s vector and embedding entry, still yields valid citations. Golden-set faithfulness scoring narrows part of that gap.

Storing and searching with pgvector and HNSW

Keep chunks, embeddings and tenant IDs in one PostgreSQL table: one transaction, backup and policy cover all three. pgvector adds the vector type and HNSW, an approximate index with a better speed-recall trade-off than IVFFlat, paid for in build time and memory.

Tune against your own queries, starting with hnsw.ef_search: 40 by default, it sizes the candidate list per query, trading speed for recall; SET LOCAL changes it per transaction. m and ef_construction are fixed at build time; the README advises their defaults unless recall, measured against exact search, is low.

As of October 2026, the latest pgvector release is 0.8.7 (changelog), Aurora PostgreSQL 18.6 lists 0.8.2, and an engine upgrade does not upgrade the extension. Check SELECT extversion FROM pg_extension WHERE extname = 'vector' before relying on a feature.

Hybrid search: merging the two lists with reciprocal rank fusion

Embeddings blur exact strings (product names, identifiers, error codes), so a keyword search runs beside the vector search: PostgreSQL full-text search, with a generated tsvector column, a GIN index and websearch_to_tsquery, which accepts raw user input.

The two scores are not comparable, a cosine distance against a ts_rank_cd score, so merge by rank. Reciprocal rank fusion scores each passage as the sum of 1/(k + rank) over the lists it appears in. Its authors fixed k = 60 and found it not critical, on TREC runs rather than RAG passages: let your golden set confirm it.

A reranker then reads the query and each candidate together, which usually improves the top of the list, the part the generator sees. Reranking adds latency, so it gets dozens of candidates, not hundreds; its score doubles as the refusal threshold. Cost: two queries and a model call per question. Test: golden cases on exact identifiers that vector search alone misses.

Tenant isolation with Row-Level Security

In multi-tenant RAG, isolation belongs in the database: a policy filters every query on the table, vector search included. The minimum, checked against the PostgreSQL 18 docs:

-- as the table owner
ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;
ALTER TABLE chunks FORCE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON chunks
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

-- as an administrator: API login, not the owner, no bypass
CREATE ROLE retrieval_api LOGIN NOSUPERUSER NOBYPASSRLS;
GRANT SELECT ON chunks TO retrieval_api;

BEGIN;
SELECT set_config('app.tenant_id', $1, true);
SELECT id, content FROM chunks ORDER BY embedding <=> $2 LIMIT 40;
COMMIT;

With no WITH CHECK clause, the USING expression also checks new rows, so ingestion cannot write into another tenant. A request that set no tenant fails: current_setting raises an error if the setting does not exist, and in my reading a connection where an earlier transaction set it locally keeps an empty string, which the ::uuid cast rejects.

Superusers and BYPASSRLS roles always skip policies, and table owners do unless the table has FORCE ROW LEVEL SECURITY. The API connects as none of them, and must not be able to SET ROLE to one. Test: SELECT rolname FROM pg_roles WHERE (rolsuper OR rolbypassrls) AND pg_has_role('retrieval_api', oid, 'SET') returns nothing, pg_class.relforcerowsecurity is true for the table, and one query under two tenant IDs returns no shared row.

Setting the tenant per transaction, and what connection pooling breaks

Set the tenant inside the transaction that runs the search, never on the session. set_config(..., true) lasts for the current transaction only, like SET LOCAL, and as an ordinary SELECT takes the tenant ID as a bind parameter.

SET LOCAL outside a transaction block emits a warning and has no effect, and under autocommit a transaction-scoped setting is gone before the search runs. A plain SET lasts for the session, while PgBouncer in transaction mode lends a server connection per transaction and does not support SET: in my reading, one missed reset then answers tenant B with tenant A’s setting. And since PostgreSQL accepts any two-part setting name, in my reading any session on the API’s role can set any tenant, so only the API should hold the connection.

Cost: a transaction and one more statement per request. Test: run the search in a separate statement and expect an error.

Recall under a filter: when a tenant gets fewer results than it asked for

With an approximate index, pgvector filters after the index scan, and the README warns that in a shared index one tenant’s vectors can affect another’s recall. In my reading, the tenant policy is such a filter whenever the plan uses the shared HNSW index: a tenant with a small share of it can get fewer rows than the LIMIT, silently.

Which tenants suffer depends on the plan, not on size alone. pgvector 0.8.0 improved cost estimation when filtering, in my reading letting the planner scan a small tenant exactly, but after five executions a prepared statement may switch to one generic plan for every tenant.

Three fixes:

  • Iterative index scans (pgvector 0.8.0 or later): SET LOCAL hnsw.iterative_scan = relaxed_order scans on until enough rows pass, up to hnsw.max_scan_tuples; the reranker absorbs the slight disorder.
  • An index on tenant_id, which per the README can give fast, exact search in many cases.
  • List partitioning by tenant, as the README suggests, at the cost of a partition and an index per tenant.

Test: run golden queries for tenants of several sizes through the production path, prepared statements and pooler included. Run the exact baseline as a plain query, or with SET LOCAL plan_cache_mode = force_custom_plan, under SET LOCAL enable_indexscan = off: its EXPLAIN must show no HNSW index, which makes it exact. A production run without one is already exact. Compare as sets: expect the LIMIT, or a smaller tenant’s total; the overlap is your recall.

The owned retrieval API is where the contract is enforced

Every guarantee runs in one place: an API the platform owns. Products send a question and the user’s token, and hold no database credentials. The API takes the tenant from the token, never the request body, then searches, fuses, reranks, applies the threshold, calls the model through the AI gateway and checks citations. Products get one retrieval contract, and each search improvement reaches all of them. It is layer 2 of the enterprise AI platform architecture.

Cost: an owner, an availability target and the risk of becoming every product’s bottleneck. Test the boundary: a product’s service account must fail to connect to the database.

Testing the contract in CI with a golden set

Each guarantee becomes a kind of case: questions with expected source documents, then two kinds that must be declined, questions no document answers and questions answerable only in another tenant. Name documents, not chunk IDs, so re-chunking keeps the cases valid. Run it on any change to chunking, embeddings, index settings, k, the reranker or the prompt, deterministic checks first. The method is in the evaluation guide. Green means yesterday’s cases still pass; real conversations need a human reader.

In practice

On the ENSO platform, I built the RAG architecture on Aurora PostgreSQL with pgvector (HNSW, hybrid search and reranking) behind an owned retrieval API with a strict citation policy: every answer cites its sources or declines, and tenant isolation is enforced in the database with Row-Level Security. The surrounding architecture is described in the ENSO case study, and retrieval of this kind is part of the enterprise LLM platform service.

Trade-offs and limits

Putting isolation and retrieval in PostgreSQL ties the design to one engine and its scaling limits; a very large corpus may need a dedicated vector store. A tenant needing its own storage encryption key needs its own Aurora cluster, since a cluster’s storage uses one KMS key: a separate design to run.

Strict citation increases refusals. Users will see “I can’t answer that from the available sources” more often, which is intended but needs a clear message and a path to add sources.

Hybrid search and reranking add latency and moving parts, and every parameter needs evaluation by the golden set in CI.

Ingestion, jobs and admin scripts must set the tenant per transaction too.

Row-Level Security is a tenant boundary, not an authorisation design: a folder only HR may read needs its own rule. Bypass roles see everything, a shared index can cost a tenant recall, and unique and foreign-key checks ignore policies, so a tenant that can insert can infer a key exists elsewhere.

Common questions

How do you make a RAG system cite its sources?

Return each retrieved passage with a stable identifier and ask the model to tag every sentence with the identifiers it used. Then check the answer in code: reject unretrieved identifiers and uncited sentences, and turn an answer with no valid citation into a refusal.

Does Row-Level Security work with pgvector?

Yes. The policy filters a vector search like any other query. Two caveats: superusers, BYPASSRLS roles and, without FORCE ROW LEVEL SECURITY, table owners skip it; and when the plan uses a shared HNSW index, the policy filters after the scan, so a tenant with a small share of it can get fewer results.

Why does pgvector return fewer results with a WHERE clause?

With an approximate index such as HNSW, the filter applies after the index scan, which returns at most hnsw.ef_search candidates, 40 by default, so fewer rows can come back. Enable iterative index scans (pgvector 0.8.0 or later), index the filter column or partition by tenant.

Why should a RAG system refuse to answer?

Because an answer without a supporting passage comes from the model’s memory, which nobody can check. A refusal says the documents do not cover the question, which is true and fixable: add the source. Strict citation raises refusals, so the message should say what to do next.

Sources