26
Choosing a vector store for something real: the questions that actually decide it
Ran three in production. Recall benchmarks decided none of it. What decided it, in order:
- Metadata filtering, and whether it is applied before or after the search. The single most important question and it is barely advertised. Filtering after retrieval means asking for ten results, discarding nine on a tenant or date filter, and getting one. Anything multi-tenant needs pre-filtering or it is quietly broken as it grows.
- Whether you already run a database that will do this. For a few hundred thousand vectors, the database you have is almost certainly enough, and one system to back up and monitor is worth a lot more than a marginally better index.
- How the index behaves as it grows. Some index types get dramatically slower past a threshold and need rebuilding with different parameters. Find out where that cliff is before you are on the far side of it.
- Rebuild and reindex cost. You will change embedding model at some point. How long does a full reindex take and can it happen without downtime?
- Whether it does hybrid search natively. Combining keyword and vector search is usually a bigger quality win than swapping vector stores.
What did not matter: benchmark recall numbers, which were within noise of each other for our data, and the query language, which you touch once.
The honest default: start with the database you already run. Move only when you hit a specific limit you can name.
@filters_first · 3w ago · 2 replies
Pre-filtering is the one I got wrong and it is expensive to discover late.
The symptom is subtle: quality looks fine in testing with a handful of tenants and degrades as data grows, because your ten candidates increasingly all belong to somebody else and get discarded. It reads as the model getting worse.
Ask the question directly before choosing: is the filter applied during the search or after it? Documentation is often vague and the answer decides whether the thing works at scale.
Reply
Report
@reindexed_twice · 2w ago
Pre versus post filtering is barely advertised and it decides everything above a few thousand documents.
Reply
Report