Pre-filtering is the one I got wrong and it is expensive to discover late.
The symptom is subtle: quality looks fine in testing with a handful of tenants and degrades as data grows, because your ten candidates increasingly all belong to somebody else and get discarded. It reads as the model getting worse.
Ask the question directly before choosing: is the filter applied during the search or after it? Documentation is often vague and the answer decides whether the thing works at scale.