meilisearch/milli/src/search/new
meili-bors[bot] e9bf4c43a4
Merge #4649
4649: Don't store the vectors in the documents database r=dureuill a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4607

## What does this PR do?
- Ensure that anything falling under `_vectors` is NOT searchable, filterable or sortable
- [x] per embedder, add a roaring bitmap of documents that provide "userProvided" embeddings
- [x] in the indexing process in extract_vector_points, set the bit corresponding to the document depending on the "userProvided" subfield in the _vectors field.
- [x] in the document DB in typed chunks, when writing the _vectors field, remove all keys corresponding to an embedder

Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-17 12:32:03 +00:00
..
logger Fixes for Rust v1.79 2024-06-13 17:47:44 +02:00
matches Add ranking_score_threshold to milli 2024-05-29 14:17:09 +02:00
query_term Fix the indexing of the searchable 2024-05-14 17:00:02 +02:00
ranking_rule_graph apply all style review comments 2024-05-15 15:02:26 +02:00
tests Merge #4649 2024-06-17 12:32:03 +00:00
bucket_sort.rs Changes after review 2024-06-03 10:30:00 +02:00
db_cache.rs get back to what we were doingb efore in the DB cache and with the restricted field id 2024-05-15 18:00:39 +02:00
distinct.rs Move to the v0.20.0-alpha.9 of heed 2023-11-27 11:52:22 +01:00
exact_attribute.rs Stops returning an option in the internal searchable fields 2024-05-14 17:00:02 +02:00
geo_sort.rs chore: fix some typos in comments 2024-04-18 14:12:52 +08:00
graph_based_ranking_rule.rs Implement Frequency matching strategy 2024-05-29 13:59:08 +02:00
interner.rs Make clippy happy 2023-11-23 14:11:38 +01:00
limits.rs Remove comments and add documentation 2023-06-14 12:39:42 +02:00
mod.rs Log more stuff around filtering 2024-06-06 11:00:07 -04:00
query_graph.rs Add tests and fix matching strategy 2024-05-30 12:02:42 +02:00
ranking_rules.rs RankingRuleOutput now contains a Score 2023-06-22 12:39:14 +02:00
resolve_query_graph.rs Remove comments and add documentation 2023-06-14 12:39:42 +02:00
small_bitmap.rs SmallBitmap: Consistently panic on incoherent universe lengths 2023-03-29 08:45:38 +02:00
sort.rs Move to the v0.20.0-alpha.9 of heed 2023-11-27 11:52:22 +01:00
vector_sort.rs Add support functions for accessing arroy writers and readers 2024-05-28 15:27:43 +02:00