9755 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Clément Renault	b03ec3f603	Another algorithm for the ARC cache	2024-07-21 18:56:50 +02:00
Clément Renault	91ec0bdaf4	Replace the SmallVec by a Vec	2024-07-21 18:22:50 +02:00
Clément Renault	56329633d5	Implement a really adaptative cache	2024-07-21 17:46:51 +02:00
Clément Renault	507bce791b	Make sure all keys are prefixed	2024-07-21 15:40:32 +02:00
Clément Renault	7adc715783	Also count the direct_inserts	2024-07-20 17:15:33 +02:00
Clément Renault	f355cf6985	Use sled to count the write insertions	2024-07-20 11:16:57 +02:00
Clément Renault	2603d8d0d0	Add a way to disable the cache	2024-07-20 10:45:16 +02:00
Clément Renault	9b3d303b08	First ArcCache version	2024-07-20 10:45:15 +02:00
Clément Renault	16b4545d23	Prefix Redis keys to avoid false negative excessive writes	2024-07-20 10:45:15 +02:00
Clément Renault	0e08906fcb	Use the sorter cache when collection prefix docids	2024-07-20 10:45:15 +02:00
Clément Renault	a3beaa90c5	Use the sorter cache when extracting the facet number docids	2024-07-20 10:45:15 +02:00
Clément Renault	02fff51902	Use the sorter cache when extracting the facet string docids	2024-07-20 10:45:15 +02:00
Clément Renault	54e2e2aa4a	Use the sorter cache when extracting the word counts	2024-07-20 10:45:14 +02:00
Clément Renault	092a383419	Use the sorter cache when extracting the word pair proximity docids	2024-07-20 10:45:14 +02:00
Clément Renault	98d55e0d4d	Use the sorter cache when extracting the word position docids	2024-07-20 10:45:14 +02:00
Clément Renault	8319552e7d	Use the sorter cache in the word docids extractor	2024-07-20 10:45:14 +02:00
Clément Renault	5d5769fd8a	Introduce a new Sorter Cache for CboRoaringBitmaps	2024-07-20 10:45:14 +02:00
Clément Renault	eafc097a85	Measure much more places where we insert in sorters	2024-07-20 10:45:13 +02:00
Clément Renault	f17cb2ef5b	Use Redis to measure the Sorter insertions	2024-07-20 10:45:13 +02:00
meili-bors[bot]	48bc797dce	Merge #4812 ... 4812: Allow `MEILI_NO_VERGEN` env var to skip vergen r=irevoire a=dureuill - vergen checks the state of the `.git` directory to embed commit information into the `meilisearch` binary and the `cargo xtask bench` invocations. - This check unfortunately results in too many recompilation of the `meilisearch` binary. - This PR allows skipping vergen when the `MEILI_NO_VERGEN` variable is present in the environment Co-authored-by: Louis Dureuil <louis@meilisearch.com>	2024-07-18 16:16:01 +00:00
Louis Dureuil	c6b33fd407	Allow MEILI_NO_VERGEN env var to skip vergen	2024-07-18 17:28:01 +02:00
meili-bors[bot]	6e9d0de8b7	Merge #4806 ... 4806: Update rustls as much as possible r=Kerollmops a=irevoire # Pull Request ## Related issue Part of https://github.com/meilisearch/meilisearch/issues/4753 ## What does this PR do? - Update rustls as much as possible ## What is missing In rustls-0.22.0 two structures we were using have been removed with no explanation or workaround <img width="518" alt="image" src="https://github.com/user-attachments/assets/fa112db1-3400-4163-8819-7913f22d6b87"> Co-authored-by: Tamo <tamo@meilisearch.com>	2024-07-17 17:00:01 +00:00
Tamo	1bfb16386c	Update rustls as much as possible	2024-07-17 18:21:26 +02:00
meili-bors[bot]	ea73615abf	Merge #4804 ... 4804: Implements the experimental contains filter operator r=irevoire a=irevoire # Pull Request Related PRD: (private link) https://www.notion.so/meilisearch/Contains-Like-Filter-Operator-0d8ad53c6761466f913432eb1d843f1e Public usage page: https://meilisearch.notion.site/Contains-filter-operator-usage-3e7421b0aacf45f48ab09abe259a1de6 ## Related issue Fixes https://github.com/meilisearch/meilisearch/issues/3613 ## What does this PR do? - Extract the contains operator from this PR: https://github.com/meilisearch/meilisearch/pull/3751 - Gate it behind a feature flag - Add tests Co-authored-by: Tamo <tamo@meilisearch.com>	2024-07-17 15:47:11 +00:00
Tamo	02c61eabfa	fix the range reported when the experimental feature has not been set	2024-07-17 16:54:33 +02:00
Tamo	56b60ec7a0	apply review comment	2024-07-17 16:13:40 +02:00
meili-bors[bot]	8f416e8f34	Merge #4805 ... 4805: Log the time to index a batch of task r=Kerollmops a=irevoire This was proposed by `@qdequele` in a private conversation and I think it’s a nice addition. Co-authored-by: Tamo <tamo@meilisearch.com>	2024-07-17 11:45:39 +00:00
Tamo	cf760cbfb1	Log the time to index a batch of task	2024-07-17 11:56:57 +02:00
Tamo	2af9481804	Implements the experimental contains filter operator«	2024-07-17 11:13:37 +02:00
meili-bors[bot]	7a292b572a	Merge #4801 ... 4801: AI quality-of-life improvements r=irevoire a=dureuill # Pull Request ## Related issue Fixes #4802 ## What does this PR do? This PR implements several quality-of-life improvements described in the [public usage](https://meilisearch.notion.site/v1-10-AI-search-changes-737c9d7d010d4dd685582bf5dab579e2#ece824a1814e47a0a986d786baff1be9) Co-authored-by: Louis Dureuil <louis@meilisearch.com>	2024-07-17 09:00:47 +00:00
Louis Dureuil	8d6ac261ae	Add tests on various failure modes for embedders	2024-07-16 13:39:02 +02:00
Louis Dureuil	b4c8b01c88	Update existing snapshots	2024-07-16 13:39:01 +02:00
Louis Dureuil	24240934f9	Improve errors when indexing documents with a user provided embedder	2024-07-16 13:39:01 +02:00
Louis Dureuil	f4c94ac57f	manual embedders: limit max size of errors to 250	2024-07-16 13:39:01 +02:00
Louis Dureuil	4087a88dbe	rest|ollama|openai: increase tries to 10 + randomize retry duration	2024-07-16 13:39:00 +02:00
Louis Dureuil	5adacf2f45	OpenAI: embed only the first MAX_TOKENS tokens	2024-07-16 13:39:00 +02:00
Louis Dureuil	65d0c32aa7	Allow overriding OpenAI's url	2024-07-16 13:39:00 +02:00
Louis Dureuil	82647bcded	When retrieveVectors is true, retrieve _vectors.embedder even if there are no vector for that embedder	2024-07-16 13:39:00 +02:00
meili-bors[bot]	1582c7e788	Merge #4769 ... 4769: Federated search r=ManyTheFish a=dureuill # Pull Request ## Related issue Fixes #4747 [Usage](https://meilisearch.notion.site/v1-10-federated-search-698dfe36ab6b4668b044f735fb40f0b2) ## What does this PR do? - multi-search now allows a top-level federation object. When not `null`, the results of multi-search are modified to be a single list of results rather than a list of a list of results - changed lifetimes around tokenizer et al. to be able to make hits one by one rather than using a vector - adds `roaring` to Meilisearch itself. As the federated search happens at the Meilisearch level (reuses the search functions declared at the Meilisearch level + merge happens after the hits were created), `RoaringBitmap`s are needed to track the candidates: hits that were seen, all candidates. - Refactor `make_hits` to allow for an individual, optimized `make_hit` - Score details comparison no longer fail when sorting on different field names or target point (for geo) Co-authored-by: Louis Dureuil <louis@meilisearch.com>	2024-07-16 08:14:46 +00:00
Louis Dureuil	20094eba06	Apply review comments	2024-07-15 12:43:29 +02:00
Louis Dureuil	c35904d6e8	search::federated::ranking_rules -> search::ranking_rules	2024-07-15 08:43:22 +02:00
Louis Dureuil	2cacc448b6	Rename src/search.rs -> src/search/mod.rs	2024-07-15 08:43:21 +02:00
Louis Dureuil	a61b852695	Add tests	2024-07-15 08:43:21 +02:00
Louis Dureuil	3167411e98	Analytics	2024-07-15 08:43:21 +02:00
Louis Dureuil	83d71662aa	Changes to multi_search route	2024-07-15 08:43:21 +02:00
Louis Dureuil	5c323cecc7	search: introduce federated search	2024-07-15 08:43:21 +02:00
meili-bors[bot]	77b9347fff	Merge #4783 ... 4783: Update minimal ubuntu version used from 18.04 to 20.04 r=curquiza a=curquiza Fixes #4782 Co-authored-by: curquiza <clementine@meilisearch.com> Co-authored-by: Tamo <tamo@meilisearch.com>	2024-07-11 16:44:30 +00:00
Tamo	c85dd9f635	install a default stable toolchain before cargo build tries to install cross	2024-07-11 18:43:47 +02:00
curquiza	7da95d62e2	Add DEBIAN_FRONTEND to avoid interaction with tzdata	2024-07-11 18:43:47 +02:00
curquiza	2cda1360ee	Remove ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION in CI	2024-07-11 18:43:47 +02:00