mirror of
https://github.com/meilisearch/meilisearch.git
synced 2024-11-22 10:07:40 +08:00
19e6f675b3
4900: Indexer edition 2024 r=Kerollmops a=dureuill This PR is implementing the indexer edition 2024, largely inspired by [the ideas from this blog post](https://blog.kerollmops.com/meilisearch-is-too-slow). Fixes https://github.com/meilisearch/meilisearch/issues/4985 ## Features - Stream-first approach to reading documents. - Minimum disk write operations. - RAM usage-first approach to avoid modifying common bitmaps on disk but in memory. - Reduced LMDB fragmentation by writing entries only once... - ...computing the final version of the entries in parallel... - ...and storing them in write-optimized data structures before sending them to the BTree (LMDB). - Indexing in multiple transactions to improve large dataset support (dumps). Co-authored-by: ManyTheFish <many@meilisearch.com> Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Louis Dureuil <louis@meilisearch.com> |
||
|---|---|---|
| .. | ||
| benchmarks | ||
| build-info | ||
| dump | ||
| file-store | ||
| filter-parser | ||
| flatten-serde-json | ||
| fuzzers | ||
| index-scheduler | ||
| json-depth-checker | ||
| meili-snap | ||
| meilisearch | ||
| meilisearch-auth | ||
| meilisearch-types | ||
| meilitool | ||
| milli | ||
| permissive-json-pointer | ||
| tracing-trace | ||
| xtask | ||