10696 Commits

Author SHA1 Message Date
Louis Dureuil
34d8c1a903
Make offline upgrade more flexible 2025-01-20 10:43:47 +01:00
Louis Dureuil
3c9483b6e0
meilitool dumps old-style dump for older DBs, otherwise new-style 2025-01-20 10:43:47 +01:00
Louis Dureuil
11458eefd9
Handle empty payloads 2025-01-20 09:51:07 +01:00
Louis Dureuil
289eb92bef
Fix warnings 2025-01-20 09:51:07 +01:00
ManyTheFish
cea0c89212
Change format of update file when importing dump 2025-01-20 09:51:07 +01:00
Louis Dureuil
1cadab9ad8
Also fix dump import from meilitool 2025-01-20 09:51:07 +01:00
Louis Dureuil
6383f8f19e
Do not explode on missing content file if the task has no docs 2025-01-20 09:51:06 +01:00
Louis Dureuil
8a9f952bda
Create update files in new format 2025-01-20 09:51:06 +01:00
meili-bors[bot]
a5c44b4d79
Merge #5242
Some checks failed
Test suite / Tests on ubuntu-20.04 (push) Failing after 13s
Test suite / Tests almost all features (push) Has been skipped
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Tests on ${{ matrix.os }} (windows-2022) (push) Failing after 14s
Test suite / Run tests in debug (push) Failing after 11s
Test suite / Run Clippy (push) Successful in 7m16s
Test suite / Run Rustfmt (push) Successful in 2m41s
Test suite / Tests on ${{ matrix.os }} (macos-13) (push) Has been cancelled
5242: Fix infinite loop r=Kerollmops a=dureuill

- Fix possible infinite loop by releasing `writer_receiver` as soon as writing to DB panics
- Demote panic to error log

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
v1.12.4
2025-01-16 14:13:07 +00:00
Louis Dureuil
8c35744848
Improve error log 2025-01-16 14:33:53 +01:00
meili-bors[bot]
c0d414fc3c
Merge #5243
5243: Update version for the next release (v1.12.4) in Cargo.toml r=dureuill a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one and Cargo.lock has been updated before merging.

Co-authored-by: dureuill <dureuill@users.noreply.github.com>
2025-01-16 10:57:55 +00:00
dureuill
b56358f606 Update version for the next release (v1.12.4) in Cargo.toml 2025-01-16 10:47:17 +00:00
Louis Dureuil
b84c0a5390
Demote panic to error log 2025-01-16 11:38:21 +01:00
Louis Dureuil
ce621e447e
Release writer_receiver as soon as writing to db panics 2025-01-16 11:37:37 +01:00
Clément Renault
aee74f47aa
Merge pull request #5229 from meilisearch/improve-unknown-entry-deletion-report
Improve the panic message when deleting an unknown entry
v1.12.3
2025-01-13 14:20:24 +01:00
Clément Renault
be2717edbd
Merge pull request #5224 from meilisearch/fix-facet-distribution
Fix facet distribution
2025-01-13 14:20:09 +01:00
Louis Dureuil
c66841626e
Update after review 2025-01-13 10:43:26 +01:00
Kerollmops
d0bc8c755a
Improve the panic message when deleting an unknown entry 2025-01-13 10:30:53 +01:00
meili-bors[bot]
031abfd281
Merge #5227
5227: Update version for the next release (v1.12.3) in Cargo.toml r=dureuill a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one and Cargo.lock has been updated before merging.

Co-authored-by: dureuill <dureuill@users.noreply.github.com>
2025-01-13 09:29:18 +00:00
dureuill
27169bc7b4 Update version for the next release (v1.12.3) in Cargo.toml 2025-01-13 08:47:07 +00:00
Louis Dureuil
181a01f8d8
Skip rebuilding field distribution if not coming from v1.12 2025-01-13 09:31:27 +01:00
Louis Dureuil
1d153c1867
write stats after rebuilding facet distribution 2025-01-09 18:13:36 +01:00
Louis Dureuil
5fde2a3ee1
Add support to upgrade to v1.12.3 in meilitool 2025-01-09 15:25:44 +01:00
Louis Dureuil
4465a1a3c9
Fix test 2025-01-09 13:26:17 +01:00
Louis Dureuil
e342ae1b46
Add currently failing test 2025-01-09 13:26:13 +01:00
meili-bors[bot]
dcb4c49cf2
Merge #5205
Some checks failed
Test suite / Tests on ${{ matrix.os }} (macos-13) (push) Waiting to run
Test suite / Tests almost all features (push) Has been skipped
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Tests on ubuntu-20.04 (push) Failing after 12s
Test suite / Run tests in debug (push) Failing after 11s
Test suite / Tests on ${{ matrix.os }} (windows-2022) (push) Failing after 30s
Test suite / Run Rustfmt (push) Successful in 1m40s
Test suite / Run Clippy (push) Successful in 5m50s
5205: Incremental facets on v1.12 r=curquiza a=dureuill

# Pull Request

## Related issue

Fixes https://github.com/meilisearch/meilisearch/issues/5213

## What does this PR do?
- Add `new_incremental`module that computes incremental facet indexing for the new indexer
- Change heuristics for incremental vs bulk facet choice: under 1000 operations is now always incremental, over 100_000 operations is now always bulk)
- Add sanity checks in debug

## Future improvements

- Use multi ops from Roaring to decrease the number of allocations
- Consider removing or adding multiple levels at once instead of max once per update
- Consider using information about the tree structure + the operations that were done (e.g. only addition) to avoid recomputing the group from all children 
- Consider making the algorithm parallel and looking into the roaring values to know which actually changed



Co-authored-by: Louis Dureuil <louis@meilisearch.com>
v1.12.2
2025-01-08 21:52:02 +00:00
Louis Dureuil
e83c021755
When spilling on the next fid, no longer ignore children 2025-01-08 16:50:05 +01:00
Louis Dureuil
7ec7200378
Check valid_facet_value as part of a filter of the iterator 2025-01-08 16:25:44 +01:00
Louis Dureuil
6a577254fa
No longer ignore the first child without parent 2025-01-08 16:25:30 +01:00
Louis Dureuil
fd88c834c3
Modernize valid_lmdb_key 2025-01-08 15:22:11 +01:00
Louis Dureuil
b4005593f4
Switch to an iterative algorithm for find_changed_parents 2025-01-08 14:57:14 +01:00
Louis Dureuil
8ee3793259
Update after review 2025-01-08 13:58:14 +01:00
Louis Dureuil
3648abbfd5
Remove unused FacetFieldIdOperation 2025-01-07 15:26:09 +01:00
Louis Dureuil
4d2433de12
center groups prototype-incremental-facets-0 2025-01-06 18:23:35 +01:00
Louis Dureuil
28cc6df7a3
Fix uselessly deep stack trace 2025-01-06 18:07:49 +01:00
meili-bors[bot]
7b14cb10a1
Merge #5207
Some checks failed
Test suite / Tests on ${{ matrix.os }} (macos-13) (push) Waiting to run
Test suite / Tests almost all features (push) Has been skipped
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Tests on ubuntu-20.04 (push) Failing after 13s
Test suite / Tests on ${{ matrix.os }} (windows-2022) (push) Failing after 13s
Test suite / Run tests in debug (push) Failing after 12s
Test suite / Run Rustfmt (push) Successful in 1m40s
Test suite / Run Clippy (push) Successful in 5m49s
5207: Update version for the next release (v1.12.2) in Cargo.toml r=dureuill a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one and Cargo.lock has been updated before merging.

Co-authored-by: dureuill <dureuill@users.noreply.github.com>
2025-01-06 16:04:03 +00:00
Louis Dureuil
34f4602ae8
Update snapshot 2025-01-06 16:55:12 +01:00
dureuill
12e21a177b Update version for the next release (v1.12.2) in Cargo.toml 2025-01-06 14:11:58 +00:00
Louis Dureuil
7a9290aaae
Use new incremental facet indexing and enable sanity checks in debug 2025-01-06 15:08:48 +01:00
Louis Dureuil
5d219587b8
Add new incremental facet indexing 2025-01-06 15:08:36 +01:00
Louis Dureuil
6e9aa49893
add valid_facet_value utility function 2025-01-06 15:08:07 +01:00
Louis Dureuil
6b3a2c7281
Add sanity checks for facet values 2025-01-06 15:07:55 +01:00
meili-bors[bot]
5908aec6cb
Merge #5192
Some checks failed
Test suite / Tests almost all features (push) Has been skipped
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Tests on ubuntu-20.04 (push) Failing after 12s
Test suite / Run tests in debug (push) Failing after 12s
Test suite / Tests on ${{ matrix.os }} (windows-2022) (push) Failing after 28s
Test suite / Run Rustfmt (push) Successful in 2m16s
Test suite / Run Clippy (push) Successful in 6m20s
Test suite / Tests on ${{ matrix.os }} (macos-13) (push) Has been cancelled
5192: Fix empty document addition r=irevoire a=irevoire

# Pull Request

## Related issue
Fixes #5190

## What does this PR do?
- Improve a test just to make sure this issue never arises again
- Fix the issue

For the reviewer: Calling `add_documents` with an empty `mmap` seems to work, but does it impact the perf in a significant way? / 

Co-authored-by: Tamo <tamo@meilisearch.com>
v1.12.1
2024-12-31 17:11:10 +00:00
Tamo
19f48c15fb
Fix the addition of empty payload 2024-12-31 18:00:14 +01:00
Tamo
47b484c07c
update the test to ensure it works when specifying the primary key or not: it doesn't work 2024-12-31 17:24:32 +01:00
meili-bors[bot]
7d5e28b475
Merge #5193
5193: Update version for the next release (v1.12.1) in Cargo.toml r=irevoire a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one and Cargo.lock has been updated before merging.

Co-authored-by: curquiza <curquiza@users.noreply.github.com>
2024-12-31 09:40:31 +00:00
curquiza
0648e06aa2 Update version for the next release (v1.12.1) in Cargo.toml 2024-12-30 17:36:46 +00:00
Tamo
33921747b7
stop skipping empty tasks when adding documents 2024-12-30 17:48:25 +01:00
Tamo
970a489dcc
add a test reproducing the bug 2024-12-30 16:21:06 +01:00
meili-bors[bot]
ba11121cfc
Merge #5159
Some checks failed
Test suite / Tests almost all features (push) Has been skipped
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Tests on ubuntu-20.04 (push) Failing after 11s
Test suite / Run tests in debug (push) Failing after 10s
Test suite / Tests on ${{ matrix.os }} (windows-2022) (push) Failing after 22s
Test suite / Run Rustfmt (push) Successful in 1m18s
Test suite / Run Clippy (push) Successful in 5m30s
Test suite / Tests on ${{ matrix.os }} (macos-13) (push) Has been cancelled
5159: Fix the New Indexer Spilling r=irevoire a=Kerollmops

Fix two bugs in the merging of the spilled caches. Thanks to `@ManyTheFish` and `@irevoire` 👏

Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
v1.12.0 v1.12.0-rc.6
2024-12-12 17:16:53 +00:00