Kerollmops
97e17f52a1
Add more logs to see calls to the embedders
2025-01-30 16:36:12 +01:00
Kerollmops
62ced0e3f1
Make cargo fmt happy
2025-01-30 11:09:54 +01:00
Clément Renault
71bb24f17e
Throw and error when the index is not found
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-30 11:07:43 +01:00
Clément Renault
c72f114b33
Fix english in the comments
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-30 11:07:09 +01:00
meili-bors[bot]
8ed39f5de0
Merge #5300
...
5300: Improve unexpected panic message r=irevoire a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5273
## What does this PR do?
- When an unexpected panic happens in the index-scheduler we catch it and rebuild an error message from the join_error
- Same when the upgrade index-scheduler fails
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-01-30 09:23:17 +00:00
Kerollmops
424c5bde40
Move the embedding computation and extraction log to debug
2025-01-29 16:40:36 +01:00
Tamo
bdd3005d10
Log the progress when a batch fails
2025-01-29 16:36:23 +01:00
Kerollmops
cb1b7513af
Log the memory metrics only once
2025-01-29 15:21:52 +01:00
Clément Renault
a9d0f4a002
Improve english comments
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-29 15:16:40 +01:00
Kerollmops
db032079d8
Show indexation allocated memory
2025-01-29 14:21:02 +01:00
Clément Renault
a00796c46a
Improve the naming in the log message
2025-01-29 14:21:02 +01:00
Kerollmops
6112bd8caa
Display the channel congestion
2025-01-29 14:21:02 +01:00
Kerollmops
cec88cfc29
Measure the bbqueue congestion
2025-01-29 14:21:02 +01:00
Tamo
8439aeb7cf
improve error message in case of unexpected panic while processing tasks
2025-01-29 11:51:06 +01:00
Tamo
1beda3b9af
fix another flaky test
2025-01-28 16:53:50 +01:00
Tamo
8676e94f5c
fix the flaky tests
2025-01-28 16:53:50 +01:00
Tamo
ef47a0d820
apply review comment
2025-01-28 16:53:50 +01:00
Tamo
e0f0da57e2
make sure the batches we snapshots actually all contains an enqueued_at
2025-01-28 16:53:50 +01:00
Tamo
485e3127c7
use the remove_n_tasks_datetime_earlier_than function when updating batches
2025-01-28 16:53:50 +01:00
Tamo
58f90b70c7
store the enqueued at to eases the batch deletion
2025-01-28 16:53:50 +01:00
Tamo
508db9020d
update the snapshots
2025-01-28 16:53:50 +01:00
Kerollmops
6ff37c6fc4
Fix the insta snapshots
2025-01-28 16:53:50 +01:00
Kerollmops
f21ae1f5d1
Remove the batch id from the date time databases
2025-01-28 16:53:50 +01:00
Kerollmops
19bc885b07
Fix the milli logo
2025-01-27 14:30:59 +01:00
Kerollmops
47f70e3d79
Debug the first vector sort fill buffer
2025-01-27 14:22:29 +01:00
Kerollmops
0f8eb3b506
Improve the logs of the search with AI
2025-01-27 14:22:22 +01:00
Kerollmops
4a5923a55e
log the time arroy took to insert embeddings
2025-01-27 14:22:17 +01:00
meili-bors[bot]
de98656ed1
Merge #5210
...
Test suite / Tests almost all features (push) Has been skipped
Run the indexing fuzzer / Setup the action (push) Failing after 6s
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Tests on ubuntu-20.04 (push) Failing after 14s
Test suite / Run tests in debug (push) Failing after 2s
Test suite / Run Rustfmt (push) Failing after 8s
Test suite / Tests on windows-2022 (push) Failing after 20s
Test suite / Run Clippy (push) Successful in 5m48s
Indexing bench (push) / Run and upload benchmarks (push) Has been cancelled
Benchmarks of indexing (push) / Run and upload benchmarks (push) Has been cancelled
Benchmarks of search for geo (push) / Run and upload benchmarks (push) Has been cancelled
Benchmarks of search for songs (push) / Run and upload benchmarks (push) Has been cancelled
Benchmarks of search for Wikipedia articles (push) / Run and upload benchmarks (push) Has been cancelled
Test suite / Tests on macos-13 (push) Has been cancelled
5210: Improve test performance of get_index.rs r=irevoire a=DerTimonius
# Pull Request
## Related issue
related to #4840
## What does this PR do?
This PR aims to improve the performance of the tests in `get_index.rs`.
There is a small issue though:
the `list_multiple_indexes` test works great when ran alone, but when running with other tests it fails with a `corrupted task queue` error. I guess this has something to do with using a shared server, but I was not really able to pinpoint the issue.
Also, the `no_index_return_empty_list` does not work a shared server (as there now will always be at least one index on the server) and I was not really sure if rebuilding the whole suite for `get_and_paginate_indexes` should be viable? While waiting for feedback on the issue mentioned above, I'll try to change the `get_and_paginate_indexes` test so that it can use the shared server
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Timon Jurschitsch <timon.jurschitsch@gmail.com>
Co-authored-by: Timon Jurschitsch <103483059+DerTimonius@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-01-27 10:04:08 +00:00
Tamo
da7469be38
removed unrelated files
2025-01-27 10:35:34 +01:00
manojks1999
528d9d6d8b
Removed CouldNotUpgrade from error file
2025-01-26 21:04:57 +05:30
Timon Jurschitsch
4fb5c39b92
resolve merge conflicts
2025-01-24 14:35:54 +01:00
Louis Dureuil
50280bf02b
Support offline upgrade up to v1.12.7
2025-01-24 12:25:33 +01:00
Clément Renault
9b579069df
Comment the max grant of the bbqueue
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-24 12:18:32 +01:00
Louis Dureuil
f5a4a1c8b2
Give more RAM to bbqueue.
...
- bbqueue buffers used to have (5% * 2%) / num_threads
- they now have 5% / num_threads
2025-01-24 12:18:32 +01:00
Kerollmops
5ab4cdb1f3
Reduce the maximum grant possible we can store in the BBQueue
2025-01-24 12:18:32 +01:00
Louis Dureuil
73d8a4eace
Remove db.snapshot
2025-01-23 17:21:42 +01:00
Louis Dureuil
c1e5897076
Do not assume v1.12 when there is no index-scheduler version
2025-01-23 17:16:53 +01:00
Louis Dureuil
718a98fbbf
remove : char from filenames
2025-01-23 17:08:35 +01:00
Louis Dureuil
86bf231d29
Change to meilitool after rebase
2025-01-23 16:59:32 +01:00
Louis Dureuil
182c3f4b80
Write assumed version to the index-scheduler version db when it is missing
2025-01-23 16:51:25 +01:00
Tamo
c1eba66443
introduce a corruption in the v1.12 data.ms field distribution
2025-01-23 16:51:24 +01:00
Tamo
7197ced673
fix the bad index version on opening
2025-01-23 16:51:24 +01:00
Tamo
4f21ee6c66
update the data.ms snapshot
2025-01-23 16:51:24 +01:00
Tamo
787472453d
write the version of the index while upgrading it
2025-01-23 16:51:24 +01:00
Tamo
8f65f35de9
rewrite part of the index-scheduler upgrade test
2025-01-23 16:51:23 +01:00
Tamo
c27c923439
introduce a trait to upgrade the indexes
2025-01-23 16:51:23 +01:00
Tamo
fd5649091d
add the upgradeTo field in the details
2025-01-23 16:51:23 +01:00
Tamo
9a57736773
fix the early exit when rewriting a batch
2025-01-23 16:51:23 +01:00
Tamo
7740997ea8
reintroduce the unrecoverable error and use it where its supposed to be used
2025-01-23 16:51:22 +01:00
Tamo
7eb23f73ba
add the version to the index-scheduler snapshots + fix a bug when opening an index scheduler for the first time
2025-01-23 16:51:22 +01:00
