Clément Renault
ba85959642
Support filtering the documents to edit with lua
2024-07-10 16:23:21 +02:00
Clément Renault
1702b5cf44
Prepare for processing documents edition
2024-07-10 16:23:21 +02:00
Clément Renault
9d6885793e
Upgrade dependencies
2024-07-10 13:46:24 +02:00
karribalu
47e526f5ea
Add index exists function in index_scheduler
2024-07-08 22:27:10 +01:00
Tamo
6afa578688
update most incompatible dependencies
2024-07-08 18:31:15 +02:00
Tamo
300bdfc2a7
update most dependencies
2024-07-08 18:09:12 +02:00
ManyTheFish
015d90a962
merge main
2024-07-01 11:50:36 +02:00
Louis Dureuil
27496354e2
Grow by 1TB instead of 1MB
2024-06-25 09:01:11 +02:00
Tamo
1693332cab
Update arroy and always build the tree that need to be built
2024-06-24 10:14:03 +02:00
Louis Dureuil
3f212a8202
Update tests
2024-06-12 18:13:34 +02:00
Louis Dureuil
3bc8f81abc
user_provided => regenerate
2024-06-12 18:12:20 +02:00
Louis Dureuil
34fabed214
Add test for vector writeback
2024-06-12 17:09:34 +02:00
Louis Dureuil
fca9fe39b3
Update test snapshots
2024-06-12 14:50:55 +02:00
Tamo
3493093c4f
add a batch of tests
2024-06-11 16:03:54 +02:00
Tamo
0502b17501
log the state of the index-scheduler in all failed tests
2024-06-10 10:52:49 +02:00
Tamo
ea61e5cbec
makes clippy happy x2
2024-06-06 11:39:29 +02:00
Tamo
31a793d226
fix the regeneration of the embeddings in the search
2024-06-06 11:39:29 +02:00
Tamo
d85ab23b82
rename all occurences of user_defined to user_provided for consistency
2024-06-06 11:39:29 +02:00
Tamo
b7349910d9
implements mor review comments
2024-06-06 11:39:29 +02:00
Tamo
49fa41ce65
apply first round of review comments
2024-06-06 11:39:29 +02:00
Tamo
b867829ef1
remove useless dbg
2024-06-06 11:39:29 +02:00
Tamo
cc5dca8321
fix two bug and add a dump test
2024-06-06 11:39:29 +02:00
Tamo
5d50850e12
always push the user defined vectors in arroy
2024-06-06 11:39:29 +02:00
Tamo
9eb6f522ea
wraps the index embedding config in a struct
2024-06-06 11:37:30 +02:00
Tamo
04f6523f3c
expose a new parameter to retrieve the embedders at search time
2024-06-06 11:36:11 +02:00
Tamo
30d66abf8d
fix the test
2024-06-06 11:36:11 +02:00
Tamo
84e498299b
Remove the vectors from the documents database
2024-06-06 11:36:11 +02:00
Clément Renault
487431a035
Fix tests
2024-05-27 16:12:20 +02:00
Clément Renault
b6d450d484
Remove puffin experimental feature
2024-05-27 15:59:28 +02:00
Clément Renault
dc949ab46a
Remove puffin usage
2024-05-27 15:59:14 +02:00
Clément Renault
7f3e51349e
Remove puffin for the dependencies
2024-05-27 15:53:06 +02:00
Louis Dureuil
8a941c0241
Smaller review changes
2024-05-22 14:44:42 +02:00
Louis Dureuil
eccbcf5130
Increase index-scheduler test timeouts
2024-05-21 14:59:08 +02:00
Louis Dureuil
9969f7a638
Add test on index-scheduler
2024-05-20 14:44:10 +02:00
Louis Dureuil
02714ef5ed
Add vectors from vector DB in dump
2024-05-20 10:36:18 +02:00
Tamo
897d25780e
update milli to latest version
2024-05-16 18:31:32 +02:00
Tamo
9fffb8e83d
make clippy happy
2024-05-14 17:36:32 +02:00
Clément Renault
ac4bc143c4
Bump ureq to v2.9.7
2024-05-07 10:39:38 +02:00
meili-bors[bot]
4d5971f343
Merge #4621
...
4621: Bring back changes from v1.8.0 into main r=curquiza a=curquiza
Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2024-05-06 13:46:39 +00:00
ManyTheFish
7468c1cf8d
Introduce WildcardSetting that are serialized as wildcards by default
2024-04-24 18:15:03 +02:00
writegr
ab43a8a949
chore: fix some typos in comments
...
Signed-off-by: writegr <wellweek@outlook.com>
2024-04-18 14:12:52 +08:00
ManyTheFish
a1ea224da9
Fix tests
2024-04-16 17:29:34 +02:00
ManyTheFish
5ab901dd30
Fix tests
2024-04-16 14:39:30 +02:00
ManyTheFish
bad46f88d6
Fix embedder test
2024-04-16 14:39:30 +02:00
meili-bors[bot]
56bf8503db
Merge #4537
...
4537: Expose distribution shift in settings r=ManyTheFish a=dureuill
See [usage page](https://meilisearch.notion.site/v1-8-AI-search-API-usage-135552d6e85a4a52bc7109be82aeca42#d652adc0890445658aaf36352dbc8802 )
# Changes
- Distribution shift added to all embedders.
- Exposed in settings
- Changed the reindexing logic to not trigger a reindex operation when only the distribution shift or API key change
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-04-03 09:08:58 +00:00
redistay
182cb42953
chore: fix some typos in conments
...
Signed-off-by: redistay <wujunjing@outlook.com>
2024-04-02 19:37:55 +08:00
meili-bors[bot]
78668584cd
Merge #4533
...
4533: Hide api key in settings and task queue r=dureuill a=dureuill
# Pull Request
See [Usage page](https://meilisearch.notion.site/v1-8-AI-search-API-usage-135552d6e85a4a52bc7109be82aeca42#117f5ff7b19f4d95bb3ae0005f6c6633 )
## Motivation
See [slack discussion (internal link)](https://meilisearch.slack.com/archives/C06GQP7FQ6P/p1709804022298749 )
## Changes
- The value of the `apiKey` parameter is now hidden in the settings and the details of the task queue.
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-03-28 16:02:53 +00:00
Bruno Casali
8f2606d79d
fixes typos
2024-03-27 14:26:47 -03:00
Louis Dureuil
92224f109a
Fix tests
2024-03-27 12:19:10 +01:00
Louis Dureuil
9a95ed619d
Add tests
2024-03-26 10:36:56 +01:00
Louis Dureuil
f82d056072
Hide secrets in settings and task queue
2024-03-26 10:36:24 +01:00
Tamo
f2f1367ec3
add a timeout to the webhook
2024-03-20 13:59:43 +01:00
Tamo
b130917933
add the content type in the webhook + improve the test
2024-03-05 11:22:29 +01:00
Louis Dureuil
452a343a2b
Fix imports
2024-02-28 18:09:40 +01:00
meili-bors[bot]
b005eb3289
Merge #4435
...
4435: Make update file deletion atomic r=Kerollmops a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4432
Fixes https://github.com/meilisearch/meilisearch/issues/4438 by adding the logs the user asked
## What does this PR do?
- Adds a bunch of logs to help debug this kind of issue in the future
- Delete the update files AFTER committing the update in the `index-scheduler` (thus, if a restart happens, we are able to re-process the batch successfully)
- Multi-thread the deletion of all update files.
Co-authored-by: Tamo <tamo@meilisearch.com>
2024-02-26 17:54:40 +00:00
Tamo
0562818c2a
fix and remove the file-store hack of /dev/null
2024-02-26 13:59:41 +01:00
Tamo
36c27a18a1
implement the dry run ha parameter
2024-02-26 13:58:04 +01:00
Tamo
1eb1c043b5
disable the auto deletion of tasks when the ha mode is enabled
2024-02-26 13:58:04 +01:00
Tamo
eb25b07390
let you specify your task id
2024-02-26 13:56:31 +01:00
Tamo
066a7a3cde
takes only one read transaction per thread
2024-02-26 10:43:04 +01:00
Tamo
91cdd502f8
When processing tasks, make the update file deletion atomic
2024-02-22 14:56:22 +01:00
Tamo
3b6544db6d
Implement the experimental log mode cli flag
2024-02-13 18:09:15 +01:00
Louis Dureuil
ef994d84d0
Change error messages and fix tests
2024-02-08 15:04:06 +01:00
Tamo
f70a615ed9
update the github discussion links
2024-02-08 15:04:05 +01:00
Tamo
7ff722b72e
get rids of the log dependencies everywhere
2024-02-08 15:04:05 +01:00
Tamo
e23ec4886d
fix the tests and add tests on the experimental features
2024-02-08 15:04:03 +01:00
Tamo
7793ba67a4
hide the route logs behind a feature flag
2024-02-08 15:03:33 +01:00
Tamo
2f1abd2c03
nelson is not used anymore
2024-02-08 15:03:32 +01:00
Louis Dureuil
02e6c8a440
Add tracing to index-scheduler
2024-02-08 15:03:31 +01:00
Louis Dureuil
05edd85d75
Stabilize scoreDetails
2024-02-06 11:15:19 +01:00
Clément Renault
5869ca7716
Upgrade all compatible dependencies
2024-01-16 15:05:03 +01:00
meili-bors[bot]
1ccde9bf0b
Merge #4316
...
4316: Autobatch the task deletions r=curquiza a=irevoire
# Pull Request
## Related issue
Fix part of https://github.com/meilisearch/meilisearch-support/issues/69
Fix #4315
## What does this PR do?
- Autobatch the task deletions
Co-authored-by: Tamo <tamo@meilisearch.com>
2024-01-15 17:54:50 +00:00
Tamo
b4d7d80ad9
autobatch the task deletions
2024-01-11 14:58:07 +01:00
Louis Dureuil
97bb1ff9e2
Move currently_updating_index to IndexMapper
2024-01-09 15:37:27 +01:00
meili-bors[bot]
658ec6e0a4
Merge #4279
...
4279: Check experimental feature on setting update query rather than in the task. r=ManyTheFish a=dureuill
Improve the UX by checking for the vector store feature and returning an error synchronously when sending a setting update, rather than in the indexing task.
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2023-12-22 11:36:12 +00:00
Louis Dureuil
ee54d3171e
Check experimental feature at query time
2023-12-21 15:26:12 +01:00
Clément Renault
fa2b96b9a5
Add an Authorization Header along with the webhook calls
2023-12-19 12:18:45 +01:00
Tamo
4fb25b8782
fix clippy
2023-12-19 10:35:51 +01:00
Tamo
c83a33017e
stream and chunk the data
2023-12-19 10:35:51 +01:00
Tamo
be72326c0a
gzip the tasks
2023-12-19 10:35:51 +01:00
Tamo
0b2fff27f2
update and fix the test
2023-12-19 10:35:51 +01:00
Tamo
3adbc2b942
return a task view instead of a task
2023-12-19 10:35:51 +01:00
Tamo
fbea721378
add a first working test with actixweb
2023-12-19 10:35:51 +01:00
Tamo
d78ad51082
Implement the webhook
2023-12-19 10:35:50 +01:00
Many the fish
9e1b458010
Merge branch 'main' into change-proximity-precision-settings
2023-12-18 09:08:47 +01:00
Louis Dureuil
e0cc775dc4
Various changes
...
- DistributionShift in Search object (to be set from model in embed?)
- Fix issue where embedder index wasn't computed at search time
- Accept as default embedder either the "default" one, or the only embedder when there is only one
2023-12-14 16:08:41 +01:00
Louis Dureuil
922a640188
WIP multi embedders
...
fixed template bugs
2023-12-14 16:08:41 +01:00
Louis Dureuil
abbe131084
Cosmetic change
2023-12-14 16:08:41 +01:00
Louis Dureuil
13c2c6c16b
Small commit to add hybrid search and autoembedding
2023-12-14 16:07:48 +01:00
ManyTheFish
35e1981488
Remove proximityPrecision form the experimental feature
2023-12-14 15:52:42 +01:00
Clément Renault
7e259cb0d2
Expose the --max-number-of-batched-tasks argument
2023-12-11 16:08:39 +01:00
ManyTheFish
1f4fc9c229
Make the feature experimental
2023-12-06 15:49:05 +01:00
meili-bors[bot]
6376c342c1
Merge #4223
...
4223: Update to heed 0.20 r=dureuill a=Kerollmops
This PR brings the v0.20-alpha.9 version of heed into Meilisearch 🎉 The main goal is to test it in a real environment to make the necessary changes if needed. We also want to merge it as soon as possible during the pre-release phase to ensure we catch bugs before the release.
Most of the calls to heed are the same as before, except:
- The `PolyDatabase` has been replaced with a `Database<Unspecified, Unspecified>`. We replaced the `get<T, U>()` by a `remap<T, U>().get()` calls.
- The `Database` `append(...)` method has been replaced with a `put_with_flags(PutFlags::APPEND, ...)`.
- The `RwTxn<'e, 'p>` has been simplified into a `RwTxn<'e>`.
- The `BytesEncode/Decode` traits return a `Result<_, BoxedError>` instead of an `Option<_>`.
- We no longer need to wrap and unwrap the `BEU32` integer when storing/getting them from heed.
### TODO
- [x] Create actual, simple error types instead of using strings in the codecs.
### Follow-up work
- Move the codecs into another member crate (we depend on the uuid one in the meilitool crate).
- Display the internal decoding error in the `SerializationError` internal error variant.
Co-authored-by: Clément Renault <clement@meilisearch.com>
2023-11-28 13:39:44 +00:00
Clément Renault
ec9b52d608
Rename copy_to_path to copy_to_file
2023-11-28 14:32:30 +01:00
Clément Renault
34c67ac389
Remove the possibility to fail fetching the env info
2023-11-28 14:31:23 +01:00
Tamo
5751f5c640
fix puffin in the index scheduler
2023-11-27 15:18:33 +01:00
Clément Renault
0dbf1a16ff
Make clippy happy
2023-11-23 14:11:38 +01:00
Clément Renault
462b4c0080
Fix the tests
2023-11-23 12:07:35 +01:00
Clément Renault
0d4482625a
Make the changes to use heed v0.20-alpha.6
2023-11-23 11:43:58 +01:00
Clément Renault
7cb7e37ba8
Merge branch 'main' into tmp-release-v1.5.0
2023-11-21 16:30:46 +01:00
