snowykami/meilisearch
1
0
Fork 0
mirror of https://github.com/meilisearch/meilisearch.git synced 2024-11-27 12:35:05 +08:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity
4,485 Commits 168 Branches 387 Tags 81 MiB
25e39edc7e
Commit Graph

257 Commits

Author SHA1 Message Date
Irevoire
e0821ad4b0
remove an useless dbg 2022-10-27 11:35:13 +02:00
Irevoire
61f0940f8c
fix an issue with the dates 2022-10-27 11:35:13 +02:00
Irevoire
241300d2d8
add more naive tests around the document addition + remove the old unused snapshot files 2022-10-27 11:35:13 +02:00
Irevoire
570b2d1167
add some naive document addition tests 2022-10-27 11:35:12 +02:00
Loïc Lecrenier
d92425658e
Add index scheduler tests for task cancelation 2022-10-27 11:35:12 +02:00
Loïc Lecrenier
16fac10074
Fix crash when batching an index swap task containing 0 swaps 2022-10-27 11:35:12 +02:00
Irevoire
0aca5e84b9
rename received_document_ids to matched_documents in the DocumentDeletion task type (reimplementation of #2826) 2022-10-27 11:35:12 +02:00
Irevoire
7ed3f00b1e
reformat 2022-10-27 11:35:12 +02:00
Irevoire
9c00b159ba
fix clippy 2022-10-27 11:35:11 +02:00
Irevoire
7e52f1effb
remove a lot of unecessary clone and ref 2022-10-27 11:35:11 +02:00
Loïc Lecrenier
4d25c159e6
Apply code review suggestions 2022-10-27 11:35:11 +02:00
Loïc Lecrenier
e9cd6cbbee
Revert implementation of get_status to query only the database 2022-10-27 11:35:11 +02:00
Loïc Lecrenier
424202d773
Pause the index scheduler for one second when a fatal error occurs 2022-10-27 11:35:11 +02:00
Loïc Lecrenier
4a35eb9849
Fix (hopefully) queries that include processing tasks 2022-10-27 11:35:11 +02:00
Loïc Lecrenier
493a8cff31
Adjust task details correctly following index swap 2022-10-27 11:35:10 +02:00
Loïc Lecrenier
4de445d386
Start testing unexpected errors and panics in index scheduler 2022-10-27 11:35:10 +02:00
Loïc Lecrenier
e3848b5f28
Add assert method to verify validity of index scheduler state 2022-10-27 11:35:10 +02:00
Irevoire
ecf4e43b3d
rename the dumpExport to dumpCreation 2022-10-27 11:35:10 +02:00
Irevoire
8a23e707c1
fix the task view and forward the task db size 2022-10-27 11:35:09 +02:00
Irevoire
eb4bdde432
fix clippy 2022-10-27 11:35:09 +02:00
Irevoire
1d04ce611d
remove ununsed function 2022-10-27 11:35:08 +02:00
Irevoire
e9055f5572
fix clippy 2022-10-27 11:35:08 +02:00
Irevoire
874499a2d2
fix all the snapshots 2022-10-27 11:35:08 +02:00
Irevoire
ecdcbf350f
update all the snapshots with the new kind name 2022-10-27 11:35:08 +02:00
Irevoire
c8ee453b6c
fix the autobatched document deletion 2022-10-27 11:35:07 +02:00
Irevoire
a8de5368e5
fix the index creation in case an index already exists 2022-10-27 11:35:07 +02:00
Irevoire
9bb2e3c790
fix the failed document addition with a primary key 2022-10-27 11:35:07 +02:00
Irevoire
8d1408c65e
fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting 2022-10-27 11:35:05 +02:00
Irevoire
131fe30934
fix the error messages and the index stats 2022-10-27 11:35:05 +02:00
Irevoire
50386921df
fix the index creation 2022-10-27 11:35:05 +02:00
Clément Renault
32cfac0cfd
Sort the TOML dependencies 2022-10-27 11:35:05 +02:00
Clément Renault
80b2e70ee7
Introduce a rustfmt file 2022-10-27 11:35:05 +02:00
Clément Renault
2a7ef3b352
Reapply #2830 2022-10-27 11:34:18 +02:00
Clément Renault
72ec4ce96b
Fix allow_index_creation useless field 2022-10-27 11:34:17 +02:00
Clément Renault
75857bf476
Fix the insta tests 2022-10-27 11:34:17 +02:00
Irevoire
0bbf80186f
push the snapshot files 2022-10-27 11:34:17 +02:00
Irevoire
b6a0abea9f
fix the index deletion when the index doesn’t exists but would be created by one of the autobatched tasks 2022-10-27 11:34:16 +02:00
Irevoire
5303bbffab
fix the last rule about merging the allow_index_creation 2022-10-27 11:34:16 +02:00
Irevoire
fc944c39a5
simplify the code A LOT and create less false positive 2022-10-27 11:34:16 +02:00
Irevoire
a1d4cc673d
add a whole new batch of tests around the index already exists / allow_index_creation 2022-10-27 11:34:16 +02:00
Irevoire
28d9f2c041
fix all the snapshot tests 2022-10-27 11:34:16 +02:00
Irevoire
d9218578e3
it probably works but it's also horrendous 2022-10-27 11:34:16 +02:00
Loïc Lecrenier
11fee30f47
Apply review suggestions and stop using rtxn.commit 2022-10-27 11:34:15 +02:00
Loïc Lecrenier
17cd2a4aa0
Implement POST /indexes-swap 2022-10-27 11:34:15 +02:00
Loïc Lecrenier
28bd8b6c6b
Remove key from index_tasks database when the value is empty 2022-10-27 11:34:15 +02:00
Loïc Lecrenier
169f386418
Add some documentation to the index scheduler 2022-10-27 11:34:15 +02:00
Loïc Lecrenier
bdb17954d2
Fix bug where assert used != instead of == 
And update snapshot tests.
 2022-10-27 11:34:15 +02:00
Loïc Lecrenier
23b01a58df
cargo fmt 2022-10-27 11:34:14 +02:00
Loïc Lecrenier
ec3391808d
Fix date parsing for task queries 
Use rfc3339 or YYYY-MM-DD.

Add a day to the parsed date when it is an excluded lower bound
and the YYYY-MM-DD was used.

Also the Query type does not need to be serialisable anymore
 2022-10-27 11:34:14 +02:00
Loïc Lecrenier
10a547df4f
Apply suggestions from code review 
Co-authored-by: Clément Renault <clement@meilisearch.com>

Apply suggestions from code review

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>

Apply suggestions from code review

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>

Apply code review suggestion

Co-authored-by: Clément Renault <clement@meilisearch.com>
 2022-10-27 11:34:14 +02:00
Loïc Lecrenier
22cf0559fe
Implement task date filters 
before/after enqueued/started/finished at
 2022-10-27 11:34:14 +02:00
Irevoire
5765883600
fix the auto-generated details 2022-10-27 11:34:14 +02:00
Tamo
cff003c928
remove the unused variants from the autobatcher 2022-10-27 11:34:14 +02:00
Tamo
ab8f1c2865
fix a bunch of snapshot tests 2022-10-27 11:34:13 +02:00
Kerollmops
50b8b9df6a
Delete the tasks content file once the transaction has been successfully committed 2022-10-27 11:34:13 +02:00
Kerollmops
ec0a5a9f01
Remove the useless r#union thing 2022-10-27 11:34:13 +02:00
Kerollmops
6460b78e08
Clean up the delete_persisted_task_data function 2022-10-27 11:34:13 +02:00
Kerollmops
d21651c968
Throw the error if we can't register the tasks in the store 2022-10-27 11:34:13 +02:00
Kerollmops
6e904d0997
Introduce a ProcessingTasks constructor 2022-10-27 11:34:12 +02:00
Kerollmops
b373d19831
Extract the must_stop flag out of the RwLock 2022-10-27 11:34:12 +02:00
Kerollmops
3cbfacb616
Prefer using an u64 instead of a usize in some places 2022-10-27 11:34:12 +02:00
Kerollmops
79c4275bfc
Delete the persisted data when we cancel a task 2022-10-27 11:34:12 +02:00
Kerollmops
c2ec4a089b
Put the original URL query in the tasks details 2022-10-27 11:34:12 +02:00
Kerollmops
751e9bac3b
Add the tasks cancel route to cancel tasks 2022-10-27 11:34:11 +02:00
Kerollmops
290945e258
Update the canceledBy and finishedAt fields 2022-10-27 11:34:11 +02:00
Kerollmops
725158b454
Introduce the core algorithm of task cancelation 2022-10-27 11:34:11 +02:00
Kerollmops
b2c5bc67b7
Add more enum-iterator related stuff 2022-10-27 11:34:11 +02:00
Kerollmops
591527a99d
Prefer using TaskDeletion in the dumps 2022-10-27 11:34:11 +02:00
Kerollmops
1ca9a67c49
Introduce the task cancelation task type 2022-10-27 11:34:11 +02:00
Kerollmops
703ba7a1fb
Introduce the ProcessingTasks struct 2022-10-27 11:34:10 +02:00
Loïc Lecrenier
ea60d35c71
Delete a task's persisted data when appropriate 2022-10-27 11:34:10 +02:00
Tamo
9e85f050b2
fix the tests 2022-10-27 11:34:09 +02:00
Tamo
2f748480a1
share the rtxn between the access to the tasks and to the indexes 2022-10-27 11:34:09 +02:00
Tamo
6bd6321226
dump the content of the dump tasks instead of recreating at import time with wrong API keys 2022-10-27 11:34:08 +02:00
Tamo
655705eb2b
remove useless todo 2022-10-27 11:34:08 +02:00
Tamo
83f3c5ec57
flush the dump-writer only once everything has been inserted 2022-10-27 11:34:08 +02:00
Tamo
78ce29f461
apply most style comments of the review 2022-10-27 11:34:08 +02:00
Tamo
d0e91555d1
rebase on index-scheduler 2022-10-27 11:34:08 +02:00
Tamo
e0221fc0a3
fix a synchronization bug while importing tasks 2022-10-27 11:34:07 +02:00
Tamo
ba150f2127
commit after creating an index 2022-10-27 11:34:07 +02:00
Tamo
e9295c03ce
the index-scheduler needs to wake-up after importing a dump 2022-10-27 11:34:06 +02:00
Tamo
d481669b7e
fix the content_file import 2022-10-27 11:34:06 +02:00
Tamo
dd506e5d87
stop dumping the current dumping task as enqueued so it's not looping for ever 2022-10-27 11:34:06 +02:00
Tamo
d976e680c5
first mostly working version 2022-10-27 11:34:06 +02:00
Tamo
c051166bcc
update the API a little bit 2022-10-27 11:34:05 +02:00
Tamo
72a906ae75
fix the tests 2022-10-27 11:34:05 +02:00
Tamo
b7f9c94f4a
write the dump export 2022-10-27 11:34:05 +02:00
Loïc Lecrenier
8954b1bd1d
Fix number of deleted tasks details after duplicate task deletion 2022-10-27 11:34:05 +02:00
Loïc Lecrenier
8defad6c38
Add task deletion tests where the same task is deleted twice 2022-10-27 11:34:05 +02:00
Loïc Lecrenier
f32b973945
Return an error when calling DELETE /tasks with an empty query 2022-10-27 11:34:04 +02:00
Loïc Lecrenier
fbd2be2ec8
Apply suggested changes from PR review 2022-10-27 11:34:04 +02:00
Loïc Lecrenier
441417447e
Avoid creating two read txn at the same time 2022-10-27 11:34:04 +02:00
Loïc Lecrenier
8c6aeaada5
Update snapshot tests following git rebase that fixes a bug 2022-10-27 11:34:04 +02:00
Loïc Lecrenier
8bb0fcd144
Finish first draft of the DELETE /tasks route 2022-10-27 11:34:04 +02:00
Loïc Lecrenier
9522b75454
Continue implementation of task deletion 
1. Matched tasks are a roaring bitmap
2. Start implementation in meilisearch-http
3. Snapshots use meili-snap
4. Rename to TaskDeletion
 2022-10-27 11:34:03 +02:00
Kerollmops
e4d461ecba
Make sure that we do not batch tasks from different indexes 2022-10-27 11:34:03 +02:00
Kerollmops
b029369653
Add a test to check different indexes autobatching 2022-10-27 11:34:03 +02:00
Kerollmops
408d00136c
Extract index creation rights and simplify the autobatcher rules 2022-10-27 11:34:03 +02:00
Kerollmops
2c24c7d403
Fix invalid import of tasks types 2022-10-27 11:34:03 +02:00
Tamo
7034803712
move the API key in meilisearch_types 2022-10-27 11:34:02 +02:00