Loïc Lecrenier
20fa103992
Add canceledBy task filter
2022-11-08 13:18:18 +01:00
Loïc Lecrenier
d5638d2c27
Use more precise error codes/message for the task routes
...
+ Allow star operator in delete/cancel tasks
+ rename originalQuery to originalFilters
+ Display error/canceled_by in task view even when they are = null
+ Rename task filter fields by using their plural forms
+ Prepare an error code for canceledBy filter
+ Only return global tasks if the API key action `index.*` is there
2022-11-08 13:18:17 +01:00
Clément Renault
932414bf72
WIP Introduce the invalid_task_uid error code
2022-11-08 13:17:56 +01:00
Tamo
c9f89d38e3
Merge branch 'main' into index-swap-error-handling
2022-10-27 18:06:45 +02:00
bors[bot]
d16ea755d8
Merge #2982
...
2982: Adapt task queries to account for special index swap rules r=irevoire a=loiclec
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/2970
## What does this PR do?
- Replace the `get_tasks` method with a `get_tasks_from_authorized_indexes` which returns the list of tasks matched by the query **from the point of view of the user**. That is, it takes into consideration the list of authorised indexes as well as the special case of `IndexSwap` which should not be returned if an index_uid is specified or if any of its associated indexes are not authorised.
- Adapt the code in other places following this change
- Add some tests
- Also the method `get_task_ids_from_authorized_indexes` now takes a read transaction as argument. This is because we want to make sure that the implementation of `get_tasks_from_authorized_indexes` only uses one read transaction. Otherwise, we could (1) get a list of task ids matching the query, then (2) one of these task ids is deleted by a taskDeletion task, and finally (3) we try to get the `Task`s associated with each returned task ids, and get a `CorruptedTaskQueue` error.
Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-10-27 14:28:04 +00:00
Loïc Lecrenier
8dd7942656
Cargo fmt
2022-10-27 16:24:09 +02:00
bors[bot]
b76f0ace26
Merge #2993
...
2993: Reconsider the Windows tests r=irevoire a=Kerollmops
This PR removes the `ignore` cfg on top of a lot of our tests. Now that we reworked the index scheduler we can make them pass again!
Fixes #2038 , fixes #1966 .
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-10-27 13:41:04 +00:00
bors[bot]
5b535a82ea
Merge #2991
...
2991: Update version for the next release (v0.30.0) in Cargo.toml files r=Kerollmops a=meili-bot
⚠️ This PR is automatically generated. Check the new version is the expected one before merging.
Co-authored-by: curquiza <curquiza@users.noreply.github.com>
2022-10-27 13:12:31 +00:00
Clément Renault
e67673bd12
Ingore the dumps v1 test on Windows
2022-10-27 14:34:45 +02:00
Clément Renault
44d6f3e7a0
Reconsider the Windows tests
2022-10-27 13:50:05 +02:00
curquiza
68f80dbacf
Update version for the next release (v0.30.0) in Cargo.toml files
2022-10-27 11:35:44 +00:00
Irevoire
4d9e9f4a9d
isolate the search in another task
...
In case there is a failure on milli's side that should avoid blocking the tokio main thread
2022-10-27 13:12:42 +02:00
Loïc Lecrenier
4f4fc20acf
Make clippy happy
2022-10-27 13:00:30 +02:00
Loïc Lecrenier
78ffa00f98
Move index swap error handling from meilisearch-http to index-scheduler
...
And make index_not_found error asynchronous, since we can't know
whether the index will exist by the time the index swap task is
processed.
Improve the index-swap test to verify that future tasks are not swapped
and to test the new error messages that were introduced.
2022-10-27 11:45:38 +02:00
Loïc Lecrenier
7b93ba40bd
Reimplement task queries to account for special index swap rules
2022-10-27 11:44:51 +02:00
Clément Renault
fae17ed590
Enable the authentication tests on Windows again
2022-10-27 11:35:24 +02:00
Clément Renault
7e355958e0
Await the last insert task
2022-10-27 11:35:24 +02:00
Irevoire
8bc602a7dd
makes clippy happy
2022-10-27 11:35:23 +02:00
Irevoire
6c2ecec4d0
fix the return of the task cancelation and task deletion
2022-10-27 11:35:23 +02:00
Irevoire
6280bd51a9
actually fix the test and the swap_indexes name resolution
2022-10-27 11:35:23 +02:00
Irevoire
54d0aff4cf
ignore a strange test that works on my machine but not on the ci
2022-10-27 11:35:23 +02:00
Irevoire
b804cba4ca
try to debug the ci
2022-10-27 11:35:23 +02:00
Irevoire
3cf8aaa4d0
reformat
2022-10-27 11:35:23 +02:00
Kerollmops
0dd8e00929
Reapply #2601
2022-10-27 11:35:22 +02:00
Irevoire
a99ddf85f7
fix clippy once again
2022-10-27 11:35:22 +02:00
Irevoire
3979c9f02b
fix all the dump snasphots
2022-10-27 11:35:20 +02:00
Irevoire
8ec3681cf8
fix clippy part1
2022-10-27 11:35:20 +02:00
Kerollmops
861a07792e
Remove useless task module
2022-10-27 11:35:20 +02:00
Kerollmops
ee6597da60
Fix all the tests
2022-10-27 11:35:20 +02:00
Kerollmops
314b89ca30
Fix insta snapshots
2022-10-27 11:35:20 +02:00
Loïc Lecrenier
9b43528bbb
Update test after fixing bug in index swap
2022-10-27 11:35:18 +02:00
Loïc Lecrenier
e641d08846
Cargo fmt
2022-10-27 11:35:18 +02:00
Loïc Lecrenier
36c9f05998
Revert "Display more than one indexUid in a task view if necessary"
...
This reverts commit 1f2e253bb6
2022-10-27 11:35:18 +02:00
Loïc Lecrenier
3b158bb966
Return invalid API key error in /swap-indexes
2022-10-27 11:35:18 +02:00
Loïc Lecrenier
08b5123380
Display more than one indexUid in a task view if necessary
2022-10-27 11:35:17 +02:00
Loïc Lecrenier
1f75caae88
Fix a few index swap bugs.
...
1. Details of the indexSwap task
2. Query tasks with type=indexUid
3. Synchronous error message for multiple index not found
2022-10-27 11:35:17 +02:00
Irevoire
a16604af80
fix all the tests
2022-10-27 11:35:17 +02:00
Irevoire
1d014a538e
comment out a test that makes the CI crash
2022-10-27 11:35:17 +02:00
Kerollmops
2de8a0711a
Cargo insta test/review
2022-10-27 11:35:16 +02:00
Kerollmops
2f577b6fcd
Patch the IndexScheduler in meilisearch-http to use the options struct
2022-10-27 11:35:16 +02:00
Tamo
eccbdb74cf
remove useless print
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-10-27 11:35:16 +02:00
Irevoire
033794d209
add tests for the task deletion and task cancelation
2022-10-27 11:35:16 +02:00
Irevoire
a85d5b4981
test the details of all tasks type
2022-10-27 11:35:16 +02:00
Kerollmops
4736e00253
Handle the CLI options related to snapshots
2022-10-27 11:35:15 +02:00
Kerollmops
942b7c338b
Compress the snapshot in a tarball
2022-10-27 11:35:15 +02:00
Kerollmops
4cafc63561
Reintroduce the versioning functions
2022-10-27 11:35:14 +02:00
Kerollmops
89e127e4f4
Declare the auth path in the index scheduler
2022-10-27 11:35:14 +02:00
Kerollmops
c063f154fb
Add the snapshots directory path to the IndexScheduler
2022-10-27 11:35:14 +02:00
Kerollmops
c641888a23
Patch the delete and cancel tasks routes
2022-10-27 11:35:13 +02:00
Irevoire
12669bf07c
rename received_documents_ids to matched_documents
2022-10-27 11:35:12 +02:00
Irevoire
0aca5e84b9
rename received_document_ids to matched_documents in the DocumentDeletion task type (reimplementation of #2826 )
2022-10-27 11:35:12 +02:00
Irevoire
7e52f1effb
remove a lot of unecessary clone and ref
2022-10-27 11:35:11 +02:00
Irevoire
ecf4e43b3d
rename the dumpExport to dumpCreation
2022-10-27 11:35:10 +02:00
Irevoire
3ea489421e
move the error types to meilisearch-http
2022-10-27 11:35:10 +02:00
Loïc Lecrenier
2808be9d45
Fix the /swap-indexes route API
...
1. payload
2. error messages
3. auth errors
2022-10-27 11:35:10 +02:00
Irevoire
8a23e707c1
fix the task view and forward the task db size
2022-10-27 11:35:09 +02:00
Irevoire
735a5da257
reformat
2022-10-27 11:35:09 +02:00
Irevoire
e9055f5572
fix clippy
2022-10-27 11:35:08 +02:00
Irevoire
2c0fde4a1a
ignore the snapshot test
2022-10-27 11:35:08 +02:00
Irevoire
c8ee453b6c
fix the autobatched document deletion
2022-10-27 11:35:07 +02:00
Irevoire
f6963f9662
ensure the indexUid is valid in most cases
2022-10-27 11:35:07 +02:00
Irevoire
b3265a8e1f
ensure the index_uid is valid when creating an index
2022-10-27 11:35:07 +02:00
Irevoire
cb48a02f94
fix the invalid index uid errors
2022-10-27 11:35:06 +02:00
Irevoire
99144b1419
fix most content file error
2022-10-27 11:35:06 +02:00
Irevoire
e107f1b282
fix the payload too large error
2022-10-27 11:35:06 +02:00
Irevoire
1bef5d119d
fix the api keys for the tasks route
2022-10-27 11:35:06 +02:00
Irevoire
ca4234b445
fix the deletion of the data.ms in case of failure
2022-10-27 11:35:06 +02:00
Irevoire
8d1408c65e
fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting
2022-10-27 11:35:05 +02:00
Irevoire
131fe30934
fix the error messages and the index stats
2022-10-27 11:35:05 +02:00
Clément Renault
32cfac0cfd
Sort the TOML dependencies
2022-10-27 11:35:05 +02:00
Clément Renault
80b2e70ee7
Introduce a rustfmt file
2022-10-27 11:35:05 +02:00
Clément Renault
52e858a588
Reapply #2890
2022-10-27 11:34:18 +02:00
Clément Renault
8b0427f0c4
Reapply #2839
2022-10-27 11:34:18 +02:00
Clément Renault
788262e588
Fix final compilation
2022-10-27 11:34:17 +02:00
Clément Renault
61edcd585a
Fix the new config file with the index scheduler
2022-10-27 11:34:17 +02:00
Loïc Lecrenier
d20b5ddda0
Don't return an error when swapping 0 indexes
2022-10-27 11:34:16 +02:00
Loïc Lecrenier
17cd2a4aa0
Implement POST /indexes-swap
2022-10-27 11:34:15 +02:00
Loïc Lecrenier
169f386418
Add some documentation to the index scheduler
2022-10-27 11:34:15 +02:00
Loïc Lecrenier
ec3391808d
Fix date parsing for task queries
...
Use rfc3339 or YYYY-MM-DD.
Add a day to the parsed date when it is an excluded lower bound
and the YYYY-MM-DD was used.
Also the Query type does not need to be serialisable anymore
2022-10-27 11:34:14 +02:00
Loïc Lecrenier
10a547df4f
Apply suggestions from code review
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
Apply suggestions from code review
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Apply suggestions from code review
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Apply code review suggestion
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-10-27 11:34:14 +02:00
Loïc Lecrenier
22cf0559fe
Implement task date filters
...
before/after enqueued/started/finished at
2022-10-27 11:34:14 +02:00
Kerollmops
3cbfacb616
Prefer using an u64 instead of a usize in some places
2022-10-27 11:34:12 +02:00
Kerollmops
f9c8fe5eaa
Use a tokio block_in_place method for potentially blocking tasks
2022-10-27 11:34:12 +02:00
Kerollmops
c2ec4a089b
Put the original URL query in the tasks details
2022-10-27 11:34:12 +02:00
Kerollmops
751e9bac3b
Add the tasks cancel route to cancel tasks
2022-10-27 11:34:11 +02:00
Kerollmops
290945e258
Update the canceledBy and finishedAt fields
2022-10-27 11:34:11 +02:00
Kerollmops
725158b454
Introduce the core algorithm of task cancelation
2022-10-27 11:34:11 +02:00
Kerollmops
1ca9a67c49
Introduce the task cancelation task type
2022-10-27 11:34:11 +02:00
Tamo
f7e546eea3
make the tests compile again
2022-10-27 11:34:10 +02:00
Tamo
b45c430165
fix the analytics
2022-10-27 11:34:10 +02:00
Tamo
634eb52926
extract the create_app function for the tests
2022-10-27 11:34:09 +02:00
Tamo
bea81ae37b
fix meilisearch-http
2022-10-27 11:34:09 +02:00
Tamo
6bd6321226
dump the content of the dump tasks instead of recreating at import time with wrong API keys
2022-10-27 11:34:08 +02:00
Tamo
9fe24fbff2
get rids of the useless Seek before creating a grenad reader
2022-10-27 11:34:08 +02:00
Tamo
78ce29f461
apply most style comments of the review
2022-10-27 11:34:08 +02:00
Tamo
e0221fc0a3
fix a synchronization bug while importing tasks
2022-10-27 11:34:07 +02:00
Tamo
554600dfd8
fix the deletion of the data.ms in case of errors
2022-10-27 11:34:07 +02:00
Tamo
d976e680c5
first mostly working version
2022-10-27 11:34:06 +02:00
Tamo
b7f9c94f4a
write the dump export
2022-10-27 11:34:05 +02:00
Loïc Lecrenier
f32b973945
Return an error when calling DELETE /tasks with an empty query
2022-10-27 11:34:04 +02:00
