Commit Graph

4536 Commits

Author SHA1 Message Date
Tamo
b44c381c2a
Store analytics for the documents deletions 2022-11-15 19:08:45 +01:00
bors[bot]
51be75a264
Merge #3056
3056: refactor the way we send the cli informations + add the analytics for the config file and ssl usage r=Kerollmops a=irevoire

Partially fix #2955

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-11-15 17:03:30 +00:00
Tamo
4953b62712
reformat, sorry @kero 2022-11-15 17:57:27 +01:00
Tamo
9473cccc27
add a comment over the new infos structure 2022-11-15 17:56:07 +01:00
Tamo
9327db3e91
Apply suggestions from code review
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-11-15 17:53:23 +01:00
Tamo
0fced6f270
refactor the way we send the cli informations + add the analytics for the config file and ssl usage 2022-11-15 16:32:31 +01:00
Tamo
b4434dcad2
rename the error codes for the sake of consistency 2022-11-14 23:27:02 +01:00
Tamo
d08d97bf43
fix the error messages and add tests 2022-11-14 23:27:02 +01:00
bors[bot]
a8991ccb64
Merge #3036
3036: Bump milli to v0.35.1 r=irevoire a=Kerollmops

This PR bumps milli to v0.35.1 which brings some fixes. You can see [the changelog of milli on the release page](https://github.com/meilisearch/milli/releases/tag/v0.35.1).

Fixes #2905
Fixes #3004
Fixes #3000
Fixes #3021
Fixes #2945

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-11-10 14:25:54 +00:00
Kerollmops
761bd3aca4
Fix the new error messages 2022-11-10 12:04:25 +01:00
Clément Renault
26ab6ab0cc
Bump milli version to 0.35.1 2022-11-10 12:04:25 +01:00
bors[bot]
379522ace3
Merge #3023
3023: Update error codes related to tasks cancelation + add canceledBy filter r=Kerollmops a=Kerollmops

<details>This PR changes the error codes [to follow the specification](https://github.com/meilisearch/specifications/pull/195).

 - [x] The `missing_filters` error code is renamed `missing_task_filters` to be more accurate and follow the `invalid_task_*` convention.
 - [x] The error code `invalid_task_uids_filter` is added.
 - [x] The error code `invalid_task_canceled_by_filter` is added.
 - [x] The error code `invalid_task_date_filter` is added.
      -  The error message is the same as for expires_at in the API Key  EXCEPT that it does not explicitly mention that a date must be given in the future.
</details>

Edit by `@loiclec` :
I have added a few more changes into this PR. The related issues are:

- Fixes https://github.com/meilisearch/meilisearch/issues/3029
- Implements https://github.com/meilisearch/meilisearch/issues/3026
- Fixes https://github.com/meilisearch/meilisearch/issues/2940
- Fixes https://github.com/meilisearch/meilisearch/issues/2939

Additionally:
- Fixes a bug where global tasks were returned by `GET /tasks` queries even if the user did not have the `index.*` API key action.
- Rename `originalQuery` to `originalFilters`
- Display `error: null` and `canceledBy: null` in the task views
- Allow using the star operator in the task filters in the `DELETE /tasks` and `POST /tasks/cancel` routes
- Make sure that the index scheduler keeps making progress even when a grave error occurs.


Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-11-10 10:51:41 +00:00
bors[bot]
1d5f17a9ea
Merge #3032
3032: Fix Index name parsing error message to fit the specification r=Kerollmops a=ManyTheFish

fixes #2924


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-11-08 16:01:53 +00:00
ManyTheFish
8bb260bf3e Fix Index name parsing error message to fit the specification 2022-11-08 16:35:58 +01:00
Loïc Lecrenier
52b38bee9d Make rustfmt happy
>:-(
2022-11-08 15:45:53 +01:00
Loïc Lecrenier
f5454dfa60 Make clippy happy
They're a happy clip now.
2022-11-08 15:31:08 +01:00
Loïc Lecrenier
1e464e87fc Update more insta-snap tests 2022-11-08 13:39:52 +01:00
Loïc Lecrenier
6126fc8d98 Rename original_query to original_filters everywhere 2022-11-08 13:18:18 +01:00
Loïc Lecrenier
2fdd814e57 Update tests following task API changes 2022-11-08 13:18:18 +01:00
Loïc Lecrenier
20fa103992 Add canceledBy task filter 2022-11-08 13:18:18 +01:00
Loïc Lecrenier
d5638d2c27 Use more precise error codes/message for the task routes
+ Allow star operator in delete/cancel tasks
+ rename originalQuery to originalFilters
+ Display error/canceled_by in task view even when they are = null
+ Rename task filter fields by using their plural forms
+ Prepare an error code for canceledBy filter
+ Only return global tasks if the API key action `index.*` is there
2022-11-08 13:18:17 +01:00
Clément Renault
932414bf72 WIP Introduce the invalid_task_uid error code 2022-11-08 13:17:56 +01:00
Kerollmops
b20025c01e Change the missing_filters error code into missing_task_filters 2022-11-08 13:17:29 +01:00
bors[bot]
3999f74f78
Merge #3022
3022: Store the `started_at` for a task that is canceled when processing r=irevoire a=Kerollmops

This PR changes the current behavior of the displayed tasks. When a processing task is canceled, the `started_at` date time is kept and displayed to the user. Otherwise, if the task was just enqueued, the `started_at` remains `null`. If a task is processing, the engine is ctrl-c, starts again, and the task becomes enqueued again, so if it is canceled, its `started_at` will be `null`.

You can read more [in this discussion](https://github.com/meilisearch/specifications/pull/195/files#r1009602335).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-11-03 20:48:24 +00:00
Kerollmops
739b9f5505
Use the content of the ProcessingTasks in the tasks cancelation system 2022-11-03 11:09:59 +01:00
bors[bot]
722a0da0c3
Merge #3019
3019: Fix error code of the "duplicate index found" error on the index swap route r=irevoire a=loiclec

According to the spec, the code of the error should be `duplicate_index_found`, but it was `bad_request` instead.

Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-11-03 09:49:55 +00:00
Loïc Lecrenier
5704a1895d Fix error code of the "duplicate index found" error 2022-11-02 09:34:50 +01:00
bors[bot]
2254bbf3bd
Merge #3002
3002: Fix dump import without instance uid r=Kerollmops a=irevoire

When creating a dump without any instance-uid (that can happen if you’ve always run meilisearch with the `--no-analytics` flag), you could get an error when trying to load the dump.


Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-10-31 12:58:37 +00:00
Irevoire
510afda590
remove unused import 2022-10-30 20:05:20 +01:00
Irevoire
fea9fdcd7e
fix the dump reader process when no instance-uid was specified 2022-10-30 20:00:27 +01:00
bors[bot]
dd1011ba76
Merge #2995
2995: merge the settings and do one indexation at the end r=irevoire a=irevoire



Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-10-27 21:24:21 +00:00
bors[bot]
20258461a8
Merge #2981 #2996
2981: Move index swap error handling from meilisearch-http to index-scheduler r=irevoire a=loiclec

And make index_not_found error asynchronous, since we can't know whether the index will exist by the time the index swap task is processed.

Improve the index-swap test to verify that future tasks are not swapped and to test the new error messages that were introduced.

## Related issue
https://github.com/meilisearch/meilisearch/issues/2973


2996: Get rids of the unecessary tasks when an index_uid is specified r=Kerollmops a=irevoire



Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-10-27 19:11:23 +00:00
Tamo
87cac158c4
Update index-scheduler/src/batch.rs 2022-10-27 18:08:21 +02:00
Tamo
c9f89d38e3
Merge branch 'main' into index-swap-error-handling 2022-10-27 18:06:45 +02:00
Irevoire
01687c87a2
Get rids of the unecessary tasks when an index_uid is specified 2022-10-27 18:00:04 +02:00
Irevoire
313f204f39
merge the settings and do one indexation at the end 2022-10-27 16:38:21 +02:00
bors[bot]
d16ea755d8
Merge #2982
2982: Adapt task queries to account for special index swap rules r=irevoire a=loiclec

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/2970 

## What does this PR do?
- Replace the `get_tasks` method with a `get_tasks_from_authorized_indexes` which returns the list of tasks matched by the query **from the point of view of the user**. That is, it takes into consideration the list of authorised indexes as well as the special case of `IndexSwap` which should not be returned if an index_uid is specified or if any of its associated indexes are not authorised.
- Adapt the code in other places following this change
- Add some tests
- Also the method `get_task_ids_from_authorized_indexes` now takes a read transaction as argument. This is because we want to make sure that the implementation of `get_tasks_from_authorized_indexes` only uses one read transaction. Otherwise, we could (1) get a list of task ids matching the query, then (2) one of these task ids is deleted by a taskDeletion task, and finally (3) we try to get the `Task`s associated with each returned task ids, and get a `CorruptedTaskQueue` error.



Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-10-27 14:28:04 +00:00
Loïc Lecrenier
8152ab5dfc Revert change in initialisation of TempDir for index scheduler tests 2022-10-27 16:26:17 +02:00
Loïc Lecrenier
8dd7942656 Cargo fmt 2022-10-27 16:24:09 +02:00
Loïc Lecrenier
2c31d7c50a Apply review suggestions 2022-10-27 16:24:08 +02:00
bors[bot]
b76f0ace26
Merge #2993
2993: Reconsider the Windows tests r=irevoire a=Kerollmops

This PR removes the `ignore` cfg on top of a lot of our tests. Now that we reworked the index scheduler we can make them pass again!

Fixes #2038, fixes #1966.

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-10-27 13:41:04 +00:00
bors[bot]
5b535a82ea
Merge #2991
2991: Update version for the next release (v0.30.0) in Cargo.toml files r=Kerollmops a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one before merging.

Co-authored-by: curquiza <curquiza@users.noreply.github.com>
2022-10-27 13:12:31 +00:00
Clément Renault
e67673bd12
Ingore the dumps v1 test on Windows 2022-10-27 14:34:45 +02:00
Clément Renault
44d6f3e7a0
Reconsider the Windows tests 2022-10-27 13:50:05 +02:00
curquiza
68f80dbacf Update version for the next release (v0.30.0) in Cargo.toml files 2022-10-27 11:35:44 +00:00
bors[bot]
08db774699
Merge #2990
2990: isolate the search in another task r=Kerollmops a=irevoire

In case there is a failure on milli's side that should avoid blocking the tokio main thread


Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-10-27 11:29:22 +00:00
Irevoire
4d9e9f4a9d
isolate the search in another task
In case there is a failure on milli's side that should avoid blocking the tokio main thread
2022-10-27 13:12:42 +02:00
Loïc Lecrenier
4f4fc20acf Make clippy happy 2022-10-27 13:00:30 +02:00
Loïc Lecrenier
78ffa00f98 Move index swap error handling from meilisearch-http to index-scheduler
And make index_not_found error asynchronous, since we can't know
whether the index will exist by the time the index swap task is
processed.

Improve the index-swap test to verify that future tasks are not swapped
and to test the new error messages that were introduced.
2022-10-27 11:45:38 +02:00
Loïc Lecrenier
7b93ba40bd Reimplement task queries to account for special index swap rules 2022-10-27 11:44:51 +02:00