3071: Analytics on the tasks route r=Kerollmops a=irevoire
Implement the missing analytics on the delete and cancel task routes.
+ Batch the analytics on the `GET tasks` route to avoid flooding ourselves while polling meilisearch.
Co-authored-by: Tamo <tamo@meilisearch.com>
3077: Don't remove DB if unreadable r=irevoire a=dureuill
# Pull Request
## Related issue
Related to #3069
Will fix it after merging into `main`
## What does this PR do?
### User standpoint
- When the DB cannot be read after opening it, Meilisearch exits with an error message like previously, but it now leaves the DB untouched
- When the DB cannot be read after importing it from a snapshot or dump, it is still removed, like previously.
### Dev standpoint
- Add new local enum `OnFailure` that is used as a parameter to the `meilisearch_builder` closure to control when to keep or remove the DB in case of failure.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
3067: Fix task details serialization r=Kerollmops a=ManyTheFish
# Pull Request
- document addition task details always contain the field `indexedDocuments`
- value is set to `null` when the task is enqueued or processing
- value is set to `0` when the task is canceled or failed
- the field `deletedDocuments` of the document deletion task details is set to `0` when the task is canceled or failed
- the field `deletedDocuments` of the document clearAll task details is set to `0` when the task is canceled or failed
- the field `deletedTasks` of the task deletion task details is set to `0` when the task is canceled or failed
- the field `canceledTasks` of the task cancelation task details is set to `0` when the task is canceled or failed
## Related issue
Fixes#3057Fixes#3058
Co-authored-by: ManyTheFish <many@meilisearch.com>
3068: Prepend question mark to the `originalFilters` of the task deletion and cancelation r=irevoire a=Kerollmops
This pull request fixes#3064 by prepending [the HTTP query question mark](https://en.wikipedia.org/wiki/Question_mark#Computing) to the `originalFilters` of the task deletion and cancelation.
Co-authored-by: Kerollmops <clement@meilisearch.com>
3061: Name spawned threads r=irevoire a=dureuill
# Pull Request
## Related issue
None, this is to improve debuggability
## What does this PR do?
- This PR replaces the raw `thread::spawn(...)` calls by `thread::Builder::new().name(...).spawn(...).unwrap()` calls so that we can give meaningful names to threads.
- This PR also setup the `rayon` thread pool to give a name to its threads.
- This improves debuggability, as the thread names are reported by debuggers:
<img width="411" alt="Capture d’écran 2022-11-16 à 10 26 27" src="https://user-images.githubusercontent.com/41078892/202141870-a88663aa-d2f8-494f-b4da-709fdbd072ba.png">
(screen showing vscode's debugger and its main/scheduler/indexing threads)
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
3036: Bump milli to v0.35.1 r=irevoire a=Kerollmops
This PR bumps milli to v0.35.1 which brings some fixes. You can see [the changelog of milli on the release page](https://github.com/meilisearch/milli/releases/tag/v0.35.1).
Fixes#2905Fixes#3004Fixes#3000Fixes#3021Fixes#2945
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
3023: Update error codes related to tasks cancelation + add canceledBy filter r=Kerollmops a=Kerollmops
<details>This PR changes the error codes [to follow the specification](https://github.com/meilisearch/specifications/pull/195).
- [x] The `missing_filters` error code is renamed `missing_task_filters` to be more accurate and follow the `invalid_task_*` convention.
- [x] The error code `invalid_task_uids_filter` is added.
- [x] The error code `invalid_task_canceled_by_filter` is added.
- [x] The error code `invalid_task_date_filter` is added.
- The error message is the same as for expires_at in the API Key EXCEPT that it does not explicitly mention that a date must be given in the future.
</details>
Edit by `@loiclec` :
I have added a few more changes into this PR. The related issues are:
- Fixes https://github.com/meilisearch/meilisearch/issues/3029
- Implements https://github.com/meilisearch/meilisearch/issues/3026
- Fixes https://github.com/meilisearch/meilisearch/issues/2940
- Fixes https://github.com/meilisearch/meilisearch/issues/2939
Additionally:
- Fixes a bug where global tasks were returned by `GET /tasks` queries even if the user did not have the `index.*` API key action.
- Rename `originalQuery` to `originalFilters`
- Display `error: null` and `canceledBy: null` in the task views
- Allow using the star operator in the task filters in the `DELETE /tasks` and `POST /tasks/cancel` routes
- Make sure that the index scheduler keeps making progress even when a grave error occurs.
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
+ Allow star operator in delete/cancel tasks
+ rename originalQuery to originalFilters
+ Display error/canceled_by in task view even when they are = null
+ Rename task filter fields by using their plural forms
+ Prepare an error code for canceledBy filter
+ Only return global tasks if the API key action `index.*` is there
2982: Adapt task queries to account for special index swap rules r=irevoire a=loiclec
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/2970
## What does this PR do?
- Replace the `get_tasks` method with a `get_tasks_from_authorized_indexes` which returns the list of tasks matched by the query **from the point of view of the user**. That is, it takes into consideration the list of authorised indexes as well as the special case of `IndexSwap` which should not be returned if an index_uid is specified or if any of its associated indexes are not authorised.
- Adapt the code in other places following this change
- Add some tests
- Also the method `get_task_ids_from_authorized_indexes` now takes a read transaction as argument. This is because we want to make sure that the implementation of `get_tasks_from_authorized_indexes` only uses one read transaction. Otherwise, we could (1) get a list of task ids matching the query, then (2) one of these task ids is deleted by a taskDeletion task, and finally (3) we try to get the `Task`s associated with each returned task ids, and get a `CorruptedTaskQueue` error.
Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2993: Reconsider the Windows tests r=irevoire a=Kerollmops
This PR removes the `ignore` cfg on top of a lot of our tests. Now that we reworked the index scheduler we can make them pass again!
Fixes#2038, fixes#1966.
Co-authored-by: Clément Renault <clement@meilisearch.com>
2991: Update version for the next release (v0.30.0) in Cargo.toml files r=Kerollmops a=meili-bot
⚠️ This PR is automatically generated. Check the new version is the expected one before merging.
Co-authored-by: curquiza <curquiza@users.noreply.github.com>
