3085: refactorize the whole test suite r=irevoire a=irevoire
1. Make a call to assert_internally_consistent automatically when snapshotting the scheduler. There is no point in snapshotting something broken and expecting the dumb humans to notice.
2. Replace every possible call to assert_internally_consistent with a snapshot of the scheduler. It uses the same amount of lines and ensures we never change something without noticing in any tests ever.
3. Name every snapshot: it's easier to debug when something goes wrong and easier to review in general.
4. Stop skipping breakpoints; it's too easy to miss something. Now you must explicitly show which path the scheduler is supposed to use.
5. Add a timeout on the channel.recv, it eases the process of writing tests; now, when something files, you get a failure instead of a deadlock.
Co-authored-by: Irevoire <tamo@meilisearch.com>
3122: Display the `dumpUid` as `null` until we create it r=irevoire a=Kerollmops
This PR fixes#3117 by displaying the `DumpCreation` `dumpUid` details field as `null` until we compute the dump and the task is finished.
Co-authored-by: Kerollmops <clement@meilisearch.com>
1. Make a call to assert_internally_consistent automatically when snapshoting the scheduler. There is no point in snapshoting something broken and expect the dumb humans to notice.
2. Replace every possible call to assert_internally_consistent by a snapshot of the scheduler. It takes as many lines and ensure we never change something without noticing in any tests ever.
3. Name every snapshots: it's easier to debug when something goes wrong and easier to review in general.
4. Stop skipping breakpoints, it's too easy to miss something. Now you must explicitely show which path is the scheduler supposed to use.
5. Add a timeout on the channel.recv, it eases the process of writing tests, now when something file you get a failure instead of a deadlock.
3110: Always display `deletedDocuments` in the `IndexDeletion` details r=ManyTheFish a=Kerollmops
This PR fixes#3108 by always displaying a `deletedDocuments` details info.
Co-authored-by: Kerollmops <clement@meilisearch.com>
3106: Fix linking error when building binaries for aarch64 r=curquiza a=Kerollmops
This PR tries to fix#3094. Please don't look too close. It will be horrifying 😱
You can look at [the status of the fix in the CI](https://github.com/meilisearch/meilisearch/actions/runs/3523723323/jobs/5908229083).
Co-authored-by: Kerollmops <clement@meilisearch.com>
3105: Fix publish release CI r=Kerollmops a=curquiza
Fix CI to avoid release creation when triggering manually the CI with `workflow_dispatch` -> only trigger the upload of binary when the event is `release` instead of different of `schedule`
Co-authored-by: curquiza <clementine@meilisearch.com>
3099: Add a dispatch to the publish binaries workflow r=curquiza a=Kerollmops
This PR adds a dispatch to the publish binaries workflow to help us debug #3094.
Co-authored-by: Kerollmops <clement@meilisearch.com>
3096: Fix total memory computation r=Kerollmops a=dureuill
# Pull Request
## Related issue
Fixes#3018
## What does this PR do?
- Don't multiply the total memory value returned by sysinfo by 1024 anymore:
According to the [changelog of sysinfo 0.26.0](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md#0260), units are now in bytes and not KBs.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
3084: Bump the `milli` and `grenad` dependencies r=irevoire a=Kerollmops
This PR is bumping the milli direct and the grenad indirect dependencies.
Co-authored-by: Clément Renault <clement@meilisearch.com>
3083: Add `workflow_dispatch` to flaky.yml r=irevoire a=curquiza
Following this PR: bringing the change into `release-v0.30.0` as well, otherwise we cannot test
<img width="346" alt="Capture d’écran 2022-11-17 à 16 55 12" src="https://user-images.githubusercontent.com/20380692/202494559-6032665c-e336-4d4b-8a84-8be5e2371370.png">
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
3080: Rename `originalFilters` into `originalFilter` on the cancelation and deletion routes r=irevoire a=Kerollmops
This PR fixes https://github.com/meilisearch/meilisearch/issues/3079.
Co-authored-by: Clément Renault <clement@meilisearch.com>
3071: Analytics on the tasks route r=Kerollmops a=irevoire
Implement the missing analytics on the delete and cancel task routes.
+ Batch the analytics on the `GET tasks` route to avoid flooding ourselves while polling meilisearch.
Co-authored-by: Tamo <tamo@meilisearch.com>
3077: Don't remove DB if unreadable r=irevoire a=dureuill
# Pull Request
## Related issue
Related to #3069
Will fix it after merging into `main`
## What does this PR do?
### User standpoint
- When the DB cannot be read after opening it, Meilisearch exits with an error message like previously, but it now leaves the DB untouched
- When the DB cannot be read after importing it from a snapshot or dump, it is still removed, like previously.
### Dev standpoint
- Add new local enum `OnFailure` that is used as a parameter to the `meilisearch_builder` closure to control when to keep or remove the DB in case of failure.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
3067: Fix task details serialization r=Kerollmops a=ManyTheFish
# Pull Request
- document addition task details always contain the field `indexedDocuments`
- value is set to `null` when the task is enqueued or processing
- value is set to `0` when the task is canceled or failed
- the field `deletedDocuments` of the document deletion task details is set to `0` when the task is canceled or failed
- the field `deletedDocuments` of the document clearAll task details is set to `0` when the task is canceled or failed
- the field `deletedTasks` of the task deletion task details is set to `0` when the task is canceled or failed
- the field `canceledTasks` of the task cancelation task details is set to `0` when the task is canceled or failed
## Related issue
Fixes#3057Fixes#3058
Co-authored-by: ManyTheFish <many@meilisearch.com>
3068: Prepend question mark to the `originalFilters` of the task deletion and cancelation r=irevoire a=Kerollmops
This pull request fixes#3064 by prepending [the HTTP query question mark](https://en.wikipedia.org/wiki/Question_mark#Computing) to the `originalFilters` of the task deletion and cancelation.
Co-authored-by: Kerollmops <clement@meilisearch.com>
3061: Name spawned threads r=irevoire a=dureuill
# Pull Request
## Related issue
None, this is to improve debuggability
## What does this PR do?
- This PR replaces the raw `thread::spawn(...)` calls by `thread::Builder::new().name(...).spawn(...).unwrap()` calls so that we can give meaningful names to threads.
- This PR also setup the `rayon` thread pool to give a name to its threads.
- This improves debuggability, as the thread names are reported by debuggers:
<img width="411" alt="Capture d’écran 2022-11-16 à 10 26 27" src="https://user-images.githubusercontent.com/41078892/202141870-a88663aa-d2f8-494f-b4da-709fdbd072ba.png">
(screen showing vscode's debugger and its main/scheduler/indexing threads)
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
