Move `meilisearch_error` to `meilisearch_types::error`
Move `meilisearch_lib::index_resolver::IndexUid` to `meilisearch_types::index_uid`
Add a new `InvalidIndexUid` error in `meilisearch_types::index_uid`
2494: Introduce the new faceting and pagination settings r=ManyTheFish a=Kerollmops
This PR introduces two new settings following the newly created spec https://github.com/meilisearch/specifications/pull/157:
- The `faceting.max_values_per_facet` one describes the maximum number of values (each with a count) associated with a value in a facet distribution query.
- The `pagination.limited_to` one describes the maximum number of documents that a search query can ever return.
Co-authored-by: Kerollmops <clement@meilisearch.com>
2460: Create custom error types for `TaskType`, `TaskStatus`, and `IndexUid` r=Kerollmops a=walterbm
# Pull Request
## What does this PR do?
Fixes#2443 by making the following changes:
- Add custom `TaskTypeError` for `TaskType::from_str`
- Add custom `TaskStatusError` for `TaskStatus::from_str`
- Add custom `IndexUidFormatError` for `IndexUid::from_str`
- Implement `From<IndexUidFormatError> for IndexResolverError` to convert between errors
- Replace all usages of `IndexUid::new` with `IndexUid::from_str`
- **NOTE** I am relatively new to Rust and I struggled a lot with this final part. This PR ended up with a messy error conversion which does not seem ideal. Please let me know if you have any suggestions for how to make this better and I'll be happy to make any updates!
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: walter <walter.beller.morales@gmail.com>
2471: Remove the connection keep-alive timeout r=MarinPostma a=Thearas
# Pull Request
## What does this PR do?
Fixes <https://github.com/meilisearch/meilisearch-go/issues/221>.
Meilisearch has a default connection keep-alive timeout for 5s, which means it will close the connections with idle time >= 5s.
This PR set actix-web keep-alive config to `KeepAlive::Os`, let the client and system to decide when to close the connection.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Co-authored-by: Thearas <thearas850@gmail.com>
2454: Unify the pagination of the index and documents route behind a common type r=curquiza a=irevoire
`@MarinPostma` wdyt of keeping the `auto_paginate_sized` until we implement the pagination on every route that needs it just to see if it could be useful to something else
Co-authored-by: Tamo <tamo@meilisearch.com>
2445: Seek-based tasks list r=Kerollmops a=Kerollmops
This PR implements the seek-based pagination for the tasks list following [the spec](https://github.com/meilisearch/specifications/pull/115).
Co-authored-by: Kerollmops <clement@meilisearch.com>
2446: rename Succeded to Succeeded r=irevoire a=MarinPostma
this pr renames `TaskEvent::Succeded` to `TaskEvent::Succeeded` and apply the migration to the dumps
Co-authored-by: ad hoc <postma.marin@protonmail.com>
- Return Documents API resources on `/documents` in an array in the the results field.
- Add limit, offset and total in the response body.
- Rename `attributesToRetrieve` into `fields` (only for the `/documents` endpoints, not for the `/search` ones).
- The `displayedAttributes` settings does not impact anymore the displayed fields returned in the `/documents` endpoints. These settings only impacts the `/search` endpoint.
Fix#2372
2410: Make dump a task r=Kerollmops a=MarinPostma
This PR transforms the dump task into a proper task.
The `GET /dumps/:dump_uid` is removed.
Some changes were made to make this work, and a bit a refactoring was necessary.
- The `dump_actor` module has been renamed do `dumps` and moved to the root
- There isn't a `DumpActor` anymore, and the dump process is handled by the `DumpHandler`.
- The `TaskPerformer` is renamed to `BatchHandler`
- The `BatchHandler` trait no longer has a `perform_job` method, but instead has a `accept` method returning whether a handler can proccess a batch
- The scheduler now accept a list of `BatchHandler`, and iterates trhough them until it finds one to accept the current batch.
- `Job` doesn't exist anymore, and everything in now inside of the `BatchContent` enum.
- The `Vec<TaskId>` from `Batch` is replaced with a `BatchContent` enum which hints at the content.
- The Scheduler is slightly modified to accept batch, and prioritize them before regular tasks.
- The `TaskList` are not identified by a `String` representing the index uid anymore, but by a `TaskListIdentifier` which also works for dumps which are not targeting any specific indexes.
- The `GET /dump/:dump_id` no longer exists
- `DumpActorError` is renamed to `DumpError`
close#2410
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2316: Add version flag r=Kerollmops a=sanders41
# Pull Request
## What does this PR do?
Fixes#2315
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Paul Sanders <psanders1@gmail.com>
2297: Feat(Search): Enhance formating search results r=ManyTheFish a=ManyTheFish
Add new settings and change crop_len behavior to count words instead of characters.
- [x] `highlightPreTag`
- [x] `highlightPostTag`
- [x] `cropMarker`
- [x] `cropLength` count word instead of chars
- [x] `cropLength` 0 is now considered as no `cropLength`
- [ ] ~smart crop finding the best matches interval~ (postponed)
Partially fixes #2214. (no smart crop)
Co-authored-by: ManyTheFish <many@meilisearch.com>
2277: fix(http): fix panic when sending document update without content type header r=MarinPostma a=MarinPostma
I found a panic when pushing documents without a content-type. This fixes is by returning unknown instead of crashing.
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2253: refactor authentication key extraction r=ManyTheFish a=MarinPostma
I am concerned that the part of the code that performs the key prefix extraction from the jwt token migh be misused in the future. Since this is a critical part of the code, I moved it into it's own function. Since we deserialized the payload twice anyway, I reordered the verifications, and we now use the data from the validated token.
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2245: Add test to validate cli r=irevoire a=MarinPostma
followup on #2242 and #2243
Add a test to make sure the cli is valid, and add a CI task to run the tests in debug to make sure we hit debug assertions.
FYI `@curquiza,` because of CI changes
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2098: feat(dump): Provide the same cli options as the snapshots r=MarinPostma a=irevoire
Add two cli options for the dump:
- `--ignore-missing-dump`
- `--ignore-dump-if-db-exists`
Fix#2087
Co-authored-by: Tamo <tamo@meilisearch.com>
2099: feat(analytics): Set the timestamp of the aggregated event as the first aggregate r=MarinPostma a=irevoire
2108: meta(auth): Enhance tests on authorization r=MarinPostma a=ManyTheFish
Enhance auth tests in order to be able to add new actions without changing tests.
Helping #2080
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2068: chore(http): migrate from structopt to clap3 r=Kerollmops a=MarinPostma
migrate from structopt to clap3
This fix the long lasting issue with flags require a value, such as `--no-analytics` or `--schedule-snapshot`.
All flag arguments now take NO argument, i.e:
`meilisearch --schedule-snapshot true` becomes `meilisearch --schedule-snapshot`
as per https://docs.rs/clap/latest/clap/struct.Arg.html#method.env, the env variable is defines as:
> A false literal is n, no, f, false, off or 0. An absent environment variable will also be considered as false. Anything else will considered as true.
`@gmourier`
`@curquiza`
`@meilisearch/docs-team`
Co-authored-by: mpostma <postma.marin@protonmail.com>
2066: bug(http): fix task duration r=MarinPostma a=MarinPostma
`@gmourier` found that the duration in the task view was not computed correctly, this pr fixes it.
`@curquiza,` I let you decide if we need to make a hotfix out of this or wait for the next release. This is not breaking.
Co-authored-by: mpostma <postma.marin@protonmail.com>
1989: Extend API keys r=curquiza a=ManyTheFish
# Pull Request
## What does this PR do?
- Add API keys in snapshots
- Add API keys in dumps
- fix QA #1979fix#1979fix#1995fix#2001fix#2003
related to #1890
Co-authored-by: many <maxime@meilisearch.com>
- Add API keys in snapshots
- Add API keys in dumps
- Rename action indexes.add to indexes.create
- fix QA #1979fix#1979fix#1995fix#2001fix#2003
related to #1890
implements:
https://github.com/meilisearch/specifications/blob/develop/text/0085-api-keys.md
- Add tests on API keys management route (meilisearch-http/tests/auth/api_keys.rs)
- Add tests checking authorizations on each meilisearch routes (meilisearch-http/tests/auth/authorization.rs)
- Implement API keys management routes (meilisearch-http/src/routes/api_key.rs)
- Create module to manage API keys and authorizations (meilisearch-auth)
- Reimplement GuardedData to extend authorizations (meilisearch-http/src/extractors/authentication/mod.rs)
- Change X-MEILI-API-KEY by Authorization Bearer (meilisearch-http/src/extractors/authentication/mod.rs)
- Change meilisearch routes to fit to the new authorization feature (meilisearch-http/src/routes/)
- close#1867
1875: Fix search post event and disk size analytics r=irevoire a=gmourier
- Branch POST search on the post_search aggregator
- Use largest disk `total_space` instead of `available_space`
1876: Update SEGMENT_API_KEY r=irevoire a=gmourier
Branch it on our Segment production stack
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
- remove the payload_error_handler in favor of a PayloadError::from
- merge the two match branch into one
- makes the accepted content type a const instead of recalculating it for every error
1768: Fix auth error r=irevoire a=MarinPostma
fix a small auth error, that set the invalid token error token to "hello". This was invilisble to the user because the invalid token is not returned.
thank you hawk-eye `@irevoire`
Co-authored-by: mpostma <postma.marin@protonmail.com>
This commit is a fix to issue #1750.
As a part of the changes to solve this issue, the following changes have
been made -
1. Route registration for static assets has been modified
2. the `mut` keyword on the `scope` has been removed.
1711: MeiliSearch refactor introducing OBKV format r=MarinPostma a=MarinPostma
This PR refactor some multiple components of meilisearch, and introduce the obkv document format to meilisearch
- [x] Split meilisearch-http and meilisearch-lib
- [x] Replace `IndexActor` and `UuidResolver` with `IndexResolver`
- [x] Remove mentions to Actor
- [x] Remove Actor traits to simplify code
- [x] Integrate obkv document format
- [x] Remove `Data`
- [x] Restore all route
- [x] Replace `Box<dyn error>` with `anyhow::Error`
- [x] Introduce update file store
- [x] Update file store error handling
- [x] Fix dumps
- [x] Fix snapshots
- [x] Fix tests
- [x] Update module documentation
- [x] add csv suppport (feat `@ManyTheFish` #1729 )
- [x] add jsonl support
- [x] integrate geosearch (feat `@irevoire` #1725)
partially implements #1691 and #1690. The error handling is very basic now, I will finish it in the next pr.
Some unit tests have been disabled, I will re-enable them ASAP, but they need a bit more work.
close#1531
P.S: sorry for this monstrous PR :'(
Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
