Commit Graph

909 Commits

Author SHA1 Message Date
ad hoc
1425d62a31
test dump v5 2022-06-09 18:35:17 +02:00
pierre-l
b8745420da Use the IndexUid and StarOr in meilisearch_auth::Key
Move `meilisearch_http::routes::StarOr` to `meilisearch_types::star_or`

Fixes #2158
2022-06-09 16:14:15 +02:00
pierre-l
36cb09eb25 Add a new meilisearch_types crate
Move `meilisearch_error` to `meilisearch_types::error`
Move `meilisearch_lib::index_resolver::IndexUid` to `meilisearch_types::index_uid`
Add a new `InvalidIndexUid` error in `meilisearch_types::index_uid`
2022-06-09 16:14:13 +02:00
bors[bot]
b9b32d65a8
Merge #2494
2494: Introduce the new faceting and pagination settings r=ManyTheFish a=Kerollmops

This PR introduces two new settings following the newly created spec https://github.com/meilisearch/specifications/pull/157:
 - The `faceting.max_values_per_facet` one describes the maximum number of values (each with a count) associated with a value in a facet distribution query.
 - The `pagination.limited_to` one describes the maximum number of documents that a search query can ever return.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-09 12:09:21 +00:00
bors[bot]
2b2e571c76
Merge #2460
2460: Create custom error types for `TaskType`, `TaskStatus`, and `IndexUid` r=Kerollmops a=walterbm

# Pull Request

## What does this PR do?
Fixes #2443 by making the following changes:

- Add custom `TaskTypeError` for `TaskType::from_str` 
- Add custom `TaskStatusError` for `TaskStatus::from_str`
- Add custom `IndexUidFormatError` for `IndexUid::from_str`
- Implement `From<IndexUidFormatError> for IndexResolverError` to convert between errors
- Replace all usages of `IndexUid::new` with `IndexUid::from_str`
    - **NOTE** I am relatively new to Rust and I struggled a lot with this final part. This PR ended up with a messy error conversion which does not seem ideal. Please let me know if you have any suggestions for how to make this better and I'll be happy to make any updates!

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: walter <walter.beller.morales@gmail.com>
2022-06-09 09:10:28 +00:00
Kerollmops
6f0d3472b1
Change the test for the new pagination.limited_to setting 2022-06-09 10:56:43 +02:00
Kerollmops
5cd13cc303
Add a test to validate the faceting.max_values_per_facet setting 2022-06-09 10:56:42 +02:00
Kerollmops
1e3dcbea3f
Plug the pagination.limited_to setting 2022-06-09 10:56:42 +02:00
Kerollmops
b96399d24b
Plug the faceting.max_values_per_facet setting 2022-06-09 10:56:42 +02:00
Kerollmops
5450b5ced3
Add the faceting.max_values_per_facet setting 2022-06-09 10:54:32 +02:00
walter
3e5d6be86b Rename TaskType::from_str parameter to 'type_' 2022-06-08 19:57:45 -04:00
Clémentine Urquizar
fc4990b968
Update version for next release (v0.28.0) 2022-06-08 17:59:18 +02:00
bors[bot]
6171f17f1d
Merge #2468
2468: Update milli 0.29 r=Kerollmops a=ManyTheFish

- [x] Update milli to 0.29
- [x] Integrate charabia
- [x] Set disabled_words to default when Index::exact_words returns None
- [x] Fix ranking rules integration test

fixes #2375
fixes #2144
fixes #2417
fixes #2407

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-08 14:29:20 +00:00
ManyTheFish
55169ff914 Fix test get_document_s_nested_attributes_to_retrieve 2022-06-08 15:56:06 +02:00
ManyTheFish
0a16f71563 Increase wait_task wainting time 2022-06-08 15:56:03 +02:00
ManyTheFish
17f30c2b2d Fix(auth): Authorization test were not testing keys unrestricted on index 2022-06-08 14:52:32 +02:00
ManyTheFish
09938c9b6f Patch ranking rules error test 2022-06-08 14:38:09 +02:00
bors[bot]
09ec8e9fca
Merge #2471
2471: Remove the connection keep-alive timeout r=MarinPostma a=Thearas

# Pull Request

## What does this PR do?
Fixes <https://github.com/meilisearch/meilisearch-go/issues/221>.
Meilisearch has a default connection keep-alive timeout for 5s, which means it will close the connections with idle time >= 5s.
This PR set actix-web keep-alive config to `KeepAlive::Os`, let the client and system to decide when to close the connection.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?


Co-authored-by: Thearas <thearas850@gmail.com>
2022-06-08 06:59:25 +00:00
ManyTheFish
6ffa222218 feat(auth): Paginate API keys listing
- [x] Update tests
- [x] Use Pagination helpers to paginate API keys

fixes #2442
2022-06-07 17:37:48 +02:00
bors[bot]
79e67df73d
Merge #2474
2474: feat(auth): remove `dumps.get` action from keys r=ManyTheFish a=ManyTheFish

- [x] Update tests
- [x] Remove `dumps.get` action


related to: #2430

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-07 15:03:41 +00:00
ManyTheFish
a7bff35e49 fix(test): Reduce default index size in tests 2022-06-07 15:16:34 +02:00
ManyTheFish
3b01ed4fe8 feat(auth): remove dumps.get action from keys 2022-06-07 10:49:28 +02:00
Thearas
9b9cbc815b fmt 2022-06-07 03:50:39 +08:00
Thearas
fd11903920 remove the connection timeout 2022-06-07 03:38:23 +08:00
Kerollmops
e5b760c59a
Fix the segment analytics tests 2022-06-06 10:44:46 +02:00
Kerollmops
277a0a7967
Bump serde-cs to simplify our usage of the star_or function 2022-06-06 10:17:33 +02:00
Kerollmops
64b5b2e1f8
Use serde-cs::CS with StarOr to reduce the logic duplication 2022-06-06 10:06:00 +02:00
Kerollmops
10d3b367dc
Simplify the const default values 2022-06-06 10:06:00 +02:00
walter
0e7e16ae72 Add custom TaskStatusError for TaskStatus 2022-06-05 00:51:08 -04:00
walter
80c156df3f Add custom TaskTypeError for TaskType 2022-06-05 00:51:08 -04:00
bors[bot]
b83455f345
Merge #2454
2454: Unify the pagination of the index and documents route behind a common type r=curquiza a=irevoire

`@MarinPostma` wdyt of keeping the `auto_paginate_sized` until we implement the pagination on every route that needs it just to see if it could be useful to something else

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-06-02 15:01:43 +00:00
bors[bot]
953a209f02
Merge #2447
2447: move index uid in task content r=Kerollmops a=MarinPostma

this pr moves the index_uid from the `Task` to the `TaskContent`. This is because the task can now have content that do not target a particular index.


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-06-02 13:54:09 +00:00
ad hoc
0c5352fc22
move index_uid from task to task_content 2022-06-02 15:30:35 +02:00
bors[bot]
8ac8fcb0c9
Merge #2433
2433: Fix the documents route r=Kerollmops a=irevoire

fix #2428

Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-06-02 13:25:34 +00:00
Irevoire
4667c9fe1a
fix(http): Fix the query parameter in the Documents route 2022-06-02 14:10:44 +02:00
Tamo
12b5eabd5d
chore(http): unify the pagination of the index and documents route behind a common type 2022-06-02 14:06:56 +02:00
bors[bot]
cf2d8de48a
Merge #2452
2452: Change http verbs r=ManyTheFish a=Kerollmops

This PR fixes #2419 by updating the HTTP verbs used to update the settings and every single setting parameter.

- [x] `PATCH /indexes/{indexUid}` instead of `PUT`
- [x] `PATCH /indexes/{indexUid}/settings`  instead of `POST`
- [x] `PATCH /indexes/{indexUid}/settings/typo-tolerance`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/displayed-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/distinct-attribute`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/filterable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/ranking-rules`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/searchable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/sortable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/stop-words`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/synonyms`  instead of `POST`


Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 11:46:17 +00:00
Kerollmops
419922e475
Make clippy happy 2022-06-02 13:38:23 +02:00
bors[bot]
c9cd1738a5
Merge #2445
2445: Seek-based tasks list r=Kerollmops a=Kerollmops

This PR implements the seek-based pagination for the tasks list following [the spec](https://github.com/meilisearch/specifications/pull/115).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 10:25:54 +00:00
Kerollmops
0258659278
Fix the get_settings tests 2022-06-02 12:24:27 +02:00
Kerollmops
ce37f53a16
Add typo-tolerance to the authorization tests 2022-06-02 12:17:53 +02:00
Kerollmops
bcb51905d7
Fix the authorization tests 2022-06-02 12:16:46 +02:00
Kerollmops
10a71fdb10
Update the /indexes/{indexUid}/settings/* verbs by adding a macro parameter 2022-06-02 11:55:47 +02:00
Kerollmops
f8d3f739ad
Update the /indexes/{indexUid}/settings verb from POST to PATCH 2022-06-02 11:55:47 +02:00
Kerollmops
bb405aa729
Update the /indexes/{indexUid} verb from PUT to PATCH 2022-06-02 11:55:47 +02:00
bors[bot]
7e3d5ebc8e
Merge #2451
2451: feat(API-keys): Change immutable_field error message r=Kerollmops a=ManyTheFish

Change the immutable_field error message to fit the recent changes in the spec:
aa0a148ee3..84a9baff68



Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-02 09:26:36 +00:00
Kerollmops
dfce9ba468
Apply suggestions 2022-06-02 11:26:12 +02:00
ManyTheFish
9eea142e2b feat(API-keys): Change immutable_field error message
Change the immutable_field error message to fit the recent changes in the spec:
aa0a148ee3..84a9baff68
2022-06-02 11:11:07 +02:00
bors[bot]
8b8c3e32f0
Merge #2450
2450: Bump the dependencies r=ManyTheFish a=Kerollmops

In order to use [the latest version of grenad](https://docs.rs/grenad) I bump the dependencies here. We also use the latest versions of all our other dependencies now.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 08:53:12 +00:00
Kerollmops
ac9e7bdbe3
Fix a test that was depending on the speed of the CPU 2022-06-02 10:21:19 +02:00
ManyTheFish
4512eed8f5 Fix PR comments 2022-06-01 18:06:20 +02:00
Kerollmops
0656df3a6d
Fix the dumps tests 2022-06-01 17:14:13 +02:00
ManyTheFish
94b32cce01 Patch errors 2022-06-01 16:17:47 +02:00
ManyTheFish
b2e2dc8558 Re-authorize master_key to access to all routes 2022-06-01 16:17:47 +02:00
ManyTheFish
c295924ea2 Patch tests 2022-06-01 16:08:42 +02:00
ManyTheFish
1f62e83267 Remove error_add_api_key_invalid_index_uid_format 2022-06-01 16:08:42 +02:00
ManyTheFish
b3c8915702 Make small changes and renaming 2022-06-01 16:08:42 +02:00
ManyTheFish
84f52ac175 Add v4 feature to uuid 2022-06-01 16:08:42 +02:00
ManyTheFish
b9a79eb858 Change apiKeyPrefix to apiKeyUid 2022-06-01 16:07:44 +02:00
ManyTheFish
a57b2d9538 Restrict master key access to /keys routes 2022-06-01 16:07:44 +02:00
ManyTheFish
34c8888f56 Add keys actions 2022-06-01 16:07:44 +02:00
ManyTheFish
96a5791e39 Add uid and name fields in keys 2022-06-01 16:07:44 +02:00
ManyTheFish
e2c204cf86 Update tests to fit to the new requirements 2022-06-01 16:07:44 +02:00
Kerollmops
d80e8b64af
Align the tasks route API to the new spec 2022-06-01 15:30:39 +02:00
Kerollmops
c11d21879a
Introduce tasks limit and after to the tasks route 2022-06-01 13:26:36 +02:00
Kerollmops
004c8b6be3
Add the new limit and after fields in the dump tests 2022-06-01 12:09:52 +02:00
Kerollmops
9d5cc88cd5
Implement the seek-based tasks list pagination 2022-06-01 12:09:52 +02:00
bors[bot]
e81c7aa2e6
Merge #2423
2423: Paginate the index resource r=MarinPostma a=irevoire

Fix #2373


Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-05-31 19:25:25 +00:00
bors[bot]
47007fa71b
Merge #2446
2446: rename Succeded to Succeeded r=irevoire a=MarinPostma

this pr renames `TaskEvent::Succeded` to `TaskEvent::Succeeded` and apply the migration to the dumps


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-31 18:27:02 +00:00
Irevoire
627f13df85
feat(http): paginate the index resource
Fix #2373
2022-05-31 18:11:45 +02:00
ad hoc
446f1f31e0
rename Succeded to Succeeded 2022-05-31 17:22:37 +02:00
Irevoire
ddad6cc069
feat(http): update the documents resource
- Return Documents API resources on `/documents` in an array in the the results field.
- Add limit, offset and total in the response body.
- Rename `attributesToRetrieve` into `fields` (only for the `/documents` endpoints, not for the `/search` ones).
- The `displayedAttributes` settings does not impact anymore the displayed fields returned in the `/documents` endpoints. These settings only impacts the `/search` endpoint.

Fix #2372
2022-05-31 16:40:40 +02:00
Kerollmops
1465b5e0ff
Refactorize the tasks filters by moving the match inside 2022-05-31 11:33:21 +02:00
Kerollmops
8800b348f0
Implement the StarOr on all the tasks filters 2022-05-31 11:33:21 +02:00
Kerollmops
082d6b89ff
Make the StarOrIndexUid Generic and call it StarOr 2022-05-31 11:33:21 +02:00
Kerollmops
b82c86c8f5
Allow users to filter indexUid with a * 2022-05-31 11:33:20 +02:00
Kerollmops
36d94257d8
Make clippy happy 2022-05-31 11:33:20 +02:00
Kerollmops
3f80468f18
Rename the Tasks Types 2022-05-31 11:33:20 +02:00
Kerollmops
8509243e68
Implement the status and type filtering on the tasks route 2022-05-31 11:33:20 +02:00
Kerollmops
3684c822f1
Add indexUid filtering on the /tasks route 2022-05-31 11:33:20 +02:00
Kerollmops
80f7d87356
Remove the /indexes/:indexUid/tasks/... routes 2022-05-31 11:33:20 +02:00
Kerollmops
d2f457a076
Rename the uid to taskUid in asynchronous response 2022-05-31 11:33:20 +02:00
Kerollmops
e5ef5a6f9c
Remove an unused updates.rs file 2022-05-31 11:33:19 +02:00
bors[bot]
3441cc6c36
Merge #2410
2410: Make dump a task r=Kerollmops a=MarinPostma

This PR transforms the dump task into a proper task.
The `GET /dumps/:dump_uid` is removed.


Some changes were made to make this work, and a bit a refactoring was necessary.
- The `dump_actor` module has been renamed do `dumps` and moved to the root
- There isn't a `DumpActor` anymore, and the dump process is handled by the `DumpHandler`.
- The `TaskPerformer` is renamed to `BatchHandler`
- The `BatchHandler` trait no longer has a `perform_job` method, but instead has a `accept` method returning whether a handler can proccess a batch
- The scheduler now accept a list of `BatchHandler`, and iterates trhough them until it finds one to accept the current batch.
- `Job` doesn't exist anymore, and everything in now inside of the `BatchContent` enum.
- The `Vec<TaskId>` from `Batch` is replaced with a `BatchContent` enum which hints at the content.
- The Scheduler is slightly modified to accept batch, and prioritize them before regular tasks.
- The `TaskList` are not identified by a `String` representing the index uid anymore, but by a `TaskListIdentifier` which also works for dumps which are not targeting any specific indexes.
- The `GET /dump/:dump_id` no longer exists
- `DumpActorError` is renamed to `DumpError`


close #2410 

Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-30 14:09:43 +00:00
Irevoire
d47b997120
chore(analytics): update the url used to send our analytics 2022-05-30 15:13:10 +02:00
ad hoc
986a99296d
remove useless dump test 2022-05-25 11:25:11 +02:00
ad hoc
0f9c134114
fix tests 2022-05-25 11:13:35 +02:00
ad hoc
61035a3ea4
create dump v5 2022-05-25 11:13:34 +02:00
ad hoc
4778884105
remove dump status route 2022-05-25 11:13:34 +02:00
ad hoc
57fde30b91
handle dump 2022-05-25 11:13:34 +02:00
ad hoc
414d0907ce
register dump handler 2022-05-25 11:13:34 +02:00
ad hoc
60a8249de6
add dump batch handler 2022-05-25 11:13:34 +02:00
ad hoc
46cdc17701
make scheduler accept multiple batch handlers 2022-05-25 11:13:34 +02:00
ad hoc
7fa3eb1003
register dump tasks 2022-05-25 11:13:33 +02:00
ad hoc
5a5066023b
introduce TaskListIdentifier 2022-05-25 11:13:33 +02:00
ad hoc
aa50acb031
make Task index_uid an option
Not all task relate to an index. Tasks that don't have an index_uid set
to None
2022-05-25 11:13:32 +02:00
bors[bot]
341756a0eb
Merge #2357
2357: chore(dump): add dump tests r=Kerollmops a=irevoire

Add tests on the import of dump v1, v2, v3 and v4.

Since the dumps are slow to decompress, I made the `flate2` crate always compile in optimized.
And since they're also slow to index, I also made the `milli` crate always compile in optimized. What do you think of this `@MarinPostma?`
Should we keep milli unoptimized in case it could help us debug some things? 👀 

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-24 12:24:29 +00:00
Tamo
5f0e9b63d2
chore(dump): add tests 2022-05-24 14:21:56 +02:00
bors[bot]
ca9ba2d90c
Merge #2406
2406: chore(search): rename in the search endpoint r=irevoire a=irevoire

Fix #2376


Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-05-24 12:02:45 +00:00
Tamo
6bf4db0bca
feat(analytics): handle the new x-meilisearch-client custom header for the analytics
Fix #2367
2022-05-23 13:51:19 +02:00