52 Commits

Author SHA1 Message Date
ManyTheFish
df776b6df7 Refactor search and facet-search
**Changes:**
The search filters are now using the FilterableAttributesFeatures from the FilterableAttributesRules to know if a field is filterable.
Moreover, the FilterableAttributesFeatures is more precise and an error will be returned if an operator is used on a field that doesn't have the related feature.
The facet-search is now checking if the feature is allowed in the FilterableAttributesFeatures and an error will be returned if the field doesn't have the related feature.

**Impact:**
- facet-search is now relying on AttributePatterns to match the locales
- search using filters is now relying on FilterableAttributesFeatures
- distinct attribute is now relying on FilterableAttributesRules
2025-02-27 17:52:17 +01:00
ManyTheFish
f770de03ed Change the filterableAttributes setting API
**Changes:**
The filterableAttributes type has been changed from a `BTreeSet<String>` to a `Vec<FilterableAttributesRule>`,
Which is a list of rules defining patterns to match the documents' fields and a set of feature to apply on the matching fields.
The rule order given by the user is now an important information, the features applied on a filterable field will be chosen based on the rule order as we do for the LocalizedAttributesRules.
This means that the list will not be reordered anymore and will keep the user defined order,
moreover, if there are any duplicates, they will not be de-duplicated anymore.

**Impact:**
- Settings API
- the database format of the filterable attributes changed
- may impact the LocalizedAttributesRules due to the AttributePatterns factorization
- OpenAPI generator
2025-02-27 17:52:16 +01:00
Clément Renault
091f989b72
Upgrade incompatible dependencies 2025-01-08 15:58:03 +01:00
Clément Renault
48a9ad4c17
Fix insta to 1.39 2025-01-08 15:18:08 +01:00
Clément Renault
b997039a91
Upgrade compatible dependencies 2025-01-08 13:52:14 +01:00
Tamo
a8ef6f08e0
Update crates/meilisearch-types/src/settings.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-07 16:31:39 +01:00
Tamo
ae5a04e85c
apply review comments 2025-01-07 16:30:14 +01:00
Tamo
8ebfc9fa92
Update crates/meilisearch-types/src/settings.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-07 16:30:14 +01:00
Tamo
dd128656cb
fix all the tests 2025-01-07 16:28:12 +01:00
Tamo
ac944f0960
review all the return type 2025-01-07 16:26:06 +01:00
Tamo
aab6ffec30
fix and review all the documents route 2025-01-07 16:26:06 +01:00
Tamo
1dd33af8a3
add the batches 2025-01-07 16:26:06 +01:00
Tamo
0bf4157a75
try my best to make the sub-settings routes works, it doesn't 2025-01-07 16:26:06 +01:00
Tamo
04e4586fb3
add the searches route and fix a few broken things 2025-01-07 16:26:06 +01:00
Tamo
78f6f22a80
implement all the /indexes/documents route 2025-01-07 16:26:06 +01:00
Tamo
13afdaf393
finish rebase and update utoipa to the latest version 2025-01-07 16:26:06 +01:00
Tamo
742d0ee531
Implements the get and delete tasks route 2025-01-07 16:26:04 +01:00
meili-bors[bot]
1fc90fbacb
Merge #5147
5147: Batch progress r=dureuill a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5068

## What does this PR do?
- ...

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [ ] Have you read the contributing guidelines?
- [ ] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Tamo <tamo@meilisearch.com>
2024-12-12 09:15:54 +00:00
Tamo
8cd3a1aa57
fmt 2024-12-11 18:18:40 +01:00
Tamo
ab9213fa94
ensure we never write the progress to the db 2024-12-11 18:16:20 +01:00
meili-bors[bot]
eaabc1af2f
Merge #5144
Some checks failed
Test suite / Tests on ${{ matrix.os }} (macos-13) (push) Waiting to run
Test suite / Tests almost all features (push) Has been skipped
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Tests on ubuntu-20.04 (push) Failing after 11s
Test suite / Run tests in debug (push) Failing after 10s
Test suite / Tests on ${{ matrix.os }} (windows-2022) (push) Failing after 47s
Test suite / Run Rustfmt (push) Successful in 1m17s
Test suite / Run Clippy (push) Successful in 5m30s
5144: Exactly 512 bytes docid fails r=Kerollmops a=dureuill

# Pull Request

## Related issue
Fixes #5050 

## What does this PR do?
- Return a user error rather than an internal one for docids of exactly 512 bytes
- Fix up error message to indicate that exactly 512 bytes long docids are not supported.
- Fix up error message to reflect that index uids are actually limited to 400 bytes in length

## Impact

- Impacts docs: 
    - update [this paragraph](https://www.meilisearch.com/docs/learn/resources/known_limitations#length-of-primary-key-values) to say 511 bytes instead of 512 

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-12-11 15:41:05 +00:00
Tamo
df9b68f8ed
inital implementation of the progress 2024-12-11 16:25:01 +01:00
Kerollmops
01bcc601be
Use a nonrandom hasher when decoding JSON 2024-12-11 12:04:29 +01:00
Kerollmops
93fbdc06d3
Use a nonrandom hasher when decoding NDJSON 2024-12-11 12:03:09 +01:00
Kerollmops
d683f5980c
Do not duplicate NDJson when unecessary 2024-12-11 12:02:48 +01:00
Kerollmops
6b269795d2
Update bumparaw-collections to 0.1.2 2024-12-10 14:25:13 +01:00
Kerollmops
89637bcaaf
Use bumparaw-collections in Meilisearch/milli 2024-12-10 11:52:20 +01:00
Louis Dureuil
866ac91be3
Fix error messages 2024-12-10 11:06:58 +01:00
ManyTheFish
a0a3b55700 Change error code 2024-12-05 14:48:29 +01:00
ManyTheFish
953a82ca04 Add new error message 2024-12-04 11:15:29 +01:00
Louis Dureuil
d040aff101 Stop allocating 1GiB for documents 2024-12-02 16:30:14 +01:00
Clément Renault
e1e76f39d0
Clean up dependencies 2024-11-27 14:30:34 +01:00
Many the fish
9008ecda3d
Update crates/meilisearch-types/src/settings.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2024-11-26 14:44:24 +01:00
ManyTheFish
d66dc363ed Test and implement settings opt-out 2024-11-25 18:23:22 +01:00
Louis Dureuil
dcc3caef0d
Remove TopLevelMap 2024-11-21 16:56:46 +01:00
ManyTheFish
36962b943b First batch of PR comment 2024-11-21 16:38:11 +01:00
Louis Dureuil
6e6acfcf1b
Merge branch 'main' into indexer-edition-2024 2024-11-20 16:59:58 +01:00
Louis Dureuil
ff9c92c409
rename documents -> substep 2024-11-20 15:12:02 +01:00
Tamo
4abcd9c04e
add some stats on the batches 2024-11-20 10:51:06 +01:00
Tamo
229fa0f902
implements the batch details 2024-11-20 10:51:06 +01:00
Tamo
15eefa4fcc
fixes a lot of small issue, the test about the cancellation is still failing 2024-11-20 10:51:05 +01:00
Tamo
a1251c3c83
Implements the get all batches route with filters working 2024-11-20 10:42:55 +01:00
Tamo
6062914654
add the batch_id to the tasks 2024-11-20 10:42:54 +01:00
Louis Dureuil
bfefaf71c2
Progress displayed in logs 2024-11-19 09:32:52 +01:00
Louis Dureuil
5f93651cef
fixes 2024-11-18 16:23:11 +01:00
Louis Dureuil
1fcd5f091e
Remove progress from task 2024-11-12 12:23:13 +01:00
Louis Dureuil
d97af4d8e6
fix field order of JSON documents 2024-11-07 22:36:52 +01:00
Tamo
2eb1801e85 reverse the order of the task queue 2024-11-07 19:19:44 +01:00
Louis Dureuil
ee03743355
Merge branch 'indexer-edition-2024' into indexer-edition-2024-doc-chunks 2024-11-06 15:50:53 +01:00
ManyTheFish
10feeb88f2 Merge branch 'main' into indexer-edition-2024 2024-11-06 15:19:18 +01:00