Commit Graph

8053 Commits

Author SHA1 Message Date
curquiza
4631f4d97f Bump milli to v0.37.2 2022-12-08 18:16:48 +01:00
Tamo
6f1c30b247 Fix the instance-uid in the data.ms
We were writing the instance-uid as bytes instead of string in the data.ms and thus we were unable to parse it later.
Also it was less practical for our user to retrieve it and send it to us.
2022-12-08 18:16:43 +01:00
bors[bot]
abba54e913
Merge #3112
3112: Rename meilisearch-http r=Kerollmops a=colbsmcdolbs

# Pull Request

## Related issue
Fixes #3073 

## What does this PR do?
- Renames all references of `meilisearch-http` to `meilisearch`
- Might need to be rebased before the 1.0.0 release

## PR checklist
Please check if your PR fulfills the following requirements:
- [X] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [X] Have you read the contributing guidelines?
- [X] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Colby Allen <colbyjayallen@gmail.com>
2022-12-08 16:32:08 +00:00
bors[bot]
c426fa1478
Merge #3212
3212: Setup COMMIT_SHA and COMMIT_DATE build args in the Docker image r=curquiza a=brunoocasali

GitHub auto-closed my PR when I synced changes with my remote 🤷‍♂️  https://github.com/meilisearch/meilisearch/pull/2550
The last PR #3205 were closed to help `@curquiza` test the CI.

In any case, the summary of changes is quite similar:

- Fix `git` usage from my last attempt (when you use `actions/checkout`) you get the `git` command to use.
- Add the `build-args` definition from https://github.com/docker/build-push-action#inputs, which is supposed to work precisely as docker build `--build-arg`. 

Fixes https://github.com/meilisearch/meilisearch/issues/2028

The result will be like this:

<img width="556" alt="image" src="https://user-images.githubusercontent.com/4116980/206019608-2713559a-1f58-4ff3-9fec-7720783993ac.png">

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
2022-12-08 16:07:50 +00:00
bors[bot]
574be942cd
Merge #3221
3221: Update README to reference Meilisearch Cloud r=curquiza a=davelarkan

# Pull Request

## Related issue
Fixes #3220

## What does this PR do?
- Updates the README to link to the Pricing page where people can choose a Meilisearch Cloud plan

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Dave Larkan <davelarkan@gmail.com>
2022-12-08 15:43:43 +00:00
Colby Allen
2262766494 chore: run fmt nightly on project 2022-12-08 08:31:15 -07:00
Colby Allen
ad2b1467da Renames meilisearch-http to meilisearch 2022-12-08 08:22:53 -07:00
Dave Larkan
ee37d5e724 Update README to reference Meilisearch Cloud 2022-12-08 15:02:34 +00:00
bors[bot]
ded2a50d14
Merge #3216
3216: Update version for the next release (v1.0.0) in Cargo.toml files r=curquiza a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one before merging.

Co-authored-by: curquiza <curquiza@users.noreply.github.com>
2022-12-08 13:49:50 +00:00
Bruno Casali
58327979f1 Use correct env vars "VERGEN_*" on Dockerfile 2022-12-08 10:48:16 -03:00
Bruno Casali
50d9fe036e Setup COMMIT_SHA and COMMIT_DATE build args in the Docker image 2022-12-08 10:48:16 -03:00
curquiza
026cf223b3 Update version for the next release (v1.0.0) in Cargo.toml files 2022-12-08 12:20:17 +00:00
bors[bot]
af6f7f8462
Merge #3215
3215: Use nightly in cargo fmt r=curquiza a=curquiza

Discussed with `@Kerollmops,` needs this change

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-12-08 10:53:24 +00:00
Clémentine Urquizar - curqui
5023d36ee7
Use nightly in cargo fmt 2022-12-08 11:51:13 +01:00
bors[bot]
1f1beae077
Merge #729
729: Fix distincted exhaustive hits r=Kerollmops a=ManyTheFish

This PR changes the name and behavior of `bucket_candidates`:
- `bucket_candidates` become `initial_candidates` that is less confusing
- `initial_candidates` is no more a simple `RoaringBitmap` but an enum allowing us to precise if the candidates are exhaustive or not
- this enum ensures that any modification is allowed only if the candidates are not already exhaustive.

The bug occurred because `initial_candidates` are modified during the bucket sort allowing the estimation to be more and more precise along the search, and this was an issue when the `initial_candidates` were already exhaustive, now, if candidates are exhaustive, then no modifications are made.

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-12-08 09:26:34 +00:00
ManyTheFish
55724f2412 Introduce an initial candidates set that makes the difference between an exhaustive count and an estimation 2022-12-08 09:41:34 +01:00
ManyTheFish
6d50ea0830 add tests 2022-12-08 08:56:57 +01:00
bors[bot]
f4dc4c5d8d
Merge #3210
3210: Fix `MDB_PAGE_FULL` by bumping LMDB r=Kerollmops a=Kerollmops

This PR fixes #3062 by upgrading LMDB to the latest version.

The changes were made in https://github.com/meilisearch/lmdb/pull/1 and https://github.com/meilisearch/lmdb-rs/pull/12. As heed directly depends on the latest main commit of https://github.com/meilisearch/lmdb-rs, we can bump the `lmdb-rkv-sys` dependency in the Meilisearch _Cargo.lock_ by doing a:

```
cargo update -p lmdb-rkv-sys
```

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-12-07 16:21:23 +00:00
Loïc Lecrenier
f37c86e0b2 Add some integration tests on the sort criterion 2022-12-07 15:59:33 +01:00
jiangbo212
717dd36547 Merge branch 'fix-3037' of github.com:jiangbo212/meilisearch into fix-3037 2022-12-07 22:54:16 +08:00
jiangbo212
538030c2da change NameTempFile to tempfile() 2022-12-07 22:47:32 +08:00
bors[bot]
098c410612
Merge #727
727: Fix bug in filter search r=Kerollmops a=loiclec

# Pull Request

## Related issue
Fixes (partially, until merged into meilisearch) https://github.com/meilisearch/meilisearch/issues/3178

## What does this PR do?
The most important change is this one:
```rust
    // in milli/src/search/facet/facet_range_search.rs, line 239
    let should_stop = {
        match self.right {
            Bound::Included(right) => right < previous_key.left_bound,
            Bound::Excluded(right) => right <= previous_key.left_bound,
            Bound::Unbounded => false,
        }
    };
```
where the operations `<` and `<=` between the two branches were switched. This caused (very few) documents to be missing from filter results.

The second change is a simplification of the algorithm for filters such as `field = value`, where we now perform a direct query into the "Level 0" of the facet db to retrieve the docids instead of invoking the full facet search algorithm. This change is done in `milli/src/search/facet/filter.rs`.

I have added yet more insta-snapshot tests, rechecked the content of the snapshots, and added some integration tests as well. 

This is purely a fix in the search algorithms. Based on this PR alone, a dump will not be necessary to switch from v0.30.1 (where this bug is present) to v0.30.2 (where this PR is merged).


Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-12-07 14:34:59 +00:00
Kerollmops
1d5294d11a
Bump lmdb version 2022-12-07 15:29:56 +01:00
bors[bot]
ee10cb8c87
Merge #726
726: Update the contributing.md r=curquiza a=irevoire



Co-authored-by: Tamo <tamo@meilisearch.com>
2022-12-07 13:59:04 +00:00
Loïc Lecrenier
d38cc73630 Add one more filter "integration" test 2022-12-07 14:38:25 +01:00
Loïc Lecrenier
e688581c36 Add tests for facet range search on different field ids 2022-12-07 14:38:21 +01:00
Loïc Lecrenier
4ac8f96342 Simplify implementation of equality condition in filters 2022-12-07 14:38:18 +01:00
Loïc Lecrenier
1c9555566e Fix bug in facet range search 2022-12-07 14:38:14 +01:00
Loïc Lecrenier
303d740245 Prepare fix within facet range search
By creating snapshots and updating the format of the existing
snapshots. The next commit will apply the fix, which will show
its effects cleanly on the old and new snapshot tests
2022-12-07 14:38:10 +01:00
bors[bot]
34c3e5ec5e
Merge #3208
3208: Stop snapshotting the version of meilisearch in the dump r=Kerollmops a=irevoire

It might change, and we don't want to update this test every time we make a new release.


Co-authored-by: Tamo <tamo@meilisearch.com>
2022-12-07 12:54:55 +00:00
Tamo
1c3a326199
stop snapshotting the version of meilisearch in the dump
It might change and we don't want to update this test everytime we make a new release.
2022-12-07 13:26:02 +01:00
Tamo
250743885d
add a sentence about installing rust-nightly 2022-12-07 12:31:43 +01:00
bors[bot]
34c0f11c26
Merge #3207
3207: Add release check when starting latest CI r=Kerollmops a=curquiza

Adding this to have the same kind of check before starting to move the latest tag

<img width="737" alt="Capture d’écran 2022-12-07 à 12 18 33" src="https://user-images.githubusercontent.com/20380692/206165868-18a2be7c-78ec-48c9-acb9-d7f60797c2e3.png">

Also, removing an un-unused script

Co-authored-by: curquiza <clementine@meilisearch.com>
2022-12-07 11:27:47 +00:00
Tamo
5eecb8489d
Update CONTRIBUTING.md
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-12-07 12:23:12 +01:00
Tamo
0e5c3b1f64
Update CONTRIBUTING.md
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-12-07 12:23:06 +01:00
curquiza
be300138e4 Add release check when starting latest CI 2022-12-07 12:22:44 +01:00
bors[bot]
2ed6017603
Merge #3204
3204: Bring back v0.30.1 changes to `main` r=curquiza a=curquiza

I was not able to just import `release-v0.30.1` to `main`, see:
<img width="1371" alt="Capture d’écran 2022-12-06 à 20 03 50" src="https://user-images.githubusercontent.com/20380692/206000844-b39b3063-7da2-475f-b3e4-1791c39a7c2f.png">

So I cherry-picked the commits.

⚠️ ⚠️ ⚠️ I had a git conflict here

<img width="730" alt="Capture d’écran 2022-12-06 à 20 09 04" src="https://user-images.githubusercontent.com/20380692/206001007-f56bc28f-c0b1-46a0-bb60-cce4e73b9584.png">


⚠️ ⚠️ ⚠️ Check out carefully how I fixed it


Co-authored-by: curquiza <curquiza@users.noreply.github.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-12-07 11:08:37 +00:00
Kerollmops
c1337f9e08
Update dump snap to new version 2022-12-07 11:48:29 +01:00
bors[bot]
9acac28574
Merge #3128
3128: Bumps cargo_toml version to most up to date r=curquiza a=colbsmcdolbs

# Pull Request

## Related issue
Fixes #3127

## What does this PR do?
- The README of this repository declares that one package is not up to date. In order to ensure Due Diligence, I have bumped the version number of the package. No test failures running on Windows.

## PR checklist
Please check if your PR fulfills the following requirements:
- [X] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [X] Have you read the contributing guidelines?
- [X] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Colby Allen <colbyjayallen@gmail.com>
2022-12-07 10:31:25 +00:00
jiangbo212
cb1d184904 fmt fix 2022-12-07 17:04:24 +08:00
jiangbo212
2841b09789
Merge branch 'meilisearch:main' into fix-3037 2022-12-07 16:30:21 +08:00
jiangbo212
35f3dd68b6 error change and tokio file use change 2022-12-07 16:20:36 +08:00
Kerollmops
f1de3aa75a Make the tests use MB to trigger page size issues 2022-12-06 20:10:10 +01:00
Kerollmops
e4e4370a3c Clamp the databases size to the page size 2022-12-06 20:09:49 +01:00
Kerollmops
24c79b79f9 Bump milli to v0.37.1 2022-12-06 20:05:52 +01:00
curquiza
5db7c4057c Update version for the next release (v0.30.1) in Cargo.toml files 2022-12-06 20:05:46 +01:00
Tamo
f53bdc4320
update the contributing.md 2022-12-06 17:41:05 +01:00
bors[bot]
0a301b5f88
Merge #723
723: Fix bug in handling of soft deleted documents when updating settings r=Kerollmops a=loiclec

# Pull Request

## Related issue
Fixes (partially, until merged into meilisearch) https://github.com/meilisearch/meilisearch/issues/3021

## What does this PR do?
This PR fixes the bug where a `missing key in documents database` internal error message could appear when indexing documents.

When updating the settings, before clearing the database and before creating the transform output, we now modify the `ExternalDocumentsIds` structure to get rid of all references to soft deleted document ids in its FSTs.

It used to be that updating the settings would clear the soft-deleted document ids, but keep the original `ExternalDocumentsIds` structure. As a consequence of this, when processing a future document addition, we could wrongly believe that a document was being replaced when, in fact, it was a completely new document. See the tests `bug_3021_first`, `bug_3021_second`, and `bug_3021` for a minimal test case that would have reproduced the issue.
 
We need to take special care to:
- evaluate how users should update to v0.30.1 (containing this fix): dump? reimporting all documents from scratch?
- understand IF/HOW this bug could have caused duplicate documents to be returned 
- and evaluate the correctness of the fix, of course :)


Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-12-06 14:37:38 +00:00
Loïc Lecrenier
a993b68684 Cargo fmt >:-( 2022-12-06 15:22:10 +01:00
Loïc Lecrenier
80c7a00567 Fix compilation error in tests of settings update 2022-12-06 15:19:26 +01:00