Commit Graph

8957 Commits

Author SHA1 Message Date
Gregory Conrad
bb9e33bf85 perf: Prevent reindex in searchable reset case when not needed 2022-11-23 22:01:46 -05:00
Gregory Conrad
7c0e544839 feat: Add all_obkv_to_json function 2022-11-23 21:18:58 -05:00
Colby Allen
6ecd486d1b Bumps cargo_toml version to most up to date 2022-11-23 16:27:54 -07:00
Gregory Conrad
d19c8672bb perf: limit reindex to when exact_attributes changes 2022-11-23 15:50:53 -05:00
Tamo
7b8641a7af
fix the dump tests
The issue was linked to the fact that the debug implementation of the PhantomData wasn't the same between rust stable and rust nightly.
This was causing an issue while snapshsotting the settings and this commit fix it by representing the settings as json which already ignores the PhantomData
2022-11-23 16:59:20 +01:00
bors[bot]
f509d81ec9
Merge #3100
3100: Add a dispatch to the publish binaries workflow r=Kerollmops a=curquiza

Add `worklfow_dispatch` event to publish binaries workflow to allow the manually trigger

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-11-21 19:31:20 +00:00
Kerollmops
8d73ae80bb Add a dispatch to the publish binaries workflow 2022-11-21 18:50:57 +01:00
bors[bot]
57c9f03e51
Merge #697
697: Fix bug in prefix DB indexing r=loiclec a=loiclec

Where the batch's information was not properly updated in cases where only the proximity changed between two consecutive word pair proximities.

Closes partially https://github.com/meilisearch/meilisearch/issues/3043



Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-11-17 15:22:01 +00:00
bors[bot]
00129e112a
Merge #3041
3041: Add `workflow_dispatch` to flaky.yml r=irevoire a=curquiza

To be able to run the job manual and don't wait for one week

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-11-17 13:21:11 +00:00
bors[bot]
467e742bd1
Merge #702
702: Update version for the next release (v0.37.0) in Cargo.toml files r=ManyTheFish a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one before merging.

Co-authored-by: curquiza <curquiza@users.noreply.github.com>
2022-11-17 12:54:27 +00:00
curquiza
cd5aaa3a9f Update version for the next release (v0.37.0) in Cargo.toml files 2022-11-17 12:50:07 +00:00
bors[bot]
8ceb199dca
Merge #696
696: Fix Facet Indexing bugs r=Kerollmops a=loiclec

1. Handle keys with variable length correctly

Closes (partially) https://github.com/meilisearch/meilisearch/issues/3042 
This issue is now easily reproducible with the updated fuzz tests, which now generate keys with variable lengths.

2. Prevent adding facets to the database if their encoded value does not satisfy `valid_lmdb_key`.

Closes (partially) https://github.com/meilisearch/meilisearch/issues/2743
This fixes an indexing failure when a document had a filterable attribute containing a value whose length is higher than ~500 bytes. For now, this fix is just meant to prevent crashes. Better handling of long values of filterable attributes will be handled in a separate PR.


Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-11-17 11:56:16 +00:00
Loïc Lecrenier
777eb3fa00 Add insta-snaps for test of bug 3043 2022-11-17 12:21:27 +01:00
Loïc Lecrenier
0caadedd3b Make clippy happy 2022-11-17 12:17:53 +01:00
Loïc Lecrenier
ac3baafbe8 Truncate facet values that are too long before indexing them 2022-11-17 11:29:42 +01:00
Loïc Lecrenier
990a861241 Add test for indexing a document with a long facet value 2022-11-17 11:29:42 +01:00
Loïc Lecrenier
d95d02cb8a Fix Facet Indexing bugs
1. Handle keys with variable length correctly

This fixes https://github.com/meilisearch/meilisearch/issues/3042 and
is easily reproducible with the updated fuzz tests, which now generate
keys with variable lengths.

2. Prevent adding facets to the database if their encoded value does
not satisfy `valid_lmdb_key`.

This fixes an indexing failure when a document had a filterable
attribute containing a value whose length is higher than ~500 bytes.
2022-11-17 11:29:42 +01:00
Loïc Lecrenier
f00108d2ec Fix name of bug in reproduction test 2022-11-17 11:29:18 +01:00
Loïc Lecrenier
f7c8730d09 Fix bug in prefix DB indexing
Where the batch's information was not properly updated in cases
where only the proximity changed between two consecutive word pair
proximities.

Closes https://github.com/meilisearch/meilisearch/issues/3043
2022-11-17 11:29:18 +01:00
bors[bot]
46c275f0e4
Merge #3070
3070: Remove core and use engine r=Kerollmops a=curquiza

Following the new team name
Not mandatory since GitHub is doing redirection, but more consistent

Co-authored-by: curquiza <clementine@meilisearch.com>
2022-11-16 16:40:44 +00:00
bors[bot]
a651397afc
Merge #685
685: ci: Use pre-compiled binaries for faster CI r=irevoire a=azzamsa

# Pull Request

## Related issue
Fixes #<issue_number>

## What does this PR do?
- ...

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [ ] Have you read the contributing guidelines?
- [ ] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: azzamsa <me@azzamsa.com>
2022-11-16 16:39:39 +00:00
curquiza
f4d4f313ea Use the right new link 2022-11-16 17:21:35 +01:00
bors[bot]
2000db8453
Merge #701
701: Remove Hacktoberfest sections r=curquiza a=meili-bot

_This PR is auto-generated._

Remove Hacktoberfest sections from CONTRIBUTING file.


Co-authored-by: meili-bot <74670311+meili-bot@users.noreply.github.com>
2022-11-15 15:17:18 +00:00
meili-bot
92cc3550d8 Update CONTRIBUTING.md 2022-11-15 16:16:40 +01:00
bors[bot]
5e1fa53354
Merge #3055
3055: Remove Hacktoberfest sections r=curquiza a=meili-bot

_This PR is auto-generated._

Remove Hacktoberfest sections from README and CONTRIBUTING files.


Co-authored-by: meili-bot <74670311+meili-bot@users.noreply.github.com>
2022-11-15 14:29:27 +00:00
meili-bot
fe59a6f628 Update README.md 2022-11-15 15:27:18 +01:00
meili-bot
647b7a20e9 Update CONTRIBUTING.md 2022-11-15 15:27:17 +01:00
bors[bot]
cd3bca06e9
Merge #699
699: Force vendoring of LMDB even if a system version is available r=Kerollmops a=dureuill

# Pull Request

## Related issue
Related to https://github.com/meilisearch/meilisearch/issues/3017: will fix once ported to milli and meilisearch.

## What does this PR do?
- Force using vendored version of LMDB
- **don't use lmdb master3 branch anymore**: this is a bit of a side effect of using a tag instead of branch for heed as a dependency, but it is wanted anyway for now as lmdb master3 was more of an experiment
- **modifies CI to run `cargo check` on the release rather than the debug artifacts**. This is an attempt to reduce the necessary disk space and avoid "out of space" failures.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-11-15 11:09:20 +00:00
Louis Dureuil
87576cf26c
Perform cargo check on the release artifacts 2022-11-15 10:25:02 +01:00
funilrys
e81b349658
Fix linting issue. 2022-11-14 18:51:34 +01:00
Louis Dureuil
6dc6a5d874
Force using vendored version of LMDB
- don't use lmdb master3 branch anymore
2022-11-14 17:17:51 +01:00
Clémentine Urquizar - curqui
a84fad5ce6
Add workflow_dispatch to flaky.yml 2022-11-14 10:10:00 +01:00
funilrys
0a102d601c
Update Task.created_at
Indeed, before this patch we weren't considering the
TaskContent::SetingsUpdate while trying to find the creation date.
2022-11-13 10:14:20 +01:00
funilrys
8a14f6f545
Add Task.processed_at. 2022-11-13 10:13:10 +01:00
funilrys
079357ee1f
Fix linting issues. 2022-11-12 20:57:27 +01:00
funilrys
06e7db7a1f
fixup! Extract the dates out of the dumpv4. 2022-11-12 18:28:23 +01:00
bors[bot]
9e189f5041
Merge #3015
3015: Replace deprecated set-output in GitHub actions r=curquiza a=funilrys

# Pull Request

This patch fixes #3011.

This patch fixes the deprecation warning regarding the usage of `set-output`.
This patch fixes the issues by switching the following format:

```
echo ::set-output name=[name]::[value]
```

into the following format:

```
echo "[name]=[value]" >> ${GITHUB_OUTPUT}
```


## Related issue
Fixes #3011

## What does this PR do?
- Fix CI/CD deprecation warnings.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: funilrys <contact@funilrys.com>
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-11-10 09:45:55 +00:00
Clémentine Urquizar - curqui
fe980f9e88
Update .github/workflows/publish-docker-images.yml 2022-11-10 10:40:21 +01:00
Clémentine Urquizar - curqui
32cd9e4852
Update .github/workflows/publish-docker-images.yml 2022-11-10 10:40:16 +01:00
Clémentine Urquizar - curqui
8d79f501f3
Update .github/workflows/publish-binaries.yml 2022-11-10 10:40:09 +01:00
Clémentine Urquizar - curqui
c05abc2b0d
Update .github/workflows/publish-binaries.yml 2022-11-10 10:40:04 +01:00
bors[bot]
e75829aded
Merge #694
694: Update version for the next release (v0.36.0) in Cargo.toml files r=curquiza a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one before merging.

Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com>
2022-11-09 11:22:24 +00:00
Kerollmops
d00d2aab3f Update version for the next release (v0.36.0) in Cargo.toml files 2022-11-09 11:03:09 +00:00
bors[bot]
f46a8ab2e2
Merge #693
693: use the lmdb-master.3 branch r=Kerollmops a=irevoire

After investigating https://github.com/meilisearch/meilisearch/issues/3017, we found out that it was due to lmdb and that, without any code change on our side, bumping using the lmdb-master-3 branch fix our issues.

But, we’re not really confident about what changed between the `mdb.master` and `mdb.master3` branches; thus this is a temporary change, and we hope we’ll be able to move to the new version of heed asap (either before the end of the pre-release or for the next release).

--------

The bug is hard to reproduce; I can reproduce it 100% of the time on my archlinux personal computer. But on a scaleway archlinux bare-metal machine, it doesn’t reproduce. It’s flaky on our test suite, but `@loiclec` was able to write a minimal test that reproduces it every time on macOS.
Basically, what happens is when there are multiple threads opening databases in a different directory at the same time.
If there are 10 or more threads running at the same time, lmdb starts throwing the `Invalid argument (os error 22)` error for no reason, we believe.
I would like to submit an issue to lmdb, but I don’t really have the time to write a test in C without heed currently.

`@hyc,` if you want to take a look at it, here is the repo that reproduces the issue on macOS: https://github.com/irevoire/heed-bug

Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-11-09 09:42:38 +00:00
funilrys
a441fe5ae5
Remove unecessary line. 2022-11-08 21:18:24 +01:00
funilrys
7331da0410
Fix auto-formater issue.
Indeed, my editor always fixes the format for me. That's why those
2 lines were changed.
2022-11-08 21:16:47 +01:00
funilrys
72c4db4553
Rewrite: ${GITHUB_OUTPUT} -> $GITHUB_OUTPUT. 2022-11-08 21:15:28 +01:00
bors[bot]
c3b75bbe5d
Merge #691
691: Update version for the next release (v0.35.1) in Cargo.toml files r=curquiza a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one before merging.

Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com>
2022-11-08 15:31:50 +00:00
Irevoire
c7711daca3
use the lmdb-master.3 branch 2022-11-08 16:28:01 +01:00
bors[bot]
f18a4581f1
Merge #692
692: Update CONTRIBUTING.md r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-11-08 14:51:30 +00:00