308: Implement a better parallel indexer r=Kerollmops a=ManyTheFish
Rewrite the indexer:
- enhance memory consumption control
- optimize parallelism using rayon and crossbeam channel
- factorize the different parts and make new DB implementation easier
- optimize and fix prefix databases
Co-authored-by: many <maxime@meilisearch.com>
337: Update version for the next release (v0.12.0) r=Kerollmops a=curquiza
Breaking because of the new indexer that implies DB changes #308
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
335: Get sortable_fields from index only if criteria present in query r=Kerollmops a=shekhirin
Seems like we don't need to retrieve `sortable_fields` from the index if there's no any `sort_criteria` in the query.
Small 🤏 optimization opportunity out there.
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
329: Run all benchmarks once every friday r=irevoire a=irevoire
All the benchmarks run every Friday on the `main` branch.
To avoid having pending benchmarks everywhere, we execute one benchmark every 8 hours.
Then the results are uploaded as if it was a normal user-run benchmark.
This PR closes#314 and #321
Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
1658: Remove COMMIT_SHA and COMMIT_DATE build arg from the Docker CIs r=irevoire a=curquiza
Since `@irevoire` add the `.git` folder in the Dockerfile, no need to compute `COMMIT_SHA` and `COMMIT_DATE` in the CI.
Can you confirm `@irevoire?`
Also, update some CIs using `checkout@v1` to `checkout@v2`
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
1652: Remove dependabot r=MarinPostma a=curquiza
Fixes#1649
Dependabot for vulnerability and security updates is still activated.
1654: Add Script for Windows r=MarinPostma a=singh08prashant
fixes#1570
changes:
1. added script for detecting windows os running git bash
2. appended `.exe` to `$release_file` for windows as listed [here](https://github.com/meilisearch/MeiliSearch/releases/)
3. removed global `$BINARY_NAME='meilisearch'` as windows require `.exe` file
1657: Bring vergen hotfix from `stable` to `main` r=MarinPostma a=curquiza
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: singh08prashant <singh08prashant@gmail.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
1656: Remove unused Arc import r=MarinPostma a=Kerollmops
This PR removes a warning introduced by #1606 which removed Sentry that was using an `Arc` but forgot to remove the scope import, we remove it here.
Co-authored-by: Kerollmops <clement@meilisearch.com>
330: Introduce the reset_sortable_fields Settings method r=irevoire a=Kerollmops
I forgot to add the `reset_sortable_fields` method on the `Settings` builder, it is no big deal as the library user (like MeiliSearch) can always call `set_sortable_fields` with an empty list of fields, it is equivalent.
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
1636: Hotfix: Log but don't panic when vergen can't retrieve commit information r=curquiza a=Kerollmops
This pull request fixes an issue we discovered when we tried to publish meilisearch v0.21 on brew, brew uses the tarball downloaded from github directly which doesn't contain the `.git` folder.
We use the `.git` folder with [vergen](https://docs.rs/vergen) to retrieve the commit and datetime information. Unfortunately, we were unwrapping the vergen result and it was crashing when the git folder was missing.
We no more panic when vergen can't find the `.git` folder and just log out a potential error returned by [the git2 library](https://docs.rs/git2). We then just check that the env variables are available at compile-time and replace it with "unknown" if not.
### When the `.git` folder is available
```
xh localhost:7700/version
HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 26 Aug 2021 13:44:23 GMT
Transfer-Encoding: chunked
{
"commitSha": "81a76eab69944de8a8d5006345b5aec7b02acf50",
"commitDate": "2021-08-26T13:41:30+00:00",
"pkgVersion": "0.21.0"
}
```
### When the `.git` folder is unavailable
```bash
cp -R meilisearch meilisearch-cpy
cd meilisearch-cpy
rm -rf .git
cargo clean
cargo run --release
<snip>
Compiling meilisearch-http v0.21.0 (/Users/clementrenault/Documents/meilisearch-cpy/meilisearch-http)
warning: vergen: could not find repository from '/Users/clementrenault/Documents/meilisearch-cpy/meilisearch-http'; class=Repository (6); code=NotFound (-3)
```
```
xh localhost:7700/version
HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 26 Aug 2021 13:46:33 GMT
Transfer-Encoding: chunked
{
"commitSha": "unknown",
"commitDate": "unknown",
"pkgVersion": "0.21.0"
}
```
Co-authored-by: Kerollmops <clement@meilisearch.com>
332: Sortable attributes in http-ui r=Kerollmops a=irevoire
- Add a `reset_sortable_attribute` method
- Add the `sortable_attributes` to http-ui
- Fix some broken test in http-ui
Co-authored-by: Tamo <tamo@meilisearch.com>
1615: Integrate the query time sort feature r=Kerollmops a=Kerollmops
This pull request integrates the sort at query time feature that was implemented on the milli side https://github.com/meilisearch/milli/pull/320. It follows the specification file https://github.com/meilisearch/specifications/blob/develop/text/0055-sort.md.
A bunch of tests has been added to ensure that the search works correctly and that the settings are fine too!
Co-authored-by: Kerollmops <clement@meilisearch.com>
