Clément Renault
759b9b1546
Introduce a new custom Lru
2024-09-25 14:49:12 +02:00
ManyTheFish
974272f2e9
Merge branch 'main' into indexer-edition-2024
2024-09-25 07:41:16 +02:00
Clément Renault
92678383d6
Update charabia
2024-09-24 15:37:56 +02:00
Clément Renault
ff931edb55
Update roaring to inline max calls
2024-09-23 16:53:42 +02:00
Clément Renault
4551abf6d4
Update roaring to the latest version
2024-09-23 14:35:33 +02:00
Clément Renault
193d7f5d34
Add the mutualized charabia normalization
2024-09-23 14:24:25 +02:00
meili-bors[bot]
462a2329f1
Merge #4941
...
4941: Implement the binary quantization in meilisearch r=irevoire a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4873
## What does this PR do?
- Add a settings for the binary quantization
- Once enabled, the bq cannot be disabled
TODO:
- [ ] Missing a bunch of tests
Co-authored-by: Tamo <tamo@meilisearch.com>
2024-09-19 15:50:24 +00:00
ManyTheFish
f77661ec44
Update Charabia v0.9.1
2024-09-19 16:08:59 +02:00
Tamo
6dde41cc46
stop using a local version of arroy and instead point to the git repo with the rev
2024-09-19 15:25:38 +02:00
Tamo
cc45e264ca
implement the binary quantization in meilisearch
2024-09-19 12:00:56 +02:00
curquiza
29ff02f3ff
Update version for the next release (v1.11.0) in Cargo.toml
2024-09-17 11:45:48 +00:00
meili-bors[bot]
4b55ba68bc
Merge #4911
...
4911: Bump quinn-proto from 0.11.3 to 0.11.8 r=Kerollmops a=dependabot[bot]
Bumps [quinn-proto](https://github.com/quinn-rs/quinn ) from 0.11.3 to 0.11.8.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/quinn-rs/quinn/releases ">quinn-proto's releases</a>.</em></p>
<blockquote>
<h2>quinn-proto 0.11.5</h2>
<h2>What's Changed</h2>
<ul>
<li>No workspace lints by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1955 ">quinn-rs/quinn#1955</a></li>
</ul>
<h2>quinn-proto 0.11.4</h2>
<h2>What's Changed</h2>
<ul>
<li>Fix panic in example due to unset default crypto provider by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1882 ">quinn-rs/quinn#1882</a></li>
<li>Fix zero-length connection IDs by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1883 ">quinn-rs/quinn#1883</a></li>
<li>Add support for NetBSD, fix OpenBSD by <a href="https://github.com/flub "><code>`@flub</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1884 ">quinn-rs/quinn#1884</a></li>
<li>docs(udp): replace AsRawFd and AsRawSocket with AsFd and AsSocket by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1890 ">quinn-rs/quinn#1890</a></li>
<li>Resolve stopped/received_reset futures on lost connections by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1886 ">quinn-rs/quinn#1886</a></li>
<li>Bump version numbers (quinn 0.11.2, -proto 0.11.3) by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1891 ">quinn-rs/quinn#1891</a></li>
<li>udp: bump version to 0.5.2 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1892 ">quinn-rs/quinn#1892</a></li>
<li>docs(quinn): Clarify effects of setting AckFrequencyConfig by <a href="https://github.com/gretchenfrage "><code>`@gretchenfrage</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1894 ">quinn-rs/quinn#1894</a></li>
<li>Apply clippy suggestions from Rust 1.79 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1895 ">quinn-rs/quinn#1895</a></li>
<li>Only send MAX_STREAMS when >1/8 of flow control window is consumed by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1898 ">quinn-rs/quinn#1898</a></li>
<li>fix: remove unused dependency tracing-attributes by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1903 ">quinn-rs/quinn#1903</a></li>
<li>proto: make initial destination cid configurable by <a href="https://github.com/thynson "><code>`@thynson</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1897 ">quinn-rs/quinn#1897</a></li>
<li>Allow configuring rng seed through <code>EndpointConfig</code> by <a href="https://github.com/aochagavia "><code>`@aochagavia</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1901 ">quinn-rs/quinn#1901</a></li>
<li>quinn: introduce waking helpers by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1908 ">quinn-rs/quinn#1908</a></li>
<li>Wake blocked streams on 0-RTT rejection by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1905 ">quinn-rs/quinn#1905</a></li>
<li>Upgrade to rustc-hash 2 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1909 ">quinn-rs/quinn#1909</a></li>
<li>Fix unnecessary Incoming warning on Endpoint drop by <a href="https://github.com/gretchenfrage "><code>`@gretchenfrage</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1907 ">quinn-rs/quinn#1907</a></li>
<li>Revise and add additional 0-rtt doc comments by <a href="https://github.com/gretchenfrage "><code>`@gretchenfrage</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1826 ">quinn-rs/quinn#1826</a></li>
<li>docs: remove reference to sendmmsg by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1914 ">quinn-rs/quinn#1914</a></li>
<li>Fix debug assert with reordered ACKs by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1893 ">quinn-rs/quinn#1893</a></li>
<li>quinn: Make <code>Endpoint::client</code> dual-stack V6 by default by <a href="https://github.com/gretchenfrage "><code>`@gretchenfrage</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1913 ">quinn-rs/quinn#1913</a></li>
<li>bench(udp): measure non-GSO & GSO on localhost by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1915 ">quinn-rs/quinn#1915</a></li>
<li>proto: avoid overflow in handshake done statistic by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1918 ">quinn-rs/quinn#1918</a></li>
<li>Use workspace dependencies for all external dependencies by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1919 ">quinn-rs/quinn#1919</a></li>
<li>Fix lack of reexport of ConnectionStats and ResetError by <a href="https://github.com/TirushOne "><code>`@TirushOne</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1920 ">quinn-rs/quinn#1920</a></li>
<li>[non-breaking] deps(udp): make tracing optional and add optional log by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1923 ">quinn-rs/quinn#1923</a></li>
<li>fix(udp): feature flag tracing in windows.rs by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1932 ">quinn-rs/quinn#1932</a></li>
<li>Bump MSRV to 1.70 following tokio 1.39 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1939 ">quinn-rs/quinn#1939</a></li>
<li>Raise default idle timeout to 30 seconds by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1938 ">quinn-rs/quinn#1938</a></li>
<li>Discard pre-handshake packets after the handshake by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1937 ">quinn-rs/quinn#1937</a></li>
<li>Apply suggestions from Clippy 1.80 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1941 ">quinn-rs/quinn#1941</a></li>
<li>chore(quinn): feature flag socket2 imports by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1933 ">quinn-rs/quinn#1933</a></li>
<li>refactor: move rust-version to workspace Cargo.toml by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1940 ">quinn-rs/quinn#1940</a></li>
<li>chore: move common package data to workspace Cargo.toml by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1943 ">quinn-rs/quinn#1943</a></li>
<li>Endpoint stats interface by <a href="https://github.com/ryleung-solana "><code>`@ryleung-solana</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1900 ">quinn-rs/quinn#1900</a></li>
<li>Expose the Handshake Confirmed state by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1944 ">quinn-rs/quinn#1944</a></li>
<li>Exclude metrics with freestanding getters from EndpointStats by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1945 ">quinn-rs/quinn#1945</a></li>
<li>Fix incorrect initial DCID indexing on retried connections by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1946 ">quinn-rs/quinn#1946</a></li>
<li>Add expect message to unwrap in PacketBuilder by <a href="https://github.com/casey "><code>`@casey</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1951 ">quinn-rs/quinn#1951</a></li>
<li>Revert "proto: yield transport error for Initial packets with no CRYPTO" by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1952 ">quinn-rs/quinn#1952</a></li>
<li>refactor(udp): introduce log facade by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1935 ">quinn-rs/quinn#1935</a></li>
<li>Update cargo-deny-action to v2 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1953 ">quinn-rs/quinn#1953</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="7c09b02073https://redirect.github.com/quinn-rs/quinn/issues/1981 ">#1981</a>)</li>
<li><a href="59bccd2e7ea8ec510fd1c26e8cd2f7e01609ccd8c292a3c6a6bb02a12a845e5cc93645cef42cccefedf16a6f10https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.3...quinn-proto-0.11.8 ">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/meilisearch/meilisearch/network/alerts ).
</details>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-16 13:32:32 +00:00
Clément Renault
72c6a21a30
Use raw JSON to read the payloads
2024-09-05 20:08:23 +02:00
meili-bors[bot]
db0cf3b2ed
Merge #4912
...
4912: Allow Meilitool to dumplessly, offline upgrade v1.9 -> v1.10 in some conditions r=Kerollmops a=dureuill
- bail early if the DB contains at least 1 REST embedder, providing the list of detected REST embedders, and without modifying the DB
- Might depend on the feature set that meilitool was compiled with and the featureset that the Meilisearch that created the DB was compiled with 💀 . In case of runtime error, try again with a different feature set (passing or not passing `-p meilitool` when building after a `cargo clean`)
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-09-05 09:11:23 +00:00
Louis Dureuil
d352b1ee83
Add serde to meilitool
2024-09-05 10:07:33 +02:00
dependabot[bot]
3f3cebf5f9
Bump quinn-proto from 0.11.3 to 0.11.8
...
Bumps [quinn-proto](https://github.com/quinn-rs/quinn ) from 0.11.3 to 0.11.8.
- [Release notes](https://github.com/quinn-rs/quinn/releases )
- [Commits](https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.3...quinn-proto-0.11.8 )
---
updated-dependencies:
- dependency-name: quinn-proto
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-09-03 20:50:30 +00:00
Clément Renault
bcb1aa3d22
Find a temporary solution to par into iter on an HashMap
...
Spoiler: Do not use an HashMap but drain it into a Vec
2024-09-02 19:39:48 +02:00
irevoire
42e7499260
Update version for the next release (v1.10.1) in Cargo.toml
2024-09-02 16:27:51 +02:00
Clément Renault
794ebcd582
Replace grenad with the new grenad various-improvement branch
2024-08-30 11:53:59 +02:00
Clément Renault
b7c77c7a39
Use the latest version of the obkv crate
2024-08-30 11:53:59 +02:00
Clément Renault
0c57cf7565
Replace obkv with the temporary new version of it
2024-08-30 11:53:58 +02:00
Clément Renault
874c1ac538
First channels types
2024-08-30 11:53:58 +02:00
Louis Dureuil
c683fa98e6
WIP
...
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2024-08-30 11:53:57 +02:00
ManyTheFish
35567b2137
Update Charabia v0.9.0
2024-07-25 16:02:14 +02:00
ManyTheFish
4fbe048cbf
Update Cargo.lock
2024-07-25 10:52:56 +02:00
ManyTheFish
cc02920f2b
Update charabia
2024-07-25 10:51:27 +02:00
Tamo
988552e178
add tests on the rest embedder
2024-07-24 14:34:17 +02:00
curquiza
50b7093f8e
Update version for the next release (v1.10.0) in Cargo.toml
2024-07-22 13:54:38 +00:00
meili-bors[bot]
6e9d0de8b7
Merge #4806
...
4806: Update rustls as much as possible r=Kerollmops a=irevoire
# Pull Request
## Related issue
Part of https://github.com/meilisearch/meilisearch/issues/4753
## What does this PR do?
- Update rustls as much as possible
## What is missing
In rustls-0.22.0 two structures we were using have been removed with no explanation or workaround
<img width="518" alt="image" src="https://github.com/user-attachments/assets/fa112db1-3400-4163-8819-7913f22d6b87 ">
Co-authored-by: Tamo <tamo@meilisearch.com>
2024-07-17 17:00:01 +00:00
Tamo
1bfb16386c
Update rustls as much as possible
2024-07-17 18:21:26 +02:00
Tamo
2af9481804
Implements the experimental contains filter operator«
2024-07-17 11:13:37 +02:00
Louis Dureuil
12a7a45930
Add roaring to meilisearch
2024-07-11 16:27:50 +02:00
Clément Renault
837274f853
Restrict even more the Rhai engine
2024-07-10 16:30:18 +02:00
Clément Renault
81ec0abad1
Use the new rayon-par-bridge library
2024-07-10 16:29:04 +02:00
Clément Renault
2d97164d9f
It works perfectly with some Rhai
2024-07-10 16:28:13 +02:00
Clément Renault
efc156a4a4
Executing Lua works correctly
2024-07-10 16:27:36 +02:00
Clément Renault
98cd6a865c
Update dependencies after removing useless ones
2024-07-10 13:37:24 +02:00
Clément Renault
5f4530ce57
Remove more unused dependencies
2024-07-10 13:36:34 +02:00
Tamo
952e742321
update charabia
2024-07-09 23:41:29 +02:00
Tamo
43db4f4242
update fxprof_processed_profile
2024-07-09 23:41:29 +02:00
Tamo
9feba5028d
update byte-unit
2024-07-09 23:41:29 +02:00
Tamo
4aa7d386d8
remove http and uses actix_web::http instead
2024-07-08 21:17:10 +02:00
Tamo
6afa578688
update most incompatible dependencies
2024-07-08 18:31:15 +02:00
Tamo
300bdfc2a7
update most dependencies
2024-07-08 18:09:12 +02:00
ManyTheFish
015d90a962
merge main
2024-07-01 11:50:36 +02:00
Tamo
a108d8f6f3
update yaup
2024-06-26 16:03:51 +02:00
Tamo
1693332cab
Update arroy and always build the tree that need to be built
2024-06-24 10:14:03 +02:00
meili-bors[bot]
e9bf4c43a4
Merge #4649
...
4649: Don't store the vectors in the documents database r=dureuill a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4607
## What does this PR do?
- Ensure that anything falling under `_vectors` is NOT searchable, filterable or sortable
- [x] per embedder, add a roaring bitmap of documents that provide "userProvided" embeddings
- [x] in the indexing process in extract_vector_points, set the bit corresponding to the document depending on the "userProvided" subfield in the _vectors field.
- [x] in the document DB in typed chunks, when writing the _vectors field, remove all keys corresponding to an embedder
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-17 12:32:03 +00:00
Clément Renault
8ec6e175e5
Replace roaring patch to the v0.10.5
2024-06-07 22:11:26 -04:00
Clément Renault
40f05fe156
Bump roaring to the latest commit
2024-06-06 10:59:55 -04:00
