mirror of
https://github.com/meilisearch/meilisearch.git
synced 2024-11-26 12:05:05 +08:00
049bd45849
4371: Fixes embedder issues r=irevoire a=dureuill # Pull Request ## Related issue Fixes #4361 Fixes #4370 ## What does this PR do? - Truncate tokens to 512 for Hugging Face embedders - Move the tokio runtime to OpenAI so that we no longer have a thread with rayon -> tokio -> rayon - Spawn a new reqwest client after each new runtime to avoid spurious runtime error ## Manual tests - embedding failing document from `@CaroFG` with hugging face - embedding movies with hugging face - embedding and searching movies with openai Co-authored-by: Louis Dureuil <louis@meilisearch.com> |
||
|---|---|---|
| .. | ||
| examples | ||
| src | ||
| tests | ||
| Cargo.toml | ||
| README.md | ||
a concurrent indexer combined with fast and relevant search algorithms
Introduction
This crate contains the internal engine used by Meilisearch.
It contains a library that can manage one and only one index. Meilisearch manages the multi-index itself. Milli is unable to store updates in a store: it is the job of something else above and this is why it is only able to process one update at a time.