mirror of
https://github.com/meilisearch/meilisearch.git
synced 2024-11-29 16:45:30 +08:00
b6fc181993
4304: Add CUDA GPU support for Hugging Face embedders r=Kerollmops a=dureuill Adds a "cuda" feature to `milli`. Compiling with this feature requires that the CUDA support library be installed (see "with CUDA support" paragraph in https://huggingface.github.io/candle/guide/installation.html), and adds CUDA support to the `huggingFace` embedder. To enable GPU support, users will need to: 1. Have a compatible NVidia GPU under Linux 2. Follow [the guide](https://huggingface.github.io/candle/guide/installation.html) to install the CUDA dependencies 3. Compile Meilisearch with the `cuda` feature: `cargo build --release --features cuda` # Impact Enabling the CUDA feature allows to use an available GPU to compute embeddings with a `huggingFace` embedder. On an AWS Graviton 2, this yields a x3 - x5 improvement on indexing time. # Technical details - I had to change the CI so that the cuda feature is not included in the `Tests all features` workflow - To achieve that, I had to add a binary following the `cargo xtask` design pattern, to list all features excepted the cuda one. - I then changed the workflow accordingly (renamed to "Tests almost all features" 😉) - A test run of the new feature was done on a temporary version of this PR that had it enabled for PRs: [See the results here](https://github.com/meilisearch/meilisearch/actions/runs/7461331929/job/20301216732) Co-authored-by: Louis Dureuil <louis@meilisearch.com> |
||
|---|---|---|
| .. | ||
| ISSUE_TEMPLATE | ||
| scripts | ||
| workflows | ||
| dependabot.yml | ||