732: Interpret synonyms as phrases r=loiclec a=loiclec

# Pull Request

## Related issue
Fixes (when merged into meilisearch) https://github.com/meilisearch/meilisearch/issues/3125

## What does this PR do?
We now map multi-word synonyms to phrases instead of loose words. Such that the request:
```
btw I am going to nyc soon
```
is interpreted as (when the synonym interpretation is chosen for both `btw` and `nyc`):
```
"by the way" I am going to "New York City" soon
```
instead of:
```
by the way I am going to New York City soon
```

This prevents queries containing multi-word synonyms to exceed to word length limit and degrade the search performance.

In terms of relevancy, there is a debate to have. I personally think this could be considered an improvement, since it would be strange for a user to search for:
```
good DIY project
```
and have a result such as:
```
{
    "text": "whether it is a good project to do, you'll have to decide for yourself"
}
```
However, for synonyms such as `NYC -> New York City`, then we will stop matching documents where `New York` is separated from `City`. This is however solvable by adding an additional mapping: `NYC -> New York`.

## Performance

With the old behaviour, some long search requests making heavy uses of synonyms could take minutes to be executed. This is no longer the case, these search requests now take an average amount of time to be resolved.

Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
This commit is contained in:
bors[bot] 2023-01-04 08:34:18 +00:00 committed by GitHub
commit 49f58b2c47
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -339,18 +339,18 @@ fn typos(word: String, authorize_typos: bool, config: TypoConfig) -> QueryKind {
/// and create the list of operations for the query tree /// and create the list of operations for the query tree
fn synonyms(ctx: &impl Context, word: &[&str]) -> heed::Result<Option<Vec<Operation>>> { fn synonyms(ctx: &impl Context, word: &[&str]) -> heed::Result<Option<Vec<Operation>>> {
let synonyms = ctx.synonyms(word)?; let synonyms = ctx.synonyms(word)?;
Ok(synonyms.map(|synonyms| { Ok(synonyms.map(|synonyms| {
synonyms synonyms
.into_iter() .into_iter()
.map(|synonym| { .map(|synonym| {
let words = synonym if synonym.len() == 1 {
.into_iter() Operation::Query(Query {
.map(|word| { prefix: false,
Operation::Query(Query { prefix: false, kind: QueryKind::exact(word) }) kind: QueryKind::exact(synonym[0].clone()),
}) })
.collect(); } else {
Operation::and(words) Operation::Phrase(synonym.into_iter().map(Some).collect())
}
}) })
.collect() .collect()
})) }))
@ -1058,9 +1058,7 @@ mod test {
AND AND
OR OR
Exact { word: "hi" } Exact { word: "hi" }
AND PHRASE [Some("good"), Some("morning")]
Exact { word: "good" }
Exact { word: "morning" }
Tolerant { word: "hello", max typo: 1 } Tolerant { word: "hello", max typo: 1 }
OR OR
Exact { word: "earth" } Exact { word: "earth" }
@ -1070,6 +1068,24 @@ mod test {
"###); "###);
} }
#[test]
fn simple_synonyms() {
let query = "nyc";
let tokens = query.tokenize();
let (query_tree, _) = TestContext::default()
.build(TermsMatchingStrategy::Last, true, None, tokens)
.unwrap()
.unwrap();
insta::assert_debug_snapshot!(query_tree, @r###"
OR
PHRASE [Some("new"), Some("york")]
PHRASE [Some("new"), Some("york"), Some("city")]
PrefixExact { word: "nyc" }
"###);
}
#[test] #[test]
fn complex_synonyms() { fn complex_synonyms() {
let query = "new york city "; let query = "new york city ";
@ -1092,16 +1108,11 @@ mod test {
AND AND
OR OR
Exact { word: "nyc" } Exact { word: "nyc" }
AND PHRASE [Some("new"), Some("york"), Some("city")]
Exact { word: "new" }
Exact { word: "york" }
Exact { word: "city" }
Tolerant { word: "newyork", max typo: 1 } Tolerant { word: "newyork", max typo: 1 }
Exact { word: "city" } Exact { word: "city" }
Exact { word: "nyc" } Exact { word: "nyc" }
AND PHRASE [Some("new"), Some("york")]
Exact { word: "new" }
Exact { word: "york" }
Tolerant { word: "newyorkcity", max typo: 1 } Tolerant { word: "newyorkcity", max typo: 1 }
"###); "###);
} }