Tamo
69edbf9f6d
Update milli/src/update/delete_documents.rs
2022-12-19 18:23:50 +01:00
Louis Dureuil
ad9937c755
Fix tests after adding DeletionStrategy
2022-12-19 10:07:17 +01:00
Louis Dureuil
171c942282
Soft-deletion computation no longer takes into account the mapsize
...
Implemented solution 2.3 from https://github.com/meilisearch/meilisearch/issues/3231#issuecomment-1348628824
2022-12-19 10:07:17 +01:00
Louis Dureuil
e2ae3b24aa
Hard or soft delete according to the deletion strategy
2022-12-19 10:00:13 +01:00
Louis Dureuil
fc7618d49b
Add DeletionStrategy
2022-12-19 09:49:58 +01:00
Loïc Lecrenier
be3b00350c
Apply review suggestions: naming and documentation
2022-12-13 10:15:22 +01:00
Loïc Lecrenier
e3ee553dcc
Remove soft deleted ids from ExternalDocumentIds during document import
...
If the document import replaces a document using hard deletion
2022-12-12 14:16:09 +01:00
Loïc Lecrenier
f2cf981641
Add more tests and allow disabling of soft-deletion outside of tests
...
Also allow disabling soft-deletion in the IndexDocumentsConfig
2022-12-05 10:51:01 +01:00
unvalley
f4ec1abb9b
Fix all clippy error after conflicts
2022-10-27 23:58:13 +09:00
unvalley
811f156031
Execute cargo clippy --fix
2022-10-27 01:00:00 +09:00
Loïc Lecrenier
54c0cf93fe
Merge remote-tracking branch 'origin/main' into facet-levels-refactor
2022-10-26 15:13:34 +02:00
Loïc Lecrenier
f198b20c42
Add facet deletion tests that use both the incremental and bulk methods
...
+ update deletion snapshots to the new database format
2022-10-26 13:47:46 +02:00
Loïc Lecrenier
e3ba1fc883
Make deletion tests for both soft-deletion and hard-deletion
2022-10-26 13:47:46 +02:00
Loïc Lecrenier
ab5e56fd16
Add document deletion snapshot tests and tests for hard-deletion
2022-10-26 13:47:46 +02:00
Loïc Lecrenier
d885de1600
Add option to avoid soft deletion of documents
2022-10-26 13:47:46 +02:00
Loïc Lecrenier
1165ba2171
Make facet deletion incremental
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
cb8442a119
Further unify facet databases of f64s and strings
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
9026867d17
Give same interface to bulk and incremental facet indexing types
...
+ cargo fmt, oops, sorry for the bad history :(
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
485a72306d
Refactor facet-related codecs
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
3d145d7f48
Merge the two <facetttype>_faceted_documents_ids methods into one
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
68cbcdf08b
Fix compile errors/warnings in http-ui and infos
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
85824ee203
Try to make facet indexing incremental
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
e8a156d682
Reorganise facets database indexing code
2022-10-26 13:46:46 +02:00
Loïc Lecrenier
39a4a0a362
Reintroduce filter range search and facet extractors
2022-10-26 13:46:14 +02:00
Loïc Lecrenier
22d80eeaf9
Reintroduce facet deletion functionality
2022-10-26 13:46:14 +02:00
Loïc Lecrenier
6cc91824c1
Remove unused heed codec files
2022-10-26 13:46:14 +02:00
Loïc Lecrenier
c3f49f766d
Prepare refactor of facets database
...
Prepare refactor of facets database
2022-10-26 13:46:14 +02:00
Ewan Higgs
6b2fe94192
Fixes for clippy bringing us down to 18 remaining issues.
...
This brings us a step closer to enforcing clippy on each build.
2022-10-25 20:49:02 +02:00
Loïc Lecrenier
264a04922d
Add prefix_word_pair_proximity database
...
Similar to the word_prefix_pair_proximity one but instead the keys are:
(proximity, prefix, word2)
2022-10-18 10:37:34 +02:00
Irevoire
4aae07d5f5
expose the size methods
2022-08-17 17:07:38 +02:00
Loïc Lecrenier
acff17fb88
Simplify indexing tests
2022-08-04 12:03:13 +02:00
Loïc Lecrenier
07003704a8
Merge branch 'filter/field-exist'
2022-07-21 14:51:41 +02:00
Loïc Lecrenier
453d593ce8
Add a database containing the docids where each field exists
2022-07-19 10:07:33 +02:00
Kerollmops
399eec5c01
Fix the indexation tests
2022-07-12 14:55:51 +02:00
Tamo
250be9fe6c
put the threshold back to 10k
2022-07-05 15:57:44 +02:00
Tamo
eaf28b0628
Apply review suggestions
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-07-05 15:30:33 +02:00
Tamo
3b309f654a
Fasten the document deletion
...
When a document deletion occurs, instead of deleting the document we mark it as deleted
in the new “soft deleted” bitmap. It is then removed from the search, and all the other
endpoints.
2022-07-05 15:30:33 +02:00
ManyTheFish
177154828c
Extends deletion tests
2022-06-13 17:34:16 +02:00
Irevoire
4f3ce6d9cd
nested fields
2022-04-07 16:58:46 +02:00
ad hoc
201fea0fda
limit extract_word_docids memory usage
2022-04-05 14:14:15 +02:00
ad hoc
ab185a59b5
fix infos
2022-04-05 09:46:56 +02:00
ad hoc
6dd2e4ffbd
introduce exact_word_prefix database in index
2022-04-04 20:54:03 +02:00
ad hoc
0a77be4ec0
introduce exact_word_docids db
2022-04-04 20:54:02 +02:00
Kerollmops
21ec334dcc
Fix the compilation error of the dependency versions
2022-03-15 11:17:45 +01:00
Irevoire
48542ac8fd
get rid of chrono in favor of time
2022-02-15 11:41:55 +01:00
Marin Postma
0c84a40298
document batch support
...
reusable transform
rework update api
add indexer config
fix tests
review changes
Co-authored-by: Clément Renault <clement@meilisearch.com>
fmt
2022-01-19 12:40:20 +01:00
Tamo
98a365aaae
store the geopoint in three dimensions
2021-12-14 12:21:24 +01:00
Clément Renault
ef59762d8e
Prefer returning None instead of the Empty Filter state
2021-12-09 11:57:52 +01:00
Marin Postma
6eb47ab792
remove update_id in UpdateBuilder
2021-11-16 13:07:04 +01:00
Marin Postma
721fc294be
improve document deletion returned meta
...
returns both the remaining number of documents and the number of deleted
documents.
2021-11-10 14:08:18 +01:00
