Commit Graph

10104 Commits

Author SHA1 Message Date
ManyTheFish
ad52c950ba
Only run word pair proximity docids extraction if proximity_precision enables it 2024-11-05 11:08:47 +01:00
Many the fish
3d29226a7f
Merge pull request #5019 from meilisearch/indexer-edition-2024-bumpalo-in-extractors
Implement facet search extraction
2024-10-23 10:42:38 +02:00
ManyTheFish
60cc09abec Implement facet search exctraction 2024-10-21 09:28:49 +02:00
Louis Dureuil
c1fcb2ebc6
add some warning 2024-10-17 09:43:11 +02:00
Louis Dureuil
0749633618
Don't sort in parallel in sorters of the new indexer 2024-10-17 09:30:18 +02:00
Louis Dureuil
0647f75e6b
Add borrow_mut_or_yield extension method 2024-10-16 17:36:41 +02:00
ManyTheFish
86a0097311 Use bumpalo in word docids 2024-10-16 14:04:44 +02:00
Louis Dureuil
c75de1f391
Remove TODO 2024-10-16 11:18:59 +02:00
Louis Dureuil
198238687f
Guess and retrieve primary key correctly in batch 2024-10-16 09:27:18 +02:00
Louis Dureuil
f9a6c624a7
Put primary key, and use provided key in operation 2024-10-16 09:27:00 +02:00
Louis Dureuil
017757004e
Add PrimaryKey::new_or_insert 2024-10-16 09:26:18 +02:00
Louis Dureuil
152683083b
Change document operation to use method in primary key 2024-10-15 14:08:37 +02:00
Louis Dureuil
c283c95f6a
Support nested primary keys 2024-10-15 14:08:37 +02:00
Clément Renault
9a0e1dc375
Fix the prefix deletion 2024-10-15 11:20:09 +02:00
Clément Renault
1e81d72b5f
Use the fixed version of the Rhai crate 2024-10-14 18:18:59 +02:00
Clément Renault
52b95c4e59
Make sure we edit the task statuses 2024-10-14 16:48:15 +02:00
Clément Renault
7e1dc8439b
Introduce the new update by function 2024-10-14 16:32:50 +02:00
Louis Dureuil
96658ec775
Make de public 2024-10-14 15:41:58 +02:00
Louis Dureuil
c01ee7b732
external changes 2024-10-14 15:41:58 +02:00
Louis Dureuil
6ad3f57bc1
Changes to de 2024-10-14 15:41:58 +02:00
Louis Dureuil
28d92c521a
External docids to &'bump str 2024-10-14 15:41:58 +02:00
Louis Dureuil
7df20d8282
Changes to primary key 2024-10-14 15:41:57 +02:00
ManyTheFish
b4102741e6 Fix duplicated fields when a document is modified 2024-10-14 14:59:40 +02:00
ManyTheFish
a525598ad6 Fix facet string indexing 2024-10-14 11:12:10 +02:00
ManyTheFish
4e97e38177 Serialize docids bitmap one time 2024-10-14 11:12:10 +02:00
ManyTheFish
d675e73af1 Finish prefix databases 2024-10-14 11:12:10 +02:00
ManyTheFish
a2fbf2ea21 set updated at at the end of the indexing 2024-10-14 11:05:25 +02:00
ManyTheFish
132916f62c Only run word pair proximity docids extraction if proximity_precision enables it 2024-10-14 11:05:25 +02:00
Louis Dureuil
8371819114
Some clippy related fixes 2024-10-14 10:58:37 +02:00
Louis Dureuil
6028d6ba43
Remove somme warnings 2024-10-10 22:42:37 +02:00
Louis Dureuil
68a2502388
Introduce indexer level bumpalo 2024-10-10 22:23:05 +02:00
Clément Renault
39b27e42be
Plug the deletion pipeline 2024-10-08 16:04:19 +02:00
Clément Renault
470c2272dd
Show much more stats about the LRU caches 2024-10-08 15:29:24 +02:00
Clément Renault
2230674c0a
Merge branch 'fix-append-only-vec' into indexer-edition-2024 2024-10-08 10:32:45 +02:00
Clément Renault
eb09dfed04
Avoid reallocation with the ThreadLocal pool 2024-10-07 16:41:17 +02:00
Clément Renault
83c09d0db0
Remove the now, useless AppendOnlyVec library 2024-10-07 16:38:45 +02:00
ManyTheFish
c11b7e5c0f Reduce number of cache created by using thread_local 2024-10-07 15:58:16 +02:00
Many the fish
dff2d54784
Merge pull request #4976 from meilisearch/fix-append-only-vec
Fix append only `Vec` by using a `LinkedList`
2024-10-03 17:26:00 +02:00
Clément Renault
58d96fbea3
Rename Node parent to next 2024-10-03 16:15:05 +02:00
Clément Renault
4665bfcb19
Move the parent assignation before the exchange operation 2024-10-03 16:14:23 +02:00
Clément Renault
a7a01646cf
Remove the useless Manually drop 2024-10-03 15:57:31 +02:00
Clément Renault
0409a26cd8
Replace the concurrent vec by a linked list 2024-10-03 15:15:29 +02:00
ManyTheFish
35f78b5423 TO REMOVE: usefull debug prints 2024-10-03 11:13:01 +02:00
ManyTheFish
14261f8f04 Integrate facet level bulk update
Only the facet bulk update has been added so far, the incremental must be completely rewritten

Factorize facet merging

Fix facet level extraction
2024-10-03 11:13:00 +02:00
ManyTheFish
774ed28539 Fix Prefix FST when a document is modified 2024-10-03 11:12:26 +02:00
ManyTheFish
d79f75f630 Compute and Write external-documents-ids database 2024-10-03 11:11:56 +02:00
Clément Renault
ccf01c2471
Merge pull request #4969 from meilisearch/indexer-edition-2024-try-map
Indexer edition 2024 try map
2024-10-02 11:25:05 +02:00
Clément Renault
b7a5ba100e
Move the ParallelIteratorExt into the parallel_iterator_ext module 2024-10-01 11:11:52 +02:00
Clément Renault
dead7a56a3
Keep the caches in the AppendOnlyVec 2024-10-01 11:11:39 +02:00
Clément Renault
0a8cb471df
Introduce the AppendOnlyVec struct for the parallel computing 2024-10-01 11:11:25 +02:00