Commit Graph

9337 Commits

Author SHA1 Message Date
mpostma
629a658c75 bors setup 2020-07-08 09:50:07 +02:00
Kerollmops
11c7fef80a
Implement a memory dumper
It moves the in memory HashMaps used when indexing to a disk based MTBL file
2020-07-07 16:48:49 +02:00
Kerollmops
b12bfcb03b
Reduce the deepness of the word position document ids
This helps reduce the number of allocations.
2020-07-07 12:30:05 +02:00
Kerollmops
7178b6c2c4
First basic version using MTBL again 2020-07-07 11:32:33 +02:00
Kerollmops
45d0d7c3d4
Clean up the README 2020-07-06 17:38:22 +02:00
Kerollmops
adb1038b26
Add a jobs parameter to set the number of threads the indexer uses 2020-07-06 12:17:17 +02:00
Kerollmops
2a3b03138b
Use heed 0.8.1 with the RwIter append method 2020-07-05 19:50:28 +02:00
Kerollmops
ec1023e790
Intersect document ids by inverse popularity of the words
This reduces the worst request we had which took 56s to now took 3s ("the best of the do").
2020-07-05 19:33:51 +02:00
Kerollmops
cd7e64b2b3
Allow users to set the arc cache size when indexing 2020-07-04 18:12:41 +02:00
Kerollmops
ac8353a64f
Merge pre-computed word attribute documents ids 2020-07-04 17:02:27 +02:00
Kerollmops
fea7cac206
Display the time it took to compute the word attribute documents ids 2020-07-04 15:18:38 +02:00
Kerollmops
46ced5c828
Introduce the RwIter append heed API 2020-07-04 12:34:10 +02:00
Clément Renault
2f6c55ef78
Merge pull request #771 from MarinPostma/placeholder-search
Placeholder search
2020-07-03 18:56:55 +02:00
mpostma
a6457718f2 update changelog 2020-07-03 17:17:28 +02:00
mpostma
3bf23a7c59 test placeholder search
move search test macro to common module
2020-07-03 17:17:28 +02:00
mpostma
bbe3a10107 implement placeholder search 2020-07-03 17:17:28 +02:00
Clément Renault
37ee0f36c1
Merge pull request #792 from MarinPostma/error-codes-in-updates
Error codes in updates
2020-07-02 16:17:57 +02:00
mpostma
e92f544fd1 add test for update errors 2020-07-02 15:18:30 +02:00
mpostma
d7b49fa671 fix potential infinite loop 2020-07-02 15:18:30 +02:00
mpostma
41707e3245 fix error on missing document id in document 2020-07-02 15:18:30 +02:00
mpostma
3c51e9f5ed Enable error code reporting for update errors 2020-07-02 15:18:30 +02:00
mpostma
7d3e937134 add tests for error codes 2020-07-02 15:18:30 +02:00
mpostma
6445eea946 update error types to be more accurate 2020-07-02 15:18:28 +02:00
mpostma
ced6cc0e23 fix bad error report when primary key exists 2020-07-02 15:16:48 +02:00
Clément Renault
944a3943e5
Merge pull request #820 from MarinPostma/readme-update
update readme
2020-07-02 15:16:37 +02:00
mpostma
d419f151a0 update readme 2020-07-02 15:14:05 +02:00
Clément Renault
b2124822a3
Merge pull request #825 from Rio/log-analytics-usage
feat(analytics): log if analytics are enabled
2020-07-02 15:02:19 +02:00
Rio Kierkels
f60b912f12
feat(analytics): log if analytics are enabled 2020-07-02 14:33:25 +02:00
Clément Renault
e1f956ce18
Merge pull request #821 from aeriksson/patch-1
Fix typo in option.rs
2020-07-02 14:05:00 +02:00
marin
ab16e2eff1
fix merge error 2020-07-02 14:04:15 +02:00
marin
3da607749f
Merge branch 'master' into patch-1 2020-07-02 13:57:52 +02:00
Clément Renault
a626e5e935
Merge pull request #737 from balajisivaraman/wip_655
Improve test suite performance using Test Dataset
2020-07-02 13:51:38 +02:00
Balaji Sivaraman
3d73a4895e cleanup movies dataset and related functions 2020-07-02 16:52:39 +05:30
Balaji Sivaraman
979b01a1c0 update index status test to use the test dataset 2020-07-02 16:52:39 +05:30
Balaji Sivaraman
38cf489acf update remaining search tests to use the test dataset 2020-07-02 16:52:39 +05:30
Balaji Sivaraman
60264763f4 update search_settings tests to use the test dataset 2020-07-02 16:52:39 +05:30
Balaji Sivaraman
d55124e524 update settings_ranking_rules tests to use the test dataset 2020-07-02 16:52:39 +05:30
Balaji Sivaraman
643933c3b0 update settings tests to use the test dataset 2020-07-02 16:52:39 +05:30
Balaji Sivaraman
44fd9384bd update stop_words tests to use the test dataset 2020-07-02 16:52:39 +05:30
Balaji Sivaraman
75d0d2df6c update documents_delete tests to use the test dataset 2020-07-02 16:52:39 +05:30
Kerollmops
7e7440c431
Finalize the LMDB indexing design 2020-07-01 22:45:43 +02:00
Kerollmops
2ae3f40971
Make the indexer ignore certain words
This is a preparation for making the indexing fully parallel by making the
indexer only be aware of certain words for each threads to avoid postings lists
conflicts for each words
2020-07-01 17:49:46 +02:00
Kerollmops
a3ac2623d5
Introduce multiple functions to clean up the code 2020-07-01 17:24:55 +02:00
Kerollmops
ac5cc7ddad
Introduce an Iterator yielding owned entries for the LruCache 2020-07-01 17:21:52 +02:00
Clément Renault
92d9283d1a
Merge pull request #823 from Rio/public-health-endpoint
chore(http): do not require auth on /health endpoint
2020-07-01 17:01:23 +02:00
Rio Kierkels
9b46887f75
chore(http): do not require auth on /health endpoint
This makes it easier to determine the health of the server using http.

closes #822
2020-07-01 16:33:01 +02:00
Clément Renault
ad267cbe59
Merge pull request #813 from Rio/remove-hardcoded-sentry-dsn
feat(sentry): make sentry dsn customizable
2020-07-01 16:15:21 +02:00
André Eriksson
029772e11f
Fix typo in option.rs 2020-07-01 13:45:00 +02:00
Rio Kierkels
2ef888d100
chore(sentry): make sentry dsn customizable
By removing the hardcoded value the sentry client will fall back to pulling
it from the SENTRY_DSN environment variable. The hardcoded value has been
moved to the default value of the commandline options so the default
behavior will be the same.

A `--no-sentry` and `MEILI_NO_SENTRY` option has also been introduced
that effectively disables sentry reporting.
2020-07-01 12:55:14 +02:00
Kerollmops
014a25697d
Use only one ARC cache based on the words 2020-07-01 12:03:18 +02:00