snowykami/meilisearch
1
0
Fork 0
mirror of https://github.com/meilisearch/meilisearch.git synced 2024-11-23 10:37:41 +08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
4,551 Commits 165 Branches 386 Tags 80 MiB
5023d36ee7
Commit Graph

1135 Commits

Author SHA1 Message Date
Irevoire
a85d5b4981
test the details of all tasks type 2022-10-27 11:35:16 +02:00
Kerollmops
4736e00253
Handle the CLI options related to snapshots 2022-10-27 11:35:15 +02:00
Kerollmops
942b7c338b
Compress the snapshot in a tarball 2022-10-27 11:35:15 +02:00
Kerollmops
4cafc63561
Reintroduce the versioning functions 2022-10-27 11:35:14 +02:00
Kerollmops
89e127e4f4
Declare the auth path in the index scheduler 2022-10-27 11:35:14 +02:00
Kerollmops
c063f154fb
Add the snapshots directory path to the IndexScheduler 2022-10-27 11:35:14 +02:00
Kerollmops
c641888a23
Patch the delete and cancel tasks routes 2022-10-27 11:35:13 +02:00
Irevoire
12669bf07c
rename received_documents_ids to matched_documents 2022-10-27 11:35:12 +02:00
Irevoire
0aca5e84b9
rename received_document_ids to matched_documents in the DocumentDeletion task type (reimplementation of #2826) 2022-10-27 11:35:12 +02:00
Irevoire
7e52f1effb
remove a lot of unecessary clone and ref 2022-10-27 11:35:11 +02:00
Irevoire
ecf4e43b3d
rename the dumpExport to dumpCreation 2022-10-27 11:35:10 +02:00
Irevoire
3ea489421e
move the error types to meilisearch-http 2022-10-27 11:35:10 +02:00
Loïc Lecrenier
2808be9d45
Fix the /swap-indexes route API 
1. payload
2. error messages
3. auth errors
 2022-10-27 11:35:10 +02:00
Irevoire
8a23e707c1
fix the task view and forward the task db size 2022-10-27 11:35:09 +02:00
Irevoire
735a5da257
reformat 2022-10-27 11:35:09 +02:00
Irevoire
e9055f5572
fix clippy 2022-10-27 11:35:08 +02:00
Irevoire
2c0fde4a1a
ignore the snapshot test 2022-10-27 11:35:08 +02:00
Irevoire
c8ee453b6c
fix the autobatched document deletion 2022-10-27 11:35:07 +02:00
Irevoire
f6963f9662
ensure the indexUid is valid in most cases 2022-10-27 11:35:07 +02:00
Irevoire
b3265a8e1f
ensure the index_uid is valid when creating an index 2022-10-27 11:35:07 +02:00
Irevoire
cb48a02f94
fix the invalid index uid errors 2022-10-27 11:35:06 +02:00
Irevoire
99144b1419
fix most content file error 2022-10-27 11:35:06 +02:00
Irevoire
e107f1b282
fix the payload too large error 2022-10-27 11:35:06 +02:00
Irevoire
1bef5d119d
fix the api keys for the tasks route 2022-10-27 11:35:06 +02:00
Irevoire
ca4234b445
fix the deletion of the data.ms in case of failure 2022-10-27 11:35:06 +02:00
Irevoire
8d1408c65e
fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting 2022-10-27 11:35:05 +02:00
Irevoire
131fe30934
fix the error messages and the index stats 2022-10-27 11:35:05 +02:00
Clément Renault
32cfac0cfd
Sort the TOML dependencies 2022-10-27 11:35:05 +02:00
Clément Renault
80b2e70ee7
Introduce a rustfmt file 2022-10-27 11:35:05 +02:00
Clément Renault
52e858a588
Reapply #2890 2022-10-27 11:34:18 +02:00
Clément Renault
8b0427f0c4
Reapply #2839 2022-10-27 11:34:18 +02:00
Clément Renault
788262e588
Fix final compilation 2022-10-27 11:34:17 +02:00
Clément Renault
61edcd585a
Fix the new config file with the index scheduler 2022-10-27 11:34:17 +02:00
Loïc Lecrenier
d20b5ddda0
Don't return an error when swapping 0 indexes 2022-10-27 11:34:16 +02:00
Loïc Lecrenier
17cd2a4aa0
Implement POST /indexes-swap 2022-10-27 11:34:15 +02:00
Loïc Lecrenier
169f386418
Add some documentation to the index scheduler 2022-10-27 11:34:15 +02:00
Loïc Lecrenier
ec3391808d
Fix date parsing for task queries 
Use rfc3339 or YYYY-MM-DD.

Add a day to the parsed date when it is an excluded lower bound
and the YYYY-MM-DD was used.

Also the Query type does not need to be serialisable anymore
 2022-10-27 11:34:14 +02:00
Loïc Lecrenier
10a547df4f
Apply suggestions from code review 
Co-authored-by: Clément Renault <clement@meilisearch.com>

Apply suggestions from code review

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>

Apply suggestions from code review

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>

Apply code review suggestion

Co-authored-by: Clément Renault <clement@meilisearch.com>
 2022-10-27 11:34:14 +02:00
Loïc Lecrenier
22cf0559fe
Implement task date filters 
before/after enqueued/started/finished at
 2022-10-27 11:34:14 +02:00
Kerollmops
3cbfacb616
Prefer using an u64 instead of a usize in some places 2022-10-27 11:34:12 +02:00
Kerollmops
f9c8fe5eaa
Use a tokio block_in_place method for potentially blocking tasks 2022-10-27 11:34:12 +02:00
Kerollmops
c2ec4a089b
Put the original URL query in the tasks details 2022-10-27 11:34:12 +02:00
Kerollmops
751e9bac3b
Add the tasks cancel route to cancel tasks 2022-10-27 11:34:11 +02:00
Kerollmops
290945e258
Update the canceledBy and finishedAt fields 2022-10-27 11:34:11 +02:00
Kerollmops
725158b454
Introduce the core algorithm of task cancelation 2022-10-27 11:34:11 +02:00
Kerollmops
1ca9a67c49
Introduce the task cancelation task type 2022-10-27 11:34:11 +02:00
Tamo
f7e546eea3
make the tests compile again 2022-10-27 11:34:10 +02:00
Tamo
b45c430165
fix the analytics 2022-10-27 11:34:10 +02:00
Tamo
634eb52926
extract the create_app function for the tests 2022-10-27 11:34:09 +02:00
Tamo
bea81ae37b
fix meilisearch-http 2022-10-27 11:34:09 +02:00
Tamo
6bd6321226
dump the content of the dump tasks instead of recreating at import time with wrong API keys 2022-10-27 11:34:08 +02:00
Tamo
9fe24fbff2
get rids of the useless Seek before creating a grenad reader 2022-10-27 11:34:08 +02:00
Tamo
78ce29f461
apply most style comments of the review 2022-10-27 11:34:08 +02:00
Tamo
e0221fc0a3
fix a synchronization bug while importing tasks 2022-10-27 11:34:07 +02:00
Tamo
554600dfd8
fix the deletion of the data.ms in case of errors 2022-10-27 11:34:07 +02:00
Tamo
d976e680c5
first mostly working version 2022-10-27 11:34:06 +02:00
Tamo
b7f9c94f4a
write the dump export 2022-10-27 11:34:05 +02:00
Loïc Lecrenier
f32b973945
Return an error when calling DELETE /tasks with an empty query 2022-10-27 11:34:04 +02:00
Loïc Lecrenier
fbd2be2ec8
Apply suggested changes from PR review 2022-10-27 11:34:04 +02:00
Loïc Lecrenier
8bb0fcd144
Finish first draft of the DELETE /tasks route 2022-10-27 11:34:04 +02:00
Loïc Lecrenier
9522b75454
Continue implementation of task deletion 
1. Matched tasks are a roaring bitmap
2. Start implementation in meilisearch-http
3. Snapshots use meili-snap
4. Rename to TaskDeletion
 2022-10-27 11:34:03 +02:00
Tamo
7034803712
move the API key in meilisearch_types 2022-10-27 11:34:02 +02:00
Tamo
2f1eb78b1d
refactor the Task a little bit 2022-10-27 11:34:02 +02:00
Tamo
510ce9fc51
start moving a lot of task types to meilisearch_types 2022-10-27 11:34:01 +02:00
Loïc Lecrenier
3e4337c91f
Add meili-snap crate to make writing snapshot tests easier 2022-10-27 11:34:01 +02:00
Tamo
667c282e19
get rids of the index crate + the document_types crate 2022-10-27 11:34:00 +02:00
Kerollmops
36e5efde0d
Update the tasks statuses 2022-10-27 11:33:49 +02:00
Kerollmops
cf6084151b
Make sure that meilisearch-http works without index wrapper 2022-10-27 11:33:48 +02:00
Kerollmops
c70f375669
Implement ErrorCode on the heed Error 2022-10-27 11:33:48 +02:00
Kerollmops
91e13c2824
Implement ErrorCode on the milli::Error type 2022-10-27 11:33:48 +02:00
Kerollmops
5d21c790ef
Make clippy happy 2022-10-27 11:33:47 +02:00
Kerollmops
3b343a930e
Fix meilisearch-http to use the new DocumentImport batch operation 2022-10-27 11:33:47 +02:00
Tamo
c2899fe9b2
bring back the IndexMeta and IndexStats in meilisearch-http 2022-10-27 11:33:45 +02:00
Tamo
c759fd6924
fix import bug 2022-10-27 11:33:45 +02:00
Tamo
fba9aa214a
remove the create_app macro 2022-10-27 11:33:45 +02:00
Tamo
2c8f1a43e9
get rids of meilisearch-lib 2022-10-27 11:33:44 +02:00
Tamo
ce2dfecc03
connect the new scheduler to meilisearch-http officially. 
I can index documents and do search
 2022-10-27 11:33:43 +02:00
Tamo
cb4feabca2
implements the get_tasks 2022-10-27 11:33:43 +02:00
Tamo
19154e48fe
fix all compilation errors 2022-10-27 11:33:42 +02:00
Irevoire
8d51c1f389
wip integrating the scheduler in meilisearch-http 2022-10-27 11:33:42 +02:00
bors[bot]
ab1800551f
Merge #2922 
2922: Add new error when using /keys without masterkey set r=ManyTheFish a=vishalsodani

# Pull Request

## Related issue
Fixes #2918 


Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?




Co-authored-by: vishalsodani <vishalsodani@rediffmail.com>
 2022-10-27 09:13:11 +00:00
vishalsodani
689bef7ad2 fmt the code 2022-10-27 14:09:38 +05:30
vishalsodani
89c40c83c3 refactor code to avoid cloning 2022-10-27 14:08:29 +05:30
vishalsodani
03ba830ab2 uncomment tests 2022-10-27 12:59:28 +05:30
vishalsodani
9cf3ff72a3 fix checking of master key as per review comment 2022-10-27 12:56:18 +05:30
bors[bot]
25ec51e783
Merge #2601 
2601: Ease search result pagination r=Kerollmops a=ManyTheFish

# Summary
This PR is a prototype enhancing search results pagination (#2577)

# Todo

- [x] Update the API to return the number of pages and allow users to directly choose a page instead of computing an offset
- [x] Change computation of `total_pages` in order to have an exact count
  - [x] compute query tree exhaustively
  - [x] compute distinct exhaustively

# Small Documentation

## Default search query

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "botman" }'
```

**result**:
```json
{
  "hits":[...],
  "query":"botman",
  "processingTimeMs":5,
  "hitsPerPage":20,
  "page":1,
  "totalPages":4,
  "totalHits":66
}
```

## Search query with offset parameter

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "botman", "offset": 0 }'
```

**result**:
```json
{
  "hits":[...],
  "query":"botman",
  "processingTimeMs":3,
  "limit":20,
  "offset":0,
  "estimatedTotalHits":66
}
```

## Search query selecting page with page parameter

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "botman", "page": 2 }'
```

**result**:
```json
{
  "hits":[...],
  "query":"botman",
  "processingTimeMs":5,
  "hitsPerPage":20,
  "page":2,
  "totalPages":4,
  "totalHits":66
}
```

# Related

fixes #2577

## In charge of the feature

Core: `@ManyTheFish` 
Docs: `@guimachiavelli` 
Integration: `@bidoubiwa` 


Co-authored-by: ManyTheFish <many@meilisearch.com>
 2022-10-26 16:10:58 +00:00
ManyTheFish
f4021273b8 Add is_finite_pagination method to SearchQuery 2022-10-26 18:08:29 +02:00
vishalsodani
f0ecacb58d add implementation for no master key set and fix tests 2022-10-25 22:41:48 +05:30
LunarMarathon
bc2a161f62 Change err mess for config 2022-10-25 16:16:34 +05:30
Morgane Dubus
079cfc70ae
Update mini-dashboard to v0.2.3 2022-10-24 15:20:59 +02:00
ManyTheFish
a2314cf436 Update analytics 2022-10-24 13:56:26 +02:00
ManyTheFish
0578aff8c9 Fix the tests 2022-10-20 17:41:13 +02:00
ManyTheFish
1d217cef19 Add some tests 2022-10-20 17:03:07 +02:00
ManyTheFish
506d08a9f4 Update analytics version 2022-10-19 14:05:42 +02:00
ManyTheFish
b423ef72be PROTO: hardcode version and interval for prototype analytics 2022-10-19 14:05:42 +02:00
ManyTheFish
77e718214f Fix pagination analytics 2022-10-19 14:05:42 +02:00
ManyTheFish
b1bf6722e8 Update API to fit the proto needs 2022-10-19 13:58:03 +02:00
vishalsodani
1a61209596 fix wrong variant returned for invalid_api_key_indexes error 2022-10-18 19:41:06 +05:30
vishalsodani
1cf6efa740 Add new error when using /keys without masterkey set 2022-10-18 10:48:45 +05:30
bors[bot]
b0749407f3
Merge #2804 
2804: Add environement variable `MEILI_CONFIG_FILE_PATH` to define the config file path r=Kerollmops a=choznerol

# Pull Request

## What does this PR do?
Fixes #2800

~This is a draft PR base on the code in #2745. I will `rebase` and mark it ready for review only after #2745 merge.~ Done rebase


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?


## Demo

With `config.toml`, `config_copy1.toml` and `config_copy2.toml` present:
> <img width="692" alt="image" src="https://user-images.githubusercontent.com/12410942/192566891-6b3c9d26-736f-4e23-a09b-687fca1cb50d.png">

`MEILI_CONFIG_FILE_PATH` works:
> <img width="773" alt="image" src="https://user-images.githubusercontent.com/12410942/192567023-f751536e-992a-4e90-a176-cb19122248be.png">

`--config-file-path` still works:
> <img width="768" alt="image" src="https://user-images.githubusercontent.com/12410942/192567318-88c80b24-7873-4cec-8d08-16fe4d228055.png">

When both present, `--config-file-path` taks precedence:
> <img width="1214" alt="image" src="https://user-images.githubusercontent.com/12410942/192567477-8a7cffe1-96f0-42a9-a348-6dbec20dc1e7.png">



Co-authored-by: Lawrence Chou <choznerol@protonmail.com>
 2022-10-17 10:40:05 +00:00