90 Commits

Author	SHA1	Message	Date
Loïc Lecrenier	4de445d386	Start testing unexpected errors and panics in index scheduler	2022-10-27 11:35:10 +02:00
Irevoire	ecf4e43b3d	rename the dumpExport to dumpCreation	2022-10-27 11:35:10 +02:00
Irevoire	e9055f5572	fix clippy	2022-10-27 11:35:08 +02:00
Irevoire	c8ee453b6c	fix the autobatched document deletion	2022-10-27 11:35:07 +02:00
Irevoire	a8de5368e5	fix the index creation in case an index already exists	2022-10-27 11:35:07 +02:00
Irevoire	9bb2e3c790	fix the failed document addition with a primary key	2022-10-27 11:35:07 +02:00
Irevoire	8d1408c65e	fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting	2022-10-27 11:35:05 +02:00
Clément Renault	80b2e70ee7	Introduce a rustfmt file	2022-10-27 11:35:05 +02:00
Clément Renault	72ec4ce96b	Fix allow_index_creation useless field	2022-10-27 11:34:17 +02:00
Irevoire	b6a0abea9f	fix the index deletion when the index doesn’t exists but would be created by one of the autobatched tasks	2022-10-27 11:34:16 +02:00
Irevoire	d9218578e3	it probably works but it's also horrendous	2022-10-27 11:34:16 +02:00
Loïc Lecrenier	11fee30f47	Apply review suggestions and stop using rtxn.commit	2022-10-27 11:34:15 +02:00
Loïc Lecrenier	17cd2a4aa0	Implement POST /indexes-swap	2022-10-27 11:34:15 +02:00
Loïc Lecrenier	169f386418	Add some documentation to the index scheduler	2022-10-27 11:34:15 +02:00
Loïc Lecrenier	22cf0559fe	Implement task date filters ... before/after enqueued/started/finished at	2022-10-27 11:34:14 +02:00
Irevoire	5765883600	fix the auto-generated details	2022-10-27 11:34:14 +02:00
Tamo	cff003c928	remove the unused variants from the autobatcher	2022-10-27 11:34:14 +02:00
Kerollmops	50b8b9df6a	Delete the tasks content file once the transaction has been successfully committed	2022-10-27 11:34:13 +02:00
Kerollmops	b373d19831	Extract the must_stop flag out of the RwLock	2022-10-27 11:34:12 +02:00
Kerollmops	3cbfacb616	Prefer using an u64 instead of a usize in some places	2022-10-27 11:34:12 +02:00
Kerollmops	79c4275bfc	Delete the persisted data when we cancel a task	2022-10-27 11:34:12 +02:00
Kerollmops	c2ec4a089b	Put the original URL query in the tasks details	2022-10-27 11:34:12 +02:00
Kerollmops	290945e258	Update the canceledBy and finishedAt fields	2022-10-27 11:34:11 +02:00
Kerollmops	725158b454	Introduce the core algorithm of task cancelation	2022-10-27 11:34:11 +02:00
Kerollmops	1ca9a67c49	Introduce the task cancelation task type	2022-10-27 11:34:11 +02:00
Kerollmops	703ba7a1fb	Introduce the ProcessingTasks struct	2022-10-27 11:34:10 +02:00
Loïc Lecrenier	ea60d35c71	Delete a task's persisted data when appropriate	2022-10-27 11:34:10 +02:00
Tamo	2f748480a1	share the rtxn between the access to the tasks and to the indexes	2022-10-27 11:34:09 +02:00
Tamo	83f3c5ec57	flush the dump-writer only once everything has been inserted	2022-10-27 11:34:08 +02:00
Tamo	78ce29f461	apply most style comments of the review	2022-10-27 11:34:08 +02:00
Tamo	dd506e5d87	stop dumping the current dumping task as enqueued so it's not looping for ever	2022-10-27 11:34:06 +02:00
Tamo	d976e680c5	first mostly working version	2022-10-27 11:34:06 +02:00
Tamo	c051166bcc	update the API a little bit	2022-10-27 11:34:05 +02:00
Tamo	72a906ae75	fix the tests	2022-10-27 11:34:05 +02:00
Tamo	b7f9c94f4a	write the dump export	2022-10-27 11:34:05 +02:00
Loïc Lecrenier	8954b1bd1d	Fix number of deleted tasks details after duplicate task deletion	2022-10-27 11:34:05 +02:00
Loïc Lecrenier	8defad6c38	Add task deletion tests where the same task is deleted twice	2022-10-27 11:34:05 +02:00
Loïc Lecrenier	fbd2be2ec8	Apply suggested changes from PR review	2022-10-27 11:34:04 +02:00
Loïc Lecrenier	8bb0fcd144	Finish first draft of the DELETE /tasks route	2022-10-27 11:34:04 +02:00
Loïc Lecrenier	9522b75454	Continue implementation of task deletion ... 1. Matched tasks are a roaring bitmap 2. Start implementation in meilisearch-http 3. Snapshots use meili-snap 4. Rename to TaskDeletion	2022-10-27 11:34:03 +02:00
Kerollmops	e4d461ecba	Make sure that we do not batch tasks from different indexes	2022-10-27 11:34:03 +02:00
Tamo	2f1eb78b1d	refactor the Task a little bit	2022-10-27 11:34:02 +02:00
Tamo	510ce9fc51	start moving a lot of task types to meilisearch_types	2022-10-27 11:34:01 +02:00
Tamo	667c282e19	get rids of the index crate + the document_types crate	2022-10-27 11:34:00 +02:00
Loïc Lecrenier	ab4e649221	Apply suggestions from code review ... Co-authored-by: Tamo <tamo@meilisearch.com>	2022-10-27 11:34:00 +02:00
Loïc Lecrenier	4c55c30027	Add a DetailsView type and improve index scheduler snapshots ... The DetailsView type is necessary because serde incorrectly deserialises the `Details` type, so the database fails to correctly decode Tasks	2022-10-27 11:33:59 +02:00
Loïc Lecrenier	dc81992eb2	Implement TaskDeletion in the index scheduler	2022-10-27 11:33:59 +02:00
Kerollmops	fe84f2648b	Allow a user to disable the auto batching system	2022-10-27 11:33:59 +02:00
Clément Renault	123f47dbc4	Create the index only if the task has the rights to do so	2022-10-27 11:33:50 +02:00
Clément Renault	068a4b2884	Correctly batch tasks with different index creation rights	2022-10-27 11:33:50 +02:00