122 Commits

Author	SHA1	Message	Date
Kerollmops	2f577b6fcd	Patch the IndexScheduler in meilisearch-http to use the options struct	2022-10-27 11:35:16 +02:00
Kerollmops	71b50853dc	Introduce an options struct to create the IndexScheduler	2022-10-27 11:35:16 +02:00
Kerollmops	e35fe33712	Fix some bugs with files	2022-10-27 11:35:15 +02:00
Kerollmops	942b7c338b	Compress the snapshot in a tarball	2022-10-27 11:35:15 +02:00
Kerollmops	4cafc63561	Reintroduce the versioning functions	2022-10-27 11:35:14 +02:00
Kerollmops	89e127e4f4	Declare the auth path in the index scheduler	2022-10-27 11:35:14 +02:00
Kerollmops	eec43ec953	Implement a first version of the snapshots	2022-10-27 11:35:14 +02:00
Kerollmops	c063f154fb	Add the snapshots directory path to the IndexScheduler	2022-10-27 11:35:14 +02:00
Kerollmops	e0548e42e7	Rename the Snapshot task into SnapshotCreation	2022-10-27 11:35:14 +02:00
Kerollmops	4d43a9f5b1	Rename the index-scheduler module into insta_snapshot	2022-10-27 11:35:14 +02:00
Loïc Lecrenier	6db90ba6cc	Make sure that we don't delete or cancel future tasks ... This should already have been the case before, but there is no harm in adding another check.	2022-10-27 11:35:13 +02:00
Irevoire	61f0940f8c	fix an issue with the dates	2022-10-27 11:35:13 +02:00
Irevoire	241300d2d8	add more naive tests around the document addition + remove the old unused snapshot files	2022-10-27 11:35:13 +02:00
Irevoire	570b2d1167	add some naive document addition tests	2022-10-27 11:35:12 +02:00
Loïc Lecrenier	d92425658e	Add index scheduler tests for task cancelation	2022-10-27 11:35:12 +02:00
Loïc Lecrenier	16fac10074	Fix crash when batching an index swap task containing 0 swaps	2022-10-27 11:35:12 +02:00
Irevoire	7ed3f00b1e	reformat	2022-10-27 11:35:12 +02:00
Irevoire	7e52f1effb	remove a lot of unecessary clone and ref	2022-10-27 11:35:11 +02:00
Loïc Lecrenier	4d25c159e6	Apply code review suggestions	2022-10-27 11:35:11 +02:00
Loïc Lecrenier	424202d773	Pause the index scheduler for one second when a fatal error occurs	2022-10-27 11:35:11 +02:00
Loïc Lecrenier	4a35eb9849	Fix (hopefully) queries that include processing tasks	2022-10-27 11:35:11 +02:00
Loïc Lecrenier	493a8cff31	Adjust task details correctly following index swap	2022-10-27 11:35:10 +02:00
Loïc Lecrenier	4de445d386	Start testing unexpected errors and panics in index scheduler	2022-10-27 11:35:10 +02:00
Loïc Lecrenier	e3848b5f28	Add assert method to verify validity of index scheduler state	2022-10-27 11:35:10 +02:00
Irevoire	ecf4e43b3d	rename the dumpExport to dumpCreation	2022-10-27 11:35:10 +02:00
Irevoire	8a23e707c1	fix the task view and forward the task db size	2022-10-27 11:35:09 +02:00
Irevoire	1d04ce611d	remove ununsed function	2022-10-27 11:35:08 +02:00
Irevoire	e9055f5572	fix clippy	2022-10-27 11:35:08 +02:00
Irevoire	8d1408c65e	fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting	2022-10-27 11:35:05 +02:00
Clément Renault	80b2e70ee7	Introduce a rustfmt file	2022-10-27 11:35:05 +02:00
Irevoire	b6a0abea9f	fix the index deletion when the index doesn’t exists but would be created by one of the autobatched tasks	2022-10-27 11:34:16 +02:00
Loïc Lecrenier	11fee30f47	Apply review suggestions and stop using rtxn.commit	2022-10-27 11:34:15 +02:00
Loïc Lecrenier	17cd2a4aa0	Implement POST /indexes-swap	2022-10-27 11:34:15 +02:00
Loïc Lecrenier	169f386418	Add some documentation to the index scheduler	2022-10-27 11:34:15 +02:00
Loïc Lecrenier	ec3391808d	Fix date parsing for task queries ... Use rfc3339 or YYYY-MM-DD. Add a day to the parsed date when it is an excluded lower bound and the YYYY-MM-DD was used. Also the Query type does not need to be serialisable anymore	2022-10-27 11:34:14 +02:00
Loïc Lecrenier	22cf0559fe	Implement task date filters ... before/after enqueued/started/finished at	2022-10-27 11:34:14 +02:00
Irevoire	5765883600	fix the auto-generated details	2022-10-27 11:34:14 +02:00
Kerollmops	6460b78e08	Clean up the delete_persisted_task_data function	2022-10-27 11:34:13 +02:00
Kerollmops	d21651c968	Throw the error if we can't register the tasks in the store	2022-10-27 11:34:13 +02:00
Kerollmops	6e904d0997	Introduce a ProcessingTasks constructor	2022-10-27 11:34:12 +02:00
Kerollmops	b373d19831	Extract the must_stop flag out of the RwLock	2022-10-27 11:34:12 +02:00
Kerollmops	751e9bac3b	Add the tasks cancel route to cancel tasks	2022-10-27 11:34:11 +02:00
Kerollmops	290945e258	Update the canceledBy and finishedAt fields	2022-10-27 11:34:11 +02:00
Kerollmops	b2c5bc67b7	Add more enum-iterator related stuff	2022-10-27 11:34:11 +02:00
Kerollmops	591527a99d	Prefer using TaskDeletion in the dumps	2022-10-27 11:34:11 +02:00
Kerollmops	1ca9a67c49	Introduce the task cancelation task type	2022-10-27 11:34:11 +02:00
Kerollmops	703ba7a1fb	Introduce the ProcessingTasks struct	2022-10-27 11:34:10 +02:00
Loïc Lecrenier	ea60d35c71	Delete a task's persisted data when appropriate	2022-10-27 11:34:10 +02:00
Tamo	9e85f050b2	fix the tests	2022-10-27 11:34:09 +02:00
Tamo	6bd6321226	dump the content of the dump tasks instead of recreating at import time with wrong API keys	2022-10-27 11:34:08 +02:00