269 Commits

Author	SHA1	Message	Date
Marin Postma	6609f9e3be	review edits	2021-05-31 18:41:37 +02:00
Marin Postma	1c4f0b2ccf	clippy, fmt & tests	2021-05-31 16:03:39 +02:00
Marin Postma	10fc870684	improve dump info reports	2021-05-31 15:49:04 +02:00
tamo	dffbaca63b	bump sentry version	2021-05-31 13:59:31 +02:00
Marin Postma	b3c8f0e1f6	fix empty index error	2021-05-31 10:58:51 +02:00
Marin Postma	bc5a5e37ea	fix dump v1	2021-05-31 10:42:31 +02:00
Marin Postma	33c6c4f0ee	add timestamos to dump info	2021-05-30 15:55:17 +02:00
Marin Postma	39c16c0fe4	fix dump import	2021-05-30 12:35:17 +02:00
Marin Postma	1cb64caae4	dump content is now only uuid	2021-05-29 00:08:17 +02:00
Marin Postma	b258f4f394	fix dump import	2021-05-27 14:30:20 +02:00
Marin Postma	c47369839b	dump meta	2021-05-27 10:51:19 +02:00
Marin Postma	b924e897f1	load index dump	2021-05-27 10:27:47 +02:00
Marin Postma	e818c33fec	implement load uuid_resolver	2021-05-26 20:42:09 +02:00
Marin Postma	9278a6fe59	integrate in dump actor	2021-05-25 18:14:11 +02:00
Marin Postma	3593ebb8aa	dump updates	2021-05-25 16:44:58 +02:00
Marin Postma	464639aa0f	udpate actor error improvements	2021-05-25 16:44:58 +02:00
Marin Postma	4acbe8e473	implement index dump	2021-05-25 16:44:58 +02:00
Marin Postma	7ad553670f	index error handling	2021-05-25 16:44:58 +02:00
Marin Postma	2185fb8367	dump uuid resolver	2021-05-25 16:44:54 +02:00
tamo	89846d1656	improve panic message	2021-05-25 15:47:57 +02:00
tamo	1a6dcec83a	crash when the actor have no inbox	2021-05-25 15:23:13 +02:00
tamo	991d8e1ec6	fix the error printing	2021-05-25 10:48:57 +02:00
tamo	49a0e8aa19	use a RwLock instead of a Mutex	2021-05-24 18:19:34 +02:00
tamo	912f0286b3	remove the dump_inner trickery	2021-05-24 18:06:20 +02:00
tamo	dcf29e1081	fix the error handling in case there is a panic while creating a dump	2021-05-24 17:33:42 +02:00
tamo	529f7962f4	handle parallel requests for the dump actor	2021-05-24 15:42:12 +02:00
tamo	8a11c6c429	Implements the legacy behaviour of the dump ... When asked if a dump exists we check if it's the current dump, and if it's not then we check on the filesystem for any file matching our `uid.dump`	2021-05-24 12:35:46 +02:00
tamo	4cbf866821	merge with main	2021-05-12 18:12:37 +02:00
tamo	e0e23636c6	fix the serializer + reformat the file	2021-05-12 17:04:24 +02:00
Marin Postma	295f496e8a	atomic index dump load	2021-05-12 16:21:37 +02:00
tamo	6d837e3e07	the route to create a dump must return a 202	2021-05-11 17:34:34 +02:00
Marin Postma	1b671d4302	fix-snapshot	2021-05-11 13:57:18 +02:00
tamo	c30b32e173	add the criterion attribute when importing dumps from the v1	2021-05-11 13:21:36 +02:00
tamo	9e798fea75	fix the import of dump without unprocessing updates	2021-05-11 13:03:47 +02:00
tamo	384afb3455	fix the way we return the settings	2021-05-11 11:47:04 +02:00
tamo	92a7c8cd17	make clippy happy	2021-05-11 00:27:22 +02:00
tamo	8b7735c20a	move the import of the updates in the v2 and ignore the v1 for now	2021-05-11 00:20:55 +02:00
tamo	7d748fa384	integrate the new Settings in the dumps	2021-05-10 20:48:06 +02:00
tamo	d767990424	fix the import of the updates in the dump	2021-05-10 20:25:12 +02:00
tamo	ef438852cd	fix the v1	2021-05-10 20:25:12 +02:00
tamo	40ced3ff8d	first working version	2021-05-10 20:25:12 +02:00
tamo	5f5402a3ab	provide a way to access the internal content path of all processing State	2021-05-10 20:25:12 +02:00
tamo	26dcb9e66d	bump milli version and fix a performance issue for large dumps	2021-05-10 20:25:12 +02:00
Marin Postma	956012da95	fix dump lock	2021-05-10 20:25:12 +02:00
tamo	24192fc550	fix tests	2021-05-10 20:25:12 +02:00
tamo	efca63f9ce	[WIP] rebase on main	2021-05-10 20:25:09 +02:00
tamo	c3552cecdf	WIP rebase on main	2021-05-10 20:24:18 +02:00
tamo	0f94ef8abc	WIP: dump	2021-05-10 20:24:18 +02:00
tamo	0275b36fb0	[WIP] rebase on main	2021-05-10 20:24:14 +02:00
tamo	1b5fc61eb6	[WIP] rebase on main	2021-05-10 20:23:12 +02:00
tamo	0fee81678e	[WIP] rebase on main	2021-05-10 20:22:18 +02:00
tamo	c4d898a265	split the dumps between v1 and v2	2021-05-10 20:20:57 +02:00
tamo	e389c088eb	WIP: rebasing on master	2021-05-10 20:20:57 +02:00
Marin Postma	0cc79d414f	add test	2021-05-10 18:34:25 +02:00
Marin Postma	8d11b368d1	implement check	2021-05-10 18:22:41 +02:00
Marin Postma	706643dfed	type setting struct	2021-05-10 17:30:09 +02:00
Marin Postma	b192cb9c1f	enable string syntax for the filters	2021-05-06 12:48:31 +02:00
Marin Postma	ec7eb7798f	remove facet setting	2021-05-04 22:36:31 +02:00
Marin Postma	a717925caa	remove filters, rename facet_filters to filter	2021-05-04 18:20:56 +02:00
Clémentine Urquizar	eb03a3ccb1	Upgrade Milli and Tokenizer	2021-05-04 17:56:19 +02:00
Marin Postma	928fb34eff	bump milli and fix tests	2021-05-04 15:10:22 +02:00
bors[bot]	1e6b40a24b	Merge #172 ... 172: Fix cors authentication issue r=MarinPostma a=MarinPostma The error was due to the middleware returning an error, instead of a response containing the error. close #110 Co-authored-by: Marin Postma <postma.marin@protonmail.com>	2021-05-03 08:38:42 +00:00
Marin Postma	78217bcf18	Fix cors authentication issue	2021-04-29 16:28:12 +02:00
Morgane Dubus	3d5fba94c2	Update mini-dashboard with version 0.1.2	2021-04-29 15:22:41 +02:00
bors[bot]	8bc7dd8b03	Merge #143 ... 143: Shared update store r=irevoire a=MarinPostma This PR changes the updates process so that only one instance of an update store is shared among indexes. This allows updates to always be processed sequentially without additional synchronization, and fixes the bug where all the first pending update for each index were reported as processing whereas only one was. EDIT: I ended having to rewrite the whole `UpdateStore` to allow updates being really queued and processed sequentially in the ordered they were added. For that purpose I created a `pending_queue` that orders the updates by a global update id. To find the next `update_id` to use, both globally and for each index, I have created another database that contains the next id to use. Finally, all updates that have been processed (with success or otherwise) are all stores in an `updates` database. The layout for the keys of these databases are such that it is easy to iterate over the elements for a particular index, and greatly reduces the amount of code to do so, compared to the former implementation. I have also simplified the locking mechanism for the update store, thanks to the StateLock data structure, that allow both an arbitrary number of readers and a single writer to concurrently access the state. The current state can be either Idle, Processing, or Snapshotting. When an update or snapshotting is ongoing, the process holds the state lock until it is done processing its task. When it is done, it sets bask the state to Idle. I have made other small improvements here and there, and have let some other for work, such as: - When creating an update file to hold a request's content, it would be preferable to first create a temporary file, and then atomically persist it when we have written to it. This would simplify the case when there is no data to be written to the file, since we wouldn't have to take care about cleaning after ourselves. - The logic for content validation must be factored. - Some more tests related to error handling in the process_pending_update function. - The issue #159 close #114 Co-authored-by: Marin Postma <postma.marin@protonmail.com>	2021-04-27 18:41:55 +00:00
Marin Postma	a961f0ce75	fix clippy warnings	2021-04-27 18:28:46 +02:00
Marin Postma	bb79a15c04	reenable ranking rules route	2021-04-27 15:29:00 +02:00
Marin Postma	4fe2a13c71	rewrite update store	2021-04-27 15:20:52 +02:00
Marin Postma	51829ad85e	review fixes	2021-04-27 15:10:57 +02:00
Marin Postma	c78f351300	fix tests	2021-04-27 15:10:57 +02:00
Marin Postma	ee675eadf1	fix stats	2021-04-27 15:10:55 +02:00
Marin Postma	33830d5ecf	fix snapshots	2021-04-27 15:09:55 +02:00
Marin Postma	2b154524bb	fix filtered out pending update	2021-04-27 15:09:23 +02:00
Marin Postma	b626d02ffe	simplify index actor run loop	2021-04-27 15:09:22 +02:00
Marin Postma	9ce68d11a7	single update store instance	2021-04-27 15:09:21 +02:00
Marin Postma	5a38f13cae	multi_index udpate store	2021-04-27 15:07:13 +02:00
Clémentine Urquizar	1ba46f8f77	Disable clippy rule	2021-04-27 12:43:00 +02:00
Clémentine Urquizar	f80ea24d2b	Add tests on every platform and fix clippy errors	2021-04-27 12:42:59 +02:00
Marin Postma	5014f74649	put mini-dashboard in out-dir	2021-04-27 09:32:17 +02:00
Clémentine Urquizar	f3b6bf55a6	Update version for the next release (alpha4)	2021-04-26 19:05:16 +02:00
Marin Postma	c2461e5066	review fixes	2021-04-26 10:20:46 +02:00
Marin Postma	e4bd1bc5ce	update actix-web-static-file rev	2021-04-22 11:42:41 +02:00
Marin Postma	90f57c1329	update CI & Dockerfile	2021-04-22 11:22:09 +02:00
Marin Postma	6af769af20	bump mini-dashboard	2021-04-22 10:45:05 +02:00
Marin Postma	6bcf20c70e	serve static site	2021-04-22 10:26:54 +02:00
Marin Postma	bb79695e44	load mini-dashboard assets	2021-04-22 10:26:54 +02:00
Marin Postma	ea5517bc8c	add mini-dashboard feature	2021-04-22 10:26:54 +02:00
Clémentine Urquizar	a72d2f66cd	use <em> tags instead of <mark> tags for highlighting	2021-04-21 19:14:55 +02:00
Marin Postma	dd2914873b	fix document fields order	2021-04-20 21:30:30 +02:00
Marin Postma	d9a29cae60	fix ignored displayed attributes	2021-04-20 21:23:35 +02:00
Marin Postma	7a737d2bd3	support wildcard	2021-04-20 21:23:35 +02:00
Marin Postma	881b099c8e	add tests	2021-04-20 21:23:34 +02:00
Marin Postma	c6bb36efa5	implement _formated	2021-04-20 21:23:28 +02:00
Marin Postma	526a05565e	add SearchHit structure	2021-04-20 21:22:48 +02:00
Clémentine Urquizar	b8e535579f	Update version for the next release (alpha3)	2021-04-20 16:11:07 +02:00
Marin Postma	f8c338e3a7	add test for dedicated distinct route	2021-04-20 15:49:17 +02:00
Marin Postma	6c470cf687	enable distinct-attribute setting route	2021-04-20 11:34:18 +02:00
Marin Postma	ec63e13896	bump actix	2021-04-20 11:29:32 +02:00
Marin Postma	1746132c7d	add test set/reset distinct attribute	2021-04-20 11:29:08 +02:00
mpostma	ec230c2835	enable distinct	2021-04-20 11:29:06 +02:00