snowykami/meilisearch
1
0
Fork 0
mirror of https://github.com/meilisearch/meilisearch.git synced 2024-11-23 10:37:41 +08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

Merge #359

Browse Source 
359: Improve the benchmark comparison script r=irevoire a=irevoire

This modification allow us to compare more than 2 benchmarks or to only print the results of one benchmark



Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
This commit is contained in:
bors[bot] 2021-09-20 12:39:59 +00:00 committed by GitHub
commit 203aa727a7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 17 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
benchmarks/README.md
View File

@ -54,7 +54,7 @@ MILLI_BENCH_DATASETS_PATH=~/datasets cargo bench --bench songs # the code is com
## Comparison between benchmarks
The benchmark reports we push are generated with `critcmp`. Thus, we use `critcmp` to generate comparison results between 2 benchmarks.
The benchmark reports we push are generated with `critcmp`. Thus, we use `critcmp` to show the result of a benchmark, or compare results between multiple benchmarks.
We provide a script to download and display the comparison report.
@ -71,12 +71,18 @@ List the available file in the DO Space:
```bash
songs_main_09a4321.json
songs_geosearch_24ec456.json
search_songs_main_cb45a10b.json
```
Run the comparison script:
```bash
# we get the result of ONE benchmark, this give you an idea of how much time an operation took
./benchmarks/scripts/compare.sh son songs_geosearch_24ec456.json
# we compare two benchmarks
./benchmarks/scripts/compare.sh songs_main_09a4321.json songs_geosearch_24ec456.json
# we compare three benchmarks
./benchmarks/scripts/compare.sh songs_main_09a4321.json songs_geosearch_24ec456.json search_songs_main_cb45a10b.json
```
## Datasets

22
benchmarks/scripts/compare.sh
View File

@ -17,21 +17,9 @@ if [[ "$?" -ne 0 ]]; then
exit 1
fi
if [[ $# -ne 2 ]]
then
echo 'Need 2 arguments.'
echo 'Usage: '
echo ' $ ./compare.sh old new'
echo 'Ex:'
echo ' $ ./compare.sh songs_main_09a4321.json songs_geosearch_24ec456.json'
exit 1
fi
old_file="$1"
new_file="$2"
s3_url='https://milli-benchmarks.fra1.digitaloceanspaces.com/critcmp_results'
for file in $old_file $new_file
for file in $@
do
file_s3_url="$s3_url/$file"
file_local_path="/tmp/$file"
@ -45,6 +33,12 @@ do
fi
done
path_list=$(echo " $@" | sed 's/ / \/tmp\//g')
if [[ ${#@} -gt 1 ]]; then
# Print the diff changes between the old and new benchmarks
# by only displaying the lines that have a diff of more than 5%.
critcmp --threshold 5 "/tmp/$old_file" "/tmp/$new_file"
critcmp --threshold 5 $path_list
else
critcmp $path_list
fi