capture metrics from arbitrary file names #1417
Description
Currently, metrics can capture from stdout/stderr as in this example from https://buildtest.readthedocs.io/en/devel/writing_buildspecs/performance_checks.html#defining-metrics
buildspecs:
metric_regex_example:
executor: generic.local.sh
type: script
description: capture result metric from output
run: echo "HPCG result is VALID with a GFLOP/s rating of=63.6515"
tags: tutorials
metrics:
hpcg_rating:
type: float
regex:
exp: '(\d+\.\d+)$'
stream: stdout
hpcg_state:
type: str
regex:
exp: '(VALID)'
stream: stdoutWe need to add support for arbitrary file names during performance checks. Therefore we can add another field such as file_regex as a property so it could be something like this
buildspecs:
metric_regex_example:
executor: generic.local.sh
type: script
description: capture result metric from output
run: echo "HPCG result is VALID with a GFLOP/s rating of=63.6515" > hpcg.txt
tags: tutorials
metrics:
hpcg_rating:
type: float
file_regex:
exp: '(\d+\.\d+)$'
file: hpcg.txt
hpcg_state:
type: str
file_regex:
exp: '(VALID)'
file: hpcg.txtif there is a regex mismatch or file doesn't exist or is not a file then metrics is undefined. This can be later used in any of the status check such as assert_ge.