Skip to content

[tmpnet] Add check for collection of logs and metrics to custom github action#3740

Merged
StephenButtolph merged 8 commits intomasterfrom
tmpnet-check-metrics-collection
Mar 4, 2025
Merged

[tmpnet] Add check for collection of logs and metrics to custom github action#3740
StephenButtolph merged 8 commits intomasterfrom
tmpnet-check-metrics-collection

Conversation

@maru-ava
Copy link
Copy Markdown
Contributor

@maru-ava maru-ava commented Feb 24, 2025

Why this should be merged

Adds checks to the run-monitored-tmpnet-cmd custom github action that verifies that logs were collected to loki and metrics were collected to prometheus for the job. A failing check will fail the job, ensuring that collection of logs and metrics doesn't silently fail to occur. This should also make reuse easier since any configuration issue will be more obvious.

How this works

  • add tmpnetctl check-metrics to enable querying prometheus for metrics labeled for a github run
  • add tmpnetctl check-logs to enable querying loki for logs labeled for a github run
  • call the new checks configured with GH_* env vars for a job at the end of run-monitored-tmpnet-cmd

How this was tested

  • Manually verified that tmpnetctl check-metrics passes for a network UUID for which metrics exist
  • Manually verified that tmpnetctl check-metrics fails for a non-existent network UUID
  • Checked that jobs for this PR pass and have metrics associated with them in grafana

Need to be documented in RELEASES.md?

N/A

TODO

@maru-ava maru-ava added the ci This focuses on changes to the CI process label Feb 24, 2025
@maru-ava maru-ava self-assigned this Feb 24, 2025
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch 3 times, most recently from ab202c2 to fd27d90 Compare February 24, 2025 14:08
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch 3 times, most recently from 3ec85e1 to 2e6404a Compare February 24, 2025 15:28
@maru-ava maru-ava force-pushed the tmpnet-managed-collectors branch from 91b64a7 to 4dce5a1 Compare February 24, 2025 15:34
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch 5 times, most recently from 453b332 to e4243fc Compare February 24, 2025 17:20
@maru-ava maru-ava changed the title [tmpnet] Add check for metrics collection to monitoring action [tmpnet] Add check for collection of logs and metrics to custom github action Feb 24, 2025
Comment thread go.sum
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA=
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is go.sum changed with no corresponding change?

Copy link
Copy Markdown
Contributor Author

@maru-ava maru-ava Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's because of my use of the prometheus client. I didn't add any deps but the go mod job failed and this was the result of running 'go mod tidy'.

@maru-ava maru-ava force-pushed the tmpnet-managed-collectors branch from 4dce5a1 to 1a5222c Compare February 24, 2025 21:33
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch 3 times, most recently from 198b68f to 049efef Compare February 24, 2025 23:31
@maru-ava maru-ava force-pushed the tmpnet-managed-collectors branch from 06869f7 to e801d02 Compare February 27, 2025 13:39
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch from 049efef to a713bbc Compare February 27, 2025 13:40
@maru-ava maru-ava force-pushed the tmpnet-managed-collectors branch from e801d02 to 284d898 Compare March 3, 2025 14:01
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch from a713bbc to d2305d5 Compare March 3, 2025 14:04
@maru-ava maru-ava force-pushed the tmpnet-managed-collectors branch from 456b45d to a9e3433 Compare March 4, 2025 14:38
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch from d2305d5 to ffc83d9 Compare March 4, 2025 14:40
@maru-ava maru-ava force-pushed the tmpnet-managed-collectors branch from a9e3433 to 9c408ff Compare March 4, 2025 17:53
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch from ffc83d9 to fb10dea Compare March 4, 2025 18:18
Base automatically changed from tmpnet-managed-collectors to master March 4, 2025 18:24
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch from fb10dea to 1d3a5f9 Compare March 4, 2025 18:30
Comment thread tests/fixture/tmpnet/check_monitoring.go
Comment thread tests/fixture/tmpnet/check_monitoring.go Outdated
@maru-ava maru-ava force-pushed the tmpnet-check-metrics-collection branch from bcb258c to 6714c80 Compare March 4, 2025 19:19
@StephenButtolph StephenButtolph added this pull request to the merge queue Mar 4, 2025
Merged via the queue into master with commit 26eb5c8 Mar 4, 2025
22 checks passed
@StephenButtolph StephenButtolph deleted the tmpnet-check-metrics-collection branch March 4, 2025 19:55
@github-project-automation github-project-automation bot moved this from In Review 👀 to Done ✅ in avalanchego Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci This focuses on changes to the CI process

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants