Skip to content

Added a basic Key configuration file for Object storage integration in Prometheus setup#15183

Open
kushalShukla-web wants to merge 3 commits intoprometheus:mainfrom
kushalShukla-web:object-config
Open

Added a basic Key configuration file for Object storage integration in Prometheus setup#15183
kushalShukla-web wants to merge 3 commits intoprometheus:mainfrom
kushalShukla-web:object-config

Conversation

@kushalShukla-web
Copy link
Copy Markdown
Contributor

@kushalShukla-web kushalShukla-web commented Oct 18, 2024

Part of the PR: prometheus/test-infra#760

This PR introduces an object storage configuration file for the PromBench project, designed to support object storage solutions like GCS, Minio. The configuration file centralizes the object storage settings, allowing PromBench to utilize it for its operations, such as storing and retrieving data from the object store.

  • Added Object Storage Config File
    A YAML configuration file has been created that includes the required details for connecting to an S3-compatible object storage (e.g., Minio).
  • Integrates with Init Containers
    The configuration is designed to be used within the init containers in the PromBench project to handle operations like uploading and downloading data from object storage.
  • Security Considerations
    Sensitive information like access_key and secret_key will not be hardcoded in the repository. Instead, these will be injected dynamically at runtime using Kubernetes Secrets to ensure security.

@kushalShukla-web kushalShukla-web force-pushed the object-config branch 5 times, most recently from fabd84b to 2f6fc87 Compare October 20, 2024 18:11
@kushalShukla-web kushalShukla-web force-pushed the object-config branch 5 times, most recently from 9a7418c to d4b6d65 Compare November 2, 2024 13:40
@kushalShukla-web kushalShukla-web changed the title Added a basic object storage configuration file for MinIO integration in Prometheus setup. Added a basic Key configuration file for MinIO integration in Prometheus setup. Nov 3, 2024
@kushalShukla-web kushalShukla-web force-pushed the object-config branch 11 times, most recently from 43460b4 to 8edadf0 Compare November 11, 2024 09:47
@kushalShukla-web kushalShukla-web changed the title Added a basic Key configuration file for MinIO integration in Prometheus setup. Added a basic Key configuration file for Object storage integration in Prometheus setup. Nov 11, 2024
@kushalShukla-web kushalShukla-web force-pushed the object-config branch 2 times, most recently from 5ebf016 to 315bc61 Compare November 11, 2024 11:28
Copy link
Copy Markdown
Member

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will use this PR for testing once Prombench is updated to read the data.

Note there is a lint error.

@kushalShukla-web kushalShukla-web force-pushed the object-config branch 2 times, most recently from c67268e to fb662fd Compare November 26, 2024 11:55
2. replaced key with path.
Signed-off-by: Kushal Shukla <[email protected]>
@bboreham
Copy link
Copy Markdown
Member

bboreham commented Apr 7, 2025

/prombench main --bench.version=8e5dd664331dda16065c4b6534cf28a4cd63edfc

@prombot
Copy link
Copy Markdown
Contributor

prombot commented Apr 7, 2025

⏱️ Welcome to Prometheus Benchmarking Tool. ⏱️

Compared versions: PR-15183 and main

Custom benchmark version: 8e5dd664331dda16065c4b6534cf28a4cd63edfc branch

After the successful deployment (check status here), the benchmarking results can be viewed at:

Available Commands:

  • To restart benchmark: /prombench restart main --bench.version=8e5dd664331dda16065c4b6534cf28a4cd63edfc
  • To stop benchmark: /prombench cancel
  • To print help: /prombench help

@bboreham
Copy link
Copy Markdown
Member

bboreham commented Apr 7, 2025

/prombench cancel

@prombot
Copy link
Copy Markdown
Contributor

prombot commented Apr 7, 2025

Benchmark cancel is in progress.

@bboreham
Copy link
Copy Markdown
Member

bboreham commented Apr 7, 2025

/prombench main --bench.version=@8e5dd664331dda16065c4b6534cf28a4cd63edfc

@prombot
Copy link
Copy Markdown
Contributor

prombot commented Apr 7, 2025

⏱️ Welcome to Prometheus Benchmarking Tool. ⏱️

Compared versions: PR-15183 and main

Custom benchmark version: 8e5dd664331dda16065c4b6534cf28a4cd63edfc commit

After the successful deployment (check status here), the benchmarking results can be viewed at:

Available Commands:

  • To restart benchmark: /prombench restart main --bench.version=@8e5dd664331dda16065c4b6534cf28a4cd63edfc
  • To stop benchmark: /prombench cancel
  • To print help: /prombench help

@bboreham
Copy link
Copy Markdown
Member

bboreham commented Apr 7, 2025

/prombench main --bench.version=@8e5dd664331dda16065c4b6534cf28a4cd63edfc

@prombot
Copy link
Copy Markdown
Contributor

prombot commented Apr 7, 2025

⏱️ Welcome to Prometheus Benchmarking Tool. ⏱️

Compared versions: PR-15183 and main

Custom benchmark version: 8e5dd664331dda16065c4b6534cf28a4cd63edfc commit

After the successful deployment (check status here), the benchmarking results can be viewed at:

Available Commands:

  • To restart benchmark: /prombench restart main --bench.version=@8e5dd664331dda16065c4b6534cf28a4cd63edfc
  • To stop benchmark: /prombench cancel
  • To print help: /prombench help

@bboreham
Copy link
Copy Markdown
Member

bboreham commented Apr 7, 2025

/prombench cancel

@prombot
Copy link
Copy Markdown
Contributor

prombot commented Apr 7, 2025

Benchmark cancel is in progress.

@bboreham
Copy link
Copy Markdown
Member

bboreham commented Apr 7, 2025

This PromBench run failed to start up in the time allowed.
https://github.com/prometheus/prometheus/actions/runs/14312141952/job/40109536237

See prometheus/test-infra#840 (review) - PromBench needs to have one set of config which is copied over to each PromBench run.

@krajorama
Copy link
Copy Markdown
Member

Hi, commented on the cover issue related to this PR: prometheus/test-infra#702 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants