Skip to content

AWS EC2 scraping not working fine on v3.7.0 #17343

@lyz-code

Description

@lyz-code

What did you do?

After upgrade prometheus to v3.7.0 the aws ec2 service discovery is not returning targets. Rolling back to 3.6.0 makes it work as expected

What did you expect to see?

No response

What did you see instead? Under which circumstances?

No targets are shown under aws ec2 service discovery scrape configs

System information

Linux 6.14.0-1014-aws x86_64

Prometheus version

prometheus, version 3.7.0 (branch: HEAD, revision: 09814effe6c3b4fb4c8e98b578f7205d0228f04d)
  build user:       root@84cfe63a2ab5
  build date:       20251015-10:17:59
  go version:       go1.25.3
  platform:         linux/amd64
  tags:             netgo,builtinassets

Prometheus configuration file

- job_name: node_exporter
    ec2_sd_configs:
      - region: us-east-1
        port: 9100
        refresh_interval: 30s
        filters:
          - name: instance-state-name
            values:
              - running
          - name: tag:environment
            values:
              - production

Alertmanager version


Alertmanager configuration file

Logs

time=2025-10-16T13:09:11.215Z level=ERROR source=refresh.go:90 msg="Unable to refresh target groups" component="discovery manager scrape" discovery=ec2 config=node_exporter err="could not describe instances: operation error EC2: DescribeInstances, get identity: get credentials: failed to refresh cached credentials, static credentials are empty"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions