Skip to content

[synthetic-qa] 003-storage-full: Validate agent identifies storage exhaustion #599

@davincios

Description

@davincios

Scenario

Run: opensre tests synthetic --scenario 003-storage-full

The RDS instance ran out of storage during a bulk INSERT archival job. FreeStorageSpace collapsed from 15 GB to 0 in 15 minutes.

Correct Answer

  • Root Cause Category: resource_exhaustion
  • Root Cause: Bulk INSERT job exhausted storage. FreeStorageSpace hit 0, writes blocked.

Human Review Checklist

  • Does the agent identify storage exhaustion as the root cause?
  • Does the agent reference the FreeStorageSpace metric trend?
  • Does the agent cite the RDS event confirming "ran out of storage space"?
  • Does the causal chain flow: bulk INSERT → WriteIOPS spike → storage fills → writes blocked?
  • Does the agent note WriteIOPS collapsing to zero (signature of full disk)?

What Good Output Looks Like

Agent traces FreeStorageSpace from 15 GB → 0 during high WriteIOPS, cites the RDS event as definitive confirmation, and explains the bulk INSERT as the driver.

Common Failure Modes

  • Agent mentions storage is low but doesn't explain what caused it (misses the bulk INSERT)
  • Agent ignores the RDS event (definitive evidence) and relies only on metric trends
  • Agent doesn't notice WriteIOPS collapsing to zero (key signal that writes are physically blocked)

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions