Skip to content
This repository was archived by the owner on Mar 23, 2026. It is now read-only.

Make cleanups fixture logging actionable#13553

Merged
joe4dev merged 1 commit intomainfrom
fix/improve-ci-cleanup-logging
Jan 7, 2026
Merged

Make cleanups fixture logging actionable#13553
joe4dev merged 1 commit intomainfrom
fix/improve-ci-cleanup-logging

Conversation

@joe4dev
Copy link
Copy Markdown
Member

@joe4dev joe4dev commented Dec 19, 2025

Motivation

A full CI run (community + pro) creates ~170 warnings "Failed to execute cleanup" (using full-text search from downloaded logs).
The AI-based analysis (likely incomplete, but the list can serve as a starting point) listed 64 occurrences linked to the following 42 test cases:

tests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_streams_on_global_tables
tests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_table_crud
tests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_route_table_association
tests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_subnet_with_tags
tests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_subnet_with_custom_id
tests/aws/services/lambda_/event_source_mapping/test_lambda_integration_dynamodbstreams.py::TestDynamoDBEventSourceMapping::test_deletion_event_source_mapping_with_dynamodb

tests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_event_source_mapping_lifecycle_delete_function
tests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_event_source_mapping_lifecycle
tests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_function_name_variations
tests/aws/services/opensearch/test_opensearch.py::TestOpensearchProvider::test_domain_lifecycle
tests/aws/services/route53/test_route53.py::TestRoute53::test_associate_vpc_with_hosted_zone
tests/aws/services/route53resolver/test_route53resolver.py::TestRoute53Resolver::test_delete_resolver_endpoint
tests/aws/services/route53resolver/test_route53resolver.py::TestRoute53Resolver::test_create_resolver_query_log_config
tests/aws/services/scheduler/test_scheduler.py::tests_create_schedule_with_valid_schedule_expression
tests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_invalid_secret_name[Inv
tests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_invalid_secret_name[
tests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_event_source_mapping_lifecycle_delete_function
tests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_event_source_mapping_lifecycle
tests/aws/services/lambda_/test_lambda_api.py::TestLambdaEventSourceMappings::test_function_name_variations

tests/aws/services/opensearch/test_opensearch.py::TestOpensearchProvider::test_domain_lifecycle
tests/aws/services/route53/test_route53.py::TestRoute53::test_associate_vpc_with_hosted_zone
tests/aws/services/route53resolver/test_route53resolver.py::TestRoute53Resolver::test_delete_resolver_endpoint
tests/aws/services/route53resolver/test_route53resolver.py::TestRoute53Resolver::test_create_resolver_query_log_config
tests/aws/services/scheduler/test_scheduler.py::tests_create_schedule_with_valid_schedule_expression
tests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_invalid_secret_name[Inv
tests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_invalid_secret_name[
tests/aws/test_multi_accounts.py::TestMultiAccounts::test_multi_accounts_dynamodb
tests/aws/services/apigateway/test_apigateway_basic.py::TestAPIGateway::test_multiple_api_keys_validate

tests/aws/services/sns/test_sns.py::TestSNSSMS::test_opt_in_phone_number
tests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_streams_on_global_tables
tests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_table_crud
tests/aws/services/lambda_/event_source_mapping/test_lambda_integration_dynamodbstreams.py::TestDynamoDBEventSourceMapping::test_deletion_event_source_mapping_with_dynamodb
tests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_streams_on_global_tables
tests/aws/services/dynamodb/test_dynamodb.py::TestDynamoDB::test_table_crud
tests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_route_table_association
tests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_subnet_with_tags
tests/aws/services/ec2/test_ec2.py::TestEc2Integrations::test_create_subnet_with_custom_id
tests/aws/services/lambda_/event_source_mapping/test_lambda_integration_dynamodbstreams.py::TestDynamoDBEventSourceMapping::test_deletion_event_source_mapping_with_dynamodb
tests/aws/test_multi_accounts.py::TestMultiAccounts::test_multi_accounts_dynamodb

Changes

  • Clarify the warning log message to include the actionable error message
  • Enrich the warning log message to suggest that the cleanup might not be necessary upon hitting 404s

Related

Follow-up from the AI Winter hackathon on Automated Log Noise Reduction

Companion PR demonstrating further fixes.

PR demonstrating a fix for ESM: https://github.com/localstack/localstack/pull/13546/changes#diff-2d1f022dc5ab81edd6a3789403f602fd45ac88ed37d8151b5b98d1ac617a2ce3

@joe4dev joe4dev added this to the 4.13 milestone Dec 19, 2025
@joe4dev joe4dev self-assigned this Dec 19, 2025
@joe4dev joe4dev added semver: patch Non-breaking changes which can be included in patch releases docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes labels Dec 19, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 19, 2025

Test Results - Preflight, Unit

22 982 tests  ±0   21 140 ✅ ±0   6m 28s ⏱️ +27s
     1 suites ±0    1 842 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit 30d8377. ± Comparison against base commit 056da83.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 19, 2025

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 1s ⏱️ +12s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit 30d8377. ± Comparison against base commit 056da83.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 19, 2025

Test Results (amd64) - Integration, Bootstrap

    5 files      5 suites   2h 33m 26s ⏱️
5 559 tests 4 999 ✅ 560 💤 0 ❌
5 565 runs  4 999 ✅ 566 💤 0 ❌

Results for commit 30d8377.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 19, 2025

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 57m 1s ⏱️ -11s
5 153 tests ±0  4 756 ✅ ±0  397 💤 ±0  0 ❌ ±0 
5 155 runs  ±0  4 756 ✅ ±0  399 💤 ±0  0 ❌ ±0 

Results for commit 30d8377. ± Comparison against base commit 056da83.

@joe4dev joe4dev marked this pull request as ready for review December 20, 2025 11:11
@joe4dev joe4dev requested a review from cloutierMat December 20, 2025 11:13
cloutierMat
cloutierMat approved these changes Dec 23, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@cloutierMat cloutierMat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is awesome @joe4dev!

Looking forward to see some actionable we can derive from that and help our pipeline to get healthier!

@joe4dev joe4dev merged commit 3bcdd33 into main Jan 7, 2026
55 checks passed
@joe4dev joe4dev deleted the fix/improve-ci-cleanup-logging branch January 7, 2026 09:10
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@dominikschubert dominikschubert Awaiting requested review from dominikschubert dominikschubert is a code owner

1 more reviewer

@cloutierMat cloutierMat cloutierMat approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@joe4dev joe4dev

Labels

docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes semver: patch Non-breaking changes which can be included in patch releases

Projects

None yet

Milestone

4.13

Development

Successfully merging this pull request may close these issues.

2 participants

@joe4dev @cloutierMat