feat(aws): Add Support for Securityhub findings #7204

bbernays · 2023-01-25T22:44:30Z

Summary

The results from GetFindings include a region and account_id which identifies where the finding was actually generated NOT where it was pulled from. I have changed the name of the fields we add to request_region and request_account_id to disambiguate...

github-actions · 2023-01-25T22:44:47Z

This PR has the following changes to source plugin(s) tables:

Table aws_securityhub_findings was added

plugins/source/aws/resources/services/securityhub/findings.go

Co-authored-by: Kemal <[email protected]>

erezrokah

Looks great 🚀 One question also related to #7203

erezrokah · 2023-01-26T08:30:29Z

plugins/source/aws/resources/services/securityhub/findings.go

+		Multiplex:   client.ServiceAccountRegionMultiplexer("securityhub"),
+		Columns: []schema.Column{
+			{
+				Name:     "request_account_id",


Can AwsAccountId and Region have different values from the ones in the multiplexer?
Might be confusing to have both so maybe keep only the struct ones (even if different)?

Yeah if I understood the PR description correctly, the values can be different from the one in the multiplexer, so I think this is a good compromise. @bbernays Maybe we can just add some details about this to the description of the table?

Ah 🤦 I missed that. Maybe we can drop the request_ ones for now?

I think it depends: can security hub findings be different depending on the account/region the request is done from? If so, I think we should include the columns (and make them part of the PK?). If not, maybe we shouldn't be multiplexing on account and region

Yes, Security Hub can aggregate findings from different regions based on user configuration

I will add a comment in the table description. Are there any other changes needed?

hermanschaaf

LGTM, will just need to run make gen again

erezrokah · 2023-01-29T11:26:18Z

/gen sha=371d504d7729ea7c1d074b16277d28aa895d9707 plugin=aws

stale

erezrokah · 2023-01-29T11:38:02Z

plugins/source/aws/resources/services/securityhub/findings.go

+	return &schema.Table{
+		Name: "aws_securityhub_findings",
+		Description: `https://docs.aws.amazon.com/securityhub/1.0/APIReference/API_GetFindings.html. 
+The 'request_account_id' and 'request_region' columns are added to show the account and region of where the request was made from. This is useful when multi region and account aggregation is enabled.`,


#### Summary The results from `GetFindings` include a `region` and `account_id` which identifies where the finding was actually generated NOT where it was pulled from. I have changed the name of the fields we add to `request_region` and `request_account_id` to disambiguate... <!--

🤖 I have created a release *beep* *boop* --- ## [13.0.0](plugins-source-aws-v12.2.1...plugins-source-aws-v13.0.0) (2023-02-01) ### ⚠ BREAKING CHANGES * **aws:** PK for `aws_eventbridge_event_bus_rules` ([#7394](#7394)) * **aws:** PKs for `aws_apigatewayv2_*` ([#7380](#7380)) * **aws:** PK for `aws_cloudtrail_trails` ([#7467](#7467)) * **aws:** PK for `aws_route53_hosted_zone_traffic_policy_instances` ([#7412](#7412)) * **aws:** PK for `aws_qldb_ledger_journal_kinesis_streams` ([#7409](#7409)) * **aws:** PK for `aws_mq_broker_configurations` ([#7407](#7407)) * **aws:** PKs for `aws_lightsail_` tables ([#7401](#7401)) * **aws:** PK for `aws_lambda_layer_versions` ([#7399](#7399)) * **aws:** PK for `aws_iam_user_groups` ([#7397](#7397)) * **aws:** PK for `aws_glue_registry_schemas` ([#7395](#7395)) * **aws:** PK for `aws_apigateway_vpc_links` ([#7379](#7379)) * **aws:** PK for `aws_apigateway_usage_plan_keys` ([#7376](#7376)) * **aws:** PK for `aws_apigateway_usage_plans` ([#7375](#7375)) * **aws:** PK for `aws_apigateway_rest_api_stages` ([#7373](#7373)) * **aws:** PK for `aws_apigateway_rest_api_resources` ([#7369](#7369)) * **aws:** PK for `aws_apigateway_rest_api_request_validators` ([#7368](#7368)) * **aws:** PK for `aws_apigateway_rest_api_models` ([#7366](#7366)) * **aws:** PK for `aws_apigateway_rest_api_gateway_responses` ([#7364](#7364)) * **aws:** PK for `aws_apigateway_rest_api_documentation_versions` ([#7360](#7360)) * **aws:** PK for `aws_apigateway_rest_api_deployments` ([#7356](#7356)) * **aws:** PK for `aws_apigateway_rest_api_documentation_parts` ([#7355](#7355)) * **aws:** PK for `aws_apigateway_rest_api_authorizers` ([#7350](#7350)) * **aws:** PK for `aws_apigateway_domain_name_base_path_mappings` * **aws:** RDS Engine Version PKs ([#7202](#7202)) ### Features * **aws:** Add Support for Securityhub findings ([#7204](#7204)) ([d96496a](d96496a)) ### Bug Fixes * **aws:** PK for `aws_apigateway_domain_name_base_path_mappings` ([8a945be](8a945be)) * **aws:** PK for `aws_apigateway_rest_api_authorizers` ([#7350](#7350)) ([33e110f](33e110f)) * **aws:** PK for `aws_apigateway_rest_api_deployments` ([#7356](#7356)) ([d5f6fd5](d5f6fd5)) * **aws:** PK for `aws_apigateway_rest_api_documentation_parts` ([#7355](#7355)) ([2d81f86](2d81f86)) * **aws:** PK for `aws_apigateway_rest_api_documentation_versions` ([#7360](#7360)) ([d6a5aea](d6a5aea)) * **aws:** PK for `aws_apigateway_rest_api_gateway_responses` ([#7364](#7364)) ([6e779ac](6e779ac)) * **aws:** PK for `aws_apigateway_rest_api_models` ([#7366](#7366)) ([aad7dd6](aad7dd6)) * **aws:** PK for `aws_apigateway_rest_api_request_validators` ([#7368](#7368)) ([0394e31](0394e31)) * **aws:** PK for `aws_apigateway_rest_api_resources` ([#7369](#7369)) ([0c8fa30](0c8fa30)) * **aws:** PK for `aws_apigateway_rest_api_stages` ([#7373](#7373)) ([b49ee55](b49ee55)) * **aws:** PK for `aws_apigateway_usage_plan_keys` ([#7376](#7376)) ([4d7a76f](4d7a76f)) * **aws:** PK for `aws_apigateway_usage_plans` ([#7375](#7375)) ([bac329d](bac329d)) * **aws:** PK for `aws_apigateway_vpc_links` ([#7379](#7379)) ([60bd130](60bd130)) * **aws:** PK for `aws_cloudtrail_trails` ([#7467](#7467)) ([f580207](f580207)) * **aws:** PK for `aws_eventbridge_event_bus_rules` ([#7394](#7394)) ([e1e7405](e1e7405)) * **aws:** PK for `aws_glue_registry_schemas` ([#7395](#7395)) ([de43500](de43500)) * **aws:** PK for `aws_iam_user_groups` ([#7397](#7397)) ([ced05f6](ced05f6)) * **aws:** PK for `aws_lambda_layer_versions` ([#7399](#7399)) ([744c124](744c124)) * **aws:** PK for `aws_mq_broker_configurations` ([#7407](#7407)) ([3b6f383](3b6f383)) * **aws:** PK for `aws_qldb_ledger_journal_kinesis_streams` ([#7409](#7409)) ([101cdb6](101cdb6)) * **aws:** PK for `aws_route53_hosted_zone_traffic_policy_instances` ([#7412](#7412)) ([4bf36e7](4bf36e7)) * **aws:** PKs for `aws_apigatewayv2_*` ([#7380](#7380)) ([48f5463](48f5463)) * **aws:** PKs for `aws_lightsail_` tables ([#7401](#7401)) ([0b218b0](0b218b0)) * **aws:** RDS Engine Version PKs ([#7202](#7202)) ([a49984e](a49984e)) * **deps:** Update github.com/gocarina/gocsv digest to 763e25b ([#7529](#7529)) ([9aaa696](9aaa696)) * **deps:** Update golang.org/x/exp digest to f062dba ([#7531](#7531)) ([59d5575](59d5575)) * **deps:** Update google.golang.org/genproto digest to 1c01626 ([#7533](#7533)) ([c549c27](c549c27)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/feature/s3/manager to v1.11.49 ([#7309](#7309)) ([16da39d](16da39d)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/accessanalyzer to v1.19.1 ([#7310](#7310)) ([3bb3d78](3bb3d78)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/acm to v1.17.1 ([#7311](#7311)) ([829e2c0](829e2c0)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/apigateway to v1.16.1 ([#7312](#7312)) ([1fd914d](1fd914d)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/apigatewayv2 to v1.13.1 ([#7313](#7313)) ([b454a62](b454a62)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/applicationautoscaling to v1.17.1 ([#7314](#7314)) ([b6a0807](b6a0807)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/autoscaling to v1.26.1 ([#7315](#7315)) ([956fddc](956fddc)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudformation to v1.25.1 ([#7316](#7316)) ([de1cb2d](de1cb2d)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudtrail to v1.22.1 ([#7317](#7317)) ([3a454c4](3a454c4)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudwatch to v1.25.1 ([#7318](#7318)) ([4103fb9](4103fb9)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs to v1.20.1 ([#7319](#7319)) ([12cdbdb](12cdbdb)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/configservice to v1.29.1 ([#7320](#7320)) ([3535303](3535303)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/databasemigrationservice to v1.23.1 ([#7321](#7321)) ([4c3f0ae](4c3f0ae)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/directconnect to v1.18.1 ([#7322](#7322)) ([0ea2664](0ea2664)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/docdb to v1.20.1 ([#7323](#7323)) ([03ca605](03ca605)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/dynamodb to v1.18.1 ([#7324](#7324)) ([6097895](6097895)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ecr to v1.18.1 ([#7325](#7325)) ([66cd4f7](66cd4f7)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ecs to v1.23.1 ([#7326](#7326)) ([7794001](7794001)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticache to v1.26.1 ([#7327](#7327)) ([4c203a5](4c203a5)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticloadbalancing to v1.15.1 ([#7328](#7328)) ([7393b13](7393b13)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to v1.19.1 ([#7329](#7329)) ([d288551](d288551)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticsearchservice to v1.18.1 ([#7330](#7330)) ([5e8e42c](5e8e42c)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/emr to v1.22.1 ([#7331](#7331)) ([d914db7](d914db7)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.32.0 ([#7334](#7334)) ([b684122](b684122)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.33.0 ([#7595](#7595)) ([c5adc75](c5adc75)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.33.1 ([#7614](#7614)) ([2fe665c](2fe665c)) * Update endpoints ([#7521](#7521)) ([004d433](004d433)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

bbernays added 6 commits January 25, 2023 15:07

security hub mocked locally

8cf066f

Update tables.md

da11064

Update findings.go

3c14f1f

Create aws_securityhub_findings.md

25a62c5

Update README.md

3143dc5

Update resolvers.go

4a10cbe

bbernays requested review from hermanschaaf and yevgenypats as code owners January 25, 2023 22:44

cq-bot added aws labels Jan 25, 2023

disq reviewed Jan 25, 2023

View reviewed changes

plugins/source/aws/resources/services/securityhub/findings.go Outdated Show resolved Hide resolved

bbernays and others added 2 commits January 25, 2023 16:53

Update plugins/source/aws/resources/services/securityhub/findings.go

c040c09

Co-authored-by: Kemal <[email protected]>

Update aws_securityhub_findings.md

d8be0d0

bbernays linked an issue Jan 26, 2023 that may be closed by this pull request

AWS Security Hub Findings #7187

Closed

erezrokah previously requested changes Jan 26, 2023

View reviewed changes

Merge branch 'main' into securityhub

39cf00e

disq approved these changes Jan 26, 2023

View reviewed changes

Merge branch 'main' into securityhub

89d0d2f

vercel bot deployed to Preview January 26, 2023 20:40 View deployment

bbernays added 4 commits January 26, 2023 16:06

Update findings_mock_test.go

64cfe3e

Update go.mod

2de6dba

Merge remote-tracking branch 'upstream/main' into securityhub

400cb9b

Update findings.go

371d504

bbernays requested a review from erezrokah January 27, 2023 18:40

hermanschaaf approved these changes Jan 28, 2023

View reviewed changes

erezrokah added the automerge Automatically merge once required checks pass label Jan 29, 2023

Merge branch 'main' into securityhub

4ba7a7c

vercel bot deployed to Preview January 29, 2023 11:30 View deployment

chore: Update code and docs

93da0c7

erezrokah reviewed Jan 29, 2023

View reviewed changes

kodiakhq bot merged commit d96496a into cloudquery:main Jan 29, 2023

cq-bot mentioned this pull request Jan 29, 2023

chore(main): Release plugins-source-aws v13.0.0 #7304

Closed

This was referenced Jan 31, 2023

chore(main): Release plugins-source-aws v13.0.0 #7513

Closed

chore(main): Release plugins-source-aws v13.0.0 #7520

Merged

bbernays deleted the securityhub branch March 8, 2024 01:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(aws): Add Support for Securityhub findings #7204

feat(aws): Add Support for Securityhub findings #7204

Uh oh!

bbernays commented Jan 25, 2023

Uh oh!

github-actions bot commented Jan 25, 2023

Uh oh!

Uh oh!

erezrokah left a comment

Uh oh!

erezrokah Jan 26, 2023

Uh oh!

hermanschaaf Jan 26, 2023

Uh oh!

erezrokah Jan 26, 2023

Uh oh!

hermanschaaf Jan 26, 2023

Uh oh!

bbernays Jan 26, 2023

Uh oh!

bbernays Jan 27, 2023

Uh oh!

hermanschaaf left a comment

Uh oh!

erezrokah commented Jan 29, 2023

Uh oh!

erezrokah Jan 29, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

feat(aws): Add Support for Securityhub findings #7204

feat(aws): Add Support for Securityhub findings #7204

Uh oh!

Conversation

bbernays commented Jan 25, 2023

Summary

Uh oh!

github-actions bot commented Jan 25, 2023

This PR has the following changes to source plugin(s) tables:

Uh oh!

Uh oh!

erezrokah left a comment

Choose a reason for hiding this comment

Uh oh!

erezrokah Jan 26, 2023

Choose a reason for hiding this comment

Uh oh!

hermanschaaf Jan 26, 2023

Choose a reason for hiding this comment

Uh oh!

erezrokah Jan 26, 2023

Choose a reason for hiding this comment

Uh oh!

hermanschaaf Jan 26, 2023

Choose a reason for hiding this comment

Uh oh!

bbernays Jan 26, 2023

Choose a reason for hiding this comment

Uh oh!

bbernays Jan 27, 2023

Choose a reason for hiding this comment

Uh oh!

hermanschaaf left a comment

Choose a reason for hiding this comment

Uh oh!

erezrokah commented Jan 29, 2023

Uh oh!

erezrokah Jan 29, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants