Skip to content

Conversation

@bbernays
Copy link
Collaborator

Summary

This is a unique resource... There is seemingly no set of fields that guarantee uniqueness... Also these values are not dependent on user input so they are 100% defined by AWS. For that reason I have just calculated a hash of the object and used that calculated hash to be the PK (along with region + account so users can determine what is available).

@bbernays bbernays linked an issue Jan 25, 2023 that may be closed by this pull request
@cq-bot cq-bot added the aws label Jan 25, 2023
@bbernays bbernays requested review from candiduslynx and disq January 25, 2023 20:28
@github-actions
Copy link

This PR has the following changes to source plugin(s) tables:

  • ⚠️ BREAKING CHANGE: Table aws_rds_engine_versions: column added with name _engine_version_hash (PK) and type String
  • Table aws_rds_engine_versions: primary key constraint removed from column db_engine_version_description
  • Table aws_rds_engine_versions: primary key constraint removed from column engine_version
  • Table aws_rds_engine_versions: primary key constraint removed from column engine

@bbernays bbernays changed the title fix(aws): RDS Engine Version PKs fix(aws)!: RDS Engine Version PKs Jan 25, 2023
@candiduslynx
Copy link
Contributor

@bbernays I think the easiest way would be just adding _cq_id to the primary key.
So, the resulting pk should be: account, region, engine, engine_version, _cq_id

@erezrokah
Copy link
Member

I think the easiest way would be just adding _cq_id to the primary key.

_cq_id changes on every sync so I'm not sure that would work in overwrite mode

@candiduslynx
Copy link
Contributor

candiduslynx commented Jan 26, 2023

_cq_id changes on every sync so I'm not sure that would work in overwrite mode

Good catch!
Still, I want to keep engine & engine_version in PK, so that the only change is replacing engine_version_description with _engine_version_hash

@hermanschaaf
Copy link
Member

I don't understand: why is it better to use a hash of the engine version as the PK, rather than the engine version directly? (Or for that matter, if any set of fields could be hashed and make a decent PK, why don't we use that set of fields as the PK directly?)

@candiduslynx
Copy link
Contributor

candiduslynx commented Jan 26, 2023

@hermanschaaf

I don't understand: why is it better to use a hash of the engine version as the PK, rather than the engine version directly? (Or for that matter, if any set of fields could be hashed and make a decent PK, why don't we use that set of fields as the PK directly?)

_engine_version_hash is calculated based on the whole structure (not just a single field).

@hermanschaaf
Copy link
Member

_engine_version_hash is calculated based on the whole structure (not just a single field).

Ah it's the whole item 🤦 Yeah sorry, I thought it was a single field.

@bbernays bbernays requested a review from candiduslynx January 26, 2023 16:39
@kodiakhq kodiakhq bot merged commit a49984e into cloudquery:main Jan 30, 2023
@bbernays bbernays deleted the DBEngineVersion-PK branch January 30, 2023 21:41
kodiakhq bot pushed a commit that referenced this pull request Feb 1, 2023
🤖 I have created a release *beep* *boop*
---


## [13.0.0](plugins-source-aws-v12.2.1...plugins-source-aws-v13.0.0) (2023-02-01)


### ⚠ BREAKING CHANGES

* **aws:** PK for `aws_eventbridge_event_bus_rules` ([#7394](#7394))
* **aws:** PKs for `aws_apigatewayv2_*` ([#7380](#7380))
* **aws:** PK for `aws_cloudtrail_trails` ([#7467](#7467))
* **aws:** PK for `aws_route53_hosted_zone_traffic_policy_instances` ([#7412](#7412))
* **aws:** PK for `aws_qldb_ledger_journal_kinesis_streams` ([#7409](#7409))
* **aws:** PK for `aws_mq_broker_configurations` ([#7407](#7407))
* **aws:** PKs for `aws_lightsail_` tables ([#7401](#7401))
* **aws:** PK for `aws_lambda_layer_versions` ([#7399](#7399))
* **aws:** PK for `aws_iam_user_groups` ([#7397](#7397))
* **aws:** PK for `aws_glue_registry_schemas` ([#7395](#7395))
* **aws:** PK for `aws_apigateway_vpc_links` ([#7379](#7379))
* **aws:** PK for `aws_apigateway_usage_plan_keys` ([#7376](#7376))
* **aws:** PK for `aws_apigateway_usage_plans` ([#7375](#7375))
* **aws:** PK for `aws_apigateway_rest_api_stages` ([#7373](#7373))
* **aws:** PK for `aws_apigateway_rest_api_resources` ([#7369](#7369))
* **aws:** PK for `aws_apigateway_rest_api_request_validators` ([#7368](#7368))
* **aws:** PK for `aws_apigateway_rest_api_models` ([#7366](#7366))
* **aws:** PK for `aws_apigateway_rest_api_gateway_responses` ([#7364](#7364))
* **aws:** PK for `aws_apigateway_rest_api_documentation_versions` ([#7360](#7360))
* **aws:** PK for `aws_apigateway_rest_api_deployments` ([#7356](#7356))
* **aws:** PK for `aws_apigateway_rest_api_documentation_parts` ([#7355](#7355))
* **aws:** PK for `aws_apigateway_rest_api_authorizers` ([#7350](#7350))
* **aws:** PK for `aws_apigateway_domain_name_base_path_mappings`
* **aws:** RDS Engine Version PKs ([#7202](#7202))

### Features

* **aws:** Add Support for Securityhub findings ([#7204](#7204)) ([d96496a](d96496a))


### Bug Fixes

* **aws:** PK for `aws_apigateway_domain_name_base_path_mappings` ([8a945be](8a945be))
* **aws:** PK for `aws_apigateway_rest_api_authorizers` ([#7350](#7350)) ([33e110f](33e110f))
* **aws:** PK for `aws_apigateway_rest_api_deployments` ([#7356](#7356)) ([d5f6fd5](d5f6fd5))
* **aws:** PK for `aws_apigateway_rest_api_documentation_parts` ([#7355](#7355)) ([2d81f86](2d81f86))
* **aws:** PK for `aws_apigateway_rest_api_documentation_versions` ([#7360](#7360)) ([d6a5aea](d6a5aea))
* **aws:** PK for `aws_apigateway_rest_api_gateway_responses` ([#7364](#7364)) ([6e779ac](6e779ac))
* **aws:** PK for `aws_apigateway_rest_api_models` ([#7366](#7366)) ([aad7dd6](aad7dd6))
* **aws:** PK for `aws_apigateway_rest_api_request_validators` ([#7368](#7368)) ([0394e31](0394e31))
* **aws:** PK for `aws_apigateway_rest_api_resources` ([#7369](#7369)) ([0c8fa30](0c8fa30))
* **aws:** PK for `aws_apigateway_rest_api_stages` ([#7373](#7373)) ([b49ee55](b49ee55))
* **aws:** PK for `aws_apigateway_usage_plan_keys` ([#7376](#7376)) ([4d7a76f](4d7a76f))
* **aws:** PK for `aws_apigateway_usage_plans` ([#7375](#7375)) ([bac329d](bac329d))
* **aws:** PK for `aws_apigateway_vpc_links` ([#7379](#7379)) ([60bd130](60bd130))
* **aws:** PK for `aws_cloudtrail_trails` ([#7467](#7467)) ([f580207](f580207))
* **aws:** PK for `aws_eventbridge_event_bus_rules` ([#7394](#7394)) ([e1e7405](e1e7405))
* **aws:** PK for `aws_glue_registry_schemas` ([#7395](#7395)) ([de43500](de43500))
* **aws:** PK for `aws_iam_user_groups` ([#7397](#7397)) ([ced05f6](ced05f6))
* **aws:** PK for `aws_lambda_layer_versions` ([#7399](#7399)) ([744c124](744c124))
* **aws:** PK for `aws_mq_broker_configurations` ([#7407](#7407)) ([3b6f383](3b6f383))
* **aws:** PK for `aws_qldb_ledger_journal_kinesis_streams` ([#7409](#7409)) ([101cdb6](101cdb6))
* **aws:** PK for `aws_route53_hosted_zone_traffic_policy_instances` ([#7412](#7412)) ([4bf36e7](4bf36e7))
* **aws:** PKs for `aws_apigatewayv2_*` ([#7380](#7380)) ([48f5463](48f5463))
* **aws:** PKs for `aws_lightsail_` tables ([#7401](#7401)) ([0b218b0](0b218b0))
* **aws:** RDS Engine Version PKs ([#7202](#7202)) ([a49984e](a49984e))
* **deps:** Update github.com/gocarina/gocsv digest to 763e25b ([#7529](#7529)) ([9aaa696](9aaa696))
* **deps:** Update golang.org/x/exp digest to f062dba ([#7531](#7531)) ([59d5575](59d5575))
* **deps:** Update google.golang.org/genproto digest to 1c01626 ([#7533](#7533)) ([c549c27](c549c27))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/feature/s3/manager to v1.11.49 ([#7309](#7309)) ([16da39d](16da39d))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/accessanalyzer to v1.19.1 ([#7310](#7310)) ([3bb3d78](3bb3d78))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/acm to v1.17.1 ([#7311](#7311)) ([829e2c0](829e2c0))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/apigateway to v1.16.1 ([#7312](#7312)) ([1fd914d](1fd914d))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/apigatewayv2 to v1.13.1 ([#7313](#7313)) ([b454a62](b454a62))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/applicationautoscaling to v1.17.1 ([#7314](#7314)) ([b6a0807](b6a0807))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/autoscaling to v1.26.1 ([#7315](#7315)) ([956fddc](956fddc))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudformation to v1.25.1 ([#7316](#7316)) ([de1cb2d](de1cb2d))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudtrail to v1.22.1 ([#7317](#7317)) ([3a454c4](3a454c4))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudwatch to v1.25.1 ([#7318](#7318)) ([4103fb9](4103fb9))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs to v1.20.1 ([#7319](#7319)) ([12cdbdb](12cdbdb))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/configservice to v1.29.1 ([#7320](#7320)) ([3535303](3535303))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/databasemigrationservice to v1.23.1 ([#7321](#7321)) ([4c3f0ae](4c3f0ae))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/directconnect to v1.18.1 ([#7322](#7322)) ([0ea2664](0ea2664))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/docdb to v1.20.1 ([#7323](#7323)) ([03ca605](03ca605))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/dynamodb to v1.18.1 ([#7324](#7324)) ([6097895](6097895))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ecr to v1.18.1 ([#7325](#7325)) ([66cd4f7](66cd4f7))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/ecs to v1.23.1 ([#7326](#7326)) ([7794001](7794001))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticache to v1.26.1 ([#7327](#7327)) ([4c203a5](4c203a5))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticloadbalancing to v1.15.1 ([#7328](#7328)) ([7393b13](7393b13))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to v1.19.1 ([#7329](#7329)) ([d288551](d288551))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/elasticsearchservice to v1.18.1 ([#7330](#7330)) ([5e8e42c](5e8e42c))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/emr to v1.22.1 ([#7331](#7331)) ([d914db7](d914db7))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v1.32.0 ([#7334](#7334)) ([b684122](b684122))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v1.33.0 ([#7595](#7595)) ([c5adc75](c5adc75))
* **deps:** Update module github.com/cloudquery/plugin-sdk to v1.33.1 ([#7614](#7614)) ([2fe665c](2fe665c))
* Update endpoints ([#7521](#7521)) ([004d433](004d433))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge Automatically merge once required checks pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aws_rds_engine_versions

5 participants