Skip to content

Use transaction in down migration for comments#23715

Merged
rijkvanzanten merged 1 commit intoretentions-p1from
comments-down-trx
Sep 26, 2024
Merged

Use transaction in down migration for comments#23715
rijkvanzanten merged 1 commit intoretentions-p1from
comments-down-trx

Conversation

@licitdev
Copy link
Member

Scope

What's changed:

  • Use transaction in down migration for comments to prevent corruption of data
  • Use a while loop to process in chunks until no comment exists before dropping the table

Potential Risks / Drawbacks

  • Would rowsLimit of 50 be an issue for tiny instances?

Review Notes / Questions

@changeset-bot

This comment was marked as resolved.

@rijkvanzanten rijkvanzanten merged commit 6bec681 into retentions-p1 Sep 26, 2024
@rijkvanzanten rijkvanzanten deleted the comments-down-trx branch September 26, 2024 08:14
ComfortablyCoding added a commit that referenced this pull request Oct 16, 2024
* Extract comments to a separate table (#22295)

* Add directus_comments migration

* Add comments controller and service

* Remove from activity

* Update system-data and types

* Refactor app with new endpoints

* Expose service

* Update app minimal permissions

* Add collection translation

* Define relations

* Allow comment creation only if there's item read access

* Patch for MSSQL double constraints issue

* Fix users service test

* Add sdk support

* Update specs

* Fix formatting

* Fix specs error

* Patch whoopsie

* Remove obsolete GraphQL mutations

* Update required fields

* Remove unused vars

* Allow edit and delete of legacy activity comments

* Remove legacy comments from SDK

* Add changeset

* Batch upwards migration

* Update SDK to use keysOrQuery

Co-authored-by: Brainslug <[email protected]>

* Update implementation for keysOrQuery

* Remove singleton check

* Update SDK to use keysOrQuery 2

Co-authored-by: Brainslug <[email protected]>

* Update keysOrQuery typedoc

* Fix import

* Update migration timestamp

* fixed import

* Update api/src/utils/get-service.ts

* utilize chunk processing in migration

* formatting

* only services extended from itemservice should be added

* remove redundant checks from comment header

* update comment service to v11 permission format

* specify missing required fields

* Mock comments in users test

* Simplify migration and update date

* WIP legacy access

* Optimise imports

* WIP app cleanup

* Update loadUserPreviews typing

Co-authored-by: Daniel Biegler <[email protected]>

* Read legacy comments

* Parse using comments service

* Perform migration directly

* Fix legacy app sort query which uses id

* Migrate legacy comments in mutations

* Reduce api semver

* Update app recommended permissions

* Recommend updating of comment only

* replace hardcoded type with existing one

* Allow users to update or delete their own comments

Co-authored-by: daedalus <[email protected]>

* Skip further access validation for non-existent collections

* Check if collection exists before the admin check

Co-authored-by: daedalus <[email protected]>

* Fix incorrect legacy check

Co-authored-by: Daniel Biegler <[email protected]>

* Fix merging of count when db returns count as string type

* Remove unused import

---------

Co-authored-by: Brainslug <[email protected]>
Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: Brainslug <[email protected]>
Co-authored-by: daedalus <[email protected]>
Co-authored-by: Daniel Biegler <[email protected]>

* Consolidate content versioning (#22413)

* Add migration

* Use the new delta field

* Add cast-json flag

* Fix typing

* Fetch existing deltas if version created during migration

* Add changeset

* Add version delta field into sdk schema

* Update migration timestamp

* Update versions.save() to return finalVersionDelta

Co-authored-by: Pascal Jufer <[email protected]>

* Sort on DB level

* Update migration date

* Disallow passing delta via create/update

* Update docs & specs

* Fix save response

* Remove unnecessary access check

Already checked by the subsequent itemsService.readOne call

* Update changeset

* Don't require update perms on versions for save

* Optimize validateCreateData

* update to new validateAccess

* Update docs/reference/system/versions.md

* Remove migration of delta

* Rename to legacy

* Add missed changes for Remove migration of delta in 2e2f50f

* Update docs/reference/system/versions.md

---------

Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: daedalus <[email protected]>

* Update migration dates

* api change should be major for versioning

* Remove comment paths from activity reference

* Added comments reference

* Added directus_comments to table of system collections

* The linter demands newline

* Revert function renaming for patch semver

* Use transaction in down migration for comments (#23715)

---------

Co-authored-by: Brainslug <[email protected]>
Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: Brainslug <[email protected]>
Co-authored-by: daedalus <[email protected]>
Co-authored-by: Daniel Biegler <[email protected]>
Co-authored-by: Kevin Lewis <[email protected]>
Co-authored-by: Rijk van Zanten <[email protected]>
DanielBiegler added a commit that referenced this pull request Nov 18, 2024
* Extract comments to a separate table (#22295)

* Add directus_comments migration

* Add comments controller and service

* Remove from activity

* Update system-data and types

* Refactor app with new endpoints

* Expose service

* Update app minimal permissions

* Add collection translation

* Define relations

* Allow comment creation only if there's item read access

* Patch for MSSQL double constraints issue

* Fix users service test

* Add sdk support

* Update specs

* Fix formatting

* Fix specs error

* Patch whoopsie

* Remove obsolete GraphQL mutations

* Update required fields

* Remove unused vars

* Allow edit and delete of legacy activity comments

* Remove legacy comments from SDK

* Add changeset

* Batch upwards migration

* Update SDK to use keysOrQuery

Co-authored-by: Brainslug <[email protected]>

* Update implementation for keysOrQuery

* Remove singleton check

* Update SDK to use keysOrQuery 2

Co-authored-by: Brainslug <[email protected]>

* Update keysOrQuery typedoc

* Fix import

* Update migration timestamp

* fixed import

* Update api/src/utils/get-service.ts

* utilize chunk processing in migration

* formatting

* only services extended from itemservice should be added

* remove redundant checks from comment header

* update comment service to v11 permission format

* specify missing required fields

* Mock comments in users test

* Simplify migration and update date

* WIP legacy access

* Optimise imports

* WIP app cleanup

* Update loadUserPreviews typing

Co-authored-by: Daniel Biegler <[email protected]>

* Read legacy comments

* Parse using comments service

* Perform migration directly

* Fix legacy app sort query which uses id

* Migrate legacy comments in mutations

* Reduce api semver

* Update app recommended permissions

* Recommend updating of comment only

* replace hardcoded type with existing one

* Allow users to update or delete their own comments

Co-authored-by: daedalus <[email protected]>

* Skip further access validation for non-existent collections

* Check if collection exists before the admin check

Co-authored-by: daedalus <[email protected]>

* Fix incorrect legacy check

Co-authored-by: Daniel Biegler <[email protected]>

* Fix merging of count when db returns count as string type

* Remove unused import

---------

Co-authored-by: Brainslug <[email protected]>
Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: Brainslug <[email protected]>
Co-authored-by: daedalus <[email protected]>
Co-authored-by: Daniel Biegler <[email protected]>

* Consolidate content versioning (#22413)

* Add migration

* Use the new delta field

* Add cast-json flag

* Fix typing

* Fetch existing deltas if version created during migration

* Add changeset

* Add version delta field into sdk schema

* Update migration timestamp

* Update versions.save() to return finalVersionDelta

Co-authored-by: Pascal Jufer <[email protected]>

* Sort on DB level

* Update migration date

* Disallow passing delta via create/update

* Update docs & specs

* Fix save response

* Remove unnecessary access check

Already checked by the subsequent itemsService.readOne call

* Update changeset

* Don't require update perms on versions for save

* Optimize validateCreateData

* update to new validateAccess

* Update docs/reference/system/versions.md

* Remove migration of delta

* Rename to legacy

* Add missed changes for Remove migration of delta in 2e2f50f

* Update docs/reference/system/versions.md

---------

Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: daedalus <[email protected]>

* Update migration dates

* api change should be major for versioning

* Remove comment paths from activity reference

* Added comments reference

* Added directus_comments to table of system collections

* The linter demands newline

* Revert function renaming for patch semver

* Use transaction in down migration for comments (#23715)

* Remove outdated logic for versioning

* Fix migration

* Update api/src/database/migrations/20240924B-populate-versioning-deltas.ts

* add changeset

* reword changeset

---------

Co-authored-by: Brainslug <[email protected]>
Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: Brainslug <[email protected]>
Co-authored-by: daedalus <[email protected]>
Co-authored-by: Daniel Biegler <[email protected]>
Co-authored-by: Kevin Lewis <[email protected]>
Co-authored-by: Rijk van Zanten <[email protected]>
DanielBiegler added a commit that referenced this pull request Nov 19, 2024
* Extract comments to a separate table (#22295)

* Add directus_comments migration

* Add comments controller and service

* Remove from activity

* Update system-data and types

* Refactor app with new endpoints

* Expose service

* Update app minimal permissions

* Add collection translation

* Define relations

* Allow comment creation only if there's item read access

* Patch for MSSQL double constraints issue

* Fix users service test

* Add sdk support

* Update specs

* Fix formatting

* Fix specs error

* Patch whoopsie

* Remove obsolete GraphQL mutations

* Update required fields

* Remove unused vars

* Allow edit and delete of legacy activity comments

* Remove legacy comments from SDK

* Add changeset

* Batch upwards migration

* Update SDK to use keysOrQuery

Co-authored-by: Brainslug <[email protected]>

* Update implementation for keysOrQuery

* Remove singleton check

* Update SDK to use keysOrQuery 2

Co-authored-by: Brainslug <[email protected]>

* Update keysOrQuery typedoc

* Fix import

* Update migration timestamp

* fixed import

* Update api/src/utils/get-service.ts

* utilize chunk processing in migration

* formatting

* only services extended from itemservice should be added

* remove redundant checks from comment header

* update comment service to v11 permission format

* specify missing required fields

* Mock comments in users test

* Simplify migration and update date

* WIP legacy access

* Optimise imports

* WIP app cleanup

* Update loadUserPreviews typing

Co-authored-by: Daniel Biegler <[email protected]>

* Read legacy comments

* Parse using comments service

* Perform migration directly

* Fix legacy app sort query which uses id

* Migrate legacy comments in mutations

* Reduce api semver

* Update app recommended permissions

* Recommend updating of comment only

* replace hardcoded type with existing one

* Allow users to update or delete their own comments

Co-authored-by: daedalus <[email protected]>

* Skip further access validation for non-existent collections

* Check if collection exists before the admin check

Co-authored-by: daedalus <[email protected]>

* Fix incorrect legacy check

Co-authored-by: Daniel Biegler <[email protected]>

* Fix merging of count when db returns count as string type

* Remove unused import

---------

Co-authored-by: Brainslug <[email protected]>
Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: Brainslug <[email protected]>
Co-authored-by: daedalus <[email protected]>
Co-authored-by: Daniel Biegler <[email protected]>

* Consolidate content versioning (#22413)

* Add migration

* Use the new delta field

* Add cast-json flag

* Fix typing

* Fetch existing deltas if version created during migration

* Add changeset

* Add version delta field into sdk schema

* Update migration timestamp

* Update versions.save() to return finalVersionDelta

Co-authored-by: Pascal Jufer <[email protected]>

* Sort on DB level

* Update migration date

* Disallow passing delta via create/update

* Update docs & specs

* Fix save response

* Remove unnecessary access check

Already checked by the subsequent itemsService.readOne call

* Update changeset

* Don't require update perms on versions for save

* Optimize validateCreateData

* update to new validateAccess

* Update docs/reference/system/versions.md

* Remove migration of delta

* Rename to legacy

* Add missed changes for Remove migration of delta in 2e2f50f

* Update docs/reference/system/versions.md

---------

Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: daedalus <[email protected]>

* Update migration dates

* api change should be major for versioning

* Remove comment paths from activity reference

* Added comments reference

* Added directus_comments to table of system collections

* The linter demands newline

* Revert function renaming for patch semver

* Use transaction in down migration for comments (#23715)

* Remove outdated logic for comments

* remove duplicate app access permission

* remove unused import/params in gql

* remove remaining comment code in activity

* remove remaining activity logic in comment

* add changeset

* reword changeset

---------

Co-authored-by: Brainslug <[email protected]>
Co-authored-by: Pascal Jufer <[email protected]>
Co-authored-by: Brainslug <[email protected]>
Co-authored-by: daedalus <[email protected]>
Co-authored-by: Daniel Biegler <[email protected]>
Co-authored-by: Kevin Lewis <[email protected]>
Co-authored-by: Rijk van Zanten <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 26, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants