Skip to content

Fixing growing bifros DB issue#3275

Merged
vinay-satish merged 2 commits intomainfrom
zei/INFS-138/fixing_growing_bifrost_db_issue
Jun 6, 2022
Merged

Fixing growing bifros DB issue#3275
vinay-satish merged 2 commits intomainfrom
zei/INFS-138/fixing_growing_bifrost_db_issue

Conversation

@vinay-satish
Copy link
Copy Markdown
Contributor

@vinay-satish vinay-satish commented Apr 21, 2022

Signed-off-by: Vinay Satish [email protected]

Description

The bifrost DB is growing due to a bug in the sandbox APIs. We use sandboxes APIs while uploading the cookbooks to bookshelf to make sure that partial cookbook content isn't served up.
When creating the sandbox, we also generate the authz_id for that sandbox. In the second half of the sandbox process, after we upload the cookbooks, we use an API to verify the checksum and delete the sandbox. In this API we are not deleting the authz_id of the sandbox. Because of this the data is growing in the bifrost database.

Issues Resolved

https://chefio.atlassian.net/browse/INFS-138

Issues Related

https://github.com/chef/umbrella/pull/253

Check List

@vinay-satish vinay-satish requested review from a team as code owners April 21, 2022 08:53
@netlify
Copy link
Copy Markdown

netlify bot commented Apr 21, 2022

👷 Deploy Preview for chef-server processing.

Name Link
🔨 Latest commit e345898
🔍 Latest deploy log https://app.netlify.com/sites/chef-server/deploys/629dc3426078880008d672cd

@vinay-satish vinay-satish changed the title Fixing growing bifros DB issue WIP/Fixing growing bifros DB issue Apr 21, 2022
@vinay-satish
Copy link
Copy Markdown
Contributor Author

ChecksumsToCommit = [ C || {C, false} <- Checksums ],
chef_db:mark_checksums_as_uploaded(Ctx, OrgId, ChecksumsToCommit),
chef_db:commit_sandbox(Ctx, Sandbox).
chef_db:commit_sandbox(Ctx, Sandbox),
Copy link
Copy Markdown
Member

@marcparadise marcparadise Apr 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this logic -- including the call to chef_db:commit_sandbox `` into chef_object_db:delete? It is sort of the centralized place for deleting top-level objects including the database entry, any indexing, and authz entry.

So the mark_checksums, the commit_sandbox, and the oc_chef_authz:delete_resource call.

chef_db_context = DbContext,
resource_state = #sandbox_state{chef_sandbox = Sandbox}} = State) ->
requestor_id = RequestorId,
resource_state = #sandbox_state{chef_sandbox = Sandbox, sandbox_authz_id = SBAuthz}} = State) ->
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems odd to me that we don't have the authz id in the #sandbox{} record itself, like we do for other objects -- even though we include it in the data we write to the DB. Have you been able to confirm that the authz id is available in the #sandbox_state{} record at this point in time?

Copy link
Copy Markdown
Contributor

@lbakerchef lbakerchef left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's still a WIP. Any test results yet to show this working? In addition to standard umbrella, I'm thinking some kind of log dump or screen shot showing that the db doesn't grow when subjected to the same steps to reproduce the issue.

@vinay-satish vinay-satish force-pushed the zei/INFS-138/fixing_growing_bifrost_db_issue branch 2 times, most recently from 08e7190 to d69763b Compare April 27, 2022 05:20
@vinay-satish
Copy link
Copy Markdown
Contributor Author

Yet to be tested on the hosted data

@vinay-satish vinay-satish force-pushed the zei/INFS-138/fixing_growing_bifrost_db_issue branch from d69763b to 512785a Compare May 16, 2022 14:17
@vinay-satish vinay-satish changed the title WIP/Fixing growing bifros DB issue Fixing growing bifros DB issue May 17, 2022
@vinay-satish vinay-satish force-pushed the zei/INFS-138/fixing_growing_bifrost_db_issue branch from 512785a to a1d601a Compare May 19, 2022 11:46
@vinay-satish
Copy link
Copy Markdown
Contributor Author

@vinay-satish vinay-satish force-pushed the zei/INFS-138/fixing_growing_bifrost_db_issue branch from a1d601a to c7b947e Compare May 27, 2022 10:04
@vinay-satish vinay-satish force-pushed the zei/INFS-138/fixing_growing_bifrost_db_issue branch from c7b947e to e345898 Compare June 6, 2022 09:05
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Jun 6, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@vinay-satish vinay-satish merged commit 507c0cd into main Jun 6, 2022
@vinay-satish vinay-satish deleted the zei/INFS-138/fixing_growing_bifrost_db_issue branch June 6, 2022 09:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants