Skip to content

Conversation

@mrdrivingduck
Copy link
Contributor

Some interface impl accidentally update all columns of metadata. Although it is okay for SQL-based metadata engine, it is not necessary and may cause error on some distributed variant, like Citus for PostgreSQL, because the SQL updates the distributed column, even though the column value is not changed, just like:

UPDATE jfs_chunk SET inode = 1 ... WHERE inode = 1;

@CLAassistant
Copy link

CLAassistant commented May 28, 2025

CLA assistant check
All committers have signed the CLA.

@zhijian-pro zhijian-pro requested a review from eltonxiao124 May 29, 2025 02:11
Copy link
Contributor

@jiefenghuang jiefenghuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov
Copy link

codecov bot commented May 29, 2025

Codecov Report

Attention: Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.

Project coverage is 38.11%. Comparing base (e3d162f) to head (4b668f2).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
pkg/meta/sql.go 0.00% 12 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #6158       +/-   ##
===========================================
+ Coverage   21.32%   38.11%   +16.79%     
===========================================
  Files          30      165      +135     
  Lines       19642    49323    +29681     
===========================================
+ Hits         4188    18801    +14613     
- Misses      14958    28415    +13457     
- Partials      496     2107     +1611     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mrdrivingduck
Copy link
Contributor Author

mrdrivingduck commented May 29, 2025

@jiefenghuang Am I breaking something? Unit test errors from something which is not touched.

@jiefenghuang
Copy link
Contributor

@jiefenghuang Am I breaking something? Unit test errors from something which is not touched.

This was a unit test issue; fixed in #6159.

@mrdrivingduck
Copy link
Contributor Author

@jiefenghuang Am I breaking something? Unit test errors from something which is not touched.

This was a unit test issue; fixed in #6159.

OK. I will rebase after #6159 is merged.

@jiefenghuang
Copy link
Contributor

hi @mrdrivingduck , #6159 has merged.

pkg/meta/sql.go Outdated

c2.Slices = append(append(c2.Slices[:skipped*sliceBytes], marshalSlice(pos, id, size, 0, size)...), c2.Slices[len(origin):]...)
if _, err := s.Where("Inode = ? AND indx = ?", inode, indx).Update(c2); err != nil {
if _, err := s.Cols("slices").Update(c2, &chunk{Inode: inode, Indx: indx}); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indx will be ingored if it is zero

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we'd better keep the Where as origin, and add Cols only? 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, PTAL.

Some interface impl accidentally update all columns of metadata.
Although it is okay for SQL-based metadata engine, it is not necessary
and may cause error on some distributed variant, like Citus for
PostgreSQL, because the SQL updates the distributed column, even though
the column value is not changed, just like:

UPDATE jfs_chunk SET inode = 1 ... WHERE inode = 1;
@mrdrivingduck mrdrivingduck requested a review from davies May 29, 2025 08:11
@jiefenghuang jiefenghuang merged commit 0451f63 into juicedata:main May 29, 2025
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants