Skip to content

Conversation

@VGalaxies
Copy link
Contributor

… commit (apache#15598)

Consider the historical data export snapshot scenario: 

1. The events delivered upstream are, in order, the tsfile event and the termination event. 
2. The tsfile event is parsed into multiple tablet events, and then the reference count of the tsfile event is set to 0 (should report as false). 
3. Assuming that for some reason the tablet events were not sent to the peer in time, the reference count of the transfer termination event is set to 0 (should report as true).
4. At this point, because the tablet events were not enriched with a commit id (see Subscription: fully managed tsfile parsing process for tsfile format topic apache#15524), the termination event successfully marks the corresponding DR complete, which in turn leads to data loss.
@SteveYurongSu SteveYurongSu self-assigned this May 29, 2025
@SteveYurongSu SteveYurongSu merged commit c8bff00 into apache:dev/1.3 May 29, 2025
25 checks passed
@VGalaxies VGalaxies deleted the cp-15598 branch May 30, 2025 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants