Skip to content

fix(dev): flush and sync files in file source tests#24243

Merged
thomasqueirozb merged 7 commits intomasterfrom
file-source-tests-flush-sync
Nov 14, 2025
Merged

fix(dev): flush and sync files in file source tests#24243
thomasqueirozb merged 7 commits intomasterfrom
file-source-tests-flush-sync

Conversation

@thomasqueirozb
Copy link
Copy Markdown
Contributor

@thomasqueirozb thomasqueirozb commented Nov 14, 2025

Summary

We should flush/sync instead of sleeping to ensure that files are written to disk properly. Sleeping is just hoping the OS will do our job for us

Vector configuration

NA

How did you test this PR?

cargo test -- sources::file::

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@thomasqueirozb thomasqueirozb requested a review from a team as a code owner November 14, 2025 18:50
@github-actions github-actions Bot added the domain: sources Anything related to the Vector's sources label Nov 14, 2025
Comment thread src/sources/file.rs
Copy link
Copy Markdown
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix. I see the pattern in many places:

file.flush().unwrap();
sleep_500_millis().await;

Could we remove some of these arbitrary sleeps?

You can test this by running cargo test -- --test-threads=8 repeatedly.

If it's too involved we can go ahead with this PR as is and follow-up in a separate PR.

@thomasqueirozb thomasqueirozb added this pull request to the merge queue Nov 14, 2025
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 14, 2025
@thomasqueirozb thomasqueirozb added this pull request to the merge queue Nov 14, 2025
Merged via the queue into master with commit b972620 Nov 14, 2025
45 checks passed
@thomasqueirozb thomasqueirozb deleted the file-source-tests-flush-sync branch November 14, 2025 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: sources Anything related to the Vector's sources

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test file_start_position_server_restart_acknowledged is flakey on non-Linux

2 participants