Skip to content

Conversation

@birschick-bq
Copy link
Contributor

Refactors and (significantly) improves performance of the file exporter.

  • Adds use of System.Threading.Channels to improve clarity and shutdown handling.
  • Now keeps open the current trace file until full or disposed.
  • Only checks for deleting old files when a new file is created.
  • Adds a handler for OnForceFlush to try to flush messages when needed.
  • Improved shutdown/dispose reliability to flush messages.

Benefits

  • orders of magnitude improvement in performance
  • huge improvement in concurrent use in the same folder

@github-actions github-actions bot added this to the ADBC Libraries 20 milestone Sep 5, 2025
Copy link
Contributor

@CurtHagenlocher CurtHagenlocher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Feedback left.

@CurtHagenlocher
Copy link
Contributor

@birschick-bq can you rebase this to see if that resolves the problem with the failing tests?

@birschick-bq
Copy link
Contributor Author

birschick-bq commented Sep 15, 2025

@birschick-bq can you rebase this to see if that resolves the problem with the failing tests?

@CurtHagenlocher
The build failures were in code that wasn't changed in this PR. This is because I tried to add CI tests for the file exporter. If I remove the change to that script, the other builds do not run and therefore don't fail.

Thoughts?

@CurtHagenlocher
Copy link
Contributor

Thoughts?

I'll check this in as-is and let's create a separate PR just to enable the tests.

Copy link
Contributor

@CurtHagenlocher CurtHagenlocher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@CurtHagenlocher CurtHagenlocher merged commit dc358d5 into apache:main Sep 15, 2025
6 checks passed
@birschick-bq birschick-bq deleted the dev/birschick-bq/improve-file-exporter branch October 28, 2025 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants