Skip to content

feat: Make streaming the default LightGBM data transfer mode #2088

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
svotaw merged 12 commits into from
Oct 21, 2023
Merged

feat: Make streaming the default LightGBM data transfer mode #2088

svotaw merged 12 commits into from
Oct 21, 2023

Conversation

@svotaw
Copy link
Collaborator

@svotaw svotaw commented Oct 2, 2023

Make streaming the default LightGBM data transfer mode.

Both streaming and bulk are tested still. This only changes the default for users.

@github-actions
Copy link

github-actions bot commented Oct 2, 2023

Hey @svotaw 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

  • fix: Fix LightGBM crashes with empty partitions
  • feat: Make HTTP on Spark back-offs configurable
  • docs: Update Spark Serving usage
  • build: Add codecov support
  • perf: improve LightGBM memory usage
  • refactor: make python code generation rely on classes
  • style: Remove nulls from CNTKModel
  • test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

@svotaw svotaw changed the title Make streaming the default LightGBM data transfer mode feat: Make streaming the default LightGBM data transfer mode Oct 3, 2023
@svotaw
Copy link
Collaborator Author

svotaw commented Oct 3, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 3, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@codecov-commenter
Copy link

codecov-commenter commented Oct 3, 2023 •
edited
Loading

Codecov Report

Merging #2088 (1624f5f) into master (b08b33d) will decrease coverage by 0.01%.
The diff coverage is 93.33%.

@@            Coverage Diff             @@
##           master    #2088      +/-   ##
==========================================
- Coverage   86.28%   86.27%   -0.01%     
==========================================
  Files         310      311       +1     
  Lines       16254    16267      +13     
  Branches      818      818              
==========================================
+ Hits        14025    14035      +10     
- Misses       2229     2232       +3
Files Coverage Δ
...rosoft/azure/synapse/ml/lightgbm/SharedState.scala 100.00% <ø> (ø)
...e/synapse/ml/lightgbm/StreamingPartitionTask.scala 97.04% <100.00%> (+0.01%) ⬆️
...re/synapse/ml/lightgbm/params/LightGBMParams.scala 100.00% <100.00%> (ø)
...oft/azure/synapse/ml/lightgbm/GroupIdManager.scala 91.66% <91.66%> (ø)

... and 4 files with indirect coverage changes

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 6, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 6, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 9, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 9, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 11, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 11, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 12, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 12, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 16, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 16, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 16, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 16, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 16, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 16, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 17, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 17, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@mhamilton723
Copy link
Contributor

mhamilton723 commented Oct 20, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 20, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw
Copy link
Collaborator Author

svotaw commented Oct 21, 2023

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Oct 21, 2023

Azure Pipelines successfully started running 1 pipeline(s).

@svotaw svotaw merged commit 2f31f74 into microsoft:master Oct 21, 2023
@operte operte mentioned this pull request May 28, 2025
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@mhamilton723 mhamilton723 mhamilton723 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@svotaw @codecov-commenter @mhamilton723