Skip to content

BENCH newton-lsmr solver for GLMs - inner stopping criterion #26707

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
lorentzenchr wants to merge 40 commits into
base: main
Choose a base branch
from
Draft

BENCH newton-lsmr solver for GLMs - inner stopping criterion #26707

lorentzenchr wants to merge 40 commits into from

Conversation

@lorentzenchr
Copy link
Member

@lorentzenchr lorentzenchr commented Jun 26, 2023 •
edited
Loading

Reference Issues/PRs

This PR is not for merge, but for analysis of #25462.

The newton-lsmr solver needs a good inner stopping criterion with a good forcing sequence. This is investigated here and benchmarks are run.

For stopping criteria of inexact Newton methods (mainly conjugate gradient for inner solver) see:

Any other comments?

See added notebook Bench_Newton_LSMR.ipynb in this PR for analysis results.

@lorentzenchr
ENH add NewtonLSMRSolver
bfb29a0
@lorentzenchr
TST add test_solver_on_ill_conditioned_X
9c3fd7f
@lorentzenchr
ENH add multinomial to LSMR
3874810
@lorentzenchr
ENH add newton-lsmr to LogisticRegression
86da909
@lorentzenchr
CLN fix dtype and tests
1d13089 
- keep dtype float32 after LSMR
- lower test precision in
 test_NewtonLSMRSolver_multinomial_A_b_on_3_classes
@lorentzenchr
ENH speed up LDL by better handling q==0
4e1a696
@lorentzenchr
ENH speed up LDL by using einsum and q_inv
7ef5877
@lorentzenchr
Merge branch 'main' into glm_newton_lsmr_only
65d7a87
@lorentzenchr
TST fix atol in test_multinomial_identifiability_properties
74dab94
@lorentzenchr
DOC add whatsnew
0dc87cf
@lorentzenchr
TST reduce tolerances
8e78465
@lorentzenchr
TST skip LinearOperator transpose for scipy<1.4
3da89d5
@lorentzenchr
TST fix skipif
298e63e
@lorentzenchr
TST loosen rtol a bit
27ddf56
@lorentzenchr
TST make tests pass for all random seeds
d26004d 
* SKLEARN_TESTS_GLOBAL_RANDOM_SEED="all" pytest -x
@lorentzenchr
DOC add comment about initial A_norm
59c1322
@lorentzenchr
Merge branch 'main' into glm_newton_lsmr_only
a84b938
@lorentzenchr
FIX warning condition and cleaner for loop
c50c859
@lorentzenchr
ENH improve Multinomial_LDL_Decomposition by precomputation
cf9facc
@lorentzenchr
DOC improve docstrings and comments
fc1cb24
@lorentzenchr
ENH inner stopping criterion for LSMR with forcing sequence
e7368e7
@lorentzenchr
DOC improve docs and comments of Newton solvers
7295409
@lorentzenchr
ENH set initial A_norm based on n_samples, n_features and l2_reg_stre…
3ea7d98 
…ngth
@lorentzenchr
CLN address review comments
99a9508
@lorentzenchr
TST set lsmr to xfail in test_solver_on_ill_conditioned_X like other …
d108ffc 
…solvers
@lorentzenchr
Doc docstring solver arg in LogisticRegression
ab075fe
@lorentzenchr
ENH set btol back to self.tol
e4c0ee3
@lorentzenchr
TST higher tol in test_glm_regression_hstacked_X
39030c4
@lorentzenchr
ENH increase conlim to make more tests pass
83ce34f
@lorentzenchr
Merge branch 'main' into glm_newton_lsmr_only
8846903
@github-actions
Copy link

github-actions bot commented Jun 26, 2023 •
edited
Loading

❌ Linting issues

Merging with upstream/main might fix / improve the issues if you haven't done that since 21.06.2023.

This PR is introducing linting issues. Here's a summary of the issues. Note that you can avoid having linting issues by enabling pre-commit hooks. Instructions to enable them can be found here.

You can see the details of the linting issues under the lint job here

ruff

ruff detected issues. Please run ruff --fix --show-source . locally, fix the remaining issues, and push the changes. Here you can see the detected issues. Note that the installed ruff version is ruff=0.0.275.

Generated for commit: 56e4953. Link to the linter CI: here

This was referenced Jun 26, 2023
Open
Open
@lorentzenchr lorentzenchr marked this pull request as draft July 2, 2023 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

module:linear_model module:utils

Projects

Losses and solvers
Status: Todo

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@lorentzenchr