Skip to content

examples: Speedup the lake_problem function by ~30x#301

Merged
EwoutH merged 1 commit intomasterfrom
lake_model_performance
Oct 30, 2023
Merged

examples: Speedup the lake_problem function by ~30x#301
EwoutH merged 1 commit intomasterfrom
lake_model_performance

Conversation

@EwoutH
Copy link
Copy Markdown
Collaborator

@EwoutH EwoutH commented Oct 30, 2023

Speedup the lake_problem function in the example_lake_model.py example.

This is an one-on-one adaptation of https://github.com/quaquel/epa1361/pull/4.

Speed up the lake_problem function by around 30x, by flattening a loop and computing those values in NumPy arrays.

performance_boxplot Average duration of 100 lake_model experiments: 0.940 seconds vs 0.030 seconds (speed up: 31.41).

I validated the output and it is consistent with the previous function:

def validate_new_function():
    result_original = lake_problem()
    result_optimized = lake_problem_fast()

    # Check if the results are the same within a given tolerance
    assert np.allclose(result_original, result_optimized, rtol=1e-3, atol=1e-3), "Outputs are not the same: {} vs. {}".format(result_original, result_optimized)

for _ in range(10):
    try:
        validate_new_function()
    except AssertionError as e:
        print(e)

It prints nothing, which means all outputs are withing 0.1% of each other (due to the stochastic nature of the random.lognormal() draw it's never exactly the same).

The full analysis can be viewed in this notebook.

This over an order of magnitude speedup will help students iterate faster when learning and playing with the exercises, and might enable some new assignments and examples previously out of scope due to performance limitations.

Also add a warning on the decisions KeyError.

Speed up the lake_problem function by around 30x.

Benchmark: Average duration of 100 lake_model experiments: 0.940 seconds vs 0.030 seconds (speed up: 31.41).

I validated the output and it is consistent with the previous output.

The improved performance might allow more and broader analysis to be done in the exercises.

Also print a warning on the decisions KeyError
@EwoutH EwoutH requested a review from quaquel October 30, 2023 15:44
@quaquel
Copy link
Copy Markdown
Owner

quaquel commented Oct 30, 2023

Looks good, I think I made basically this change to the code used in teaching

@coveralls
Copy link
Copy Markdown

Coverage Status

coverage: 80.893%. remained the same when pulling 464a669 on lake_model_performance into ff192a1 on master.

@EwoutH EwoutH merged commit 9ccdb99 into master Oct 30, 2023
@quaquel quaquel deleted the lake_model_performance branch December 4, 2023 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants