Skip to content

Conversation

@PGijsbers
Copy link
Collaborator

Two functions which make it easier to switch between a test environment and a production environment.
This is so we can create examples which operate on the test server and upload things, and have them easily be reproducible for people trying to follow the examples.

In the old style, you would have to set the server and apikey yourself. This is a little bit more verbose, possibly requires having to look up what the apikey is, and having the apikey in more plain sight.

With the changes we can instead use:

>>> import openml
>>> len(openml.tasks.list_tasks())
18742
>>> openml.config.start_use_example_configuration()
>>> len(openml.tasks.list_tasks())
1527
>>> openml.config.stop_use_example_configuration()
>>> len(openml.tasks.list_tasks())
18742

The apikey for the test server is from the openml tutorials, and is different from the one we use for unit testing. This is in the event we later need to purge specific subset of data from the test server (e.g. we find someone abused the example apikey and it is affecting unit tests).

Yay or nay?

I'll add a unit test if the concept is approved.

@PGijsbers PGijsbers requested a review from mfeurer April 18, 2019 12:52
@PGijsbers
Copy link
Collaborator Author

I cancelled the pr tests because its not final yet, to free resources for other PRs.

@codecov-io
Copy link

codecov-io commented Apr 18, 2019

Codecov Report

Merging #680 into develop will increase coverage by 0.35%.
The diff coverage is 100%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #680      +/-   ##
===========================================
+ Coverage    90.82%   91.17%   +0.35%     
===========================================
  Files           36       36              
  Lines         3573     3897     +324     
===========================================
+ Hits          3245     3553     +308     
- Misses         328      344      +16
Impacted Files Coverage Δ
openml/config.py 92.4% <100%> (+2.93%) ⬆️
openml/study/functions.py 92.57% <0%> (-4.31%) ⬇️
openml/study/study.py 100% <0%> (ø) ⬆️
openml/study/__init__.py 100% <0%> (ø) ⬆️
openml/datasets/functions.py 96.08% <0%> (+0.7%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4152f91...0d2a769. Read the comment docs.

Copy link
Collaborator

@mfeurer mfeurer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, this all looks great, but I have a remark about the naming. ExampleConfiguration reads like it is an example for the configuration, but it actually is the configuration for the examples. Could you please switch the order of the two in the class name and all the function names?

@PGijsbers
Copy link
Collaborator Author

Changed it, but to ConfigurationForExamples as I thought this would be even clearer? If you disagree, I will just make it ConfigurationExample.

openml/config.py Outdated
cache_directory = cachedir


start_use_example_configuration = ConfigurationForExamples.start_use_example_configuration
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for not being specific enough, could you please also change the name of these functions to make their use clearer?

@mfeurer
Copy link
Collaborator

mfeurer commented Apr 19, 2019

Taking over this PR.

@mfeurer mfeurer merged commit 46ec3ab into develop Apr 19, 2019
@mfeurer mfeurer deleted the easy_access_test_server branch April 19, 2019 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants