Remove dtype checking for prediction comparison #1177
Conversation
It looks like the predictions loaded from an arff file are read as floats by the arff reader, which results in a different type (float v int). Because "equality" of values is already checked, I figured dtype is not as imported. That said, I am not sure why there are so many redundant comparisons in the first place? Anyway, the difference should be due to pandas inference behavior, and if that is what we want to test, then we should make a small isolated test case instead of integrating it into every prediction unit test.
mfeurer
left a comment
mfeurer
left a comment
There was a problem hiding this comment.
Fine by me, but why did this fail in the first place 😕
|
If I remember correctly, the numbers are integer. When openml-python executes the experiments, they are also integer (from |
|
That description makes sense, but why did this work at some point at all? The behavior of the ARFF reader is correct as ARFF does not specify integers. Maybe OpenML should also move away from ARFF for internal data structures. |
My best guess is different behaviour of Pandas, but I haven't spent time nailing it down exactly (it's interesting, but didn't seem important). |
It looks like the predictions loaded from an arff file are read as floats by the arff reader, which results in a different type (float v int). Because "equality" of values is already checked, I figured dtype is not as imported. That said, I am not sure why there are so many redundant comparisons in the first place? Anyway, the difference should be due to pandas inference behavior, and if that is what we want to test, then we should make a small isolated test case instead of integrating it into every prediction unit test. Finally, over the next year we should move away from ARFF.
It looks like the predictions loaded from an arff file are read as floats by the arff reader, which results in a different type (float v int). Because "equality" of values is already checked, I figured dtype is not as imported. That said, I am not sure why there are so many redundant comparisons in the first place? Anyway, the difference should be due to pandas inference behavior, and if that is what we want to test, then we should make a small isolated test case instead of integrating it into every prediction unit test.