Skip to content

Conversation

@jjerphan
Copy link
Member

Reference Issues/PRs

None.

What does this implement/fix? Explain your changes.

Some fixtures for Numpy 1.3 and Numpy 1.4 were included.
They are no longer needed.

Pointers to vec and mat arrays have also been dropped.

vec and mat are now memoryviews, and the state
for pickling has been adapted to explicitely cast
memoryview as numpy arrays to recover the same
behavior.

Any other comments?

Pointers to `vec` and `mat` arrays have also been
dropped.

`vec` and `mat` are now memoryviews, and the state
for pickling has been adapted to explicitely cast
memoryview as numpy arrays to recover the same
behavior.
@jjerphan
Copy link
Member Author

Here are the result on those benchmark comparing pairwise distances computation on main (1cd282d) on on the branch of this PR (d7ff130).

       before           after         ratio
     [1cd282d6]       [d7ff1305]
     <main>           <remove-old-np-compat-fixtures>
+         882±9μs         1.27±0ms     1.43  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(100, 10)
+      88.2±0.3ms        125±0.2ms     1.42  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(1000, 10)
+      9.29±0.01s       13.0±0.05s     1.40  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(10000, 10)
+       316±0.2μs          398±2μs     1.26  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(100, 5)
+      31.6±0.1ms       39.1±0.1ms     1.24  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(1000, 5)
-      8.95±0.5ms      8.02±0.03ms     0.90  dist_metrics.ChebyshevDistanceBenchmark.time_pairwise(1000, 5)
-      9.80±0.2ms      8.75±0.06ms     0.89  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(1000, 5)
-       143±0.1μs        127±0.5μs     0.89  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(100, 10)
-      98.2±0.2μs       86.7±0.3μs     0.88  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(100, 5)
-      9.21±0.3ms      8.12±0.03ms     0.88  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(1000, 5)
-      13.9±0.5ms      12.2±0.05ms     0.88  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(1000, 10)
-        796±20ms         694±10ms     0.87  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(1000, 10)
-      155±0.08μs        134±0.5μs     0.87  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(100, 10)
-        4.52±2ms      3.91±0.01ms     0.87  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(100, 5)
-      7.26±0.03s       6.29±0.03s     0.87  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(1000, 100)
-      7.17±0.02s          6.21±0s     0.87  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(1000, 100)
-      95.9±0.1μs       82.2±0.3μs     0.86  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(100, 5)
-     8.15±0.06ms      6.98±0.06ms     0.86  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(100, 10)
-     4.55±0.04ms      3.89±0.04ms     0.86  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(100, 5)
-        817±20ms          696±9ms     0.85  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(1000, 10)
-     8.04±0.03ms      6.82±0.02ms     0.85  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(100, 10)
-      1.48±0.05s       1.24±0.02s     0.84  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(10000, 5)
-     16.1±0.07ms      13.1±0.08ms     0.82  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(1000, 10)
-       2.15±0.3s       1.69±0.01s     0.79  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(10000, 10)
-         1.48±0m          1.16±0m     0.79  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(10000, 10)
-       161±0.9ms        115±0.4ms     0.71  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(1000, 100)
-        1.61±0ms         1.15±0ms     0.71  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(100, 100)
-       1.82±0.2s       1.23±0.02s     0.68  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(10000, 5)
-       105±0.6ms       70.9±0.2ms     0.67  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(100, 100)
-        189±40ms        115±0.2ms     0.60  dist_metrics.ChebyshevDistanceBenchmark.time_pairwise(1000, 100)
-       183±0.4ms          110±1ms     0.60  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(1000, 100)
-       3.30±0.2s       1.61±0.01s     0.49  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(10000, 10)
-       809±100ms          387±5ms     0.48  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(1000, 5)

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE DECREASED.

As reported, performances have decreased but for MahalanobisDistance solely.
On all the others DistanceMetric, it has introduce significant improvement, judging that DistanceMetric.pairwise is a good proxy to evaluate the performance.

I've used a shared machine to perform the benchmarks. Though it was not overwhelmed with others tasks, I would be glad if someone could report their results using the benchmark script linked above as well.

Full report
 → asv continuous -b DistanceBenchmark main remove-old-np-compat-fixtures
· Creating environments
· Discovering benchmarks.
·· Uninstalling from conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl...
·· Installing d7ff1305  into conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl..
· Running 14 total benchmarks (2 commits * 1 environments * 7 benchmarks)
[  0.00%] · For scikit-learn commit d7ff1305  (round 1/1):
[  0.00%] ·· Benchmarking conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl
[  7.14%] ··· dist_metrics.ChebyshevDistanceBenchmark.time_pairwise                            ok
[  7.14%] ··· ======= ============= ============= ============
              --                         d
              ------- ----------------------------------------
                 n          5             10          100
              ======= ============= ============= ============
                100     91.9±0.5μs    138±0.4μs     1.19±0ms
                1000   8.02±0.03ms   13.2±0.05ms   115±0.2ms
               10000    1.20±0.03s    1.75±0.03s   10.6±0.01s
              ======= ============= ============= ============

[ 14.29%] ··· dist_metrics.EuclideanDistanceBenchmark.time_pairwise ok
[ 14.29%] ··· ======= ============= ============= ===========
-- d
------- ---------------------------------------
n 5 10 100
======= ============= ============= ===========
100 86.7±0.3μs 127±0.5μs 1.15±0ms
1000 8.12±0.03ms 12.2±0.05ms 115±0.4ms
10000 1.24±0.02s 1.61±0.01s 10.6±0s
======= ============= ============= ===========

[ 21.43%] ··· dist_metrics.MahalanobisDistanceBenchmark.time_pairwise 1/9 failed
[ 21.43%] ··· ======= ============ ============ ============
-- d
------- --------------------------------------
n 5 10 100
======= ============ ============ ============
100 398±2μs 1.27±0ms 130±0.2ms
1000 39.1±0.1ms 125±0.2ms 11.3±0.03s
10000 4.34±0.05s 13.0±0.05s failed
======= ============ ============ ============

[ 28.57%] ··· dist_metrics.ManhattanDistanceBenchmark.time_pairwise ok
[ 28.57%] ··· ======= ============= ============= ===========
-- d
------- ---------------------------------------
n 5 10 100
======= ============= ============= ===========
100 82.2±0.3μs 134±0.5μs 1.09±0ms
1000 8.75±0.06ms 13.1±0.08ms 110±1ms
10000 1.23±0.02s 1.69±0.01s 10.1±0.2s
======= ============= ============= ===========

[ 35.71%] ··· dist_metrics.MinkowskiDistanceBenchmark.time_pairwise 1/9 failed
[ 35.71%] ··· ======= ============= ============= ============
-- d
------- ----------------------------------------
n 5 10 100
======= ============= ============= ============
100 3.89±0.04ms 6.82±0.02ms 70.9±0.2ms
1000 387±5ms 694±10ms 6.21±0s
10000 39.6±0.3s 1.20±0.01m failed
======= ============= ============= ============

[ 42.86%] ··· dist_metrics.SEuclideanDistanceBenchmark.time_pairwise failed
[ 42.86%] ··· ======= ======== ======== ========
-- d
------- --------------------------
n 5 10 100
======= ======== ======== ========
100 failed failed failed
1000 failed failed failed
10000 failed failed failed
======= ======== ======== ========

[ 50.00%] ··· dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise 1/9 failed
[ 50.00%] ··· ======= ============= ============= =============
-- d
------- -----------------------------------------
n 5 10 100
======= ============= ============= =============
100 3.91±0.01ms 6.98±0.06ms 71.5±0.06ms
1000 413±9ms 696±9ms 6.29±0.03s
10000 38.5±0.1s 1.16±0m failed
======= ============= ============= =============

[ 50.00%] · For scikit-learn commit 1cd282d6

(round 1/1):
[ 50.00%] ·· Building for conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl.....
[ 50.00%] ·· Benchmarking conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl
[ 57.14%] ··· dist_metrics.ChebyshevDistanceBenchmark.time_pairwise ok
[ 57.14%] ··· ======= ============ ============ ==========
-- d
------- ------------------------------------
n 5 10 100
======= ============ ============ ==========
100 88.0±0.9μs 150±1μs 1.15±0ms
1000 8.95±0.5ms 14.4±0.3ms 189±40ms
10000 1.26±0.02s 1.83±0.2s 12.2±2s
======= ============ ============ ==========

[ 64.29%] ··· dist_metrics.EuclideanDistanceBenchmark.time_pairwise ok
[ 64.29%] ··· ======= ============ ============ ===========
-- d
------- -------------------------------------
n 5 10 100
======= ============ ============ ===========
100 98.2±0.2μs 143±0.1μs 1.61±0ms
1000 9.21±0.3ms 13.9±0.5ms 161±0.9ms
10000 1.48±0.05s 3.30±0.2s 14.0±1s
======= ============ ============ ===========

[ 71.43%] ··· dist_metrics.MahalanobisDistanceBenchmark.time_pairwise 1/9 failed
[ 71.43%] ··· ======= ============ ============ ===========
-- d
------- -------------------------------------
n 5 10 100
======= ============ ============ ===========
100 316±0.2μs 882±9μs 132±0.7ms
1000 31.6±0.1ms 88.2±0.3ms 10.8±0.3s
10000 6.10±0.9s 9.29±0.01s failed
======= ============ ============ ===========

[ 78.57%] ··· dist_metrics.ManhattanDistanceBenchmark.time_pairwise ok
[ 78.57%] ··· ======= ============ ============= ============
-- d
------- ---------------------------------------
n 5 10 100
======= ============ ============= ============
100 95.9±0.1μs 155±0.08μs 1.10±0.4ms
1000 9.80±0.2ms 16.1±0.07ms 183±0.4ms
10000 1.82±0.2s 2.15±0.3s 11.7±1s
======= ============ ============= ============

[ 85.71%] ··· dist_metrics.MinkowskiDistanceBenchmark.time_pairwise 1/9 failed
[ 85.71%] ··· ======= ============= ============= ============
-- d
------- ----------------------------------------
n 5 10 100
======= ============= ============= ============
100 4.55±0.04ms 8.04±0.03ms 105±0.6ms
1000 809±100ms 796±20ms 7.17±0.02s
10000 49.6±3s 1.59±0.04m failed
======= ============= ============= ============

[ 92.86%] ··· dist_metrics.SEuclideanDistanceBenchmark.time_pairwise failed
[ 92.86%] ··· ======= ======== ======== ========
-- d
------- --------------------------
n 5 10 100
======= ======== ======== ========
100 failed failed failed
1000 failed failed failed
10000 failed failed failed
======= ======== ======== ========

[100.00%] ··· dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise 1/9 failed
[100.00%] ··· ======= ========== ============= ============
-- d
------- -------------------------------------
n 5 10 100
======= ========== ============= ============
100 4.52±2ms 8.15±0.06ms 71.8±20ms
1000 440±1ms 817±20ms 7.26±0.03s
10000 45.2±1s 1.48±0m failed
======= ========== ============= ============

   before           after         ratio
 [1cd282d6]       [d7ff1305]
 <main>           <remove-old-np-compat-fixtures>
  •     882±9μs         1.27±0ms     1.43  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(100, 10)
    
  •  88.2±0.3ms        125±0.2ms     1.42  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(1000, 10)
    
  •  9.29±0.01s       13.0±0.05s     1.40  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(10000, 10)
    
  •   316±0.2μs          398±2μs     1.26  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(100, 5)
    
  •  31.6±0.1ms       39.1±0.1ms     1.24  dist_metrics.MahalanobisDistanceBenchmark.time_pairwise(1000, 5)
    
  •  8.95±0.5ms      8.02±0.03ms     0.90  dist_metrics.ChebyshevDistanceBenchmark.time_pairwise(1000, 5)
    
  •  9.80±0.2ms      8.75±0.06ms     0.89  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(1000, 5)
    
  •   143±0.1μs        127±0.5μs     0.89  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(100, 10)
    
  •  98.2±0.2μs       86.7±0.3μs     0.88  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(100, 5)
    
  •  9.21±0.3ms      8.12±0.03ms     0.88  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(1000, 5)
    
  •  13.9±0.5ms      12.2±0.05ms     0.88  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(1000, 10)
    
  •    796±20ms         694±10ms     0.87  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(1000, 10)
    
  •  155±0.08μs        134±0.5μs     0.87  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(100, 10)
    
  •    4.52±2ms      3.91±0.01ms     0.87  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(100, 5)
    
  •  7.26±0.03s       6.29±0.03s     0.87  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(1000, 100)
    
  •  7.17±0.02s          6.21±0s     0.87  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(1000, 100)
    
  •  95.9±0.1μs       82.2±0.3μs     0.86  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(100, 5)
    
  • 8.15±0.06ms      6.98±0.06ms     0.86  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(100, 10)
    
  • 4.55±0.04ms      3.89±0.04ms     0.86  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(100, 5)
    
  •    817±20ms          696±9ms     0.85  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(1000, 10)
    
  • 8.04±0.03ms      6.82±0.02ms     0.85  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(100, 10)
    
  •  1.48±0.05s       1.24±0.02s     0.84  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(10000, 5)
    
  • 16.1±0.07ms      13.1±0.08ms     0.82  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(1000, 10)
    
  •   2.15±0.3s       1.69±0.01s     0.79  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(10000, 10)
    
  •     1.48±0m          1.16±0m     0.79  dist_metrics.WMinkowskiDistanceBenchmark.time_pairwise(10000, 10)
    
  •   161±0.9ms        115±0.4ms     0.71  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(1000, 100)
    
  •    1.61±0ms         1.15±0ms     0.71  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(100, 100)
    
  •   1.82±0.2s       1.23±0.02s     0.68  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(10000, 5)
    
  •   105±0.6ms       70.9±0.2ms     0.67  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(100, 100)
    
  •    189±40ms        115±0.2ms     0.60  dist_metrics.ChebyshevDistanceBenchmark.time_pairwise(1000, 100)
    
  •   183±0.4ms          110±1ms     0.60  dist_metrics.ManhattanDistanceBenchmark.time_pairwise(1000, 100)
    
  •   3.30±0.2s       1.61±0.01s     0.49  dist_metrics.EuclideanDistanceBenchmark.time_pairwise(10000, 10)
    
  •   809±100ms          387±5ms     0.48  dist_metrics.MinkowskiDistanceBenchmark.time_pairwise(1000, 5)
    

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE DECREASED.

@jjerphan jjerphan marked this pull request as ready for review June 23, 2021 11:49
@jjerphan jjerphan changed the title MNT Remove fixtures for Numpy 1.3 and 1.4 compatibilities MNT Remove fixtures for Numpy 1.3 and 1.4 compatibility Jun 23, 2021
@jjerphan
Copy link
Member Author

jjerphan commented Jun 30, 2021

As of 125e4b6, performances have not changed (thanks @thomasjpfan, this was also the answer of the conversation).

ASV Benchmark results
 → asv continuous -b Distance main remove-old-np-compat-fixtures
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl.
·· Building 125e4b6b <remove-old-np-compat-fixtures> for conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl..............................................................
·· Installing 125e4b6b <remove-old-np-compat-fixtures> into conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl.
· Running 14 total benchmarks (2 commits * 1 environments * 7 benchmarks)
[  0.00%] · For scikit-learn commit 125e4b6b <remove-old-np-compat-fixtures> (round 1/1):
[  0.00%] ·· Benchmarking conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl
[  7.14%] ···  distance_metric.ChebyshevDistanceBenchmark.time_pairwise                                                                                                                                                                                     ok
[  7.14%] ··· ======= ============= ============ ============
              --                         d                   
              ------- ---------------------------------------
                 n          5            10          100     
              ======= ============= ============ ============
                100     81.1±0.5μs   133±0.8μs     949±2μs   
                1000   7.80±0.06ms   12.9±0.2ms   94.7±0.6ms 
               10000     1.07±0s     1.59±0.01s   10.7±0.01s 
              ======= ============= ============ ============

[ 14.29%] ···  distance_metric.EuclideanDistanceBenchmark.time_pairwise                                                                                                                                                                                     ok
[ 14.29%] ··· ======= ============= ============= ============
              --                         d                    
              ------- ----------------------------------------
                 n          5             10          100     
              ======= ============= ============= ============
                100     78.9±0.3μs    115±0.6μs    979±0.8μs  
                1000   7.42±0.04ms   10.9±0.05ms   97.9±0.2ms 
               10000     1.05±0s       1.40±0s     10.9±0.03s 
              ======= ============= ============= ============

[ 21.43%] ···  distance_metric.MahalanobisDistanceBenchmark.time_pairwise                                                                                                                                                                           1/9 failed
[ 21.43%] ··· ======= ============ ============ =============
              --                         d                   
              ------- ---------------------------------------
                 n         5            10           100     
              ======= ============ ============ =============
                100    292±0.9μs     877±5μs     85.3±0.07ms 
                1000   29.2±0.2ms   88.2±0.5ms    8.33±0.01s 
               10000    3.19±0s     8.98±0.01s      failed   
              ======= ============ ============ =============

[ 28.57%] ···  distance_metric.ManhattanDistanceBenchmark.time_pairwise                                                                                                                                                                                     ok
[ 28.57%] ··· ======= ============= ============= =============
              --                          d                    
              ------- -----------------------------------------
                 n          5             10           100     
              ======= ============= ============= =============
                100     81.7±0.3μs    115±0.4μs     940±0.9μs  
                1000   7.63±0.02ms   11.1±0.04ms   93.8±0.04ms 
               10000    1.06±0.01s     1.41±0s      10.4±0.01s 
              ======= ============= ============= =============

[ 35.71%] ···  distance_metric.MinkowskiDistanceBenchmark.time_pairwise                                                                                                                                                                             1/9 failed
[ 35.71%] ··· ======= ============ ============= =============
              --                         d                    
              ------- ----------------------------------------
                 n         5             10           100     
              ======= ============ ============= =============
                100     1.59±0ms    2.76±0.02ms   24.6±0.03ms 
                1000   158±0.8ms     274±0.9ms     2.42±0.01s 
               10000   16.1±0.01s    28.0±0.01s      failed   
              ======= ============ ============= =============

[ 42.86%] ···  distance_metric.SEuclideanDistanceBenchmark.time_pairwise                                                                                                                                                                                failed
[ 42.86%] ··· ======= ======== ======== ========
              --                  d             
              ------- --------------------------
                 n       5        10      100   
              ======= ======== ======== ========
                100    failed   failed   failed 
                1000   failed   failed   failed 
               10000   failed   failed   failed 
              ======= ======== ======== ========

[ 50.00%] ···  distance_metric.WMinkowskiDistanceBenchmark.time_pairwise                                                                                                                                                                            1/9 failed
[ 50.00%] ··· ======= ============= ============= =============
              --                          d                    
              ------- -----------------------------------------
                 n          5             10           100     
              ======= ============= ============= =============
                100    1.59±0.01ms   2.74±0.01ms   24.6±0.09ms 
                1000     158±1ms      278±0.9ms     2.41±0.01s 
               10000    16.2±0.02s    28.2±0.03s      failed   
              ======= ============= ============= =============

[ 50.00%] · For scikit-learn commit b5e5db4a <main> (round 1/1):
[ 50.00%] ·· Building for conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl..
[ 50.00%] ·· Benchmarking conda-py3.9-cython-joblib-numpy-scipy-threadpoolctl
[ 57.14%] ···  distance_metric.ChebyshevDistanceBenchmark.time_pairwise                                                                                                                                                                                     ok
[ 57.14%] ··· ======= ============= ============= ============
              --                         d                    
              ------- ----------------------------------------
                 n          5             10          100     
              ======= ============= ============= ============
                100     81.5±0.4μs    132±0.9μs     950±2μs   
                1000   7.71±0.03ms   12.7±0.08ms   94.8±0.2ms 
               10000     1.08±0s      1.59±0.01s   10.6±0.03s 
              ======= ============= ============= ============

[ 64.29%] ···  distance_metric.EuclideanDistanceBenchmark.time_pairwise                                                                                                                                                                                     ok
[ 64.29%] ··· ======= ============= ============= =============
              --                          d                    
              ------- -----------------------------------------
                 n          5             10           100     
              ======= ============= ============= =============
                100     78.0±0.5μs    113±0.4μs     979±0.8μs  
                1000   7.45±0.04ms   11.0±0.03ms   97.9±0.06ms 
               10000     1.05±0s      1.42±0.01s    10.9±0.01s 
              ======= ============= ============= =============

[ 71.43%] ···  distance_metric.MahalanobisDistanceBenchmark.time_pairwise                                                                                                                                                                           1/9 failed
[ 71.43%] ··· ======= ============ ============ ============
              --                        d                   
              ------- --------------------------------------
                 n         5            10          100     
              ======= ============ ============ ============
                100     292±1μs      875±3μs     85.1±0.2ms 
                1000   28.8±0.1ms   86.9±0.2ms   8.35±0.02s 
               10000   3.22±0.01s   9.01±0.01s     failed   
              ======= ============ ============ ============

[ 78.57%] ···  distance_metric.ManhattanDistanceBenchmark.time_pairwise                                                                                                                                                                                     ok
[ 78.57%] ··· ======= ============= ============= ============
              --                         d                    
              ------- ----------------------------------------
                 n          5             10          100     
              ======= ============= ============= ============
                100     81.4±0.2μs    114±0.4μs    941±0.6μs  
                1000   7.74±0.07ms   11.1±0.04ms   93.7±0.2ms 
               10000    1.07±0.01s    1.42±0.01s   10.4±0.01s 
              ======= ============= ============= ============

[ 85.71%] ···  distance_metric.MinkowskiDistanceBenchmark.time_pairwise                                                                                                                                                                             1/9 failed
[ 85.71%] ··· ======= ============= ============= =============
              --                          d                    
              ------- -----------------------------------------
                 n          5             10           100     
              ======= ============= ============= =============
                100    1.59±0.01ms   2.76±0.02ms   24.6±0.05ms 
                1000    157±0.5ms      277±2ms      2.40±0.01s 
               10000     16.2±0s      28.0±0.02s      failed   
              ======= ============= ============= =============

[ 92.86%] ···  distance_metric.SEuclideanDistanceBenchmark.time_pairwise                                                                                                                                                                                failed
[ 92.86%] ··· ======= ======== ======== ========
              --                  d             
              ------- --------------------------
                 n       5        10      100   
              ======= ======== ======== ========
                100    failed   failed   failed 
                1000   failed   failed   failed 
               10000   failed   failed   failed 
              ======= ======== ======== ========

[100.00%] ···  distance_metric.WMinkowskiDistanceBenchmark.time_pairwise                                                                                                                                                                            1/9 failed
[100.00%] ··· ======= ============= ============= =============
              --                          d                    
              ------- -----------------------------------------
                 n          5             10           100     
              ======= ============= ============= =============
                100    1.60±0.02ms   2.78±0.01ms   24.6±0.08ms 
                1000    159±0.9ms      277±2ms      2.41±0.01s 
               10000    16.2±0.03s    28.2±0.1s       failed   
              ======= ============= ============= =============


BENCHMARKS NOT SIGNIFICANTLY CHANGED.

Copy link
Member

@glemaitre glemaitre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think that it is worth merging this then.

Copy link
Member

@thomasjpfan thomasjpfan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thomasjpfan thomasjpfan merged commit 03aa496 into scikit-learn:main Jul 22, 2021
@jjerphan jjerphan deleted the remove-old-np-compat-fixtures branch July 22, 2021 12:21
TomDLT pushed a commit to TomDLT/scikit-learn that referenced this pull request Jul 29, 2021
samronsin pushed a commit to samronsin/scikit-learn that referenced this pull request Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants