According to timings below the issue is already resolved by rework of subset by Matt, already published in 1.12.0.

Code to reproduce

library(data.table)
NN = 1e9
set.seed(304093)
DT = data.table(grp = sample(8L, NN, TRUE),
                V = rpois(NN, 10), key = 'grp')
(default_threads = getDTthreads())
system.time(DT[ , mean(V), keyby = grp])
setDTthreads(1)
system.time(DT[ , mean(V), keyby = grp])

Results

machine A (20 cores)

current latest 1.12.1

> (default_threads = getDTthreads())
[1] 20
> 
> system.time(DT[ , mean(V), keyby = grp])
   user  system elapsed 
 49.004   4.952   4.818 
> 
> setDTthreads(1)
>  
> system.time(DT[ , mean(V), keyby = grp])
   user  system elapsed 
  9.702   3.120  12.822

1.11.6

> (default_threads = getDTthreads())
[1] 20
> system.time(DT[ , mean(V), keyby = grp])
   user  system elapsed 
  6.533   1.244   7.777 
> setDTthreads(1)
> system.time(DT[ , mean(V), keyby = grp])
   user  system elapsed 
  6.543   1.224   7.767

machine B (40 cores)

current latest 1.12.1

> (default_threads = getDTthreads())
[1] 40
> 
> system.time(DT[ , mean(V), keyby = grp])
   user  system elapsed 
 63.085   9.026   4.519 
> 
> setDTthreads(1)
>  
> system.time(DT[ , mean(V), keyby = grp])
   user  system elapsed 
 10.810   4.994  15.805

1.11.6

> (default_threads = getDTthreads())
[1] 40
> system.time(DT[ , mean(V), keyby = grp])
   user  system elapsed 
  7.290   2.008   9.298 
> setDTthreads(1)
> system.time(DT[ , mean(V), keyby = grp])
   user  system elapsed 
  7.582   2.264   9.847

Multi-threaded group by #2533

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions