You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With --enable-sharding, every array gets one shard that covers the whole array (conversion/utils.py:673-697). For 75 arrays in one product, the shard holds only one inner chunk (for example 9 × 9 meteorology arrays and 23 × 23 angle arrays). A reader then makes two requests (index, then chunk) where one is enough. In 66 arrays the shard is larger than the array (for example 2048 × 2048 for 1830 × 1830). That is valid, but it is not necessary.
Proposal
In build_dataarray_encoding, set shards = None when the array fits in one inner chunk.
Tests
Add a unit test for the shard rule. Regenerate the snapshots.
Activity
[-]#TBD-04 Do not shard arrays that fit in one chunk[/-][+]Do not shard arrays that fit in one chunk[/+]