Repository navigation
Add Metadata::retain to filter entries in place #10683
Description
Is your feature request related to a problem or challenge?
- Found while testing upgrade of arrow in DataFusion: Update to arrow-rs main (arrow 60 pre-release testing) datafusion#24366
- Related to Use
Metadatastruct instead ofHashMap<String, String>for metadata: Ordered; cheap to clone #10075
While updating DataFusion to arrow 60 I found the new Metadata struct introduced in #10075 has no way to remove entries based on a predicate. Code that previously called HashMap::retain on field/schema metadata now has to convert to a BTreeMap, filter, and convert back.
For example, DataFusion computes the intersection of metadata across UNION inputs, which now looks like:
let mut intersected: Option<BTreeMap<String, String>> = None;
for metadata in metadatas {
match &mut intersected {
None => {
// deep copy into a BTreeMap so we can filter it below
intersected =
Some(metadata.iter().map(|(k, v)| (k.clone(), v.clone())).collect());
}
Some(current) => {
current.retain(|k, v| metadata.get(k) == Some(&*v));
}
}
}
intersected.map(Metadata::from).unwrap_or_default()Describe the solution you'd like
A Metadata::retain(|k, v| ...) method, copy-on-write like the existing insert/remove. The code above would then stay in terms of Metadata and avoid the deep copy and round-trip:
let mut intersected: Option<Metadata> = None;
for metadata in metadatas {
match &mut intersected {
None => intersected = Some(metadata.clone()), // cheap clone
Some(current) => current.retain(|k, v| metadata.get(k) == Some(&*v)),
}
}
intersected.unwrap_or_default()Describe alternatives you've considered
Converting through BTreeMap as shown above; it works but adds a copy and obscures the intent.
Additional context
Activity
implement `Metadata::retain` (apache#10695)
implement `Metadata::retain` (apache#10695)
implement `Metadata::retain` (apache#10695)
implement `Metadata::retain` (apache#10695)
implement `Metadata::retain` (apache#10695)
implement `Metadata::retain` (apache#10695)
implement `Metadata::retain` (apache#10695)
implement `Metadata::retain` (apache#10695)