Skip to content

Make KeyCondition usable outside *MergeTree when analyzer is enabled #53536

@al13n321

Description

@al13n321

If I'm reading the code correctly:

  • when analyzer is enabled, SelectQueryInfo::filter_asts doesn't have sets, and is probably missing some casts,
  • so, KeyCondition shouldn't use it,
  • instead it should use SelectQueryInfo::filter_actions_dag,
  • which is assigned only in ReadFromMergeTree::applyFilters().

So, any code using KeyCondition without a merge tree won't work. Currently that's just StorageHive, which uses KeyCondition to skip files based on their min/max statistics. (I didn't test if it's actually broken with analyzer, so maybe I'm misreading the code and it actually works.) And #52951 added KeyCondition to Parquet reader (to skip row groups), and I'm not sure how to make it work with analyzer.

What's the intended way for this to work? Should filter_actions_dag be built by some non-merge-tree code, before IStorage::read() calls? Or should it be built lazily by whoever needs it first? Should everything work completely differently (e.g. non-merge-tree filter pushdowns shouldn't use KeyCondition?)?

Metadata

Metadata

Assignees

Labels

developmentDevelopement process & source code & implementation details

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions