Description

Support a new, optional pushFileNamePattern parameter in the pushJobSpec section of the job yaml. This will filter segments in the outputDir that are pushed to Pinot.

Upgrade Notes

Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

• Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

• Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

• New configuration options
• Deprecation of configurations
• Signature changes to public methods/interfaces
• New plugins added or old plugins removed

• Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes

Support for name-based filtering of segments being pushed to the Pinot cluster.

Documentation

In the https://docs.pinot.apache.org/configuration-reference/job-specification#push-job-spec section, add:

pushFileNamePattern | segment name pattern for which segments to push, supported glob and regex patterns. E.g. 
'glob:stats_* will push all segment files under the outputDirURI whose names start with 'stats_'. 
'glob:*2022-01*' will push all the segment files under the outputDirURI whose names contain '2022-01'.