Skip to content

Infinite loop in iterative optimizer when pushing down row dereference within expression into Hive connector #11559

Closed

Description

@findepi

Reproducible on current master with HiveQueryRunner, so affects 374 too (and maybe all versions since #7994, i.e. 372+)

trino:tpch> CREATE TABLE t(r row(a varchar, b int));
CREATE TABLE

trino:tpch> SET SESSION iterative_optimizer_timeout = '10s';
SET SESSION

trino:tpch> SELECT r, r.a LIKE 'a%' FROM t;
Query 20220318_092354_00036_63qnj failed: The optimizer exhausted the time limit of 10000 ms: Top rules: {
		io.trino.sql.planner.iterative.rule.PushProjectionIntoTableScan@46d83ec4: 3652 ms, 39605 invocations, 39605 applications,
		io.trino.sql.planner.iterative.rule.PruneTableScanColumns@c651286: 1404 ms, 39605 invocations, 39604 applications,
		io.trino.sql.planner.iterative.rule.PushProjectionThroughUnion@78de0f6f: 192 ms, 39605 invocations, 0 applications,
		io.trino.sql.planner.iterative.rule.RemoveRedundantIdentityProjections@2ccb8db3: 150 ms, 39605 invocations, 0 applications,
		io.trino.sql.planner.iterative.rule.PushProjectionThroughExchange@588614f9: 132 ms, 39605 invocations, 0 applications }
io.trino.spi.TrinoException: The optimizer exhausted the time limit of 10000 ms: Top rules: {
		io.trino.sql.planner.iterative.rule.PushProjectionIntoTableScan@46d83ec4: 3652 ms, 39605 invocations, 39605 applications,
		io.trino.sql.planner.iterative.rule.PruneTableScanColumns@c651286: 1404 ms, 39605 invocations, 39604 applications,
		io.trino.sql.planner.iterative.rule.PushProjectionThroughUnion@78de0f6f: 192 ms, 39605 invocations, 0 applications,
		io.trino.sql.planner.iterative.rule.RemoveRedundantIdentityProjections@2ccb8db3: 150 ms, 39605 invocations, 0 applications,
		io.trino.sql.planner.iterative.rule.PushProjectionThroughExchange@588614f9: 132 ms, 39605 invocations, 0 applications }
	at io.trino.sql.planner.iterative.IterativeOptimizer$Context.checkTimeoutNotExhausted(IterativeOptimizer.java:376)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreNode(IterativeOptimizer.java:156)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:132)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:251)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:134)
	at io.trino.sql.planner.iterative.IterativeOptimizer.optimize(IterativeOptimizer.java:117)
	at io.trino.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:238)
	at io.trino.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:215)
	at io.trino.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:210)
	at io.trino.execution.SqlQueryExecution.doPlanQuery(SqlQueryExecution.java:473)
	at io.trino.execution.SqlQueryExecution.planQuery(SqlQueryExecution.java:454)
	at io.trino.execution.SqlQueryExecution.start(SqlQueryExecution.java:395)
	at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:243)
	at io.trino.dispatcher.LocalDispatchQuery.lambda$startExecution$7(LocalDispatchQuery.java:143)
	at io.trino.$gen.Trino_testversion____20220318_092222_3.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

cc @martint @wendigo @hashhar @assaf2 @losipiuk

Rel: #11560

Activity

  1. findepi commented on Mar 18, 2022

    @findepi
    MemberAuthor

    Note that disabling complex_expression_pushdown is not a workaround:

    trino:tpch> SET SESSION complex_expression_pushdown = false;
    SET SESSION
    trino:tpch> SELECT r, r.a LIKE 'a%' FROM t;
    Query 20220318_092511_00038_63qnj failed: The optimizer exhausted the time limit of 10000 ms: Top rules: {
    		io.trino.sql.planner.iterative.rule.PushProjectionIntoTableScan@46d83ec4: 3349 ms, 93717 invocations, 93717 applications,
    		io.trino.sql.planner.iterative.rule.PruneTableScanColumns@c651286: 1311 ms, 93717 invocations, 93716 applications,
    		io.trino.sql.planner.iterative.rule.PushProjectionThroughUnion@78de0f6f: 210 ms, 93717 invocations, 0 applications,
    		io.trino.sql.planner.iterative.rule.RemoveRedundantIdentityProjections@2ccb8db3: 149 ms, 93717 invocations, 0 applications,
    		io.trino.sql.planner.iterative.rule.PushProjectionThroughExchange@588614f9: 149 ms, 93717 invocations, 0 applications }
    
  2. findepi commented on Mar 18, 2022

    @findepi
    MemberAuthor
  3. findepi commented on Mar 18, 2022

    @findepi
    MemberAuthor

    Repro: #11561

  4. added this to the 375 milestone on Mar 21, 2022
  5. added theissue type on Jan 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions