Skip to content

[enhancement](memtracker) Optimize readability of mem exceed limit error message - #11877

Merged
yiguolei merged 1 commit into
apache:masterfrom
xinyiZzz:mem_tracker_factor_v2_fix9
Aug 18, 2022
Merged

yiguolei merged 1 commit into
apache:masterfrom
xinyiZzz:mem_tracker_factor_v2_fix9

Conversation

@xinyiZzz

@xinyiZzz xinyiZzz commented Aug 17, 2022 •

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close #11876

Problem summary

  1. Mem exceed limit error message after optimization
memory limit exceeded:<exceeded_tracker=Query#queryId=1720e4ca3ac541c0-8dee8f4667ee7176, limit=80000, peak_used=133111504, current_used=133168448>, executing:<exec node:, vsort, while open.>, backend=172.24.47.117 free memory left=12.01 GB. If is query, can change the limit by `set exec_mem_limit=xxx`, details mem usage see be.INFO.
  1. Optimize the constructor of mem tracker.
  2. Restore consume_local, which has better applicability.

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@xinyiZzz
xinyiZzz force-pushed the mem_tracker_factor_v2_fix9 branch from aad9df6 to 87a3013 Compare August 17, 2022 14:22
// this will cause coredump for ASAN build when running regression test,
// disable temporarily.
doris::ExecEnv::GetInstance()->task_pool_mem_tracker_registry()->logout_task_mem_tracker();
doris::ExecEnv::GetInstance()->process_mem_tracker()->enable_print_log_usage();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this method has to be called and it only set a property value. I think the property value should be a config.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enable_print_log_usage is reset to true every 1s, this is to avoid frequent log printing when process mem tracker exceeds mem limit.

@yiguolei yiguolei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] Improve readability of mem exceed limit error message

2 participants