-
Notifications
You must be signed in to change notification settings - Fork 5k
Use percentage to represent memory/cpu usage #13896
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use percentage to represent memory/cpu usage #13896
Conversation
davidzollo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
eb662f6 to
05c9980
Compare
Codecov Report
@@ Coverage Diff @@
## dev #13896 +/- ##
============================================
- Coverage 38.91% 38.87% -0.05%
+ Complexity 4465 4453 -12
============================================
Files 1158 1158
Lines 42424 42414 -10
Branches 4772 4776 +4
============================================
- Hits 16510 16488 -22
- Misses 24097 24106 +9
- Partials 1817 1820 +3
... and 4 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Radeity
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, @ruanwenjun , i just wonder why we use percentage representation? For me, physical memory is better.
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/OSUtils.java
Show resolved
Hide resolved
...worker/src/main/java/org/apache/dolphinscheduler/server/worker/task/WorkerHeartBeatTask.java
Outdated
Show resolved
Hide resolved
0a8dfbc to
60cdf22
Compare
Since the instance's memory/cpu might different, use percentage is more suitable, otherwise we need to set different config. |
60cdf22 to
7d164f5
Compare
|
SonarCloud Quality Gate failed. |
If physical memory of different worker vary significantly, such as workerA has 8G memory and workerB has 64G, it's better to set different config, WDYT? Anyway, we can still use different percentage to represent it if you think percentage is more intuitive :D |
I think we may more concern about the percentage of usage. |
Actually I think both make sense to me. May I ask whether it is possible to keep both of them and give users an option to configure whether by physical memory or by percentage? |
This is possible, we can provide different strategy, e.g percentage, absolute value. This will need to change the whole config. avoid-overload:
strategy: Percentage
max-memory-usage: 70%
max-cpu-usage: 80%
# strategy: AbsoluteValue
# max-memory-usage: 6G
# max-cpu-usage: 6CIf this is needed, I will improve this in the next PR. |
Maybe we can simply provide two settings, if satisfy both, we consider the node is overload, like the memory usage is higher than 70% and reserved memory is less than 6G. |
+1 |
caishunfeng
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍








Purpose of the pull request
Brief change log
Verify this pull request
This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(or)
If your pull request contain incompatible change, you should also add it to
docs/docs/en/guide/upgrede/incompatible.md