Skip to content

Masking glibc symbols for better portability - #4180

Merged
yangzhg merged 2 commits into
apache:masterfrom
amosbird:master
Mar 5, 2021
Merged

yangzhg merged 2 commits into
apache:masterfrom
amosbird:master

Conversation

@amosbird

@amosbird amosbird commented Jul 24, 2020 •

Copy link
Copy Markdown
Contributor

Proposed changes

In ClickHouse we have glibc symbols masked to backport down towards CentOS 5 (glibc 2.5). It's quite useful for creating portable executables. I'd love to have this capability shipped into Doris :)

This PR contains glibc related symbols extracted from https://github.com/bminor/musl/commit/593caa456309714402ca4cb77c3770f4c24da9da . It's been verified in ClickHouse and works perfectly. I've also shipped https://github.com/skywind3000/FastMemcpy together to have a hosted memcpy implementation. Otherwise it requires [email protected] which is not available on most CentOS 6 machines.

Btw, I've also disabled libssh2 and nghttp2 when building libcurl. It somehow introduces weird dependency issues on my dev machine (gentoo). Please verify if they are indeed necessary.

After this PR, palo_be only requires glibc 2.10. Related license files are attached too.

Types of changes

What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I have create an issue on Doris's issues, and have described the bug/feature there in detail
  • Commit messages in my PR start with the related issues ID, like "Add pull request template to doris project #4071 Add pull request template to doris project"
  • Compiling and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • If this change need a document change, I have updated the document
  • Any dependent changes have been merged

@morningman

Copy link
Copy Markdown
Contributor

Good work!
I am not familiar with this part of the logic, but I read some of the cited articles you gave.
I have some questions:

  1. What is difference between src/glibc-compatibility/FastMemcpy.h and src/FastMemcpy/FastMemcpy.h?
  2. Does it compatible with ARM?

@amosbird

Copy link
Copy Markdown
Contributor Author

What is difference between src/glibc-compatibility/FastMemcpy.h and src/FastMemcpy/FastMemcpy.h?

Oops, those are redundant. I'll get rid of the FastMemcpy folder.

Does it compatible with ARM?

Not yet in this PR, but it's doable. I've done ARM-compatible stuff for ClickHouse. I'll send another PR if this one gets accepted :)

@morningman

Copy link
Copy Markdown
Contributor

Not yet in this PR, but it's doable. I've done ARM-compatible stuff for ClickHouse. I'll send another PR if this one gets accepted :)

Thanks!
But we are going the release version 0.13, I think its better to merge this PR after we create 0.13 branch.
It will be soon.

@amosbird

amosbird commented Jul 27, 2020 •

Copy link
Copy Markdown
Contributor Author

But we are going the release version 0.13 ...

Great!

@yangzhg yangzhg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yangzhg
yangzhg merged commit d6ac8f4 into apache:master Mar 5, 2021
EmmyMiao87 pushed a commit to EmmyMiao87/incubator-doris that referenced this pull request Apr 21, 2021
* Masking glibc symbols for better portability

* Remove redundant files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants