Skip to content

Fix missing _ZN6google8protobuf2io17SafeDoubleToFloatEd#191

Merged
h-vetinari merged 5 commits intoconda-forge:mainfrom
h-vetinari:symbol
May 31, 2023
Merged

Fix missing _ZN6google8protobuf2io17SafeDoubleToFloatEd#191
h-vetinari merged 5 commits intoconda-forge:mainfrom
h-vetinari:symbol

Conversation

@h-vetinari
Copy link
Copy Markdown
Member

@h-vetinari h-vetinari commented May 27, 2023

Check out what's going on in #190

Closes #190

@conda-forge-webservices
Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@xylar
Copy link
Copy Markdown
Contributor

xylar commented May 27, 2023

@h-vetinari, thanks for taking a look.

@h-vetinari h-vetinari marked this pull request as draft May 27, 2023 21:47
@h-vetinari
Copy link
Copy Markdown
Member Author

I'm scratching my head why that symbol wouldn't be found (or not in the linked protobuf in the first place). If it's urgent we can switch from the CPP to the python builds, then at least things would run (if perhaps a bit slower)

@xylar
Copy link
Copy Markdown
Contributor

xylar commented May 28, 2023

@h-vetinari, absolutely no rush on my part. I had just noticed this issue in a few different repos so I decided it was time to report it. I think you're right that it would be better to take the time to debug the CPP build than to resort back to the python one. I'm following along what you're trying in the hope that I can be more helpful at debugging these kinds of issues in the future.

@h-vetinari h-vetinari changed the title Debug Debug missing _ZN6google8protobuf2io17SafeDoubleToFloatEd May 30, 2023
@h-vetinari
Copy link
Copy Markdown
Member Author

@coryan, if you have a moment, I feel I have need of your C++ foo here. I cannot figure out for the life of me why that symbol would not be found. To recap:

  • The source doesn't do anything unusual (well, setup.py doesn't correctly link abseil, but that's not it and I'm "fixing" this here already)
  • That file definitely gets included in the build of libprotobuf
  • libprotobuf gets linked correctly(?) to _message.cpython-PYVER-x86_64-linux-gnu.so:
    INFO (protobuf,lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so): Needed DSO lib/libprotobuf.so.23.2.0 found in conda-forge::libprotobuf-4.23.2-hd1fb520_1
    
  • The symbol being asked for by _message.cpython-PYVER-x86_64-linux-gnu.so matches the signature of what's in libprotobuf
    >llvm-cxxfilt _ZN6google8protobuf2io17SafeDoubleToFloatEd
    google::protobuf::io::SafeDoubleToFloat(double)
    

@h-vetinari
Copy link
Copy Markdown
Member Author

PS. There's a tiny hint on OSX:

Expected in: flat namespace

but googling for that, the one thing I found makes it look like building against different stdlibs, which we're not doing here.

@coryan
Copy link
Copy Markdown

coryan commented May 30, 2023

if you have a moment, I feel I have need of your C++ foo here.

Happy to help. Don't take anything I say as gospel. I am good at expressing myself with more confidence than I should.

I cannot figure out for the life of me why that symbol would not be found.

Maybe this:

https://github.com/protocolbuffers/protobuf/blob/6609bea792af9624f460dad0213bb47fd1ccf623/cmake/libprotobuf.cmake#L42

In combination with a missing PROTOBUF_EXPORT here:

https://github.com/protocolbuffers/protobuf/blob/0ce1fce37f5663552448a35ef1cb84e28e764547/src/google/protobuf/io/strtod.h#L68

I do not know if it is intentional that this symbol is hidden, but at least on my build it is (the lowercase t as opposed to T is the giveaway):

nm /usr/local/lib64/libprotobuf.so.23.2.0 | c++filt | grep SafeDoubleToFloat
000000000018f350 t google::protobuf::io::SafeDoubleToFloat(double)

Contrast with an exported symbol:

nm /usr/local/lib64/libprotobuf.so.23.2.0 | c++filt | grep SimpleDtoa
000000000018f450 T google::protobuf::io::SimpleDtoa[abi:cxx11](double)

We need to ask the Protobuf team if they meant for these symbols to be used from outside the .so.

To recap: The symbol in question

  • The source doesn't do anything unusual (well, setup.py doesn't correctly link abseil, but that's not it and I'm "fixing" this here already)
  • That file definitely gets included in the build of libprotobuf
  • libprotobuf gets linked correctly(?) to _message.cpython-PYVER-x86_64-linux-gnu.so:
    INFO (protobuf,lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so): Needed DSO lib/libprotobuf.so.23.2.0 found in conda-forge::libprotobuf-4.23.2-hd1fb520_1
    
  • The symbol being asked for by _message.cpython-PYVER-x86_64-linux-gnu.so matches the signature of what's in libprotobuf
    >llvm-cxxfilt _ZN6google8protobuf2io17SafeDoubleToFloatEd
    google::protobuf::io::SafeDoubleToFloat(double)
    

That all seems correct, but for the hidden symbols flag.

@h-vetinari
Copy link
Copy Markdown
Member Author

Ah, that's amazing, thanks!

We need to ask the Protobuf team if they meant for these symbols to be used from outside the .so.

Well, it's their own python bindings that want to access it! 😅

It seems they're not covering the build of the C++ backend against a shared libprotobuf (anymore? what with the move to upb) ... another issue to open?

@h-vetinari h-vetinari changed the title Debug missing _ZN6google8protobuf2io17SafeDoubleToFloatEd Fix missing _ZN6google8protobuf2io17SafeDoubleToFloatEd May 30, 2023
@coryan
Copy link
Copy Markdown

coryan commented May 30, 2023

Ah, that's amazing, thanks!

No worries.

We need to ask the Protobuf team if they meant for these symbols to be used from outside the .so.

Well, it's their own python bindings that want to access it! 😅

HA! Yeah, that kind of strongly suggests they should be public.

It seems they're not covering the build of the C++ backend against a shared libprotobuf (anymore? what with the move to upb) ... another issue to open?

I created: protocolbuffers/protobuf#12932

Comment thread recipe/patches/0007-add-PROTOBUF_EXPORT-for-google-protobuf-io-SafeDoubl.patch Outdated
@h-vetinari h-vetinari marked this pull request as ready for review May 31, 2023 04:23
@h-vetinari
Copy link
Copy Markdown
Member Author

Thanks a lot for your help on this @coryan! 🙏

@h-vetinari h-vetinari merged commit a29f81c into conda-forge:main May 31, 2023
@h-vetinari h-vetinari deleted the symbol branch May 31, 2023 04:25
@xylar
Copy link
Copy Markdown
Contributor

xylar commented May 31, 2023

Thanks so much @h-vetinari, and @coryan as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Undefined symbol _ZN6google8protobuf2io17SafeDoubleToFloatEd

3 participants