Skip to content

Update to LevelDB 1.20 #10544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
laanwj merged 4 commits into from
Jun 13, 2017
Merged

Update to LevelDB 1.20 #10544

laanwj merged 4 commits into from
Jun 13, 2017

Conversation

@sipa
Copy link
Member

@sipa sipa commented Jun 6, 2017

gmaxwell
gmaxwell reviewed Jun 6, 2017
View reviewed changes
Copy link
Contributor

@gmaxwell gmaxwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

@sipa
Copy link
Member Author

sipa commented Jun 6, 2017 •
edited
Loading

Major improvement: adds the ability to use hardware-accelerated crc32, which this PR is not exploiting yet.

@gmaxwell
Copy link
Contributor

gmaxwell commented Jun 7, 2017

Travis is bombing with

leveldb/util/env_posix.cc:29:10: fatal error: 'util/env_posix_test_helper.h'

  file not found

#include "util/env_posix_test_helper.h"

@paveljanik
Copy link
Contributor

paveljanik commented Jun 7, 2017

On OS X:

Making all in src
  CXXLD    bitcoind
Undefined symbols for architecture x86_64:
  "leveldb::port::AcceleratedCRC32C(unsigned int, char const*, unsigned long)", referenced from:
      leveldb::crc32c::Extend(unsigned int, char const*, unsigned long) in libleveldb.a(leveldb_libleveldb_a-crc32c.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

@paveljanik
Copy link
Contributor

paveljanik commented Jun 7, 2017

But util/env_posix_test_helper.h is there:

94fe30a#diff-c439a67281cf7c414d51630edca309c9

Strange. This Travis issue does not happen locally.

@paveljanik
Copy link
Contributor

paveljanik commented Jun 7, 2017

My issue is this one 8) NuxiNL/cloudabi-ports#32 reported by @laanwj ;-)

@jonasschnelli
Copy link
Contributor

jonasschnelli commented Jun 8, 2017

Gitian complains:

  CXX      leveldb/util/leveldb_libleveldb_a-status.o
  CXX      leveldb/port/leveldb_libleveldb_a-port_posix.o
  CXX      leveldb/helpers/memenv/leveldb_libmemenv_a-memenv.o
  CXX      test/test_test_bitcoin-arith_uint256_tests.o
  AR       libbitcoin_cli.a
leveldb/util/env_posix.cc:29:10: fatal error: 'util/env_posix_test_helper.h' file not found
#include "util/env_posix_test_helper.h"
         ^
1 error generated.
make[2]: *** [leveldb/util/leveldb_libleveldb_a-env_posix.o] Error 1
make[2]: *** Waiting for unfinished jobs....
  CXX      test/test_test_bitcoin-addrman_tests

@laanwj
Copy link
Member

laanwj commented Jun 8, 2017

My issue is this one 8) NuxiNL/cloudabi-ports#32 reported by @laanwj ;-)

There the SSE-accelerated CRC function was referenced, but not linked in.

@sipa
Copy link
Member Author

sipa commented Jun 8, 2017

@theuni fixed it, yo

@paveljanik
Copy link
Contributor

paveljanik commented Jun 8, 2017

compiles OK here, only one new warning:

leveldb/db/version_set.cc:1039:21: warning: comparison of integers of different signs: 'uint64_t' (aka 'unsigned long long') and 'int' [-Wsign-compare]
      manifest_size >= TargetFileSize(options_)) {
      ~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.

@maflcko
Copy link
Member

maflcko commented Jun 9, 2017

@sipa I suspect you need to fix the out-of-tree builds by something like

diff --git a/src/Makefile.leveldb.include b/src/Makefile.leveldb.include
index b61088e..ac38141 100644
--- a/src/Makefile.leveldb.include
+++ b/src/Makefile.leveldb.include
@@ -77,6 +77,7 @@ leveldb_libleveldb_a_SOURCES += leveldb/table/merger.h
 leveldb_libleveldb_a_SOURCES += leveldb/table/format.h
 leveldb_libleveldb_a_SOURCES += leveldb/table/iterator_wrapper.h
 leveldb_libleveldb_a_SOURCES += leveldb/util/crc32c.h
+leveldb_libleveldb_a_SOURCES += leveldb/util/env_posix_test_helper.h
 leveldb_libleveldb_a_SOURCES += leveldb/util/arena.h
 leveldb_libleveldb_a_SOURCES += leveldb/util/random.h
 leveldb_libleveldb_a_SOURCES += leveldb/util/posix_logger.h

@paveljanik
Copy link
Contributor

paveljanik commented Jun 9, 2017

@MarcoFalke Good catch!

@sipa
Copy link
Member Author

sipa commented Jun 9, 2017

@MarcoFalke Thanks! I've added a commit with you as author.

sipa and others added 4 commits June 9, 2017 19:24
@sipa
Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0
cf44e4c 
196962ff0 Add AcceleratedCRC32C to port_win.h
1bdf1c34c Merge upstream LevelDB v1.20
d31721eb0 Merge #17: Fixed file sharing errors
fecd44902 Fixed file sharing error in Win32Env::GetFileSize(), Win32SequentialFile::_Init(), Win32RandomAccessFile::_Init() Fixed error checking in Win32SequentialFile::_Init()
5b7510f1b Merge #14: Merge upstream LevelDB 1.19
0d969fd57 Merge #16: [LevelDB] Do no crash if filesystem can't fsync
c8c029b5b [LevelDB] Do no crash if filesystem can't fsync
a53934a3a Increase leveldb version to 1.20.
f3f139737 Separate Env tests from PosixEnv tests.
eb4f0972f leveldb: Fix compilation warnings in port_posix_sse.cc on x86 (32-bit).
d0883b600 Fixed path to doc file: index.md.
7fa20948d Convert documentation to markdown.
ea175e28f Implement support for Intel crc32 instruction (SSE 4.2)
95cd743e5 Including <limits> for std::numeric_limits.
646c3588d Limit the number of read-only files the POSIX Env will have open.
d40bc3fa5 Merge #13: Typo
ebbd772d3 Typo
a2fb086d0 Add option for max file size. The currend hard-coded value of 2M is inefficient in colossus.

git-subtree-dir: src/leveldb
git-subtree-split: 196962ff01c39b4705d8117df5c3f8c205349950
@sipa
Update to LevelDB 1.20
e4030ab
@theuni @sipa
leveldb: enable runtime-detected crc32 instructions
2424989
@sipa
Add extra LevelDB source to Makefile
3ee3d04
gmaxwell
gmaxwell approved these changes Jun 13, 2017
View reviewed changes
Copy link
Contributor

@gmaxwell gmaxwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@laanwj
Copy link
Member

laanwj commented Jun 13, 2017

After merging bitcoin-core/leveldb-subtree#1

src/leveldb in HEAD was last updated to upstream commit 196962ff01c39b4705d8117df5c3f8c205349950 (tree d5a89851da0a8ab07c40966ac137f4e299c80927)
src/leveldb in HEAD currently refers to tree d5a89851da0a8ab07c40966ac137f4e299c80927
GOOD

@laanwj laanwj merged commit 3ee3d04 into bitcoin:master Jun 13, 2017
laanwj added a commit that referenced this pull request Jun 13, 2017
@laanwj
Merge #10544: Update to LevelDB 1.20
a4fe077 
3ee3d04 Add extra LevelDB source to Makefile (MarcoFalke)
2424989 leveldb: enable runtime-detected crc32 instructions (Cory Fields)
cf44e4c Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0 (Pieter Wuille)

Tree-SHA512: 19ade77e3f6265507b3ab7b9aa5150d378aa0751e24ac7a61567b0f720a566cedc6c3d3336da17a3bd2b5d068ee86600d96a15228f78bd20ccf98c8fc9041a91
@theuni
Copy link
Member

theuni commented Jun 13, 2017

post-merge utACK.

@fanquake fanquake mentioned this pull request Jun 14, 2017
Closed
3 tasks
@sipa sipa deleted the leveldb120 branch June 23, 2017 00:34
sickpig referenced this pull request in sickpig/BitcoinUnlimited Aug 17, 2017
@laanwj @sickpig
Backport of Core #10544: Update to LevelDB 1.20
1380391 
3ee3d04 Add extra LevelDB source to Makefile (MarcoFalke)
2424989 leveldb: enable runtime-detected crc32 instructions (Cory Fields)
cf44e4c Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0 (Pieter Wuille)
sickpig referenced this pull request in sickpig/BitcoinUnlimited Aug 17, 2017
@laanwj @sickpig
Port Core #10544: Update to LevelDB 1.20
521cfbf 
3ee3d04 Add extra LevelDB source to Makefile (MarcoFalke)
2424989 leveldb: enable runtime-detected crc32 instructions (Cory Fields)
cf44e4c Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0 (Pieter Wuille)

Tree-SHA512: 19ade77e3f6265507b3ab7b9aa5150d378aa0751e24ac7a61567b0f720a566cedc6c3d3336da17a3bd2b5d068ee86600d96a15228f78bd20ccf98c8fc9041a91
@sickpig sickpig mentioned this pull request Aug 17, 2017
Merged
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Jul 9, 2019
@laanwj @PastaPastaPasta
Merge bitcoin#10544: Update to LevelDB 1.20
9463881 
3ee3d04 Add extra LevelDB source to Makefile (MarcoFalke)
2424989 leveldb: enable runtime-detected crc32 instructions (Cory Fields)
cf44e4c Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0 (Pieter Wuille)

Tree-SHA512: 19ade77e3f6265507b3ab7b9aa5150d378aa0751e24ac7a61567b0f720a566cedc6c3d3336da17a3bd2b5d068ee86600d96a15228f78bd20ccf98c8fc9041a91
@str4d str4d mentioned this pull request Sep 22, 2020
Merged
zkbot added a commit to zcash/zcash that referenced this pull request Sep 25, 2020
@zkbot
Auto merge of #4739 - str4d:update-leveldb, r=str4d
514d868 
Update LevelDB to upstream commit f545dfabf

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#7911
- bitcoin/bitcoin#7982
- bitcoin/bitcoin#8133
- bitcoin/bitcoin#8784
  - Only the missing changes.
- bitcoin/bitcoin#8826
- bitcoin/bitcoin#8613
- bitcoin/bitcoin#10544
- bitcoin/bitcoin#10633
  - Only the changes to files and code we have.
- bitcoin/bitcoin#10806
- bitcoin/bitcoin#10958
- bitcoin/bitcoin#12451
- bitcoin/bitcoin#13925
- bitcoin/bitcoin#15270

This upgrades LevelDB in the exact same commit progression as upstream, up to January 2019.
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@laanwj laanwj laanwj approved these changes

+1 more reviewer

@gmaxwell gmaxwell gmaxwell approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@sipa @gmaxwell @paveljanik @jonasschnelli @laanwj @maflcko @theuni