Bitcoin full node and query server

Jun 13, 2024

View in context in the libbitcoin Slack #general mirror.

evoskuil18:07

Also, there is a command line option on bn that allows you to see what your compiler supports (without the need to compile with the option of course)... platform zero means not available, compiled zero means you didn't compile for it.

$ bn -c f:/bn.cfg --hardware
Intrinsics...
arm..... platform:0.
intel... platform:1.
avx512.. platform:0 compiled:0.
avx2.... platform:1 compiled:1.
sse41... platform:1 compiled:1.
shani... platform:0 compiled:0.
neon.... platform:0 compiled:0.

21 replies · parsevalbtc, evoskuil, pmienk · last Jun 16, 2024

parsevalbtc11:25

got avx2 and sse4 as well. Will re-compile also including additional flags reported in the messages above and let you know hows test will be going.

 $ ./bn -c ../etc/libbitcoin/libbitcoin.conf -d
Intrinsics...
arm..... platform:0.
intel... platform:1.
avx512.. platform:0 compiled:0.
avx2.... platform:1 compiled:0.
sse41... platform:1 compiled:0.
shani... platform:0 compiled:0.
neon.... platform:0 compiled:0.

evoskuil12:08

The latest stem build includes the changes to haves.hpp (as of Thursday).

👍 1

parsevalbtc16:51

Unfortunately, it seems that by adding $ CFLAGS="-03" CXXFLAGS="-03" to ./install.sh ... the compilation stops at:

configure:3495: error: in `/home/user11/gitprojects2/libbitcoin4/build-libbitcoin-node/secp256k1':
configure:3497: error: C compiler cannot create executables
See `config.log' for more details

(edited)

parsevalbtc16:56

By removing these optimization "-03" flags the compliation process goes smoothly till the end. However, looking at the messages: configure: WARNING: unrecognized options: --enable-avx2, --enable-sse4

evoskuil18:33

You can ignore those.

parsevalbtc21:59

you mean ignore avx2 and ss4 unrecognized flag warnings?

(edited)

evoskuil23:11

Yes. The install.sh arguments get passed to each build, but these only apply to the libbitcoin-system build, so you get warnings on the others.

👍 1

parsevalbtc11:08

...Stupid error. It's -O3 (with o not zero). Now compiling.

parsevalbtc12:04

Ok the compilation went fine with -O3 flags. However avx2 and ss4 optimization where not compiled:

$ ./bn -c ../libbitcoin.conf -d
Intrinsics...
arm..... platform:0.
intel... platform:1.
avx512.. platform:0 compiled:0.
avx2.... platform:1 compiled:0.
sse41... platform:1 compiled:0.
shani... platform:0 compiled:0.
neon.... platform:0 compiled:0.

parsevalbtc12:12

First impression is that the node is faster than previous compilations

evoskuil13:09

Did you build from latest?

evoskuil13:59

I did extensive testing, including disassembly for comparison with C implementations, and detailed performance testing. The implementation relies heavily on compiler optimization. That’s the nature of C++. The developer manages the code such that the compiler can reduce it to efficient machine code while retaining the advantageous abstractions. Hashing (for example) is like 10x slower without full optimization.

(edited)

👍 1

parsevalbtc14:04

yeah latest build:

libbitcoin-node$ git log
commit 0e7853934e778f7ba03e74b887b6a90c004596d1 (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Eric Voskuil <eric@voskuil.org>
Date:   Wed Jun 12 21:02:09 2024 -0400

    Merge pull request #643 from evoskuil/master
    
libbitcoin-system$ git log
commit b619ff2dd6c2e77679ce935cb84a177fbb7ae918 (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Eric Voskuil <eric@voskuil.org>
Date:   Thu Jun 13 21:12:39 2024 -0400

    Merge pull request #1480 from evoskuil/master

libbitcoin-network$ git log
commit 19177766c968917e3d38306e84f0d7c398d5c7e6 (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Eric Voskuil <eric@voskuil.org>
Date:   Thu Jun 6 18:04:51 2024 -0400

    Merge pull request #414 from evoskuil/master
    
libbitcoin-database$ git log
commit ed3e0a26b7072266d8b3b81134497ea9f575525b (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Eric Voskuil <eric@voskuil.org>
Date:   Thu Jun 13 00:47:13 2024 -0400

    Merge pull request #493 from evoskuil/master

parsevalbtc14:06

any idea why it didn't optimize the sse4 and avx2?

parsevalbtc14:06

I can provide the full compilation log in case

(edited)

parsevalbtc14:31

From a quick look it seems that the compiler has embedded the flags in the compilation stream:

checking whether C++ compiler accepts -mavx -mavx2... yes
checking whether C++ compiler accepts -msse4.1... yes

evoskuil15:24

This is a better question for @pmienk

pmienk15:43

Full log would be helpful.

evoskuil15:59

FYI in a recent -system change I removed all of the runtime conditionality, so the builds are not portable but are maximally benefit from hash vectorization. The conditionality is only used now to provide bn command line feedback on the availability of the optimizations.

evoskuil16:01

The sse4 and avx2 options cut CPU by 4x and 8x respectively for merkle root computations. avx512 would be 16x.

parsevalbtc19:29

here is the log @pmienk. The launched coommand was:

CFLAGS="-O3" CXXFLAGS="-O3" ./install.sh --prefix=/home/tester/gitprojects2/libbitcoin4 --build-boost --enable-ndebug --disable-shared --enable-avx2 --enable-sse4

BINAinstall_20240616_5_mod.logBinary · 121.8 KB