Bitcoin full node and query server
Apr 23, 2024
View in context in the libbitcoin Slack #general mirror.
Rich Phillips14:17
@evoskuil can’t seem to download the Mac binary for v3.2 of explorer, get a 404 error. Can that be restored or is there someplace else I can grab it from?
157 replies · Rich Phillips, evoskuil, pmienk · last Apr 23, 2024
Rich Phillips14:20
😬 ok, was hoping that would be the simple button since the build is sucking right now. What’s the flag to build without boost library? Or do I have to hack it out of the install script?
evoskuil14:22
The default install.sh does not build boost, but commonly the local install of boost isn’t compatible.
Rich Phillips20:27
Ok either I’m being stupid here or I’m doing this wrong but running either auto gen /configure or just install results in a check for boostlib being present. I can’t force it to be ignored. I’ve even forced a false flag on the install scripts etc to try and skip it but as soon as the linker tries to build the system section it falls over here.
evoskuil20:33
Right, the question is only whether you build boost or rely on your preinstalled version.
evoskuil20:35
$ ./install.sh --with-icu --build-icu --build-boost --build-zmq --prefix=/home/me/myprefix
evoskuil20:35
evoskuil20:37
This can be reduced to the following since you probably don't need ICU.
$ ./install.sh --build-boost --build-zmq --prefix=/home/me/myprefix
Rich Phillips20:39
I’ve built it, installed boost a hundred times, added to path. And even with all that the linked won’t find the binaries/libraries. Its the last thing that seems to fall over
Rich Phillips20:55
Ok moved along now it won’t build or find the secp256 library. That worked before.
evoskuil21:01
BTW, you should also set
--disable-shared as we cannot fully control the linker and it tends to to try and dynamically link to installed libs like boost despite the build isolation and direction. Static linking prevents that.evoskuil21:04
FYI, this is the most recent bx static/osx/clang/build-boost/build-zmq CI build. https://github.com/libbitcoin/libbitcoin-explorer/actions/runs/8117844403/job/22190912983
(edited)Rich Phillips21:18
output
Libbitcoin installer configuration. -------------------------------------------------------------------- OS : Darwin PARALLEL : 10 CC : clang CXX : clang++ CPPFLAGS : CFLAGS : CXXFLAGS : LDFLAGS : LDLIBS : WITH_ICU : WITH_PNG : WITH_QRENCODE : BUILD_ICU : BUILD_ZLIB : BUILD_PNG : BUILD_QRENCODE : BUILD_BOOST : yes PREFIX : /Users/richphillips/bx/ BUILD_DIR : build-libbitcoin-explorer DISABLE_SHARED : yes DISABLE_STATIC : with_boost : --with-boost=/Users/richphillips/bx/ with_pkgconfigdir : --with-pkgconfigdir=/Users/richphillips/bx//lib/pkgconfig -------------------------------------------------------------------- Initialized empty Git repository in /Users/richphillips/libbitcoin-explorer-3.2.0/build-libbitcoin-explorer/.git/ ********************** Download boost_1_85_0.tar.bz2 ********************** Libbitcoin boost configuration. -------------------------------------------------------------------- variant : release threading : multi toolset : clang cxxflags : linkflags : link : static boost.locale.iconv : on boost.locale.posix : onfull file
evoskuil21:33
@pmienk Going back to /version4 there was no versioning in secp256k1, just [0.1] even with changes, which is why we retain branches for versioning.
evoskuil21:37
Master shows
0.4.2.0. As of our most recent /version8 they were still showing 0.1.0.0. But the install is checking for 0.1.0.19.evoskuil21:42
I'm not sure what's up, but the log output doesn't match our current /version3 CI build.
pmienk21:42
@evoskuil you are referring to constraints that preexist the build project merging versions 3 and 4. I do not recall those limits
pmienk21:45
Correct. The .19 ending has been generated for more than 3 years by my quick check of the file history
evoskuil21:45
Notice that in our latest /version3 CI the following text:
********************** Prepairing to aquire libbitcoin/secp256k1/version7 ********************** Cloning libbitcoin/secp256k1/version7...
evoskuil21:45
evoskuil21:50
And now see /version8 is actually at 0.1.0.20. It looks like we did transition to using the libsecp versioning, which I forgot.
evoskuil21:51
But the install script posted above doesn't match latest /version3, and is pulling /version4 branch libsecp (which is unversioned).
pmienk21:52
Agreed, had assumed no other alteration of a fresh install.sh. any url substitution should be performed on latest version3 install.sh
👍🏻 1
evoskuil21:54
So is that the master install.sh? Confused because of this:
********************** Download libbitcoin/secp256k1/version4 **********************
evoskuil22:11
Boost hits incompat with new compilers etc. It is not always easy to get a version that works. You may even have to downgrade your compiler.
(edited)evoskuil22:13
But this line emitted from a v3.2 installer doesn't make sense.
checking for libsecp256k1 >= 0.1.0.19... no
Rich Phillips22:14
I’m gonna try the 3.8 tag instead. Start there. I think as long as it’s v3 should be ok
evoskuil22:16
Major versions maintain compatibility. So v3 latest is where you should be. I am still curious though how a 7 year old build is trying to find a very recent version of libsecp256k1.
(edited)Rich Phillips22:16
That part I couldn’t tell you. I tried to trace where it was looking for v4 and couldn’t find it
evoskuil22:18
That part is correct. /version4 secp is not the same as /version4 libbitcoin. They aren't aligned.
evoskuil22:20
So that build would never work if looking for 0.1.0.19 - it didn't even exist until 5+ years later.
(edited)Rich Phillips22:20
As I said, no idea why it was looking for that version, didn’t edit those parts. Just the boost url
evoskuil22:27
This is from the /version3 head (v3.8) configure.ac:
# Require secp256k1 of at least version 0.1.0.19 and output ${secp256k1_CPPFLAGS/LIBS/PKG}.
#------------------------------------------------------------------------------
PKG_CHECK_MODULES([secp256k1], [libsecp256k1 >= 0.1.0.19],
[secp256k1_INCLUDEDIR="`$PKG_CONFIG --variable=includedir "libsecp256k1 >= 0.1.0.19" 2>/dev/null`"
secp256k1_OTHER_CFLAGS="`$PKG_CONFIG --cflags-only-other "libsecp256k1 >= 0.1.0.19" 2>/dev/null`"],
[AC_MSG_ERROR([libsecp256k1 >= 0.1.0.19 is required but was not found.])])
AC_SUBST([secp256k1_PKG], ['libsecp256k1 >= 0.1.0.19'])
AC_SUBST([secp256k1_CPPFLAGS], [${secp256k1_CFLAGS}])
AS_IF([test x${secp256k1_INCLUDEDIR} != "x"],
[AC_SUBST([secp256k1_ISYS_CPPFLAGS], ["-isystem${secp256k1_INCLUDEDIR} ${secp256k1_OTHER_CFLAGS}"])],
[AC_SUBST([secp256k1_ISYS_CPPFLAGS], [${secp256k1_OTHER_CFLAGS}])])
AC_MSG_NOTICE([secp256k1_CPPFLAGS : ${secp256k1_CPPFLAGS}])
AC_MSG_NOTICE([secp256k1_ISYS_CPPFLAGS : ${secp256k1_ISYS_CPPFLAGS}])
AC_MSG_NOTICE([secp256k1_OTHER_CFLAGS : ${secp256k1_OTHER_CFLAGS}])
AC_MSG_NOTICE([secp256k1_INCLUDEDIR : ${secp256k1_INCLUDEDIR}])
AC_MSG_NOTICE([secp256k1_LIBS : ${secp256k1_LIBS}])
AS_CASE([${enable_isystem}],[yes],
[AC_SUBST([secp256k1_BUILD_CPPFLAGS], [${secp256k1_ISYS_CPPFLAGS}])],
[AC_SUBST([secp256k1_BUILD_CPPFLAGS], [${secp256k1_CPPFLAGS}])])
AC_MSG_NOTICE([secp256k1_BUILD_CPPFLAGS : ${secp256k1_BUILD_CPPFLAGS}])evoskuil22:28
When you use the install.sh it does not build from tags, it builds latest from branches. So by using a v3.2 install.sh you were mixing updates to v3 that are 7 years apart.
Rich Phillips22:29
I literally trashed the folders and downloaded fresh from the tag version for each build
evoskuil22:30
The installer is designed to build the latest of the branch it resides in, which means the latest from all corresponding branches in the other libbitcoin repos (not secp256k1).
evoskuil22:32
So when you get the v3.2 installer it builds external dependencies like boost/zeromq/secp from its recorded versions/URLs. But all the libbitcoin repos are built from their /version3 branch tops. And /libbitcoin-system/version3/ (v3.8) requires a much more recent libsecp than the v3.2 install.sh is building.
(edited)Rich Phillips22:49
Ok moved on from that, building 3.8 explorer, now we get past most stuff , finds the boost 1.73 content ok but, can’t find version for the Chrono library. It’s definitely there in the boost directory, have rebuilt boost a few times to make sure.
Rich Phillips22:50
checking for boostlib >= 1.73.0 (107300) includes in "/Users/richphillips/boost_1_73_0//include"... no
checking for boostlib >= 1.73.0 (107300)... yes
configure: boost_CPPFLAGS :
configure: boost_ISYS_CPPFLAGS :
configure: boost_LDFLAGS :
configure: boost_BUILD_CPPFLAGS :
checking whether the Boost::Chrono library is available... yes
configure: error: Could not find a version of the Boost::Chrono library!
Rich Phillips22:55
If I have the installer build boost instead of using one already built it throws build errors
pmienk22:55
Reporting is spotty from those boost checks. The m4s commonly report the wrong library. Likely it's the one immediately preceeding
Rich Phillips22:57
If I force using —with-boost and give a path, it finds it but can’t version chrono.
Rich Phillips22:57
If I have install build it, I get out of range error for a static cast for an enumerator
(edited)pmienk23:00
The supplied m4s have this as a known issue. What other output are you relying on to say it's found with the provided parameterization? Did the boost build share static/dynamic parameterization as your libbitcoin build?
evoskuil23:00
As I said, reliance on local installs is problematic because parts are often missing or compiled differently.
evoskuil23:01
So I would ask, what errors are you seeing when you use the installer to build it into a prefix lib, are you building static and not into a system directory.
evoskuil23:02
And what is your compiler version. There are known issues with boost and recent compilers.
Rich Phillips23:04
And because I’m
Setting the boost url, it’s building from
The same source tarball as I use for a Static build
evoskuil23:07
There are different components and build parameters. This is why preinstalled packages aren't generally usable.
Rich Phillips23:09
In file included from libs/thread/src/pthread/thread.cpp:11:
In file included from ./boost/thread/thread_only.hpp:17:
In file included from ./boost/thread/pthread/thread_data.hpp:12:
In file included from ./boost/thread/lock_types.hpp:18:
In file included from ./boost/thread/thread_time.hpp:11:
In file included from ./boost/date_time/posix_time/posix_time_types.hpp:16:
In file included from ./boost/date_time/posix_time/posix_time_duration.hpp:15:
In file included from ./boost/numeric/conversion/cast.hpp:33:
In file included from ./boost/numeric/conversion/converter.hpp:13:
In file included from ./boost/numeric/conversion/conversion_traits.hpp:13:
In file included from ./boost/numeric/conversion/detail/conversion_traits.hpp:18:
In file included from ./boost/numeric/conversion/detail/int_float_mixture.hpp:19:
In file included from ./boost/mpl/integral_c.hpp:32:
./boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for this enumeration type [-Wenum-constexpr-conversion]
./boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
# define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
Rich Phillips23:09
I get two of the same errors on this file. I can edit the cast to fix but next build those changes are overwritten again
evoskuil23:10
It's the compiler. The warning has been elevated to an error by clang and cannot be suppressed.
evoskuil23:11
Boost hasn't fixed it yet, so the only option is to regress the compiler, which is what we did in CI.
evoskuil23:19
I've got it down to libsecp and a much smaller boost subset, and zeromq for client/server and optional ICU.
Rich Phillips23:51
Well I got further again but now it throws 20 errors and fatal errors out. Gonna have to work through each and see if I can work around them.
Rich Phillips23:51
Looks like this is driven by compiler version but don’t have a choice (thanks Apple :))
evoskuil23:56
Yeah, it sucks that they just decided to break this, and that boost didn't get it resolved.
evoskuil23:58
We've been lucky to be able to remove a lot of boost as that code has worked it's way into c++20 etc. But we'll probably at least have ASIO for some time.