Bitcoin full node and query server
August 2020
Archive of the libbitcoin Slack #general channel.
Monday, August 3, 2020
2 replies · neillm, gh · last Aug 04, 2020
neillm01:46
Rushed public release of software not ready for the public 😅 https://github.com/thecodefactory/megabit @gh Please fix it!
Tuesday, August 4, 2020
N
slack-term10:25
where is the getblocks network code located? I couldn't find it in -network repo. Is it in -node?
1 reply · evoskuil · last Aug 05, 2020
evoskuil05:05
libbitcoin-network includes network protocols for minimal network traffic. blocks require a blockchain, so blocks protocols are added by libbitcoin-node, which incorporates a libbitcoin-blockchain dependency.
Wednesday, August 5, 2020
K
Kaichieh Ku01:08
Hi, I am trying to deploying v4 libbitcoin servers. I noticed that if I set
flush_writes = true, the database will always corrupt with lock failure error. I also read the implementation of class flush_lock , and seems the failure is indeed can be triggered easily. Because flush_lock::try_lock only check the existence of the lock file without creating it, while flush_lock::lock_shared fails if it cannot create lock file immediately. Is this intentional?
I think semantic behavior is try_lock for non-blocking lock (this is what flush_lock::lock_shared does currently), and lock_shared for blocking lock.3 replies · evoskuil · last Aug 05, 2020
evoskuil05:24
Hi,
v4 is not production code and deployment is definitely not recommended. I believe write flushing in v4 fails purposely because it is not actually supported yet due to changes to the store structure.
The flush_lock semantics are as intended. lock_shared blocks on creation of the lock file. If creation fails there is an error, not simply a lock to wait on.
(edited)🙏 1
evoskuil05:27
The flush lock exists only to cause a lock file to be left behind in the event of an uncontrolled shutdown. The lock is tested on startup, preventing execution in the case of a preceding uncontrolled shutdown.
(edited)evoskuil05:40
The original simple startup/shutdown use of the lock is straightforward. However write flushing is essentially transactional commit. In v3 this could be guarded in each data file write independently. However v4 incorporates multi-file transactions, materially complicating write flushing. As you can see the lock attempts are not guarded, so the class is not thread safe across the shared lock file resource. In order to properly support transactional commit the class needs to accept a flush mutex and use it to apply an upgradeable lock around the lock file access. This will allow the flush lock to become an actual guard for transactional commit.
(edited)evoskuil05:05
replied to a thread: view
libbitcoin-network includes network protocols for minimal network traffic. blocks require a blockchain, so blocks protocols are added by libbitcoin-node, which incorporates a libbitcoin-blockchain dependency.
evoskuil05:24
replied to a thread: view
Hi,
v4 is not production code and deployment is definitely not recommended. I believe write flushing in v4 fails purposely because it is not actually supported yet due to changes to the store structure.
The flush_lock semantics are as intended. lock_shared blocks on creation of the lock file. If creation fails there is an error, not simply a lock to wait on.
(edited)🙏 1
evoskuil05:27
replied to a thread: view
The flush lock exists only to cause a lock file to be left behind in the event of an uncontrolled shutdown. The lock is tested on startup, preventing execution in the case of a preceding uncontrolled shutdown.
(edited)evoskuil05:40
replied to a thread: view
The original simple startup/shutdown use of the lock is straightforward. However write flushing is essentially transactional commit. In v3 this could be guarded in each data file write independently. However v4 incorporates multi-file transactions, materially complicating write flushing. As you can see the lock attempts are not guarded, so the class is not thread safe across the shared lock file resource. In order to properly support transactional commit the class needs to accept a flush mutex and use it to apply an upgradeable lock around the lock file access. This will allow the flush lock to become an actual guard for transactional commit.
(edited)1 reply · evoskuil · last Aug 08, 2020
if (!(block_height >= prevout.height)) { std::cout << "IS_LOCKED block_height=" << block_height ... << "prevout.height: " << prevout.height << std::endl; } BITCOIN_ASSERT(block_height >= prevout.height);Here is what I get:
bool input::is_locked(size_t block_height, uint32_t median_time_past) const
{
if ((sequence_ & relative_locktime_disabled) != 0)
return false;
// bip68: a minimum block-height constraint over the input's age.
const auto minimum = (sequence_ & relative_locktime_mask);
const auto& prevout = previous_output_.metadata;
if ((sequence_ & relative_locktime_time_locked) != 0)
{
// Median time past must be monotonically-increasing by block.
BITCOIN_ASSERT(median_time_past >= prevout.median_time_past);
const auto age_seconds = median_time_past - prevout.median_time_past;
return age_seconds < (minimum << relative_locktime_seconds_shift);
}
std::cout << "bh: " << block_height << ", ph: " << prevout.height << std::endl;
if (!(block_height >= prevout.height)) {
std::cout << "IS_LOCKED block_height=" << block_height
<< " prevout = " << encode_base16(previous_output_.hash()) << ":"
<< previous_output_.index() << std::endl;
std::cout << "prevout.height: " << prevout.height << std::endl;
std::cout << std::endl << boost::stacktrace::stacktrace() << std::endl;
}
BITCOIN_ASSERT(block_height >= prevout.height);
const auto age_blocks = block_height - prevout.height;
return age_blocks < minimum;
}This line: prevout.height = 2147483648
@neillm I have a fast server which has 2 ssds that can store 1 active and 1 backup blockchain now. I have a script to switch between then (using sed). You can deactivate my user unless I ask. Thanks
👍 2
Monday, August 10, 2020
N
slack-term09:46
@evoskuil where does the tx metadata get populated? I look in organize_block::validate() which calls accept() but I don't see where it happens exactly.
jungly09:49
hey @narodnik - I think transaction_database class in libbitcoin-database has a
get_block_metadata method that fetches the metadata for a transaction that is in the database.It is called get_block_metadata, but it sets metadata for the tx too
Tuesday, August 11, 2020
evoskuil11:41
The idea is to isolate validation to a pure functional operation over a data set. The presumption is that the data set (chain) is stored and reproduced faithfully (presumed to be consensus behavior). Validation of an object is broken into context free and contextual (requiring population from other objects), with all validation methods on the objects themselves. The validation classes invoke validation and manage the determination of relative strength of a given branch. By objects self-validating against their own metadata, testing full validation becomes straightforward (functional).
The State class (chain state) provides historical context which is rolled forward through organizations, eliminating the need to hit the store, and provides a means to easily inject test data.
L
lucaferr12:04
Shouldn’t validation be implemented in Script so that everyone could share the same library 🙂
1 reply · evoskuil · last Aug 18, 2020
evoskuil05:00
Script execution validates only an input script against an output script and a few contextual parameters. There is a lot more to validation than that.
Monday, August 17, 2020
N
narodnik15:02
Validation is in the library, Eric is talking about the algorithm for downloading blocks fast
Tuesday, August 18, 2020
Wednesday, August 19, 2020
I would like to play with libbitcoin-web
so how do I build libbitcoin server v4
pmienk22:09
Don't believe a release has been made. But master is built with install.sh just like all libbitcoin projects.
☝️ 1
bs -v says I am on 3.6.0
pmienk22:17
if you're using master, then the version probably hasn't been updated since the last release
by 'wss stuff' do you mean neil's work on a web portal or endpoint?
web sockets
Thursday, August 20, 2020
X
xraid01:27
In all libbitcoin-server repositories there is a NEWS file with the content of "See https://libbitcoin.org"
and that server domain is not serving . as we know ...
merry joy
Does anybody here have a example bs.cfg file with the section
[websockets]
handy ... thanks
need NEWS
where is it
where is the milestones
it's where the mbedtls dependency enters.
The file in data/bs.cfg has the proper configuration in it
Not really caught up, but hope that helps 😉
Install example:
$ ./install.sh --disable-shared --build-boost --build-zmq --prefix=/home/me/myprefixneillm01:58
Some additional/unofficial websocket related notes/examples: https://github.com/thecodefactory/libbitcoin-web
X
xraid01:58
I already have boost and zmq installed in system via brew OSX . since build boost version linked don't build on OSX ?
(edited)yes @neilim . I have that on disk here
neillm02:00
If you don't want to build the supported dependency versions, you can leave those options out, but if you encounter errors, you're on your own since those are untested and not known to be compatible versions.
(edited)I think I seen somewhere it mentioned that OSX needed a earlier version of boost
anyhow the brew one builds the server so that's JOY
what's the --disable-shared and why is --shared a default build
neillm02:04
This OS X build timed out, but it looks like boost built ok: https://travis-ci.org/github/libbitcoin/libbitcoin-server/jobs/669466920
travis-ci.org
Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you'll be testing your code in minutes.
X
xraid02:17
@neilim in libbitcoin-web there is mention of [websocket] section in bs.cfg and my github=>libbitcoin-server/data/bs.cfg do not have that section ? but Your linked bs.cfg has it
Use this version of
(edited)install.sh: https://github.com/libbitcoin/libbitcoin-server/blob/master/install.sh (not the one from the version3 branch)^ The readme recommends that you
wget the version3 install script because version3 is stable/productionX
xraid02:43
Building Boost.Build engine with toolset darwin...
Failed to build Boost.Build build engine
Consult 'bootstrap.log' for more details
make.c:132:13: error: implicit declaration of function 'out_printf' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
out_printf( "...found %d target%s...\n", counts->targets,
seems invalid in C99
(edited)X
xraid04:19
libbitcoin-server master install.sh Darwin wants c11, and its not one change but all packages install wants c11 .
jungly04:44
In my day job, I have been playing around with docker a bit. So, I was thinking of writing a Docker script for libbitcoin to help first time users get off the starting blocks faster.
Will that be something useful?
👍 1
X
xraid08:40
@evoskuil yes now building master don't work from install.sh v3.6 did build from a install.sh
and as being new to libbitcoin I am trying to build master
that's all
then I report my OS and roadblocks for doing so
thanks
ohhhh and I report broken links in docs as well
@jungly maybe before making a docker build . clean up stale URL links in codebase
29 replies · jungly, xraid, evoskuil, neillm · last Aug 21, 2020
jungly09:20
I saw you reported a link broken to libbitcoin.org. Any others please drop them in here. Thanks!
X
xraid09:30
hi, it seems there is a lot of mention to libbitcoin.org that do not resolve . a duplicate content libbitcoin.info do resolve ...
jungly10:17
OK. So it is a find and replace fix. Thanks for pointing this out.
@evoskuil should we change these over to the new domain? Happy to do a quick PR.
evoskuil16:10
Yeah, it’s been a couple years. There was an attempt to get .org over to me but that got stalled.
X
xraid01:40
@evoskuil I can try contact owner of domain and offer 10$ for transfer to You ...or I will say ICANN UDRP will help in transfer it to You ...
(edited)neillm01:59
I'd say for now, @jungly see if it's easily accomplished via build, otherwise the find/replace as mentioned. It's been years, and mostly down during that period.
👍🏾 1
X
xraid02:08
it should be possible ask owner for dns access and so it can be managed by the project or amir need change admin
evoskuil02:10
A transfer to me was attempted twice, just doesn’t get completed. It’s not a priority of mine. He owns it and it’s up to him what to do with it.
X
xraid02:14
it diffuses the brand is all . kind of obfuscating . maybe good to be not so accessible for all . kind of camoflaged 🙂
neillm02:16
A stable site would be great, even if a bit out of date. A place to point to development at least.
X
xraid02:21
For these reasons Bitcoin is used herein as a label for Cryptodynamic Principles. Implementations are referred to by their brands, such as "Bitcoin Core" or "Libbitcoin";
bitcoin.org
neillm02:21
I don't know what that means really. We're not selling anything -- we're offering bitcoin, as implemented, which to me is bitcoin. If that refers to a brand, I'm ok with that.
X
xraid02:56
ahhh ok a brand do not have to be commercial but it can be used for carrying specific characteristics in a labeled product as in this case an implementation
(edited)X
xraid11:42
when building libbitcoin-system from git using install.sh it build but fails when exec libbitcoin-system-test_runner.sh that references file ./test/libbitcoin-system-test that do not exist ?
(edited)CXXLD test/libbitcoin-system-test /Library/Developer/CommandLineTools/usr/bin/make check-TESTS PASS: libbitcoin-system-test_runner.sh ============================================================================ Testsuite summary for libbitcoin-system 4.0.0 ============================================================================ # TOTAL: 1 # PASS: 1 # SKIP: 0 # XFAIL: 0 # FAIL: 0 # XPASS: 0 # ERROR: 0 ============================================================================ Running 2310 test cases... Platform: Mac OS Compiler: Clang version 11.0.3 (clang-1103.0.32.62) STL : libc++ version 9000 Boost : 1.73.0 real 15m30.648s user 42m43.729s sys 3m54.655s
Friday, August 21, 2020
This message was deleted.08:37
9 replies · jungly, xraid · last Aug 21, 2020
jungly08:57
You’ll need to install full xcode it seems :)
Also, since you are using brew. Which version of boost is brew installing for you? Please make sure that it matches the version required by install.sh, i.e. 1.62.0
https://github.com/libbitcoin/libbitcoin-system/blob/master/install.sh#L83
jungly09:11
You might run into problems down the road. Try to get the same version as specified in install.sh to avoid surprises later
jungly09:14
I think installing xcode will fix you compiler issue too. On my mac,
clang++ -v says
Apple LLVM version 10.0.0 (clang-1000.11.45.5) Target: x86_64-apple-darwin17.7.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin(edited)
X
xraid09:16
bash-3.2$ clang++ -v Apple clang version 12.0.0 (clang-1200.0.31.1) Target: x86_64-apple-darwin19.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Wednesday, August 26, 2020
F
Federico21:09
Hello there, I'd like to ask for a clarification for a project I'm working on which make use of libbitcoin-system v3.
I need to sign a coinjoin tx, which can spend utxos of types p2pkh, p2sh-p2wpkh, bech32(native-p2wpkh).
As per my understanding, if I don't want to use the SIGHASH_ANYONECANPAY every input has to be signed "togheter with his siblings".
To do this I need to create a input::list and then call tx.set_inputs(that_list) and then sign them one by one using script::create_endorsement. I saw that what create_endorsement do in the generate_signature_hash step is basically creating a copy of the tx stripping away the scriptSig from the others inputs and replacing the scriptSig of the input that has to be signed with the prev_out.scriptPubKey, which is perfectly fine.
Now that I have the endorsement... how can I safely set the "real" input (the one in the tx.inputs_) scriptSig? The transaction header file says that using a reference to the tx.inputs_ is deprecated and unsafe (https://github.com/libbitcoin/libbitcoin-system/blob/version3/include/bitcoin/bitcoin/chain/transaction.hpp line 130).
My question is: how can I sign multiple inputs with SIGHASH_ALL (without making use of SIGHASH_ANYONECANPAY) using safe endpoints only? I feel like I'm missing something, but can't guess what.
Sorry for my terrible english and maybe a bit confused explanation.
evoskuil21:13
The reason that it is unsafe is that it bypasses cache invalidation.
And of course you can copy from the tx, modify, and move back to the tx. Ideally this is a single copy.
Or you maintain an external list, modify it, and copy to the tx. Same efficiency.
F
Federico21:24
so for let's say 3 inputs I have to:
1) tx.set_inputs(_3inputs_)
2) create_endorsement(_index_, ...)
3) copy back tx.inputs()
4) update the copy for the input at index _index_
5) _index_++ goto: 1)
Right?
F
Federico21:27
Perfect thank you very much 😃 can I ask you if it does make difference when having witness data? As per my understanding witness data should not change nothing, but I'm not completely sure about that
So any state change should be done through setters. That’s why the const refs are okay.
To be clear, this doesn’t apply to metadata, but that’s not considered public state.
thank you again, good night!!