Bitcoin full node and query server
August 2018
Archive of the libbitcoin Slack #general channel.
Wednesday, August 1, 2018
Fil10:07
@evoskuil is the feature issue written down somewhere? I'm happy to add that feature because it would be useful for me atm.
neillm12:05
@Fil This looks to be related: https://github.com/libbitcoin/libbitcoin-explorer/issues/291
(edited)
vdeurzen13:04
I'm trying to (manually, i.e. not through install.sh) build libbitcoin but I'm getting a linking error
src/.libs/libbitcoin.so: undefined reference to `secp256k1_ec_pubkey_parse'
looking at the scripts for building and the documentation I'm not quite sure what's going wrong, any ideas?
(edited)neillm13:17
Hrm, something environmental appears incorrect. Are you installing each repo manually to the same (non-standard) dir, and did you end up setting the PKG_CONFIG_PATH to that dir/lib?
on a related note, any plans on making these libs available as conan packages?
neillm13:21
Interesting, I'm not familiar with that, but looks like it could be useful. I can look at in 'free time', but if you're experienced with it, it may make more sense for you to take a look for us?
vdeurzen13:21
I'd be making a package anyway if we decide to move forward with libbitcoin (looking good so far, quite impressed!)
Saturday, August 4, 2018
M
Mlmikael04:26
A question not relating to LibBitcoin but Bitcoin generally, while of course the bloom filters was a bad idea, at least it's nice to be able to understand why it was written as it was. :))
Can you please have a look at https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki#Partial_Merkle_branch_format with me :))
So the idea here is, you're a thin client and you tell the bloom filter-supporting node your bloom filter, and in response instead of feeding you full blocks, it'll send you block extracts that are filtered for matching transactions only.
AND, here comes the first SPV proofs (tx merkle proof against block's merkle root) definition, documented in a BIP.
It's in the "Constructing a partial merkle tree object" and "Parsing a partial merkle tree object" subsections.
What they do is that they traverse the whole merkle tree - as in merkle nodes, including but not limited to leaves - if I understand it right
and then, they devise a "flag bit list"
and add one bit per merkle node to it??? 😮
i would understand that there's a bitmap of which transactions were matched or not, but this does not seem to be it.
on first sight this design appears a bit redundant.
here's a merkle tree:
if i want to produce a merkle proof for a tx in a block against its merkle root, aka an "SPV proof" for a single tx,
say tx H -
then all I need is the SHA256 hashes, first of H itself of course - the recipient can produce that because he has H's full data already - and then he needs the hashes G, EF, ABCD, and IJKLMOP.
meaning, an equal number of hashes as the merkle tree has layers below the root.
do you agree that this is all you need to produce an SPV proof?
M
Mlmikael18:01
Also one more question here, for its merkle node calculations, why does Bitcoin use the child hashes in reverse order when calculating a merkle node hash? (pointed out at https://medium.com/coinmonks/how-to-manually-verify-the-merkle-root-of-a-bitcoin-block-command-line-7881397d4db1)
also are you aware of any particular rationale for using double SHA256:s instead of one
Medium
Let’s do it again! (see How to manually verify block hash) for a similar article
Tuesday, August 7, 2018
neillm13:24
@fpelliccioni Looks like standard allocated mem lifetime management. Is it causing an issue?
Wednesday, August 8, 2018
N
narodnik06:54
@fpelliccioni because the subscriber needs to access a reference to itself using shared_from_this() and there are other classes which need the subscriber to be a shared_ptr (because it is instantiated after the constructor or the reference is shared among multiple classes).
2 replies · fpelliccioni · last Aug 09, 2018
fpelliccioni01:39
template <typename... Args>
void subscriber<Args...>::relay(Args... args)
{
// This enqueues work while maintaining order.
dispatch_.ordered(&subscriber<Args...>::do_invoke,
this->shared_from_this(), args...);
}
neillm18:57
FYI, mainnet2/testnet2 has been updated with the latest version3 server code, including improved tx/block subscription. If you use (bx) subscribe-tx or the python subscribe-tx.py script, you can see a huge difference in what's coming out of mainnet1/mainnet3
👍 2
Thursday, August 9, 2018
M
Mlmikael01:11
where is OP_CHECKSIG's implementation?
i'd like to understand what signed data OP_CHECKSIG checks is being signed. the tx outputs should be included there, right (as well as a hash constant-stripped version of the current transaction input)?
libbitcoin-consensus/src/clone/script/interpreter.cpp: case OP_CHECKSIG: hm
yey.
in non-segwit transactions, data stripping is done as follows: scriptCode.FindAndDelete(CScript(vchSig));
I got the impression the stripping would be to overwrite with zeroes rather than actually remove bytes, but i guess i didn't catch that right
You may find it a little easier to follow than libbitcoin-consensus (bitcoind).
(edited)@evoskuil you mean that Bitcoin Core's code for OP_CHECKSIG and other opcodes is difficult to read or even convoluted?
Thank you guys for LibBitcoin, this is just phenomenal.
A lot of this stuff is technically undocumented still, so LibBitcoin's source even serves as an essential manual today
M
Mlmikael02:02
Just a code reading question -
This is the code that generates the signtext that OP_CHECKSIG checks the P2PKH signature against:
https://github.com/libbitcoin/libbitcoin-consensus/blob/master/src/clone/script/interpreter.cpp#L1120
For each of the now-being-checked transaction's transaction outputs, the code here runs "::Serialize(s, txTo.vout[nOutput]);".
Where is the "Serialize" logics for the transaction output (CTxOut)?
Or just grep
@evoskuil wait, what's the GitHub URL to Bitcoin Core's 'original' copy of the same?
As long as you realize this isn't related to libbitcoin 😉
Guys, actually P2PKH lock scripts could be the "OP_CHECKSIG" operation only, and the reason for the "OP_DUP OP_HASH160 <pubKeyHash> OP_EQUALVERIFY" warup is actually a DDOS-protection measure, right -
the OP_CHECKSIG is a really expensive operation so we're trying to save the network from the headache of needing to validate signatures provided by people who can't prove ownership of the public key, right?
*warmup
M
Mlmikael03:22
Where in LibBitcoin's code are the constraints on what scripts are allowed, implemented? E.g., gossiped transaction's lock scripts must be either of P2PK, P2PKH, multisig, nulldata or P2SH in order to be considered valid and be gossiped?
vdeurzen13:32
does libbitcoin have any support for regnet? I.e. if I'd want to experiment with a tool built on top of libbitcoin locally, how would I do that?
neillm13:34
Regtest is supported, although admittedly, I haven't used it yet myself (been meaning to for a while). Be sure to update your config accordingly (search for regtest). If you run into issue, let me know, probably about time I also look at it 😉
vdeurzen13:37
I'm setting up a service for hd-wallet management and wante to test that the addresses I'm generating work 🙂 hence the question
neillm13:39
I don't know your exact setup, but hd generation can be done without a server using libbitcoin (programmatically) or libbitcoin-explorer (bx) on the command line.
Oh, I see, you want to transact with them to be sure they work. Any particular derivation you're using?
Alternatively, you could use the testnet community servers and do generation/testing using testnet.
vdeurzen13:52
I'm effectively following bip44, I want to ensure that the service generates usable addresses and indeed I can create transactions with them
preferably locally to get results as soon as possible 🙂
testnet is my next step
skaht13:53
Libbitcoin bx HD functinality can faithfully and reliably mimic the offline key synthesis and address synthesis processes deployed by Trezor, Ledger, Keepkey, Jaxx, Coinomi and many other wallet implementations. However, there are a number of bx commands dealing with transaction construction and broadcasting that require bitcoin-server connectivity. HD bx brain wallets are also very easy to implement.
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64
02e10cc063a5fb88779510cae84b0eec4e6daa10503b374c60f7258b20d3a27f
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new
actual anchor copy shrug garden service tattoo pear slab drastic senior sudden traffic steak donor alter dance glide differ enrich rare hawk measure walnut
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new | bx mnemonic-to-seed -p "sucks" | bx hd-new | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-private -d -i 0 | bx hd-private -i 0 | bx hd-private -i 0 | bx hd-to-ec | sed 's/$/01/' | bx base58check-encode -v 128
KwcmrcW31rBBL8iYxZfZVaKELpMTsD3fA3bFteSRpqkJKTceTzrz
skaht14:02
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new | bx mnemonic-to-seed -p "sucks" | bx hd-new | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-private -d -i 0 | bx hd-private -i 0 | bx hd-private -i 0 | bx hd-to-ec | bx ec-to-public | bx ec-to-address -v 0
1Cs8vGgxaKpYWwSiubW5QziU5EAH8akTV3
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new | bx mnemonic-to-seed -p "sucks" | bx hd-new | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-private -d -i 0 | bx hd-private -i 0 | bx hd-private -i 0 | bx hd-to-ec | bx ec-to-public -u | bx ec-to-address -v 0
19C9xjgKa5K1X8FjNi5QdojEf4ra9ueyGd
Look at this example very carefully:
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new | bx mnemonic-to-seed -p "sucks" | bx hd-new | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-public -d -i 0 | bx hd-public -i 0 | bx hd-public -i 0 | bx hd-to-ec | bx ec-to-address -v 0
1Cs8vGgxaKpYWwSiubW5QziU5EAH8akTV3
vdeurzen14:25
does the regtest mode use different address encoding / prefixes or does it use testnet prefix?
skaht14:27
About to answer your question, bur let me finish this point…
Here is how bx can synthesize BIP 49 segwit style Bitcoin private key and the associated addresses (BIP 49 P2WPKH-nested-in-P2SH):
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new | bx mnemonic-to-seed -p "sucks" | bx hd-new | bx hd-private -d -i 49 | bx hd-private -d -i 0 | bx hd-private -i 0 | bx hd-private -i 0 | bx hd-to-ec | sed 's/$/01/' | bx base58check-encode -v 128
KzTmMJpH9hK9UVzWcnf4zuxnPDwfqtia8MamFpV8xFwg6SQ3QNX6
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new | bx mnemonic-to-seed -p "sucks" | bx hd-new | bx hd-private -d -i 49 | bx hd-private -d -i 0 | bx hd-public -d -i 0 | bx hd-public -i 0 | bx hd-public -i 0 | bx hd-to-ec | bx bitcoin160
2228fd761aaf4da7471d3f08aeb33bf05d3ec2b5
% echo "0 [2228fd761aaf4da7471d3f08aeb33bf05d3ec2b5]" | bx script-to-address -v 5
3M6Wkk3Edxreat9FyJR5XNMgzckJnC3Bb9
skaht14:31
When specifying version numbers for WIF keys and Addresses for different coin types, get the information from the table at https://github.com/libbitcoin/libbitcoin/wiki/Altcoin-Version-Mappings#bip44-altcoin-version-mapping-table. Look at the examples starting at https://github.com/libbitcoin/libbitcoin/wiki/Altcoin-Version-Mappings#8-dash-dash-bip-3944-technology-examples.
(edited)neillm14:32
@vdeurzen yes, they are definitely testnet addresses: https://github.com/bitcoin/bitcoin/blob/master/src/chainparams.cpp#L344
(edited)@skaht oh yes, definitely a swiss army knife like tool, awesome unix style usage too 🙂
skaht15:02
Computing m/44'/0'/0'/0/0 and M/44'/0'/0'/0/0 for Testnet:
Private Compressed Key in WIF encoded:
% echo “This is a really bad low entropy brain wallet” | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new | bx mnemonic-to-seed -p “sucks” | bx hd-new | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-private -d -i 0 | bx hd-private -i 0 | bx hd-private -i 0 | bx hd-to-ec | sed ‘s/$/01/’ | bx base58check-encode -v 239
cMymKXVtSusSVaBpLyUgrtpHy3esXf9ME5jj14twKxQJaCga9HLv
echo “This is a really bad low entropy brain wallet” | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new | bx mnemonic-to-seed -p “sucks” | bx hd-new | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-private -d -i 0 | bx hd-private -i 0 | bx hd-private -i 0 | bx hd-to-ec | bx ec-to-public | bx ec-to-address -v 111
miP6BawYTMZBntPf1BWCs8ANtUZp17BnxL@vdeurzen - One final example set:
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-32 | bx mnemonic-new
pulp obtain diary that ramp ring share anchor visual silk wrestle sugar
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-48 | bx mnemonic-new
pulp obtain diary that ramp ring share anchor visual silk wrestle submit sword host ramp gadget entry roast
% echo "This is a really bad low entropy brain wallet" | bx base16-encode | bx sha256 | cut -c 1-64 | bx mnemonic-new
pulp obtain diary that ramp ring share anchor visual silk wrestle submit sword host ramp gadget entry sauce bind entire shell total obtain gentle
Friday, August 10, 2018
M
Mlmikael03:22
Question, where in LibBitcoin are the P2SH-specific logics trigged?
If I got it right, it should look something like this:
if (utxo's scriptPubKey is in format "OP_HASH160 <scriptHash> OP_EQUAL") {
evaluaste scriptSig;
var redeemScript = pull topmost stack element;
return evaluateBitcoin(redeemScript);
}
vdeurzen11:12
I'm trying to get bitcoind (bitcoin-core client) to sign a transaction with a key I generated using libbitcoin, but I'm getting 'Invalid private key' messages 😕 I'm using
wallet::ec_private(key.secret(), network, true).encoded(); to get the encoded private key, the key variable is a hd_private object. Any ideas what am I doing wrong?neillm12:11
You'd have to send an example of what you're doing. I've never worked with HD keys in core, and am guessing there's a mismatch between what you're getting from libbitcoin and what you're trying to send to core.
vdeurzen12:16
I seem to have at least found 1 issue, namely I was using the wrong constant for the test net
I'll dig around a bit more to see if I can figure it out otherwise I'll surely return with more questions 😜
👍 1
Saturday, August 11, 2018
skaht02:11
@vdeurzen - Bitcoin-core will not care how the keys were generated. Just be sure you are using the final elliptic curve keys and not the interim extended HD keys.
(edited)@vdeurzen - The bx command line interface can be used to construct, sign, and initially transmit signed transactions. However, bx requires interactions with bs to accomplish such tasking. The bx interface does not care if how the private keys, public keys, and addresses were generated.
Its been a few years, but I was able to use bx to successfully construct and transmits these types of transactions: P2PKH, P2SH, and OP_RETURN associated with sending funds. Was also able to successfully redeem funds sent to P2PKH, and P2SH addresses. I suggest understanding how this is done prior to digging into C++ code to roll your own apps.
Sunday, August 12, 2018
evoskuil03:56
Is anyone familiar with the electrum style of payment indexing? Specifically I’m seeking the key format, which I understand is some hash of the output script.
Monday, August 13, 2018
M
Mlmikael04:31
oh, https://bitcoincore.org/en/2016/01/26/segwit-benefits/ writes " Bitcoin Core does not check signatures for transactions prior to the most recent checkpoint by default".
is this the case with libbitcoin too?
if so maybe one would very occasionally run a libbitcoin node without checkpoints just to do the absolutely full work that way.
M
Mlmikael06:11
Can I make a segwit transaction that has an output that does NOT match the P2WPKH pkscript pattern (which is 0x0014{20-byte-key-hash}) and also does NOT match the P2WSH pkscript pattern (which is 0x0020{32-byte-hash}), BUT still contains witness data - would a transaction with such an output be relayed?
(in other words, a "null-data segwit output")
vdeurzen07:50
@skaht Thanks for the tips. I am indeed trying to get things running 'by hand' before coding it up in C++. As it turned out I was using the wrong version prefix for the WIF formatted key -.-' I might consider writing up a simple tutorial or blog post detailing my findings 🙂
skaht17:10
@vdeurzen - For WIF keys for a given coin type, use the third column of https://github.com/libbitcoin/libbitcoin/wiki/Altcoin-Version-Mappings#bip44-altcoin-version-mapping-to specify the version number for the associated base58check encoding.
As Neillm indicated the values populating this table have traceability to source code. The last column that is hidden provides such traceability for the various columns of information, albeit, the line numbers referenced are deprecated since all such source code is a moving target. Also, much of the content of the table referenced here is approaching 3 years of age.
skaht17:16
@vdeurzen - When you start to dig into the source code see:
1.
http://aaronjaramillo.org/libbitcoin-working-with-altcoins
2. https://github.com/jachiang/LibbitcoinDocumentation/blob/master/AddressesWallets/AddressesWallets.md1 reply · James C. · last Aug 14, 2018
James C.06:01
Hi @skaht, please refer to https://github.com/libbitcoin/libbitcoin/wiki/Addresses-and-HD-Wallets , I haven't kept up that repo, I now apply documentation updates directly in the libbitcoin wiki page(s).
(edited)Tuesday, August 14, 2018
vdeurzen08:31
Wow, this is awesome! I managed to figure out where my main issue was. I was using the wrong version constants (I was under the impression there was 1, but there are several in different, but logical, places). I'm now testing my addresses / HD wallet by trying to get some funds transferred through the bitcoin core client. When I have that working I'll be doing the same with a simple libbitcoin based set-up 🙂
thank you very much for these pointers! @skaht
P
Phil11:22
According to some guy in the irc the signature size can vary between 68-71 bytes in btc.
But the Libbitcoin type signature is a 64-byte according to https://github.com/libbitcoin/libbitcoin/wiki/ECDSA-and-DER-Signatures
How can that happen? Am I missing something?
James C.11:51
Hi @Phil: The 64-byte bc::ec_signature type represents R, S values, each as 32 bytes. When such a signature is DER encoded, the leading null bytes are omitted, so that the DER encoded R&S values may vary in length.
-Consider
(edited)bc::der_signature in Libbitcoin
-DER encoding in BIP66: https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki.In Particular:
// * R: arbitrary-length big-endian encoded R value. It must use the shortest
// possible encoding for a positive integers (which means no null bytes at
// the start, except a single one when the next byte has its highest bit set).
// * S: arbitrary-length big-endian encoded S value. The same rules apply.P
Phil11:54
@James C. I see it can vary. But I don’t see how can Libbitcoin use a fixed 64 amount of bytes to the signature, once the DER encoded may vary.
James C.11:59
The Libbitcoin class
(edited)bc::der_signature is a variable length data_chunk (https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin/math/elliptic_curve.hpp#L55)Fil15:36
Guys, I was wondering if I find mistakes in the dev docs on the wiki, am I supposed to fix them or report them? I just don't know the workflow and don't want to interfere with existing conventions that's why I'm asking.
Thursday, August 16, 2018
M
Mlmikael04:19
i think there is an index of standardized altcoin names, and maybe even each one of them has its own integer descriptor.. where is it again
where is there an authoritative source confirming that Bitcoin signatures are max 72 bytes indeed?
https://bitcoin.stackexchange.com/questions/12554/why-the-signature-is-always-65-13232-bytes-long discusses this but only says "71 on average".
Bitcoin Stack Exchange
A signature created by the Bitcoin-Qt client can always be decoded (base64) to a 65-byte array. This array seems (according to https://github.com/bitcoin/bitcoin/blob/master/src/key.cpp#L217) to co...
M
Mlmikael05:30
also, the P2PKH pubkeys in the 65 byte (uncompressed) form, those will stick around as long as P2PKH is used, or?
neillm13:30
@Mlmikael I think what you're looking for is the max length of a DER encoded ECDSA signature, as used in bitcoin. I could be wrong, but don't think that's formally described anywhere (although there definitely are formal definitions of "DER encoding" and "ECDSA signatures", outside of the bitcoin sphere). If bitcoin made changes to how the signatures could be DER encoded, the length could vary more, but while it doesn't, there's a cap. In other words, if you wanted to be 'wasteful' and provide a buffer far larger than this max value, it shouldn't hurt anything, so long as you respect the actual encoding lengths of the format when working with the data. There are numerous posts about the length as used in bitcoin, such as the one you provided as well as this one: https://bitcoin.stackexchange.com/questions/77191/what-is-the-maximum-size-of-a-der-encoded-ecdsa-signature
Bitcoin Stack Exchange
I've been told by a seasoned Bitcoin contributor that signatures in Bitcoin could be up to 75 bytes. I'm curious to find out how that maximum comes to pass. According to this answer on Why the sig...
evoskuil15:41
No. BIP62 was withdrawn: https://github.com/bitcoin/bips/blob/master/bip-0062.mediawiki#Low_S_values_in_signatures
(edited)BIP62 is referenced by BIP141 but this limit was not applied to segwit scripts. https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#motivation
(edited)Friday, August 17, 2018
Saturday, August 18, 2018
phitsc22:04
Hello Everyone
I am conducting research on the usability of Bitcoin APIs from a developer's point of view. I am intending to analyse the API usability of Bitcoin libraries with respect to usability heuristics and guidelines published in the academic literature. The goal of my research is to provide a static code analysis tool allowing to detect usability violations in applications using libraries such as libbitcoin.
Is anyone aware of any open source projects that rely on libbitcoin, besides the one's available from https://github.com/libbitcoin itself?
Thanks, Philipp
👍 1
1 reply · vdeurzen · last Aug 20, 2018
vdeurzen13:02
You could have a look at bitprim, a fork of libbitcoin, they list some projects using their fork on their webpage: http://bitprim.org
Monday, August 20, 2018
James C.10:07
I have added some documentation examples to...
https://github.com/libbitcoin/libbitcoin-server/wiki/Heartbeat-Service
https://github.com/libbitcoin/libbitcoin-server/wiki/Transaction-Service
https://github.com/libbitcoin/libbitcoin-server/wiki/Block-Service
Thx to @neillm for looking into dropped subscription messages internally: https://github.com/libbitcoin/libbitcoin-server/pull/469#issuecomment-411203064
(edited)👍 2
James C.15:35
Quick question on regtest testing for BS: Is the best way to obtain regtest blocks by generating these on bitcoind
(edited)bitcoin-cli -regtest generate, and then have them propagate to the peer-connected Libbitcoin Server? https://github.com/libbitcoin/libbitcoin-server/issues/404 Thanks!neillm22:39
@James C. I haven't tried this myself, but yes, that is the approach that I would take anyway.
👍 1
Tuesday, August 21, 2018
M
Mlmikael04:33
just for absolute clarity, "static BC_CONSTEXPR size_t max_der_signature_size = 72;" says https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin/math/elliptic_curve.hpp#L54 .
this is EXCLUDING the sighhash byte, meaning max DER signature size is 73 bytes, right?
i had this brief conversation with Sipa:
"the R and S value are 256-bit values, which are at most 33 bytes when encoded in DER
then there is 6 bytes of DER headers
and bitcoin adds a 1 byte sighash to the end
so together 33+33+6+1 = 73 max
however, since the introduction of the low-s rule (which is just policy, not a consensus rule), S values are at most 255 bits, which encodes to at most 32 bytes, reducing the maximum to 72"
(I asked the clarifying question "aha, so when low-s is not applied, 33B R + 33B S + 6B headers + 1B sighash = 73 bytes" and he answered "right".)
..and then applied to P2PKH and P2KH sigscripts: "the way this max of 72/73B applies to P2PKH and P2PK sigscripts is... P2PK is only <signature>, since this is <= 75 only one byte for PUSHDATA is needed, and then follows up to **73** bytes of signature data, meaning P2PK may take **74** bytes at most? and, P2PKH is <signature> <pubkey>, pubkey is normally 33B or for legacy uncompressed 65B, both push-datas are single-byte due to <=75B, and the signature may be **73B** so 1 + 73 + 1 + 33/65 = 108 or 140B sigscript max for P2PKH?"
.."are the calculations I made about max size sigscript for P2PKH and P2PK (108/140B and 74B) correct?"
and he said he thinks they're right.
would you mind confirming this one?
evoskuil22:17
Max DER signature size excludes the signature hash type byte. This byte has nothing to do with DER encoding.
(edited)Wednesday, August 22, 2018
evoskuil02:30
The computation of script max sizes is straightforward once you pin down endorsement max (DER max + sighash byte = 73), which is the bc constant max_endorsement_size.
M
Mlmikael02:57
@evoskuil aha. where are these two structures used respectively, that is, where in Bitcoin do you have "contract endorsements" and where do you have DER signatures without endorsements?
E.g. sigscripts of P2PKH, P2PK, etc.
M
Mlmikael04:18
Oh, i just understood the Bitcoin network level implications of the merkle tree CVE now (https://github.com/bitcoin/bitcoin/blob/master/src/consensus/merkle.cpp#L32):
The issue is that if miner M mines a block and sends it to fully synced Bitcoin node N, and on its way a third party would modify the block's data, in such a way that N when it receives the block, he gets a repeat of some transactions, he would consider the block invalid obviously - because there's an attempt to double-spend the inputs in the repeated transactions, right -
then, if he identifies a block by its block hash only, and puts the hash on some kind of "known to be invalid" blocks list, then indeed he would fork off !
how has this been mitigated?
1 reply · James C. · last Aug 22, 2018
James C.17:06
Interesting, this part of Bitcoin history was new to me. I found this referring to Gavin's patch: https://bitcointalk.org/?topic=102395 https://github.com/bitcoin/bitcoin/commit/be8651dde7b59e50e8c443da71c706667803d06d
(edited)evoskuil04:26
Consensus code validates signatures. The EC functions implement signing and validation.
M
Mlmikael17:28
One question, when you produce the human-readable hex string form of a Bitcoin block hash or a Bitcoin transaction hash, then you produce the string in **reverse byte order**.
The original motivation for this regarded the block hash and was didactic: you want the leading zeroes to be printed first.
Where in LibBitcoin's code are these reverse-order output logics located?
evoskuil17:34
We never reverse byte order internally. It’s cosmetic formatting for human readability. It’s only applied in textual serializations, which you may only see in config (settings, command line, and supporting classes).
and where is LibBitcoin producing and parsing such textual serialization?
notably this applies for block hashes and transaction hashes right?
bc::config, bc::wallet
Similar in concept to textual script in/out.
Maybe work backwards from bx commands.
(I guess LibBitcoin implements the deserialization in only one place at least for the same kind of value e.g. block hash, looking next hrs.)
(Bitcoin Core does this in "uint256's ToString() method", will look closer at that later also.)
evoskuil18:36
The above is the only place where the conversions take place. This is invoked in other places in the code. Generally speaking encode_hash() is used independently mostly for logging. Otherwise config classes wrap tx/block hash serialization:
In complex bx text output these are serialized via property trees:
But the only place in libbitcoin (system) that encode_hash/decode_hash are used is config::hash256 (above) and config::checkpoint: https://github.com/libbitcoin/libbitcoin/blob/master/src/config/checkpoint.cpp#L135-L154
(edited)Since checkpoint has a custom single-token serialization format for the combination of block hash and height.
S
Steve Barbour21:44
Eric, didn't want to spam you on twitter. I'm curious to learn about your thoughts on block size, have you written about it anywhere? Was reading this wiki article and it came to mind in regards to how it relates minimum utility threshold for value transfer. Wondering what you think of bigger blocks, when/why the size should be changed, how it relates to the relevant principals described in the Wiki.
https://github.com/libbitcoin/libbitcoin/wiki/Utility-Threshold-Property
(edited)evoskuil22:42
Hi Steve. There is an inverse relationship between block size and utility threshold. However there is also an inverse relationship between block size and miner and merchant anonymity. Block size increases that are big enough to have a material impact on utility threshold seem likely to have a materially adverse impact on security as well. The block size has been "changed" (i.e. in the context of new coins). I try to avoid telling people what they should do 🙂.
👍 1
evoskuil23:28
It's also worth pointing out that the reduced security of non-anonymity is not necessarily offset by higher aggregate fees.
👍 1
Thursday, August 23, 2018
M
Mlmikael02:36
@evoskuil thank you for pointing out. so the reverse order representation/deserialization only regards block hasehs and transaction hashes right?
ah, Bitcoin Core's hd wallet serialization uses reverse order too: https://github.com/bitcoin/bitcoin/blob/master/src/wallet/rpcwallet.cpp#L3059
ah, and Core prints the merkle root in reverse order.
M
Mlmikael02:50
Maxwell clarifies,
"the point is that bitcoin's UI uses the same stuff to print all these 256 bit hashes.
block hash is interpreted as a 256-bit little endian number for target comparison as luke says, so then bitcoin needed to print it that way too, otherwise the 0s would have been on the right which would have been confusing... and the same code was then used to print other hashes."
interesting.
evoskuil02:52
Presentation order is ultimately irrelevant, so we keep it out of all internals - which was not originally the case and led to bugs and greater confusion.
@evoskuil yeap i agree, great to have it out of any internals. indeed presentation order is really a corner thing.
the idea that if you sort a list of transaction hashes or block hashes, that they get sorted in their alphanumeric order, that's an interesting one (even if not worth to fight for, er).
M
Mlmikael02:54
@evoskuil does LibBitcoin ever sort a list of hashe seg block hash or transaction hash?
if anyone would, does LibBitcoin implement a comparator? i guess gmaxwell's thought is that the hashes are logical uint256:es so by sorting them in "integer order", you would get an alphanumeric list
M
Mlmikael02:55
so like, HEX STRING -----(funny reverse order deserialization)----> uint256 structure which is actually a 32B byte vector ------(sort function, compares the uint256:es by normal < > logic) ----> sorted list ----- (serialize with funny reverse order again) ----> alphanumeric order sorted list, lol.
however that only makes sense if the < > operator is little endian? 🙂
ah, Bitcoin Core does this:
friend inline bool operator<(const base_blob& a, const base_blob& b) { return a.Compare(b) < 0; }
inline int Compare(const base_blob& other) const { return memcmp(data, other.data, sizeof(data)); }
evoskuil02:57
Hashes have no meaningful comparison apart from equality. PoW converts a hash to a number for an inequality relation (less than).
(edited)I believe bitcoind manipulates bitcoin hashes as numbers, which is a bit strange, since the values are arbitrary except in one pow function.
That function reinterprets an arbitrary byte array as a number (in that it applies a less than relation to it).
(edited)Hashes have no meaningful comparison apart from equality.
i agree, any other operation than equality is functionally meaningless, it would be only cosmetic
I believe bitcoind manipulates bitcoin hashes as numbers, which is a bit strange, since the values are arbitrary except in one pow function.
right, they call it "uint256", i agree this is kind of strange particularly as integer representation normally is a platform-specific thing - which it's a byte vector just like in libbitcoin
PoW converts a hash to a number for an inequality relation (less than).
wait, where is this done?
are you talking about counting zeroes in the hash to validate work here, or something else?
evoskuil03:10
There is similarly a summation of branch work in blockchain, with comparison.
But that does not convert hashes. That happens only in the one line above.
M
Mlmikael03:14
@evoskuil thanks again for clarifying. wait, hashes are such are binary/byte concepts so the whole idea of relating to them as comparable numbers, is a concept that Bitcoin adds on top of them, right?
the inventor of SHA256 never related to hashing as SHA256(byte string) => 256bit integer which may be little or big endian, right?
There is similarly a summation of branch work in blockchain, with comparison.
wait just for ref, where is that one?
interesting.
No, it’s converted to a number for work computation.
M
Mlmikael03:26
@evoskuil: the line "uint256_t target(bits);" (https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/header.cpp#L457) does what, create a 256bit integer with.. "bits" number of zeroes and the rest 1:s?
(..looking up boost uint256_t's documentation..)
(edited)M
Mlmikael03:31
aha the uint256_t is defined in Boost's multiprecision (https://www.boost.org/doc/libs/1_66_0/libs/multiprecision/doc/html/)'s cppint, https://github.com/boostorg/multiprecision/blob/develop/include/boost/multiprecision/cpp_int.hpp (y)
(edited)M
Mlmikael03:41
@evoskuil: https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/header.cpp#L451 , to get it:
(edited)const auto bits = compact(bits_);
bits_ is the difficulty target that should be matched exactly, and it's a uint32 number of bits.
this integer value is copied from the block header.
compact here makes a 256bit uint that's ... bits_ number of 1:s?
then:
uint256_t target(bits);
this just copies the "bits" uint256_t value in the variable "bits", to a new varible "target", right - so it's just a copy operation done (e.g. to clarify what's going on to c++' generics type system)
then:
return to_uint256(hash()) <= target;
here, you calculate the block hash as a 32B byte vector, and you then feed it to "to_uint256" which converts it to the LibBitcoin/Boost uint256_t big integer type, via a "little endian" logic.
and then you compare that (the big integer representation of the little-endian-decoded hash) with the "target" value, which is a binary value 1111.... , bits_ count of 1:s.
@evoskuil did i get what the code here does righT?
M
Mlmikael04:18
ah right, "bits" is in Bitcoin's internal 32bit floating point format, described here https://github.com/bitcoin/bitcoin/blob/master/src/pow.cpp#L80 .
M
Mlmikael04:55
i talked through the reverse order thing with the Bitcoin Core guys and what's going on in their code base is simply that they only have one single hex printing function in Bitcoin Core, for uint256 and uint160,
and that one prints in reverse order.
derp derp.
so anytime they add a new uint256 value to their software, and then print it, then it arrives to the world as a reverse order hex string, derp.
so last year they started printing a (256bit) hash of the UTXO set at current block height, derp!
guess what order it's presented in derp derp.
if this becomes a value that the whole ecosystem likes to share to check between nodes, maybe everyone will do it in reverse order, derp.
maxwell concludes, "don't think, just do it" (print in reverse order) lol.
vdeurzen17:36
I've gotten a simple node up and running, after wresting with a whole bunch of config values (specifically database settings for regtest, which have no defaults afais), I managed to start it and get it to connect to a local bitcoind node. However, I'm not sure it's actually connecting to the node as I'm getting these messages:
2018-08-23T17:35:14.871481 INFO [network] Connected manual channel [127.0.0.1:18444] as [127.0.0.1:18444] (1) 2018-08-23T17:35:14.872014 INFO [node] The chain is stale, send sendcmcpt low bandwidth [127.0.0.1:18444] 2018-08-23T17:35:14.936866 WARNING [network] Failure fetching new address: object does not exist 2018-08-23T17:35:14.937004 WARNING [network] Failure fetching new address: object does not exist
I've used port 18444 in my bitcoind set-up and have my own inbound port set to 18445 (and the bitcoind config using that as a peer)
Now, when I run generate with bitcoin-cli, nothing seems to happen 😕 anyone set-up something like this before?
evoskuil17:53
All settings have defaults. There is no need to change database settings when working with regtest.
(edited)"Connected manual channel [127.0.0.1:18444]" means you are connected to the node at "[127.0.0.1:18444]".
"Failure fetching new address: object does not exist" means that the node cannot get an address from the address pool because the pool is empty. This implies you have no address pool configured yet you have outgoing (non-manual) connections configured.
This is not a libbitcoin message, it appears you are running a modified node:
2018-08-23T17:35:14.872014 INFO [node] The chain is stale, send sendcmcpt low bandwidth [127.0.0.1:18444]
though, for the database settings, when using regtest, the defaults are empty if I understand this correctly: https://github.com/libbitcoin/libbitcoin-database/blob/a36dfa017464975c6342cb7005196d1a3ec24f91/src/settings.cpp#L66
(edited)evoskuil18:16
Re: database settings, yes, regtest bug, resolved here: https://github.com/libbitcoin/libbitcoin-database/pull/221
(edited)I don't know what's happening with the bitprim compact blocks sync. You might just set up a libbitcoin node or server and test the same configuration to verify.
Do this from version3, master is the staging branch.
Otherwise you should probably start with bitprim support, since they have modified the sync code, which is where you appear to be having trouble.
Friday, August 24, 2018
vdeurzen10:46
Sooooo, I've managed to get the two to connect (bitcoind and my binary) but what seems to be happening is compact block sync is continuously requesting the same block without progressing -.-
anyone experience something similar?
Z
zoranc12:27
what is the recommended machine configuration for running a libbitcoin server? SSD size, RAM, processor type... i tried running it at home on a 2TB SSD with 2-core processor 4. or 5. gen intel, 8GB RAM, and it takes weeks to sync the chain...
@zoranc I recently sync'd version3 in 5 days (but with 32GB RAM). One thing that helps speed things up is if you have an already sync'd node on the local network that you can peer with (see peer line in configuration).
neillm12:34
Yes, the peer can be a sync'd bitcoin core node. Both help, but RAM is probably the bottleneck during sync.
You could make a swap file on the SSD to help speed it along
I've done this on small mem machines with SSDs for building larger projects that require more mem and it worked pretty well. May help here to expand RAM to something more substantial since you have the space
neillm13:01
Give it a shot, 5 days for sync is already a lot (v4 will improve this substantially), but weeks for initial sync is something I've also experienced in sub-optimal configurations -- and it's painful 😉
Z
zoranc13:07
i am trying to figure a good config for v3 to run at work (made a small POC project with libbitcoin there). to properly demo it for customers i'd like to run our own server...
neillm13:11
Without knowing what the project's about, just showing fastest local sync would probably be to setup a server with multiple peers to already sync'd nodes on the same switch/subnet. If RAM can't be extended, the SSD swap will definitely help. If you mean after sync, I think the hardware you have is plenty sufficient for any other kinds of demoing (i.e. showing great concurrent query performance, etc).
vdeurzen13:13
hmm, seems it has trouble with empty transactions, the generated transactions from bitcoin-core only contain the coinbase transaction ... which might be the problem 😕
neillm13:15
@vdeurzen You mean the generated blocks from core only contain the coinbase tx? Sounds about right if you're just generating and not creating spends. Did you ever stand up the non-modified v3 libbitcoin node for this testing?
vdeurzen13:17
true, I do still need to do that, not trying to be lazy, just wanted to explore this path before switching to alternatives
is there a precompiled version of the libbitcoin node you are aware of?
neillm13:19
Unfortunately, not a recent one. The binary builds have been lagging and as far as I know contain unpatched issues.
I'll get to compiling then 😛
neillm13:19
The install.sh script takes time, but should be something you don't need to monitor once you get it going, no?
on branch
version3: /home/tim/src/experiments/libbitcoin-install//include/boost/multiprecision/cpp_int.hpp:548:56: error: right operand of shift expression '(1 << 63)' is >= than the precision of the left operand [-fpermissive]
In file included from ./include/bitcoin/bitcoin/machine/operation.hpp:159,
from ./include/bitcoin/bitcoin/chain/script.hpp:31,
from ./include/bitcoin/bitcoin/chain/output.hpp:28,
from ./include/bitcoin/bitcoin/chain/output_point.hpp:25,
from src/chain/output_point.cpp:19:trying master branch
when running the
install.sh scriptit's relative to my home dir
vdeurzen14:19
on master libbitcoin-blockchain fails to compile:
src/populate/populate_chain_state.cpp: In member function 'bool libbitcoin::blockchain::populate_chain_state::populate_all(libbitcoin::chain::chain_state::data&, const header&, size_t, bool) const':
src/populate/populate_chain_state.cpp:192:67: error: 'const class libbitcoin::settings' has no member named 'net_sample'
bitcoin_settings_.retargeting_interval, bitcoin_settings_.net_sample,arch linux
retrying build with version3 branch
evoskuil14:20
Master is in the midst of a multi-repo update, so yes. Also it’s sync is incomplete. Master is staging for next major release.
(edited)evoskuil14:22
Not sure about the boost error, looks like the warning level is too high, but that has been verified in the Travis builds.
Maybe Arch, dunno. We verify on Ubuntu.
evoskuil14:46
Results from the warning level, which has been made more restrictive. You can probably lower it somehow. It is probably Arch and/or the compiler that varies a little from our CI environment.
vdeurzen14:56
This implies the problem with compilation might be in boost: https://svn.boost.org/trac10/ticket/12181
vdeurzen15:05
my guess is, it's a bug in boost, which is only signaled by newer compilers, hence your own build is OK
evoskuil15:16
It's a boost bug that normally manifests as a warning but we use strict compiler settings that treat warnings as errors.
But warnings vary by compiler and version.
alright, we have a binary
took a while -.-
glorious .. same error with the regtest network:
15:27:06.158060 DEBUG [node] Rejected block [1c18eac728abe7ae6e435d00d0d11b6a89dea9f29de858ab0e06b3f496a98c53] from [127.0.0.1:18444] transaction inputs or outputs empty
that block only has a coinbase transaction
this leads to a loop continuously requesting the same blocks btw
BTW, I didn’t see this error in your previous posts.
vdeurzen15:33
I may have posted a description rather than the error, I've only just traced it to the actual error message
{
"amount": 50.00000000,
"confirmations": 104,
"generated": true,
"blockhash": "1c18eac728abe7ae6e435d00d0d11b6a89dea9f29de858ab0e06b3f496a98c53",
"blockindex": 0,
"blocktime": 1535124339,
"txid": "223e7ae1577df1afa3cec023d4a44b65868d341d523468e1ae47b19d461633e2",
"walletconflicts": [
],
"time": 1535124339,
"timereceived": 1535124339,
"bip125-replaceable": "no",
"details": [
{
"address": "miAewNuBf7omffAictCFbG6S9ZR1qkrMQa",
"category": "generate",
"amount": 50.00000000,
"vout": 0
}
],
"hex": "020000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff03510101ffffffff0200f2052a010000002321029f3b44528ce59338ec02a2ce98293c495aa0c691079c6e04aa897e12670c4dfaac0000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000"
}this is what bitcoin-cli tells me about the coinbase transaction
I’m on my phone atm.
well, I have to start moving at this point, so I'm going to continue this exploration another time. thank you for all the help so far!
Sure, glad to help.
evoskuil15:38
$ bx tx-decode 020000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff03510101ffffffff0200f2052a010000002321029f3b44528ce59338ec02a2ce98293c495aa0c691079c6e04aa897e12670c4dfaac0000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000
transaction
{
hash 223e7ae1577df1afa3cec023d4a44b65868d341d523468e1ae47b19d461633e2
inputs
{
input
{
address_hash c51b66bced5e4491001bd702669770dccf440982
previous_output
{
hash 0000000000000000000000000000000000000000000000000000000000000000
index 4294967295
}
script "1 [01]"
sequence 4294967295
witness [0000000000000000000000000000000000000000000000000000000000000000]
}
}
lock_time 0
outputs
{
output
{
address_hash 1d1191d528bdb565a139ac4eab8fc7bb7fad9074
script "[029f3b44528ce59338ec02a2ce98293c495aa0c691079c6e04aa897e12670c4dfa] checksig"
value 5000000000
}
output
{
script "return [aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf9]"
stealth
{
prefix 626186022
ephemeral_public_key 02aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836
}
value 0
}
}
version 2
}
evoskuil15:39
The "hex" is the tx. I'll run the tx through a validation test later today and see what's up. Assuming the error is reasonable then I shouldn't need the block.
vdeurzen15:40
should you need some details about the block, this is what bitcoin-cli returns:
{
"hash": "1c18eac728abe7ae6e435d00d0d11b6a89dea9f29de858ab0e06b3f496a98c53",
"confirmations": 104,
"strippedsize": 226,
"size": 262,
"weight": 940,
"height": 1,
"version": 536870912,
"versionHex": "20000000",
"merkleroot": "223e7ae1577df1afa3cec023d4a44b65868d341d523468e1ae47b19d461633e2",
"tx": [
"223e7ae1577df1afa3cec023d4a44b65868d341d523468e1ae47b19d461633e2"
],
"time": 1535124339,
"mediantime": 1535124339,
"nonce": 0,
"bits": "207fffff",
"difficulty": 4.656542373906925e-10,
"chainwork": "0000000000000000000000000000000000000000000000000000000000000004",
"nTx": 1,
"previousblockhash": "0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206",
"nextblockhash": "6e557eee16d21aec738f98ac41ef2acdd0b76f9d26800037adc898a8c11e7f42"
}and this is the hex:
0000002006226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910fe23316469db147aee16834521d348d86654ba4d423c0cea3aff17d57e17a3e227323805bffff7f200000000001020000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff03510101ffffffff0200f2052a010000002321029f3b44528ce59338ec02a2ce98293c495aa0c691079c6e04aa897e12670c4dfaac0000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000
vdeurzen15:42
[log] archive_directory = ./archive debug_file = ./debug.log error_file = ./error.log rotation_size = 100000000 minimum_free_space = 0 verbose = true [network] protocol_maximum = 70013 protocol_minimum = 31402 services = 9 #identifier = 118034699 identifier = 3669344250 outbound_connections = 2 inbound_connections = 10 inbound_port = 18445 hosts_file = hosts.cache peer = 127.0.0.1:18444 host_pool_capacity = 0 self = 0.0.0.0:0 [database] directory = blockchain flush_writes = true [blockchain] # Testnet-btc checkpoint = 0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206:0 [fork] easy_blocks = false retarget = false bip16 = true bip30 = true bip34 = false bip66 = true bip65 = true bip90 = true bip68 = true bip112 = true bip113 = true bip141 = true bip143 = true bip147 = true [node] relay_transactions=true #compact_blocks_high_bandwidth = true
thx again !
I'll be digging further, I think, tomorrow 🙂
neillm16:43
@vdeurzen The boost error building version3 is probably because of the gcc version. gcc-5 should be used, as gcc-8 can't compile that version of boost without warnings resulting in errors. If you have gcc-5 installed on your system, prefix the install.sh with
CC=gcc-5 CXX=g++-5evoskuil22:27
@vdeurzen This is the result of segwit (specifically bip141) not being active when validating a segwit tx. This is the result of the regtest activation hashes (which are at genesis) being reversed in the constants.hpp. I haven't verified full operation yet, but the PR is here: https://github.com/libbitcoin/libbitcoin/pull/1020
(edited)I haven't confirmed runtime operation, but this is certainly a bug. Once it clears I'll merge and you can retest.
evoskuil22:35
Notice the last comment on this page appears to be dated. A segwit block is being created by bitcoind at height 1. I assume bitcoind behavior has changed due to removal of bip9 activation code, but I have not looked into it. (If so this would be an undocumented bip90 style hard fork.) https://github.com/libbitcoin/libbitcoin-server/wiki/Regtest-Configuration
(edited)Thanks for raising the issue and helping us track it down. Please give me an update when you have a chance.
Saturday, August 25, 2018
S
Steve Barbour17:12
I would love to, but unfortunately I don't think I have the ability to administer funds as custodian just yet. Maybe in the future. Just looking to make a small donation as token appreciation, the Wiki has been very helpful.
jbaczuk20:36
Hey I’m really excited about this project, just trying to wrap my head around all of the repos. If I want to build everything, should I use
libbitcoin-build? What exactly is the purpose of that repo?Is it just for travis ci?
Autotools/make msvc project files, install.sh and Travis and appveyor yml files
Along with the project header
jbaczuk20:39
Thanks! So if I’m on Ubuntu should I run the
generate.sh script and then just run make && make install in each repo?
pmienk20:42
If you are not altering the repo contents, you don't need to involve the build repo. The instructions for installation are provided by the readme or repo page. They should point you towards the install.sh script on non windows platforms
👍 1
jbaczuk20:45
Ok. I would like to contribute, but I would like to have the latest release libraries installed globally on my system first and wrap my head around the whole system. So the build repo is to update the build artifacts for all systems after my changes even if I’m just developing on one?
👍 1
Sunday, August 26, 2018
jbaczuk04:17
Ok boost libraries are giving me a lot of trouble. Is there a recommended common version to use for the different libbitcoin projects and how to install it?
pmienk04:19
the install.sh will build boost with a minimum version. ./install.sh --build-boost --prefix=<absolute path>
best to utilize a prefix for installation to reduce dll/lib hell
are you utilizing the install script?
currently they are in
/usr/local/lib and /usr/local/includethose are system paths
the point of a prefix path is to avoid those locations
isolate the installation of libbitcoin and favor the libraries installed in the prefix for dependency location
pmienk04:24
not quite - prefix installation produces a bin/lib/include/share series of subdirectories rooted at the provided path
multiple libraries can be rooted at the prefix. but yes, the point of performing the prefix path is to scope/isolate some collection of projects
Monday, August 27, 2018
vdeurzen08:48
@evoskuil I'm going to test your changes today 🙂 thanks for the help and glad I was able to assist
vdeurzen10:41
Did a bit of bisecting, the error does not occur (transaction being rejected due to empty inputs/outputs) with 0.15 of the bitcoind node, with 0.16 and up it breaks
vdeurzen12:41
For the blockchain and transaction subscriptions, what is the return value used for exactly?
ah, subscription renewal it seems 🙂
(edited)James C.13:39
hi @vdeurzen there are some wiki examples on address subscription (via query service) or just subscribing to all newly accepted tx and blocks (via subscription services), perhaps helpful?
address subscription (via query service, requires subscription renewals):
https://github.com/libbitcoin/libbitcoin-server/wiki/Subscriptions-from-DEALER-sockets
new tx/block subscriptions (pure sub, no subscription renewals, via subscription services)
https://github.com/libbitcoin/libbitcoin-server/wiki/Transaction-Service
https://github.com/libbitcoin/libbitcoin-server/wiki/Block-Service
(edited)vdeurzen13:41
I have had a look at those indeed, however, they still have ZMQ in the middle and I'm trying to have as few hops as possible 🙂 Therefore I'm using the chain::subscribe_transaction() and chain::subscribe_blockchain() methods directly
thx for the tip though 😄
👍 1
I'm looking at block.hpp but really find it 😕
evoskuil15:49
If you are no longer having the original problem with 0.15 then it’s a new issue and “it breaks” needs definition.
vdeurzen15:50
fair enough, rather than 'it breaks' let's define it as, starting with bitcoind 0.16 libbitcoin will not accept a block generated by bitcoind in regtest mode, the debug log states the transaction has either no inputs or outputs as reason for the rejection.
evoskuil15:51
Need the data again, since clearly it’s not the same as with 0.15. Curious if the genesis block has changed.
Wednesday, August 29, 2018
A
Alex23:33
@Fil, I saw you are working on altcoin (bitcoin cash, litecoin, and feathercoin) support in libbitcoin. Do you have a target completion date?
evoskuil23:44
I assume this will be complete before the v4 release. I expect v4 to ship by the end of the year. I’ve begun operational testing of continuous parallel block download (the blocking feature) and it’s looking good so far.
👍 1
I’m on a flight to Tokyo, if you need a more detailed update this week :).
Thursday, August 30, 2018
Fil10:47
@Alex as Eric said the goal is to have all necessary changes upstream before the v4 release. The biggest and most pervasive change is the parametrization of the constants of which the last remaining PRs are currently in review. What's left are some minor cleanups and the implemenation of the hard forks themselves. I'm expecting to progress there faster because the changes will only affect two repos which cuts down compilation and local testing time a lot.
Maybe as an outlook, I'm confident that we will have litecoin support on master before I leave for vacation mid-September.
👍 1
great work, very nice libraries 🙂
Question though, I'm looking at the
console/executor.cpp file and I see that the node_ member is a shared_ptr, is that because there will be multiple threads accessing it and potentially trying to deallocate it when they finish or is there another reason I'm maybe missing?neillm13:01
@vdeurzen The shared pointer usage is just for allocation/de-allocation lifetime management and keeping things simplified instead of trying to do clean-up for all exit scenarios.
Friday, August 31, 2018
vdeurzen07:28
Is anyone aware of a similar library for ethereum and or other chains? Thinking of the design I need to put in place and having a solution for other chains that I could integrate would certainly change my approach 😜
lclc07:54
libbitcoin works soon also with Litecoin and Feathercoin.
Other coins like BCH, ZCash, Doge, etc are possible to add too.
vdeurzen08:13
nice, those are all based on bitcoin right? So, no similar tools / libs for ethereum yet?
still awesome to hear that 😄
neillm21:11
@Revive Thanks, I'm not sure who runs that -- if you could send mail to the mailing list, I think that's how it's been resolved in the past
(edited)👍 2