Bitcoin full node and query server

December 2021

Archive of the libbitcoin Slack #general channel.

Thursday, December 2, 2021

evoskuil05:31
I just did some reading on boost json. This looks perfect, as it’s stream oriented. I think I’d like to drop xml/info serialization support and implement json de/serialization using this. By doing so we can get serialization and deserialization directly from the existing from_data/to_data methods of each class.
✅ 1👍 1
pmienk05:34
Not looking at it myself, I don't have any objections. Json is rather friendly to the eye.
What's the current xml consumer?
(edited)
evoskuil05:36
By passing json_reader/writer that implement our bytereader/writer interfaces all of the serialization code is embedded in the streamers (reader/writer) and any io stream is supportable. The only other change would be passage of element names to the generic reader/writer methods.
Only bx
pmienk05:37
Sounds great for engineering savings. Hey, is your enlistment passing all tests?
evoskuil05:37
It’s only there because it and info were supported by boost pt.
Yeah, big savings.
No, there’s maybe 10 failing. Didn’t have a chance to fully clean up after some major rework, but wanted to get it in before I left. I’ll take care of those in a few days.
pmienk05:39
Cool, just making sure they weren't induced by my changes
evoskuil05:42
Recent changes eliminate a large amount of copying in validation code. I’ve got another big change coming, which should eliminate all other unnecessary copying in validation. As script and witness data elements enter the stack, but never actually change, op.data and stack elements become data_chunk::ptr.
🙌 1
Reviewing Core, I can see a good amount of element and script copying that we won’t have.
Shared ptrs to script/witness data just get refcounted as the pointers get added to the stack.
(edited)
pmienk05:45
Can't argue with a simple smart pointer
evoskuil05:59
I had to refactor script to make this possible, which is what instigated the recent refactor. We’ve be holding the full script as a data vector, despite parsing it. This was because scripts can terminate in unparseable data. But this must be retained for serialization, as it’s valid if not executed (eg coinbase inputs). So we now have a flag in operation that indicates that the op is raw data for serialization. Originally this was attempted with a raw_data opcode, but this overloaded a potentially valid opcode. So now we can move to pointer stacks from deserialized pointer push-data ops.
(edited)
evoskuil06:06
I’ve got another set of changes which move script, witness, input, output and transaction storage to shared_ptr, and a template for deriving their collections from std::vector, along with a template-defined ::ptr for the collection itself. These collection objects can be specialized to provide collection queries (such as obtaining the set of tx hashes), which will move a lot of code out of the primary objects.
(edited)
I’ve also removed all setters, making chain objects effectively const. Fabricating an object is possible, for tests and sighashing, by simply using initializer syntax with the property constructors. So with pointers, copy/move of existing objects into new objects becomes just a couple integer copies and pointer ref counts.
(edited)

Saturday, December 4, 2021

neillm20:47
Cypherpunk Cogitations
Testing full validation syncing performance of 7 Bitcoin node implementations.

Monday, December 6, 2021

jungly12:31
Seriously impressed by GoCoin. Given it is in Go!
The disk I/O on version3 is a serious concern, but there's tonnes of super fabulous improvements in the pipeline!
✅ 1
evoskuil14:05
Version 3 isn’t even headers first. It hasn’t changed materially in 5 years yet still hangs in there.
💪🏾 1✅ 1

Sunday, December 19, 2021

N
narodnik11:03
in electrum.hpp, I see validate_mnemonic() which when I check the code is only for new type seeds (it checks explicitly). I see a static function in the source called mnemonic_decode() but that only takes new type dictionaries:
// Electrum v1 dictionary required for v2 compatibility
extern const dictionary_v1 en_v1;
so the dictionary list is there, but there's no way to generate public keys (removed by accident)
N
narodnik11:22
Hacking an old version of Electrum, I can generate the seed
$ python mnemonic.py witch collapse practice feed shame open despair creek road again ice least
25a39767960aafbdba67ad41f5657826
But then this isn't recognized by libbitcoin
$ bx ec-to-public 25a39767960aafbdba67ad41f5657826
Error: the argument ('25a39767960aafbdba67ad41f5657826') for option '--EC_PRIVATE_KEY' is invalid
neillm13:57
If I remember right, electrum v1 support was dropped a while back. The v1 related electrum code is only there to validate v2 mnemonics.
N
narodnik14:04
ok thanks
evoskuil17:12
I restored Electrum v1 in master.
👍 1
But it's not integrated into bx yet. Electrum v1 has a bug due to a mismatch of its wordlist domain to its numeric domain. I worked out a rational way to deal with this in master. Electrum v2 requires Electrum v1 so you really have to have both and deal with the bug.
✅ 1

Monday, December 20, 2021

N
narodnik09:29
if you need working code to compare, then put this in the electrum/ repo: https://gist.github.com/narodnik/b88e4d783ece0483d2bb116126c41bf8
it will derive an address v1 style
evoskuil09:35
Thanks, but the master implementation of Electrum v1/v2 and BIP 39 is complete and fully tested.
N
narodnik09:35
Great