Bitcoin full node and query server
July 2020
Archive of the libbitcoin Slack #general channel.
Thursday, July 2, 2020
Sunday, July 5, 2020
N
narodnik07:47
I am still trying to sync the server lol, last time crashed at 650k blocks, now on 490k
I wonder how much of the initial sync speed is building indexes, and which indexes are slowest
we can actually measure that by creating a dump of raw blocks, timing iteration over them, then using raw database primitives to index them. Anyone done anything like that?
Monday, July 6, 2020
Saturday, July 18, 2020
N
narodnik16:21
I stopped a server 2 days ago and it still says stopping. Using master branch. Would be good if I could get access to a synced daemon to try to debug some of these things when I have time.
evoskuil16:23
That’s the deadlock I told you about. Debugging it doesn’t require one to be fully synced, you need to break into the process during the deadlock. I looked at it a few times but couldn’t isolate it. I’ve done this before in v3 and eventually got it.
(edited)N
narodnik16:25
@evoskuil I think it's related to the work queue. The other day I found that there's a way in boost asio to debug the work queue. I think we need a way to cancel running tasks from the work queue on shutdown... thats my theory anyway
N
narodnik16:26
the reason I want sync is so that it isn't hitting store a lot when I try to shut down, that will make it easier
The deadlock remains after store access is done.
17 replies · evoskuil, narodnik · last Jul 18, 2020
evoskuil17:22
evoskuil17:23
The problem is that, in this case, the code executed under the critical section cannot be easily visually verified to exit without reentry.
evoskuil17:50
Requires a good concurrency debugger. I’ve had it up in VS, which shows everything. Takes time to track tho. Reducing thread count helps a little.
N
narodnik18:04
I think we need to isolate which section of code the lock happens in and whether it's repeatable/deterministic across runs. It should be possible to see which sequence of invoke()s causes the lock.
evoskuil18:05
I can see all the threads idle, including the deadlock, in the debugger. That’s not the problem.
evoskuil18:15
I’ve debugged this same sort of issue before several times. V3 has been running for a couple years no with no occurrence of deadlock, and it is basically the same code. The new work in v4 is incomplete, which includes debugging this deadlock.
evoskuil19:01
Prolly not too much unfortunately. The outstanding work on v4 is not large in amount, but requires a lot of context. It takes me about a week to get my head back into it.
I can send you my ssh pubkey
That would be quicker to setup
Sunday, July 19, 2020
neillm03:40
Hrm, potential v4 mainnet issue (via libconsensus -- have not tested native):
03:37:54.008298 DEBUG [blockchain] Verify failed [628273] : invalid script libconsensus : true forks : 32767 outpoint : dc84990e4c84cd12d50466b6c2409dbd4319291ff92d1cef1e2cd33f3e85f263:1 script : a91453c5c7ea830a44a618dfca0bd79829f178778af987 value : 650723387 inpoint : cb10c3684632fa7b04a47c3977edf1ed73f7bdeb5dc7b849045eb43fa3b4b8a7:0 transaction : 020000000163f2853e3fd32c1eef1c2df91f291943bd9d40c2b66604d512cd844c0e9984dc0100000017160014183e55d4e9f0f055dc27a2a3cbf4f85b72910dc6ffffffff027cd70100000000001976a914734de9c66fcba1fc93c82f91b09627dae9d31b2c88acb709c7260000000017a914e2a744386747039cdfd9022ce0424e1f3421d6538700000000(edited)
Monday, July 20, 2020
N
narodnik18:58
Did anyone see this old code I wrote?
https://github.com/libbitcoin/libbitcoin-c
C is an amazing language. Very easy for people to use and incredibly popular. I looked for a way to parse C++ headers and autogenerate C but there is nothing ready or established out there. Instead I propose we manage these manually ourselves.
The great thing about C is that it makes generating bindings for other languages very easy. Like for example calling it directly in Python through CFFI:
https://github.com/libbitcoin/python-libbitcoin
N
narodnik18:59
If v4 is stabilized, I can get a junior dev working with me to update these codes and I will write a tutorial based off the Wiki with code samples for C/C++/Python.
Yes that's what I'm saying, but I think this is key. We have all the parts but they need to be put together. Neill is telling me he has a Qt wallet. If we could modify the server to only index selectively addresses we submit to it, then we could have a local fullnode wallet.
N
narodnik19:03
I will get these up to standard, then after v4 we can ensure changes happen across all repos in lockstep. Especially with the new schnorr/taproot changes coming in.
But a alt was already pushed most of the refactoring it required into master.
With the release out, I will approach Arthur Hayes/100x for a grant. They are funding Bitcoin development: https://www.100x.group/grants.html
In Bitcoin there's this elite holding back development. After all these years we still don't have an API. They control development and say there's only one true software.
👍 1
mileschet19:32
Hi my name is Roberto i’m also from BCN, working and learning from @narodnik a lot in the past months. I’m very happy to see libbitcoin alive and kicking !
👍 1👊 1
N
narodnik19:34
@mileschet see the mission eric write on this page https://libbitcoininstitute.org/
Libbitcoin Institute
The Libbitcoin Institute is a non-profit entity for formalizing operations, accelerating work, and sustaining future development of core Bitcoin infrastructure.
❤️ 1
R
rose19:51
hello everyone, I'm Rose, I'm a junior developer. I'll be responsible for maintaining the C bindings
👍 2👊 1
gh19:54
Hello my name ghassan, working with @narodnik . I am learning c++, rust, and bitcoin. I will help with the Qt wallet.
👍 3👊 1
hey @gh
Tuesday, July 21, 2020
neillm00:02
To clarify regarding a Qt wallet -- I worked on one for a while back in 2017 (in free time) made to work against version3, but it wasn't quite (open source) release ready, let alone 'production' ready. Aside from some UI issues, some sprawling console debug messages and needing better error handling, I recall thinking it was close, supporting either mainnet or testnet (and setup to default against the community servers as the backend). I was able to do deposits and sends using mainnet across multiple BIP44 accounts w/o issue, though I don't recommend anyone do that without review and testing, etc. I got it building recently, but I'll have to clean it up a bit before posting somewhere. Happy to have some more eyes on it though, in case some of it is useful to anyone. It wasn't the first stab at writing a wallet, so I think it benefits at least a little from past attempts.
N
narodnik08:07
OK @gh can help you once you have something ready. I can sort out some of the plumbing.
L
lucaferr12:40
Do used addresses have to be scanned for incoming funds indefinitely? I.e. there is a risk someone is reusing an address much later, after already funding it once
(edited)N
narodnik13:10
bs: src/chain/input.cpp:423: bool libbitcoin::system::chain::input::is_locked(size_t, uint32_t) const: Assertion `block_height >= prevout.height' failed. bs: src/chain/input.cpp:423: bool libbitcoin::system::chain::input::is_locked(size_t, uint32_t) const: Assertion `block_height >= prevout.height' failed.
N
slack-term18:48
I was resyncing the server from neill's last checkpoint. But I had tons of other debugging switched on, and now the VPS is offline because I broke the HD limit... waiting for it to come back online.
(that error, and VPS unrelated - VPS went offline on second run)
It was a single run
How long did it run?
Wednesday, July 22, 2020
From 628216
Thursday, July 23, 2020
mileschet10:26
hey guys, i did a fresh install of our master version and using neillm’s snapshot that i copied from narodnik, i was able to run and sync properly the chain, there was a period when the node became unavailable and i needed to restart, in the process there was this FATAL message on my log, here it’s the snippet that i was able to grab from the log
(edited)19:09:18.469839 INFO [network] Connected outbound channel [146.148.19.221:8333] (6) 19:10:01.927956 FATAL [blockchain] Failure in block organization, store is now corrupt: operation failed 19:11:29.951368 INFO [network] Connected outbound channel [45.84.42.80:8333] (1) 19:11:29.975513 INFO [network] Connected outbound channel [116.202.97.222:8333] (2) 19:11:30.027442 INFO [network] Connected outbound channel [95.217.2.55:8333] (3) 19:11:30.065452 INFO [network] Connected outbound channel [94.16.18.51:8333] (4) 19:11:30.117006 INFO [network] Connected outbound channel [68.194.92.202:8333] (5) 19:11:30.216059 INFO [network] Connected outbound channel [73.31.14.186:8333] (6) 19:11:30.232878 INFO [network] Connected outbound channel [66.114.33.90:8333] (7) 19:11:30.235557 INFO [network] Connected outbound channel [190.123.23.22:8333] (8) 19:11:42.010203 INFO [network] Connected outbound channel [13.125.181.34:8333] (9) 19:11:42.011036 INFO [network] Connected outbound channel [125.236.215.133:8333] (10) 19:11:42.014725 INFO [network] Connected outbound channel [116.202.196.182:8333] (10) 19:11:42.107018 INFO [network] Connected outbound channel [185.137.70.171:8333] (11) 19:11:42.111938 INFO [network] Connected outbound channel [188.37.47.5:8333] (12) 19:17:19.019861 INFO [network] Connected outbound channel [85.234.145.132:8333] (1) 19:17:19.057446 INFO [network] Connected outbound channel [77.120.122.118:8433] (2) 19:17:19.116593 INFO [network] Connected outbound channel [139.59.209.23:8333] (3) 19:17:19.124856 INFO [network] Connected outbound channel [34.245.72.53:8333] (4) 19:17:19.145303 INFO [node] Header #640323 [00000000000000000004f333ec3a6c357fccd74f8b5dbe73b8fb0062c365514b] 77.120.122.118:8433 32767 19:17:19.338404 INFO [network] Connected outbound channel [47.95.220.157:9595] (5)
going up on the log i have this as well, idk if this helps somehow !
18:54:55.175725 INFO [network] Connected outbound channel [163.158.243.230:8333] (2) 18:54:55.207700 INFO [network] Connected outbound channel [46.101.112.24:8333] (3) 18:54:55.278019 INFO [node] Header #640321 [00000000000000000004b52674c8a0a4687541758af2061af0174a4273b318be] 163.158.243.230:8333 32767 18:54:55.326794 INFO [node] Header #640322 [00000000000000000006d5eeb634894430340f3140eacb92d75f0565cac88cb2] 163.158.243.230:8333 32767 18:54:55.327115 INFO [network] Connected outbound channel [95.165.104.89:8333] (4) 18:54:55.411441 INFO [network] Connected outbound channel [211.110.66.114:8333] (5)
1 reply · evoskuil · last Jul 24, 2020
Friday, July 24, 2020
Sunday, July 26, 2020
N
slack-term14:03
he's talking about the first message "Failure in block organization, store is now corrupt"
1 reply · evoskuil · last Jul 27, 2020
evoskuil16:32
Oh, I missed it. That’s an unrecoverable error, the code has detected a store inconsistency. I’ve never seen that error actually happen. Should only be possible by deleting the lock file after a hard shutdown.
(edited)Monday, July 27, 2020
vdeurzen08:22
I believe I've also seen that on a first run while still syncing the chain, but certainly also after a restart
8 replies · evoskuil, vdeurzen · last Jul 27, 2020
vdeurzen16:38
I've reported it several times, didn't pursue it any further because I needed a functional node more than I needed the latest version at the time
vdeurzen16:39
Note, I've seen it a couple of times not necessarily many times, but that is mainly due to the fact that after spending a bunch of time in GDB trying to figure out where it was going wrong I decided my time was better spent on getting my system working with v3
evoskuil17:04
Is there an open issue for it? I don’t recall ever seeing this, in v3 or v4, but would expect to see it if the store had been hard closed and restarted. It certainly could be the case though, since v4 has a lot of unverified work. The assertion that Amir is seeing is an possible indication of a consistency issue.
vdeurzen17:35
I didn't report an issue because within my time constraints I failed to narrow it down enough for a clear report. I was at the time unsure whether it was my own fault or a bug. Seeing it again I figured it would at least at credibility to know I had also seen it
evoskuil17:36
Ok, any error that doesn’t result from operating with a corrupted store is worth seeing.
N
slack-term16:24
14:00:08.177003 INFO [node] Valid #640957 [0000000000000000000504ebf4c031b1c44e17f86dc43dd2f2771ae118cd6225] 1129 txs 6396 ins 3 des 23 pop 0 acc 0 scr 22 can 5 con 0.000000
14:00:08.457897 INFO [node] Valid #640958 [0000000000000000000549f02ebccaa70daa24e6bdc30a731fe9b7270bdcffc7] 1052 txs 6314 ins 3 des 17 pop 0 acc 0 scr 19 can 4 con 0.000000
bs: src/chain/input.cpp:423: bool libbitcoin::system::chain::input::is_locked(size_t, uint32_t) const: Assertion `block_height >= prevout.height' failed.
apologies, using a terminal client. I encountered this assert again.
1 reply · evoskuil · last Jul 27, 2020
evoskuil16:32
replied to a thread: view
Oh, I missed it. That’s an unrecoverable error, the code has detected a store inconsistency. I’ve never seen that error actually happen. Should only be possible by deleting the lock file after a hard shutdown.
(edited)Let me see if it repeats again
I was using a friend's server and it ended up killing his server so he pulled the plug. But I have a new one from a different provider this time with enough specs.
Tuesday, July 28, 2020
mileschet09:52
Hey, i saw that message after stating syncing from “neillm” snapshot, maybe it’s a oprtion worth to try, how can i turn on some debugs so i can resync again from there ?
(edited)N
narodnik16:07
0# libbitcoin::system::chain::input::is_locked(unsigned long, unsigned int) const in /home/narodnik/.local/lib/libbitcoin-system.so.0 1# libbitcoin::system::chain::transaction::is_locked(unsigned long, unsigned int) const in /home/narodnik/.local/lib/libbitcoin-system.so.0 2# libbitcoin::system::chain::transaction::accept(libbitcoin::system::chain::chain_state const&, bool) const in /home/narodnik/.local/lib/libbitcoin-system.so.0 3# libbitcoin::blockchain::validate_block::accept_transactions(std::shared_ptr<libbitcoin::system::message::block const>, unsigned long, unsigned long, std::shared_ptr<std::atomic<unsigned long> >, bool, bool, std::function<void (std::error_code const&)>) const in /home/narodnik/.local/lib/libbitcoin-blockchain.so.0
This is the stack trace for that assert. It's reproducible on my machine.
Going to add a bunch of debug output to isolate source of the error
Wednesday, July 29, 2020
N
slack-term07:54
IS_LOCKED block_height=640959 prevout = d3371e10309dd3da2b412a3a8d47dbda300b1ed1a0a791fd6a013e0494db9c1e:0
This is where is errors.
N
slack-term08:00
N
slack-term09:12
So I see in node, the protocol handles catch the block event then pass it the blockchain store, but where does the metadata storages get initialized? Particularly for output_point in tx inputs for validated blocks (in accept())
👍 1
2 replies · evoskuil · last Jul 29, 2020
evoskuil15:20
Maybe add debug info in the case where that assert condition would fire, to emit the relevant values. It would be good to see whether values are unpopulated or otherwise incorrect.
@evoskuil sorry to mark you, for some reason i think you’ve mentioned a taproot implementation already done on some branch of libbitcoin ? I’m willing to look into it if you can share !
(edited)evoskuil15:06
No worries. It’s in a fork in my personal repo. It’s not yet complete, but it’s pretty close.
👍 1
1 reply · mileschet · last Jul 29, 2020
mileschet15:18
ok fantastic, i’ll dig into it for sure… i was reading gmax review on core taproot worth reading to smile a bit !