Bitcoin full node and query server
July 2025
Archive of the libbitcoin Slack #general channel.
Tuesday, July 1, 2025
evoskuil02:27
I completed the implementation of block and txs announcements (though we aren't taking txs), such that we don't announce an object to a peer who has announced to us. This uses a small per-channel cache of hashes, configurable size. That required a
node::channel, derived from network::channel so that tx/block state can be hung off of the channel without hacking it back into the network lib.
This uses a simple circular buffer of hashes. That can later be optimized using a circular cuckoo filter, reducing storage space and maybe search time. But it's trivial already, so no big deal.👍 2
Wednesday, July 2, 2025
evoskuil07:15
Spent the day today implementing pruning of the prevouts cache. When the node becomes current it starts announcing blocks. The announcement is handled by the snapshot chaser and determines if the top candidate is also confirmed, at which point the prevout table can be purged - since all blocks that have been validated are also confirmed. The table purge work was previously complete, just had to integrate it. This reclaims over 22GB on validated runs. The operation takes about 50 ms.
(edited)evoskuil17:43
I verified that with a full validation run, record time of
4.88 hrs to 900,000 blocks. Automatic snapshot at recent (block 903,646), taking 119 secs, and automatic prevouts pruning at current and coalesced (block 903,650) taking 438 ms. Shutdown in 49 secs. Given that blocks are still coming in and being validated out of order at recent and current, I added a coalesced condition. This state is that in which the top candidate is also the fork block (all relevant validations confirmed). Until there is no more activity this is a race, but assured when there are no more blocks pending.🔥 1
Thursday, July 3, 2025
evoskuil14:08
The client filters are a total of
11GiB of additional storage, computed over every tx and prevout for each block.Huddle with pmienk, Marnix, Rowan Voskuil, eynhaender, evoskuil · 204 min16:56
1 reply · Rowan Voskuil · last Jul 03, 2025
eynhaender20:02
I think i messed up the labels but this is how it's going to look on Linux with disk activity.
🔥 1
evoskuil22:10
Regarding discussion on wtxid indexation, Core indexes by wtxid only for its mempool. Non-mempool txs are not announced and cannot be requested by wtxid. Of course only its confirmed txs are persistent, so malleation is really just a question of which competing witness serialization is accepted into the pool.
Friday, July 4, 2025
evoskuil16:24
New record, maxi full validation to
(edited)900,000, with client filters index, 4.80 hrs. Dropped 11 minutes. Also shutdown 50 secs, snapshot 136 secs, and pruned prevouts 302 ms. Store size with filters, to current top, 778 GiB. Additional for snapshot 13.6GiB.🔥 2
Sunday, July 6, 2025
Rowan Voskuil04:43
I have made some progress on the dashboard. The graphs are a bit messy. But it now has a pie chart to display the percent completion of the sync.
🚀 1
7 replies · neillm, Rowan Voskuil, evoskuil · last Jul 07, 2025
Rowan Voskuil16:56
No im getting the data from the event.log file for now. In the future the data will be directly sent from the node via websockets to the dashboard.
👍 1👍🏻 1
evoskuil05:16
This is looking really good Rowan.
I think it would be worth splitting the header events into their own plots, since these are so rapid initially, and so infrequent later, that they get completely buried the block plots.
Also, it might be interesting to see downloaded and validated blocks in the pie chart. Downloaded will always be <= to validated which is always <= confirmed. But unlike confirmed, these are not sequential. You would have to maintain a count. That would be fine unless the node is restarted with partial completions of download/validate, since you wouldn't have an accurate count. But it's also the case that a validated block implies that all blocks up to that height are downloaded, and a confirmed block implies all blocks up to that point are validated. So you could correct quickly.
evoskuil05:40
Correction. Downloaded can be any amount ahead of validated (up to the window size), and validated can be any amount ahead of confirmed. Not <=. But the rest is correct.
Rowan Voskuil22:06
You are able to deselect any event that you don't want to see on the graph and the graph will resize to fit the data. Do you still think it is worth it to make a separate graph. Because the way I see It most of the time the user would just keep those deselected and when they are interested in that data they can select it. But I can totally see wanting them to just exist in a separate graph if that is what you think is best and would be easy to change.
evoskuil22:12
Ah, just leave that it's good. Does the plot continue to scroll right, and is there a way to scroll back or scale the x axis so that earlier periods can be observed?
Monday, July 7, 2025
eynhaender05:19
I have just completed a sync on my 32GB/Ryzen7 Ubuntu workstation. This is how it looks, both the time per 50k chunk as well as the runtime profiling. Total sync time to 900k was 21292 secs/355mins/5.91hrs
(edited)
🚀 3
4 replies · eynhaender, cguida · last Jul 15, 2025
eynhaender10:34
I installed Windows 11 on the same machine of the above performance test and let it sync. The result is devastating indeed. Same hardware, same internet connection, half the time.
(edited)
eynhaender10:39
Time to 900k was 11,235secs/187mins/3.12hrs. So just a blink of an eye slower than the leading run on my "maxi" with 96GB RAM.
eynhaender14:12
Unfortunately, this is not the pattern we´d like to see. Network not maxed out, CPU not maxed out. The activity profile of the windows run (on a different computer, 32GB/i7) from the other day (https://libbitcoin.slack.com/files/U076JU78W91/F094BCVS744/combined_metrics.svg) looked much more like what we´d expect.
This run, recorded today on my 96GB Ubuntu, looks very good imho. A new record at this location with 900k blocks reached after 10,632seconds/177minutes/2.95hrs.
🔥 3🎉 1
1 reply · evoskuil · last Jul 09, 2025
evoskuil00:54
This looks great. I’m concerned though that the issue is intermittent, as we saw it recently with a long period of low activity.
Tuesday, July 8, 2025
Thursday, July 10, 2025
Huddle with pmienk, Marnix, Rowan Voskuil · 15 min16:56
1 reply · Rowan Voskuil · last Jul 10, 2025
Thursday, July 17, 2025
pmienk05:59
Tomorrow's meetup has been postponed due to schedule conflict. Will resume next week, 7/24
👍 2
Thursday, July 24, 2025
Huddle with pmienk · 32 min16:53
1 reply · Marnix · last Jul 24, 2025
Thursday, July 31, 2025
Huddle with pmienk, Marnix · 26 min16:55
evoskuil18:08
Hey, sorry - I was here working but forgot to log in. I'm around if anyone wants to catch up.
(edited)Huddle with pmienk, evoskuil · 117 min18:17
4 replies · pmienk, evoskuil · last Jul 31, 2025
evoskuil19:06
ubuntu@ubuntu:~$ source ~/bitcoin.sh
ubuntu@ubuntu:~$ newBitcoinKey 0x8010b1bb119ad37d4b65a1022a314897b1b3614b345974332cb1b9582cf03536
---
secret exponent: 0x8010B1BB119AD37D4B65A1022A314897B1B3614B345974332CB1B9582CF03536
public key:
X: 09BA8621AEFD3B6BA4CA6D11A4746E8DF8D35D9B51B383338F627BA7FC732731
Y: 8C3A6EC6ACD33C36328B8FB4349B31671BCD3A192316EA4F6236EE1AE4A7D8C9
compressed:
WIF: L1WepftUBemj6H4XQovkiW1ARVjxMqaw4oj2kmkYqdG1xTnBcHfC
bitcoin address: 1HV3WWx56qD6U5yWYZoLc7WbJPV3zAL6Hi
uncompressed:
WIF: 5JngqQmHagNTknnCshzVUysLMWAjT23FWs1TgNU5wyFH5SB3hrP
bitcoin address: 113Pfw4sFqN1T5kXUnKbqZHMJHN9oyjtgD
ubuntu@ubuntu:~$