Bitcoin full node and query server
Dec 4, 2024
View in context in the libbitcoin Slack #general mirror.
evoskuil16:34
Sure, I'll build a current copy and run it to verify. What's your measured bandwidth?
🔥 1
36 replies · cguida, evoskuil · last Dec 11, 2024
evoskuil17:35
Well that's only 27% of what I'm testing on, and since it's network bound on my network, it will be as well on the same machine on a slower network. So you should expect it to take around 3.7 hours.
🔥 1
C
cguida17:40
Haha it’s super fast for the country i’m in. Btw the machine cost US$530 here because of silly import duties as well.
Again, 3.7 hours on a US$530 machine is still an amazing result, looking forward to confirming 🙂
evoskuil17:41
I'm waiting for it to complete. Once that's done I'll post you a zip with the executable. The command line is simply > bn
🙏 1
evoskuil17:48
It generates a rotating pair of log files named
bn_end.log and bn_begin.log (the combined size can be configured). It starts with end and then rotat3es it over to begin and so-on. These are currently just an echo of whatever hits the console. The console and logs are buffering, so pausing the console doesn't really slow the node. The log will start with an entry that shows the name of the config file its using (or default) so you know if it has your config.
2024-11-24T19:39:57Z.0 ====================== startup ======================= 2024-11-24T19:39:57Z.0 Using config file: "m:\\bn.cfg"
There's also some logging options dumped at the start of the log. Since log levels (and their message generation) can be entirely compiled out, this gives you some good info.
2024-11-24T19:39:57Z.0 Version Information... libbitcoin-node: 4.0.0 libbitcoin-database: 4.0.0 libbitcoin-network: 4.0.0 libbitcoin-system: 4.0.0 2024-11-24T19:39:57Z.0 Press CTRL-C to stop the node. 2024-11-24T19:39:57Z.0 Log system configuration... 2024-11-24T19:39:57Z.0 [a]pplication.. compiled:1 enabled:1. 2024-11-24T19:39:57Z.0 [n]ews......... compiled:1 enabled:1. 2024-11-24T19:39:57Z.0 [s]ession...... compiled:1 enabled:1. 2024-11-24T19:39:57Z.0 [p]rotocol..... compiled:1 enabled:0. 2024-11-24T19:39:57Z.0 [x]proxy....... compiled:1 enabled:0. 2024-11-24T19:39:57Z.0 [r]emote....... compiled:1 enabled:1. 2024-11-24T19:39:57Z.0 [f]ault........ compiled:1 enabled:1. 2024-11-24T19:39:57Z.0 [q]uitting..... compiled:1 enabled:0. 2024-11-24T19:39:57Z.0 [o]bjects...... compiled:0 enabled:0. 2024-11-24T19:39:57Z.0 [v]erbose...... compiled:1 enabled:0. 2024-11-24T19:39:57Z.0 Please wait while network is starting... 2024-11-24T19:39:57Z.1 Candidate top [000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f:0]. 2024-11-24T19:39:57Z.1 Fork point (0) unassociated (0). 2024-11-24T19:39:57Z.1 Loaded (10000) addresses. 2024-11-24T19:39:57Z.1 Bypassed seeding because of sufficient (10000 of 1000) address quantity. 2024-11-24T19:39:57Z.0 Node is started. 2024-11-24T19:39:57Z.1 Not configured for inbound connections. 2024-11-24T19:39:57Z.1 Create 200 connections 5 at a time. 2024-11-24T19:39:57Z.0 Node is running.
👍 1
evoskuil17:50
Windows Defender might flag the process. I don't think it's infected, but the process behavior is obviously a bit scary. If so you might need to give the file an exclusion or turn off Defender.
👍 1
evoskuil17:59
The recommended config for a milestone sync benchmark on this machine is the default config. There are a lot of (1) config settings, (2) command line options, and (3) runtime console options. Config settings are in an "ini" file format, and established using the
--config or -c command line switch as described above. The --help or -h command dumps command line options:
>bn -c m:\bn.cfg -h
Usage: bn [-abdfhiklnrstvw] [--config value]
Info: Runs a full bitcoin node.
Options (named):
-a [--slabs] Scan and display store slab measures.
-b [--backup] Backup to a snapshot (can also do live).
-c [--config] Specify path to a configuration settings file.
-d [--hardware] Display hardware compatibility.
-f [--flags] Scan and display all flag transitions.
-h [--help] Display command line options.
-i [--information] Scan and display store information.
-k [--buckets] Scan and display all bucket densities.
-l [--collisions] Scan and display hashmap collision stats (may exceed
RAM and result in SIGKILL).
-n [--newstore] Create new store in configured directory.
-r [--restore] Restore from most recent snapshot.
-s [--settings] Display all configuration settings.
-t [--test] Run built-in read test and display.
-v [--version] Display version information.
-w [--write] Run built-in write test and display. (edited)evoskuil18:00
Many of these options are for developer troubleshooting and store sizing. The
(edited)--settings option dumps the entire set of config file settings and descriptions. It's a lot and again many of these are to aid tuning, but some are for general purpose use.evoskuil18:09
When the node is running, at the console you can enter various commands by hitting a letter key and then <enter>. The combination
m <enter> brings up the console menu. These consist of toggles and options. Toggle varies the log levels that are compiled in (all except for objects in this build. I don't recommend leaving highly verbose levels turned on for a long time, it just wastes time.
2024-12-04T18:06:13Z.0 Toggle: [a]pplication 2024-12-04T18:06:13Z.0 Toggle: [n]ews 2024-12-04T18:06:13Z.0 Toggle: [s]ession 2024-12-04T18:06:13Z.0 Toggle: [p]rotocol 2024-12-04T18:06:13Z.0 Toggle: [x]proxy 2024-12-04T18:06:13Z.0 Toggle: [r]emote 2024-12-04T18:06:13Z.0 Toggle: [f]ault 2024-12-04T18:06:13Z.0 Toggle: [q]uitting 2024-12-04T18:06:13Z.0 Toggle: [o]bjects 2024-12-04T18:06:13Z.0 Toggle: [v]erbose 2024-12-04T18:06:13Z.0 Option: [b]ackup the store 2024-12-04T18:06:13Z.0 Option: [c]lose the node 2024-12-04T18:06:13Z.0 Option: [e]rrors in store 2024-12-04T18:06:13Z.0 Option: [g]o network communication 2024-12-04T18:06:13Z.0 Option: [h]old network communication 2024-12-04T18:06:13Z.0 Option: [i]nfo about store 2024-12-04T18:06:13Z.0 Option: [m]enu of options and toggles 2024-12-04T18:06:13Z.0 Option: [t]est built-in case 2024-12-04T18:06:13Z.0 Option: [w]ork distribution 2024-12-04T18:06:13Z.0 Option: [z]eroize disk full error
evoskuil18:10
You cannot really track progress from watching the log levels. There is a high speed event log that emits messages to a file sink (
events.log) presently. You can use this to track the number of archivals (a block that's downloaded, checked, stored and indexed.
block_archived...... 551105 1217 block_archived...... 565941 1217 block_archived...... 563597 1217 block_archived...... 562631 1217 block_archived...... 571193 1217 block_archived...... 567873 1217 block_archived...... 589292 1217 block_archived...... 559177 1217 block_archived...... 572302 1217 block_archived...... 550289 1217 block_archived...... 587634 1217 block_archived...... 584910 1217 block_archived...... 564990 1217 block_archived...... 567301 1217
evoskuil18:12
Since these are out of order you should rely on the file line count to see the number of blocks downloaded. The default config is milestoned to 850k, and validate/confirm are disabled. Blocks get a tad bit slower above 850k, and also at the tail end of the chain just because there are fewer blocks remaining.
evoskuil18:14
The commands work at any time the node is running.
<ctrl>-c stops the process normally, or you can also use c <enter>. If you type multiple or wrong characters at the command line, just hit <enter> to clear them, which you'll see at the console once it clears. Note that the node is highly asynchronous, so sometimes console actions are not immediate, but generally it's pretty quick.evoskuil18:16
This is the
i <enter> command at runtime (it provides a little more at the command line):
2024-12-04T18:12:49Z.0 Body sizes... header :85577716 txs :4520884072 tx :69884209602 point :101072408544 input :390667618129 output :107407237477 puts :26885386434 candidate :2619726 confirmed :3 spend :67402563504 strong_tx :12387474660 valid_tx :0 valid_bk :0 address :0 neutrino :0 2024-12-04T18:12:49Z.0 Table records... header :873242 tx :1127164671 point :2807566904 candidate :873242 confirmed :1 spend :2808440146 strong_tx :1032289555 address :0 2024-12-04T18:12:49Z.0 Head buckets... header :524493 txs :524493 tx :551320125 point :546188501 spend :1459791875 strong_tx :551320125 valid_tx :551320125 valid_bk :524493 address :1 neutrino :1 2024-12-04T18:12:49Z.0 Collision rates... header :1.66493 txs :1.66493 tx :2.04448 point :5.14029 spend :1.92386 strong_tx :1.8724 valid_tx :2.04448 valid_bk :1.66493 address :0 neutrino :0(edited)
evoskuil18:18
Shutdown time from the full chain sync was 10 seconds:
2024-12-04T18:18:02Z.0 Database stopped successfully in 10 secs. 2024-12-04T18:18:02Z.0 Node stopped successfully. 2024-12-04T18:18:02Z.0 ====================== shutdown ====================== 2024-12-04T18:18:02Z.0 Press <enter> to exit...
evoskuil18:21
You should always terminate the process normally, or it will require a full resync from the last snapshot. You can establish snapshots while running or once it's complete, or at the command line. Automatic snapshotting is currently off by default. A snapshot pauses the network and takes a couple minutes. There are only two in rotation. Restore is automatic, or can be invoked at the command line. If the disk fills it is detected, and will resume once space is available, or can be shut down normally.
(edited)👍 1
evoskuil18:26
The store can also be hot backed-up from the file system, and will automatically recover to the last snapshot. So for hot backup simply snapshot first and then copy the directory once that completes, while it's running.
evoskuil18:35
You will probably have to configure seed nodes manually. The ones configured by default are probably not running. This can be done in the config file, any number of entries:
[network] seed = ip:port seed = host:port
👍 1
evoskuil18:37
By default the node maintains 10,000 host names in a text file named
hosts.cache . IPv6 is supported but disabled by default. To skip seeding you can also just place this file in the network directory, as long as there are at least 500 entries (by default).C
cguida00:15
Ok upon closer inspection the sync took 7605 seconds which is 126.75 minutes or 2.11 hours
C
cguida00:16
I am incredibly psyched about this result!!
I am in awe of the work you guys are doing!
🤙🏻 1
C
cguida00:16
I'm wondering if the sync time would be massively impacted with only 16GB ram instead of 32...