Bitcoin full node and query server

May 11, 2021

View in context in the libbitcoin Slack #general mirror.

@evoskuil Just curious, during full_node.chain().get_transaction() and .get_block() , is it ever the case that the async callback would be invoked *in the OS thread that this call is made*, i.e. **BEFORE** get_transaction/block() returns - Or instead, is the callback always run in one of LB’s worker threads? I was thinking that the optimal speed case would be dispatching the async callback within the self same OS thread as the get_transaction/block() call is made - e.g. in the situations where that transaction or block is already cached - here no work by LibBitcoin worker threads would be needed. This would be the highest possible logical speed possible. When work is passed from the calling thread to another OS thread, there is a tiny tiny bit of latency for the worker thread to wake up and the memory to move from core to core. Mostly unnoticeable though I guess if you make a billion tx/block requests in a *sequence*, that latency would add up. Anyhow this was mostly performance trivia. By the way separately, where in the code do I find how many worker threads full_node* actuall runs? My thought is, an SSD tends to have is it 32+ queue depth. When the memory mapping accesses the disk, the OS thread blocks. For this reason a full_node for maximum performace, I presume, should have at least 32 OS threads?

8 replies · evoskuil, Phil · last May 14, 2021

evoskuil18:57

Calls may return on any thread in the threadpool. Callbacks are queued and then dispatched as threads become available. Operation the node can operate on a single thread. It’s not clear what you distinction you intend by “OS thread”. All threads, with the exception of the single process thread, are logical objects created by the threadpool class. They may or may not correspond to distinct operating system threads of execution.

evoskuil19:05

This qualifies as premature optimization. Until you identify a problem, there is no problem to solve. Increasing the amount of work on a single thread reduces the availability of threads for other work. Load balancing is a performance consideration. Individual calls would certainly be more performant if executed synchronously, as you are essentially suggesting. However this simply blocks other calls, with a subsequent impact on latency of all calls. A trade off is made with consideration of the thread switching cost against the size of the work. Larger jobs are broken up into smaller jobs, and small jobs are aggregated into larger jobs. The optimal size of any job can only be determined by performance testing. Until you have measured this difference an informed decision on which trade off to make is not possible. Therefore, premature optimization.

evoskuil19:07

The number of logical threads in the node/network threadpool, the validation threadpool, and the query server threadpool, is exposed in the configuration file. You can trace this back to where they are created and used within the source.

evoskuil19:11

The optimal number of threads for any pool varies. I have found that on my 32 dual core (64) machines, 32 is more optimal than 64 for the validation threadpool. At some point the cost of fanning out the parallel jobs exceeds the benefit of parallelism. For this reason the node limits the number of threads configured to the number that are reported by the operating system, but that’s fairly arbitrary and more of a safeguard against gross misconfiguration.

(edited)

evoskuil19:19

Libbitcoin is both highly asynchrous and parallel. Many individual std library calls are parallelizable by the compiler, and expensive validation steps (such as script execution) are explicitly parallelized by the implementation. This is also why std lib calls are used when possible vs. for loops. Imposing order where it is not a logical requirement is an antipattern. The same holds for reducing asynchrony to increase individual call performance. This reduces the effectiveness of parallelization.

evoskuil19:21

Generally speaking the approach is to never impose order when it is not required. At that point performance tuning can optimize tradeoffs, but only through performance testing.

evoskuil19:28

I found at least one case where Satoshi imposed order in consensus evaluation. He requires transactions in a block to be ordered such that outputs that are spent by other txs in the block precede those other transactions. There is no reason for this except that he used a naïve implementation which queues outputs and verifies inputs against the outputs queue as it’s being built. So I retained our parallel implementation and added and explicit and otherwise entirely unnecessary rule called is-forward-reference. https://github.com/libbitcoin/libbitcoin-system/blob/master/src/chain/block.cpp#L821

(edited)

Phil13:55

The optimal number of threads for any pool varies. I have found that on my 32 dual core (64) machines, 32 is more optimal than 64 for the validation threadpool. At some point the cost of fanning out the parallel jobs exceeds the benefit of parallelism. For this reason the node limits the number of threads configured to the number that are reported by the operating system, but that’s fairly arbitrary and more of a safeguard against gross misconfiguration.

You mean for safety, LibBitcoin’s thread pool will be the number of CPU cores that the OS reports that the machine has? Safety as in don’t presume users will bother to set such a setting