May 2021

Was designing libbitcoin to use ZMQ for the blockchain API a mistake? Was it better to go with JSON since it's easier for frontend devs and less maintenance work? Originally the ZMQ choice was to reduce latency and optimize for performance.

No, it provides a robust internal and external framework. Not just in terms of ease and latency, but for managing high scale and/or distributed service. We've layered a light json interface over it for compat, but the http overhead is non-trivial and you lose the other benefits (though the underlying DoS management is preserved).

Thanks a lot!

👍

@neillm did you receive my signal messages?

bump @pmienk

Please give me detail cues as to which exact lines to comment out

I understand what you are hinting at, the Node will trig the logic to get the whole block. What we will do is simply to disable the request for whole blocks. Also when a whole block is pushed from a peer, we ignore it. Also when any transaction is pushed from a peer, we ignore that too. Are those in total the three points needed to degrade to SPV?

If you don't want blocks or transactions you shouldn't bother with libbitcoin-node. All you need is libbitcoin-network.

But, I do want persistence for the block header downloads. That persistence is in libbitcoin-node, though? I.e. we need that = more than solely libbitcoin-network

Dear Eric, I am looking into the symmetry of accessing the persistence (safe_chain class), symmetrically between offline mode and online mode. I’m totally clear that I can access the chain ‘live’ from full_node.chain(). How should I access the persistence as a safe_chain instance, when I am not intending to go online? (My guesses: Either I can somehow start a full_node, but without running full_node.start() or full_node.run(), this way somehow getting the full_node.chain() but without starting any peer to peer work, OR i should separately create some kind of libbitcoin-database instance for this...) And then second question, how do I switch between that online and offline mode?

Once you start accepting data from the network (or APIs) asynchronously you the synchronous database calls are unsafe. They are meant for startup and calls from within the asynchronous calls.

I found BIP173 so poorly written I had to write this up. https://github.com/libbitcoin/libbitcoin-system/wiki/BIP173-For-Humans

I have a newbie question related to Bitcoin Explorer. From what I understand, bx ec-new only accepts base16 entropy with at least 128 bits. I'm passing 1ffffffffffffffffffffffffffffffff hex which technically has 129 bits and yet bx ec-new complains that the argument is invalid. What am I missing? (I understand that we should be using bx seed but I'm just testing some features)

@evoskuil Just curious, during full_node.chain().get_transaction() and .get_block() , is it ever the case that the async callback would be invoked *in the OS thread that this call is made*, i.e. **BEFORE** get_transaction/block() returns - Or instead, is the callback always run in one of LB’s worker threads? I was thinking that the optimal speed case would be dispatching the async callback within the self same OS thread as the get_transaction/block() call is made - e.g. in the situations where that transaction or block is already cached - here no work by LibBitcoin worker threads would be needed. This would be the highest possible logical speed possible. When work is passed from the calling thread to another OS thread, there is a tiny tiny bit of latency for the worker thread to wake up and the memory to move from core to core. Mostly unnoticeable though I guess if you make a billion tx/block requests in a *sequence*, that latency would add up. Anyhow this was mostly performance trivia. By the way separately, where in the code do I find how many worker threads full_node* actuall runs? My thought is, an SSD tends to have is it 32+ queue depth. When the memory mapping accesses the disk, the OS thread blocks. For this reason a full_node for maximum performace, I presume, should have at least 32 OS threads?

(Followup on above:) Which are the synchronous database calls? What is the failure mode of read operations to those - inconsistent view, or SIGSEGV?

If my OS process is starting just now, and want to do blockchain explorer queries to the persistence built up by LibBitcoin Node in the past - alternatively my OS process has run a full_node, but it’s offline now thanks to that I did full_node.stop() (?+ .close() i presume?), then how do I do that now? Shall I somehow run a full_node* but not, uh, run .start() on it, this way it will give me blockchain explorer queries but not do networking?

If the store terminates without clearing the flush lock file, the store files are corrupt. That’s the purpose of the flush lock, to detect store corruption and prevent use of a corrupt store, which would be undefined behavior.

Calls may return on any thread in the threadpool. Callbacks are queued and then dispatched as threads become available. Operation the node can operate on a single thread. It’s not clear what you distinction you intend by “OS thread”. All threads, with the exception of the single process thread, are logical objects created by the threadpool class. They may or may not correspond to distinct operating system threads of execution.

This qualifies as premature optimization. Until you identify a problem, there is no problem to solve. Increasing the amount of work on a single thread reduces the availability of threads for other work. Load balancing is a performance consideration. Individual calls would certainly be more performant if executed synchronously, as you are essentially suggesting. However this simply blocks other calls, with a subsequent impact on latency of all calls. A trade off is made with consideration of the thread switching cost against the size of the work. Larger jobs are broken up into smaller jobs, and small jobs are aggregated into larger jobs. The optimal size of any job can only be determined by performance testing. Until you have measured this difference an informed decision on which trade off to make is not possible. Therefore, premature optimization.

The number of logical threads in the node/network threadpool, the validation threadpool, and the query server threadpool, is exposed in the configuration file. You can trace this back to where they are created and used within the source.

The optimal number of threads for any pool varies. I have found that on my 32 dual core (64) machines, 32 is more optimal than 64 for the validation threadpool. At some point the cost of fanning out the parallel jobs exceeds the benefit of parallelism. For this reason the node limits the number of threads configured to the number that are reported by the operating system, but that’s fairly arbitrary and more of a safeguard against gross misconfiguration.

Generally speaking the approach is to never impose order when it is not required. At that point performance tuning can optimize tradeoffs, but only through performance testing.

I found at least one case where Satoshi imposed order in consensus evaluation. He requires transactions in a block to be ordered such that outputs that are spent by other txs in the block precede those other transactions. There is no reason for this except that he used a naïve implementation which queues outputs and verifies inputs against the outputs queue as it’s being built. So I retained our parallel implementation and added and explicit and otherwise entirely unnecessary rule called is-forward-reference. https://github.com/libbitcoin/libbitcoin-system/blob/master/src/chain/block.cpp#L821

hey guys, looking at using cxx https://cxx.rs an rust tool to generate bindings for libbitcoin

Libbitcoin is in outer space

That's excellent

Yup, it’s a house of cards. That’s one of the several reasons it can’t change much at all. I still have a large amount to do on our code, but it’s really fun to work in.

You mean for safety, LibBitcoin’s thread pool will be the number of CPU cores that the OS reports that the machine has? Safety as in don’t presume users will bother to set such a setting

Not safety. Any number of threads is safe.

@evoskuil I’m not being able to find the definition of header_const_ptr_list_const_ptr using grep. I can only find usage of it. Where/how is it defined?

When I subscribe to header reorganizations, get branch/height, does LB have a guarantee that as long as you have **not returned** the reorg callback, that during that time frame, it wil *NOT* make another reorg callback for a reorg that happened AFTER that reorg ??

No, if that was the case your handling of that callback would block the entire node.

Are there any synchronicity guarantees? Or otherwise some way for me as api user to index the reorgs so i can sort and process them in a sequence?

The guarantee is that it is asynchronous and that callbacks are invoked in the order that events happen. You have to write asynchronous code to use it. You could of course run it on one thread, and it will still be asynchronous, but given only one thread it would not be parallel. You could also just block every callback thread and starve the network pool (to the same effect). But these are poor choices. If you require order you impose it using critical sections of your own.

full_node has a block_chain typed object and when I call node->chain() it returns a safe_chain& typed object. Is it safe for me to typecast the outcome of node->chain() back to block_chain? I wanna poll for what’s the head block’s hash & height, which I cannot get with a `safe_chain` typed object. These are available on `block_chain` only:

    /// Get highest confirmed or candidate header.
    bool get_top(system::chain::header& out_header, size_t& out_height,
        bool candidate) const;

    /// Get highest confirmed or candidate checkpoint.
    bool get_top(system::config::checkpoint& out_checkpoint,
        bool candidate) const;

    /// Get height of highest confirmed or candidate header.
    bool get_top_height(size_t& out_height, bool candidate) const; 

might want to consider polling fetch_last_height

If a cast was possible it wouldn’t be necessary. The reason you get a safe_chain is because block_chain is unsafe and cannot be used except at startup or under internal locks.

oh I see. So I’ll just use what @pmienk suggested. Thank you both for answering!

just as long as it isn't the derivation approach, that requires you to adhere to the same paradigm or seriously shoot yourself in the foot

The reorg listener, does it invoke the callback ONCE at startup?? Thus functioning as a poll/‘fetch’ operation (I.e. in addition to trigging when an actual reorg happened thanks to network input, subsequently) I am trying to figure how to poll my full_node. for what height it is at

Nodes aren’t query servers. If you want general purpose query you should build on server.

The reorg callback is fired when a reorg happens. If you have derived from or contained node you have access to safe_chain as a property of node, so you can access any chain state. However, all calls are asynchronous. A reorg may happen while you are fetching height, at which point height may have changed by the time your fetch call returns. So you would only want to initialize height before starting node.

It is possible that you could get two reorgs very close together, and end up completing the processing of the first one sent after competing processing of the second. This should be considered normal behavior and would simply appear like a reorg in the other direction. It’s even possible that the tracked chain could get shorter under this circumstance, and that a block you believe to be confirmed is actually unconfirmed. this would be resolved by the next reorg and is normal behavior, despite being rare. Race conditions are the nature of asynchronous programming.

Every bit can be one or zero, but if you reduce the domain there are fewer bits and therefore less entropy. A 132 bit buffer has 132 bits of entropy as entropy is not just the 1 bits. If you reduce the domain then it reduces the strength of the seed, even if the actual value was just 1. 1 out of 2^1 is 50%, 1 out of 2^132 is much smaller.

So bx is accepting from you both the numeric value and the domain.

Dear Eric, here is GDB’s backtrace of all threads of a full_node that I have running on OpenBSD, embedded in my custom process - really just running a full_node, very basic, like LibBitcoin’s “bn”. After running for a long time, the reception of blocks suddenly stops. According to the log this is the last block it received: https://0bin.net/paste/Syl1pi1m#H5XeVZh6aCTk4CWevFO28L6Y9WDVYbbIP419dGfA-df On Linux curiously the exact same program works. So weird. As you see, threads 15 and 16 are doing an operation at 0xda33452f878 , some kind of contention i guess. And, threads 17 and 14 are doing a mutex op on 0xda2a38a0a50. I presume this is a locking related bug, that for some magic reason only appears in this particular environment, i have no idea why =))) What do you think of the issue?

Version?

This error is reproducible, it kicks in after about 9 minutes. The reproducibility is 100%, it happens every single time in openbsd

Didn’t we discuss this? V4, not production, known issue.

hm ok. What is the timeline for V4 to be useful?

It will appear on any platform. The implementation of the node/server is incomplete

I see. bn seems to work fine, though. This issue is not present on bn. Do you know when bn is expected to be solid?

Believe me, it’s present