Bitcoin full node and query server

July 2024

Archive of the libbitcoin Slack #general channel.

Tuesday, July 2, 2024

evoskuil17:33

I've spent a few days on a performance problem that I encountered when starting to optimize validation through the caching of blocks following download for later validation. After a couple days of frustration, I think I've isolated the issue. As sometimes happens there is both a problem and an opportunity...

17:38

The behavior is that block objects (owned by std::shared_ptr) are passed form the protocols, operating concurrently, into new chaser for the purpose of indexing them. This requires sequencing them into the chaser's strand. So there are objects created on any number of different threads, sequenced into a stranded function which itself can be on one of any number of other different threads at any given instantiation. What I finally concluded is that ~block is taking up about 95% of process time, where previously it was < 1%.

evoskuil17:44

Aside from passing the block pointer around, I hadn't actually added anything else yet (no caching/indexing) of the block. I finally concluding it had to be an issue of lock contention in the memory deallocation. So I looked at whether blocks or other dynamically created objects were previously spanning threads. Turns out that they are not (with some unrelated exceptions, notably address handling and message heads). Network messages are allocated on the same thread as they notify on, and upon which all subscribers receive and process the message (apart from address, which appears to have the same but much less impactful issue). That changed when I posted the block message to another strand, but would also occur if it was to be posted to another function on the same strand.

evoskuil17:50

I was surprised that I didn't see this when I tested concurrent validation. But a quick review shows that transactions are queried, populated, validated, and destroyed on a single thread. Only integral values and a single racer object are passed around. One racer per block is a trivial object destruction, so inconsequential. In other words, through a coincidence of design, so far, object allocation and destruction has been very efficient. So we've got to handle this if we want block/tx/prevout caching for validation, but it also presents unforeseen optimization opportunities.

evoskuil18:06

I tested posting the block back to the originating strand for destruction, but this shows little or no benefit. That makes sense if the problem is in fact contention for deallocation. Also it tried performing the same posts but with a null block pointer (no performance issue). So it only occurs with block destruct on a non-originating thread. There is really no way to post back to a specific thread, and that would produce its own set of significant performance issues. So the possible resolutions consist of (1) not doing it and (2) using a custom allocator/deallocator. #1 is possible by just avoiding the optimization and simply re-reading blocks from the store as before. This performs well on the high memory machine, but not otherwise. So I’m taking a look at #2.

18:08

This may present an opportunity to take on a custom allocator. Allocation and deallocation area material cost, even apart from this issue, and custom allocators can achieve 2-10x performance advantages depending on the scenario.

18:09

I know that @jungly will be excited about this.

evoskuil18:45

One option I've considered is having the store expose an allocator that can be passed to specific objects, such as for tx construction. A store blob table is a memory manager. Allocation is from a memory map - which can be sized via configuration and resized as necessary. When we hit a space limit we would just reset the allocation pointer to the beginning of the map (unlike store tables this cannot be allowed to grow indefinitely). This implies of course that all uses of the allocated memory are first deleted. We track this already for table remapping. However in this case we wouldn't want to track at the level of granularity of each allocation. We would just want to know (for example) that there are no tx objects. That would require the tx construction to obtain the memory object that we currently use, and hold it until destruct. Then we just block new allocations until all are released. As long as work progresses, this results in a periodic delay (like table remap) but makes allocation almost free - and deallocation becomes a nop. Basically this would allow us to allocate all tx object memory on a mmap and strictly control that level of allocation. Blocks and headers are trivial by comparison, and there's really not much else. Allocation would be cheap due to the large singular reservation and deallocation would be free. Memory full behavior would be similar if not identical to virtual memory, as pages would get swapped to disk as RAM becomes unavailable. However it may very well be better, due to the memory locality.

evoskuil19:53

In C++ there are several ways to replace memory allocations. new/delete can trivially be globally replaced, and class/struct can expose static member new/delete (for itself). The major cost comes from std::vector. We barely use std::string and std::array is stack-allocated. All member integrals and arrays are allocations of the containing class/struct. We also need to consider that this technique needs to be strictly limited to objects that we can track (or more specifically can track themselves). So for example, a block can encapsulate a memory tracking pointer and new memory can thus be allocated when all of these are freed, and no new memory allocated while waiting for that to be the case. Within that block all allocations can be based on a replaced allocator.

evoskuil20:06

The downside to using custom allocators other than globally is that objects are not polymorphic. A std::vector<T> is not the same type as a std::vector<T, A> where A is a custom allocator. This is why std::pmr::polymorphic_allocator was introduced in C++17. While pmr types instantiated with a memory resource are polymorphic, they require a constructor parameter, and aren't polymorphic with std::vector<T>:

template <typename T>
using pmr_vector = std::vector<T, std::pmr::polymorphic_allocator<T>>;

class my_memory_resource : public std::pmr::memory_resource
{
    ....
}

void foo()
{
    my_memory_resource mem_res;
    auto my_vector = pmr_vector<int>(0, &mem_res);
    std::vector<int> v;
    my_vector = v;
}

(edited)

✅ 1

Wednesday, July 3, 2024

evoskuil03:10

After some thinking on this it appears that I can most easily and cleanly implement a block/tx object targeted allocator using a new constructor on the fast io streams that we use to construct all objects from the network and store. By defining an interface and optionally accepting an instance on reader construct, the reader can hold and provide the common allocator. It can also provide a shared lock object that allows for reset of the mmap when full, by blocking and waiting until all objects are cleared. Of course these cannot be long-lived objects, or they cause a deadlock. This is the same pattern used by the store presently.

✅ 1