Bitcoin full node and query server
April 2020
Archive of the libbitcoin Slack #general channel.
Thursday, April 2, 2020
vdeurzen12:46
@neillm The gai.conf changes haven't solved the problem sadly. Looks like my hosts cache was completely filled with ipv6 addresses, I checked them and it seems most of them were for an onion service. Could I use the blacklist feature to blacklist an entire ip range?
It seems to accept a prefix in ipv6 format, we'll see if that helps 🙂
N
narodnik15:03
i'm getting much better behaviour from the server on a vps with public IP than on home computer behind a NAT... damn ipv4
Monday, April 6, 2020
A
Alex05:52
I'm syncing a v3 server with DEBUG logging on and I've noticed that the "Connected block [hash] at height [number] from [ip:port]" message skips blocks. For example, although the most recent block synced is 433672, there is no DEBUG message for block number 400028. I've grepped all the logs in the archive plus the current debug.log file. Should I be worried?
Tuesday, April 7, 2020
neillm01:56
If you can query the blocks and it's progressing, there shouldn't be an issue. During sync, I don't think all blocks are logged individually
👍 1
evoskuil03:30
V4 skips logging every block especially early because it slows sync, I’m not sure about v3, but if it doesn’t stall there’s no issue. You will get more logging by using the verbose setting.
👍 1
Wednesday, April 8, 2020
N
narodnik18:19
long story short, my sysadmin rebooted the server during a live sync so now i need to restart
here's my config: https://gist.github.com/narodnik/2687348e9b47cd9acfd40b36ea496057
is there any setting i can enable to just index blocks and avoid checking signatures until last checkpoint? or is that enabled by default?
Friday, April 17, 2020
D
damian19:01
I'm new in the world of bitcoin development. Is this the right place to ask some newbie questions?
D
damian19:02
Then I will start with my first question: How can I get the UTXOs of an address which is not in a wallet?
Check the commands on the right side for more
D
damian19:33
Is there also a documentation about C++ library functions of libbitcoin-explorer available?
neillm19:34
Not exactly, bx is a thin layer around libbitcoin-system, and there are some dev docs here: https://github.com/libbitcoin/libbitcoin-system/wiki/Developer-Documentation
evoskuil19:42
The source code in bx commands is intended to be self-explanatory. One class per command, very little code.
D
damian19:44
When I include #include <bitcoin/explorer.hpp> I get the following compiler error:
(edited)/usr/local/include/bitcoin/explorer/command.hpp:58: Fehler: 'SYSCONFDIR' was not declared in this scope
58 | BC_DECLARE_CONFIG_DEFAULT_PATH("libbitcoin" / BX_PROGRAM_NAME ".cfg")
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Do I have to include anything else? Or do I have to set an environment variable?neillm19:56
Unfortunately I can't look at it right now, but if you paste/attach the code and how you're trying to compile it, I can look at it later tonight
Saturday, April 18, 2020
D
damian06:16
I just included it like this:
#include <bitcoin/explorer.hpp>
and tried to compile. If just I include
#include <bitcoin/bitcoin.hpp>
it complies without any errors.
It use it in a Qt project.
(edited)Sunday, April 19, 2020
D
damian08:20
I still get the compiler error. Do you have any idea what I could do to get rid of it:
(edited)In file included from /usr/local/include/bitcoin/bitcoin.hpp:47,
from ../btc/btcrpclistunspent.h:10,
from ../btc/btcrpclistunspent.cpp:1:
/usr/local/include/bitcoin/explorer/command.hpp: In function 'boost::filesystem::path libbitcoin::explorer::config_default_path()':
/usr/local/include/bitcoin/explorer/command.hpp:58:1: error: 'SYSCONFDIR' was not declared in this scope
58 | BC_DECLARE_CONFIG_DEFAULT_PATH("libbitcoin" / BX_PROGRAM_NAME ".cfg")
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~neillm11:55
Would you mind putting up some minimal code in a pastebin or something and show how you're building this? The error alone doesn't tell me much
D
damian13:39
I just created a new class and included <bitcoin/explorer.hpp> in the .cpp file like this:
(edited)#include <bitcoin/bitcoin.hpp>
#include <bitcoin/explorer.hpp>
#include "btctransactioninput.h"
BtcRpcListUnspent::BtcRpcListUnspent(QObject *parent) : BtcRpcEndPoint(parent)
{
}
And this is how the Qt framework compiles it:
g++ -c -pipe -g -std=gnu++11 -Wall -Wextra -D_REENTRANT -fPIC -DBITCOIN_LIBRARY -DQT_DEPRECATED_WARNINGS -DQT_QML_DEBUG -DQT_NETWORK_LIB -DQT_CORE_LIB -I../btc -I. -isystem /usr/local/include -I../number -I../bcaddress -I../arca-api-crypto -I../arca-api-protobuf/protos -I../QtCryptoHash/lib/include -isystem /usr/include/qt5 -isystem /usr/include/qt5/QtNetwork -isystem /usr/include/qt5/QtCore -I. -I/usr/lib64/qt5/mkspecs/linux-g++ -o btcrpclistunspent.o ../btc/btcrpclistunspent.cpp
In file included from /usr/local/include/bitcoin/bitcoin.hpp:47,
from ../btc/btcrpclistunspent.h:10,
from ../btc/btcrpclistunspent.cpp:1:
/usr/local/include/bitcoin/explorer/command.hpp: In function 'boost::filesystem::path libbitcoin::explorer::config_default_path()':
/usr/local/include/bitcoin/explorer/command.hpp:58:1: error: 'SYSCONFDIR' was not declared in this scope
58 | BC_DECLARE_CONFIG_DEFAULT_PATH("libbitcoin" / BX_PROGRAM_NAME ".cfg")
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If I only include #include <bitcoin/bitcoin.hpp> it complies without any errorsneillm15:41
When compiling, try using something like
g++ ... $(pkg-config --cflags --libs libbitcoin-explorer)?neillm15:49
If you're using qmake, you could add something like this to your
.pro file:
CONFIG += c++11 link_pkgconfig debug static PKGCONFIG += libbitcoin-explorer(edited)
Just make sure that you set the PKG_CONFIG_PATH if it's in a non-standard location, e.g.
export PKG_CONFIG_PATH=/path/to/libbitcoin/lib/pkgconfigMonday, April 20, 2020
D
damian03:44
I added this to the qmake .pro file:
CONFIG += c++11 link_pkgconfig debug static
PKGCONFIG += libbitcoin-explorer
libbitcoin-explorer is installed in /usr/local/lib and I exported PKG_CONFIG_PATH like this:
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
I still get the same compiler error.
I suspect that it doesn't find the file bx.cfg because the compiler error points to this line in /usr/local/include/bitcoin/command.hpp
#define BX_PROGRAM_NAME "bx"
#define BX_HELP_VARIABLE "help"
#define BX_CONFIG_VARIABLE "config"
BC_DECLARE_CONFIG_DEFAULT_PATH("libbitcoin" / BX_PROGRAM_NAME ".cfg")
Do I have to set SYSCONFDIR to the location of the bx.cfg file?neillm03:47
I don't think so, but not sure. I have a qt project that uses libbitcoin-client and it's as simple as what I provided above (except I replaced explorer for your case). Are you sure you need methods from explorer by the way? It's likely you only need libbitcoin-client, or libbitcoin-system, which almost all bx calls use internally.
I'd say you can try setting it though and see if that works
D
damian03:59
I tried it but still the same.
My plan is to read all the unspent transactions of an address which is not in the wallet. Are there methods in libbitcoin-client, or libbitcoin-system which I can use for that?
neillm03:59
Did you read the code for the command I sent when you asked a while ago? That does what you need, and yes, it's a libbitcoin-client call
bx (explorer) is a command line wrapper into the code to do certain things, but it's more or less an example for developers to expand on in their own code. Perhaps that wasn't clear though
If you're using
version3 (prod), check out this call: https://github.com/libbitcoin/libbitcoin-explorer/blob/version3/src/commands/fetch-utxo.cpp#L69In short, the code instantiates a client class and calls that method and returns the result. I recommend that if you're writing Qt code to just use that call and more or less do the same thing, rather than bringing in the bx command line wrappers, etc if not needed.
neillm04:08
No problem, good luck. I think libbitcoin-client and libbitcoin-system is much simpler to use from Qt (and more designed for usage like that). bx is a great tool to see if you can get the results you want, so running the command line program may help you find what you need -- then you can check the code to see what calls it makes and incorporate that into your own code.
That said, if you run
(edited)bx fetch-utxo on the command line and can't find the result that you need, maybe exactly what you need isn't yet supported.D
damian04:38
Since I'm new to the bitcoin world I still have some confusions.
The application which I develop stores the private keys in a HSM and therefore I don't have access to the public keys. There is an endpoint on the HSM appliance which returns the public key and stores the private key. With the public key I create a new bitcoin address. Do I have to import this new address into a wallet, that I can send BTC to this address? Can I get the UTXO's of this address even if the address is not in a wallet?
Sunday, April 26, 2020
neillm00:38
Hrm, all of the community servers apparently went down at some point today. 3 of 4 debug logs (including timestamps) end with
21:51:16.324975 DEBUG [server] Failed to receive query from [0000000000] bad data stream
Segfault
I have a non-public v4 server running that wasn't affected though (as well as a public testnet v4 instance).
flush lock is not present, looks like it's restartable
(writes are sync'd)
All started up properly and are catching up. Let's see how long before it happens again 😅
The idea being it was off during sync, but then on for prod runs
neillm00:51
Since it happened not that long ago today, I'm wondering if it was coordinated (perhaps intentional data sent to all endpoints), or just a buggy impl trying to get online. Either way, not much info at the moment. No core files, etc. If it happens often from here on, I'll see if I can monitor a bit more closely
Well, darn. testnet1 just failed with
00:51:33.198868 FATAL [blockchain] Failure writing transaction to store, is now corrupted: operation fail ed
I'll restore from a snapshot and have it back online shortly.
Doesn't help much, but here's dmesg logging:
mainnet1/testnet1: [23268097.168760] bs[28861]: segfault at ffffffffffffffff ip 000055982a15e3ce sp 00007f2e2a7fba20 error 7 in bs[55982a05a000+383000] [23268097.291379] bs[20330]: segfault at ffffffffffffffff ip 00005574d05393ce sp 00007ef5c27fba20 error 7 in bs[5574d0435000+383000] mainnet2/testnet2: [23266887.882841] bs[1094]: segfault at ffffffffffffffff ip 000055890e4b7cc9 sp 00007fefaf7fd9e0 error 7 in bs[55890e38a000+55d000] [23266888.087727] bs[13956]: segfault at ffffffffffffffff ip 000055f8aa369cc9 sp 00007efc1e7fb9e0 error 7 in bs[55f8aa23c000+55d000]
Just confirming the same times of whatever happened
Resource limitation?
"Error code 7 (0x7) is 4 + 2 + 1, a user-mode write to a read-only mapping, and is what you get if you attempt to write to a string constant in C": taken from https://utcc.utoronto.ca/~cks/space/blog/linux/KernelSegfaultMessageMeaning
But yeah, can't say without better monitoring what was happening. The services have been so reliable, I barely ever monitor, other than watching console output on occasion ;-)
evoskuil05:42
Both mainnet or testnet processes in close proximity maybe, but one mainnet and one testnet, extremely unlikely. Two mainnets and two testnets at the same time? The only bug I could imagine is one that was related to processing a certain time value.
👍 1