Bitcoin full node and query server

June 2020

Archive of the libbitcoin Slack #general channel.

Tuesday, June 2, 2020

N
narodnik16:50
what's the news on v4 release?
evoskuil16:51
How’s it running?
N
narodnik16:58
Been working on an anonymous cryptsystem for bitcoin and other currencies. how's everything?
i want to revive the C/Python bindings project when v4 is finished. I don't think they can be autogenerated and manual is the best way.
also there should be a web interface to manage the blockchain / make queries, and a way to extend blockchain indexing with custom plugins that can be hotloaded
I'm working a lot these days doing metrics and stochastic calculations / price models... that sort of thing, I have a few ideas for products
👍 1
N
narodnik18:41
Also I wrote this library for 2 devs working on wallets: https://github.com/narodnik/biji_wallet/ I think we should fix it up a bit more and include it... I'm waiting though for v4 since I couldn't figure out the v3 zmq async loop
👍 1

Monday, June 8, 2020

vdeurzen09:57
I need a bit of help, I'm encountering something I find highly suspicious and don't quite understand. For an address I own, someone transferred a small amount of BTC which was, within the same block, also transferred back to the same address. So, the output in block X is also an input in block X, which seems like it might be an issue 😕 Any thoughts? Libbitcoin, when I tried to automatically move the incoming funds stated it could not find the previous output, which is when I started investigating and found this weird situation
pmienk18:31
There's nothing wrong with a subtree of referential transactions within the same block within the protocol. I don't know what mechanism you tried to use, but it's not that weird a situation to hear about. I have on a number of occasions facilitated the commitment of a transaction by spending the output with a greater fee, thereby 'paying for confirmation'
vdeurzen18:32
I figured out the transaction indeed makes sense, thank you for confirming 🙂
pmienk18:32
mine have regularly been of the form a pays b, so b pays b to facilitate the confirmation of a pays b
vdeurzen18:33
The scary part is, it turns out someone managed to reconstruct their private key because somewhere in the set of transactions from this address an 'r' value / nonce was reused 😕
I was secretly hoping someone would say: "oh, that's the new protocol change, nothing to worry about 😛 "
evoskuil18:34
😳
vdeurzen18:35
I'm glad we had multiple extra tools watching transactions 😳
evoskuil18:37
The only issue with two related txs in one block is that the output must precede the input tx, in serialization order. It’s a stupid consensus “bug”, preventing parallelism for no reason.
(edited)
pmienk18:38
if you didn't form the spend from your address, then there's something up, but if you executed the spend the fact that they both end up in the same block isn't an error.
vdeurzen18:39
It seems like someone managed to reconstruct their private key and tried to (and kind of did) scam us out of a small amount of BTC
pmienk18:40
I don't understand your scenario. 'Reconstruct' implies they didn't have the key without giving a reason as to why they would have lost it in the first place.
vdeurzen18:44
Context: I work on an infrastructure for receiving and subsequently moving around BTC, the received BTC is moved to our cold storage, but we have an address for each user which they can deposit to. This address is reused for all their deposits (yes, this is not recommended, but a single recognizable address for a user wasn't and isn't really an issue for us, our target audience is mostly beginners so ease of use was higher on the priority list). Now, the user doesn't have their key, we manage that so we can send the received BTC to different wallets under different situation (cold wallets, trading wallets, etc). Now, after sending a couple of deposits for a particular user on to our cold wallet, it turns out one of those transactions (which are generated and broadcast with the help of libbitcoin) reused an 'r' value that was used for an earlier transaction from the same address, allowing this user to find their own private key and both deposit + withdraw their money, sadly it was credited to their account before we found the issue, but luckily it was only a small amount 😕
I'm now trying to figure out how the nonce could have been reused, or well .. mostly understand where it comes from and how it is managed 😅
pmienk18:48
sounds like a tough problem
evoskuil18:49
Signing nonce?
vdeurzen18:50
yeah, the 'r' in the signature was the same for two separate transactions
evoskuil18:50
You don’t use deterministic nonce?
(edited)
vdeurzen18:50
both seem to be transactions that have more than 1 input and per-design a single output
evoskuil18:51
We removed to non-deterministic nonce option years ago.
vdeurzen18:51
I use the create_endorsement method to sign for each single input
evoskuil18:52
Is there a nonce parameter?
I’m not looking at code atm
vdeurzen18:53
checking
don't think so though
bool script::create_endorsement(endorsement& out, const ec_secret& secret,
    const script& prevout_script, const transaction& tx, uint32_t input_index,
    uint8_t sighash_type, script_version version, uint64_t value)
it uses sign() which I'm still trying to find
vdeurzen19:01
Grep did not help me, any clue where secp256k1_nonce_function_rfc6979 is defined?
found it, ... note to self, ensure you are in the right folder
pmienk20:36
just took a look - seems like the call to secp256k1_ecdsa_sign is nullptr parameterized and should instead expose that parameter for control.
(edited)

Wednesday, June 10, 2020

vdeurzen18:27
If I sign several inputs from the same address, should I expect the r-value of the signature to be the same? because investigating my issue, I've found there are several occurences of a duplicated 'r' whenever I take several inputs from the same address and sign them individually
pmienk18:31
I took a look at secp internals - not enough to answer your question but enough to say that there is a nonce parameter which isn't required and in which our sign function is providing a nullptr. In my judgement that means that you're getting deterministic behavior
So, not definitive, but I'd expect at best some weak pseudorandomness if not the same value
vdeurzen18:33
I was indeed wondering about that parameter
That would be a pretty serious bug though
evoskuil18:34
AFAICT not an implementation bug, by design. See specification.
pmienk18:37
Certainly it's how secp is expected to behave. If they wanted to require the nonce they should fail without it. But a user controlled nonce is supported by the library to the tune of a 32byte array. I did not look at the specification though
vdeurzen18:37
The transaction being signed, I expect, should also provide a different hash for the data being signed (i.e. the signature hash has different data, though some of it may be the same (e.g. the output script given that it's a deposit to the same address))
But I've seen a number of transactions where the r is identical, specifically this happened several times for a transaction that spends two different outputs for the same address
If we are not in compliance with the RFC it’s a bug, otherwise not.
pmienk19:04
well, my quick skim says we're doing everything there. so yes, we're compliant. the nonce is deterministically constructed.
however, i have no idea how secp256k1 chooses k between [1, q-1] - that would be what impacts r according to spec.
evoskuil19:07
Well prolly something to escalate either to libsecp or the RFC.
We don’t do science, just engineering ;).
pmienk19:09
yep - if r = kG (mod q) and G is your key and q is the prime number, the choice of k is what you want - which according to the rfc isn't related to the nonce.
i guess the rfc covers alternate generation of k - not up to that section
vdeurzen19:12
Within the create endorsement and generate signature functions, the value parameter is defaulted to max_uint64, could that be an issue? I.e. could that lead to multiple hashes that are effectively the same? I mean it shouldn't because they are spending different outputs right?
pmienk19:15
Until I map the process described in section 3.3 of the rfc back to its inputs from our side I couldn't say. It's where I'd start
vdeurzen19:18
    const auto sighash = to_sighash_enum(sighash_type);
    if (input_index >= tx.inputs().size() ||
        (input_index >= tx.outputs().size() &&
            sighash == sighash_algorithm::single))
    {
        //*********************************************************************
        // CONSENSUS: wacky satoshi behavior.
        //*********************************************************************
        return one_hash;
    }
Looks like the default params get me to the 'unversioned' signature algorithm, which has this gem 😛
Slackbot19:18
👻
evoskuil19:19
That was satoshi’s brilliant idea for returning an error.
Slackbot19:19
👻
evoskuil19:20
I “fixed” it once. I was very new. @narodnik patiently corrected me.
vdeurzen19:25
I most often have 0 or 1 as index with at most 1 output and always 'single' signature, so there might be cases I hit this 😕
evoskuil19:27
Is that the condition??
vdeurzen19:40
This is:
if (input_index >= tx.inputs().size() ||
        (input_index >= tx.outputs().size() &&
            sighash == sighash_algorithm::single))
evoskuil19:42
That’s more specific, it’s an index out of bounds condition (logical error).
vdeurzen19:43
indexing is 0 based right?
evoskuil19:43
Yep
vdeurzen19:43
Ok, then I shouldn't hit that 🤔
evoskuil19:43
Right
I forget why input_index is tested against outputs. I end up re-researching it every time I see it.
(edited)
vdeurzen19:44
When the size() of inputs == 2, however, I have a number of cases where both signatures have the same 'r' value within that transaction 😕
The default is unversioned signature hashes, would it be recommended to use a versioned signature hash?
evoskuil19:51
Sometimes defaults are set to bogus because the choice matters and there isn’t a reasonable preference that should be assumed.
vdeurzen19:53
that seems like a rather big security issue to me
evoskuil19:53
What does?
vdeurzen19:54
the reasoning about the defaults
evoskuil19:54
How so?
(edited)
vdeurzen19:54
With a bogus default the program compiles when that parameter is not provided, if there is no default I'll get a compiler error to force me to actually make a choice
evoskuil19:54
You choose the parameters, and get correct results.
There is nothing insecure about the bogus version.
vdeurzen19:56
Sure, but when default parameters are used in several places, a mistake, such as not setting things to versioned or not providing the 'value' parameter, may go unnoticed. From a security perspective these defaults are dangerous, in a library that deals with currency I would expect a compile error is much preferred
evoskuil19:57
In cases where one doesn’t care to provide a version, a reasoned default is provided.
vdeurzen19:58
So, my main question is then, would using a 'versioned' signature provide better security in light of my issue with duplicated r values
evoskuil19:58
I doubt it, but not looking at the code.
vdeurzen19:59
Ok
evoskuil19:59
The version is the same for each sig you apply it to.
vdeurzen20:00
It worries me that a transaction that spends two UTXOs for the same address will possibly have two identical 'r' values for the signatures, when both UTXOs should be different and should not have the same signature hash right?
evoskuil20:02
Seems like a reasonable concern, but not an implementation issue.
IOW, how would we resolve it and remain compliant?
vdeurzen20:05
So for context, it seems that someone is running software that is trying to find and exploit duplicate 'r' values (i.e. multiple transactions coming from the same address). What we're seeing is that anything coming into a compromised address is immediately sent through to another address outside of our control. Now they are not trying to be stealthy they are using the same target address to send their stolen BTC to each time. In fact they also include an output from that address as a secondary input to the 'stealing' transaction (I find this weird).
evoskuil20:06
Beside the point
What you should do is determine whether this is an issue with the specification or an issue of implementation(s).
Either way the impact would presumably be much broader than libbitcoin.
vdeurzen20:08
That is what I'm trying to do indeed
evoskuil20:08
👍
vdeurzen20:29
Now, I'm fairly certain that this software is essentially scanning transactions and storing pairs of (address, r) and when it finds a duplicate 'r' it derives the private key and starts sending anything coming into that address to another address under their control. I'm not sure we are being targeted specifically, I figure an operation like this could work on a reasonable gaming computer with enough memory and storage for the database. If it ever hits an address with a bit more BTC it could have covered the energy / operational costs
The following paper might be interesting regarding my problem: https://link.springer.com/chapter/10.1007/978-3-030-00470-5_29
SpringerLink
We study key leakage in the context of cryptocurrencies. First, we consider the problem of explicitkey leakage occurring on open-source intelligence platforms. To do this, we monitor the Pastebin...
evoskuil20:44
Cool
vdeurzen20:46
their solution is the deterministic nonce we use though 😅
I'll try to get a small reproduction program up to see if I can reproduce generating the same nonce twice with the given inputs
(edited)

Thursday, June 11, 2020

vdeurzen13:38
On testnet, here 3 out of 4 inputs have the same value for 'r' in their signature: https://blockchair.com/bitcoin/testnet/transaction/9a16e7bbfab607a6e3cd70266726094b87248cb8378276c3174ba96df133681e
blockchair.com
Check Bitcoin (BTC) transaction, value: 0.0028416, date: 2020-06-11