Bitcoin full node and query server

August 2019

Archive of the libbitcoin Slack #general channel.

Friday, August 2, 2019

Any possibillity to use libbitcoin to produce ethereum keys? As describe above https://www.freecodecamp.org/news/how-to-create-an-ethereum-wallet-address-from-a-private-key-ae72b0eee27b/ its similar, just need to add keccak256 and 0x

Developer News

How to create an Ethereum wallet address from a private key

In the first article [https://www.freecodecamp.org/news/how-to-generate-your-very-own-bitcoin-private-key-7ad0f4936e6c/] of this series, we generated a bitcoin private key: 60cf347dbc59d31c1358c8e5cf5e45b822ab85b79cb32a9f3d98184779a9efc2. Here, we’ll use that key to get the public address and then the Ethereum wallet address of that private key. Creating the Bitcoin wallet address from the private key is a bit complicated. Here, the process will be much simpler. We need to apply one hash fu

Sunday, August 4, 2019

narodnik02:55

I have a whole bunch of cryptographic primitives I'm developing. I'm just unsure whether adding them is outside the scope of the library. Also there's nothing stopping you using the functionality needed for Ethereum related code.

evoskuil03:42

Primitives are cool. I'd prefer to not include any explicitly Ethereum features however, since that's the line we've explicitly drawn for the library (Bitcoins only, where the distinction is based on https://github.com/libbitcoin/libbitcoin-system/wiki/Cryptodynamic-Principles). Not a judgement, just a question of controlling scope creep.

narodnik03:54

The problem is that much cryptography, especially cutting edge stuff is highly dependent on application and not generalizable enough yet.

evoskuil04:04

Then probably should not include. Sounds like libetherium.

narodnik11:58

https://en.cppreference.com/w/cpp/language/attributes/nodiscard

narodnik15:23

my crypto is not ethereum related, it's for bitcoin but not core bitcoin.

narodnik20:30

@evoskuil looks like mmap api and other things coming to c++20 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1031r2.pdf

Monday, August 5, 2019

Edward Boggis-Rolfe11:17

Hi I am brand new to this site, can someone tell me how compatible libbitcoin is with other bitcoin flavours such as bitcoin sv?

neillm12:23

@Edward Boggis-Rolfe Forks can be implemented and toggled via configuration (see https://github.com/libbitcoin/libbitcoin-server/blob/master/data/bs.cfg#L189). I'm not aware of anyone adding SV fork support

evoskuil12:24

Wallet features of -system and -explorer are probably fairly complete.

(edited)

👍 1

Edward Boggis-Rolfe12:25

Thanks, I did see some breaking changes in bitcoin abc's secp256k implementation which broke some of the unit tests, which were reverted in sv's, I am interested in running tests against various test nets, but as encryption generates gobbledigook it is better to know the differences at source!

evoskuil12:27

I don't follow the comments on secp256k1, can you elaborate?

Edward Boggis-Rolfe12:27

I am a keen implementor of code that use cpp package systems, currently I am using this which works well: https://docs.hunter.sh I have done an initial version of secp256k implementations and would like to set my attention to the libbitcoin implementation.

12:27

ok one sec, let me find the break..

12:30

ABC are shoving "ECDSA+Recovery " into an optional pararameter of the call to noncefp

12:31

I do not know enough about its implications

12:31

I guess it is a nonce on the nonce

evoskuil12:32

I don't foresee us making any changes to this or other crypto primitives.

Edward Boggis-Rolfe12:33

Interestingly ABC's implementation includes a schnorr implementation

evoskuil12:34

We've added primitives, such as scrypt, but changing a published library is another issue altogether.

Edward Boggis-Rolfe12:34

I think it would be good if there was only one version of secp256k that everyone could agree on!

12:35

I rattled off a package for this library that implements a hunterised version: https://github.com/edwardbr/secp256k1

evoskuil12:35

Yes, it would be wise of those who want to add to it to layer on additions, not to fork the library.

Edward Boggis-Rolfe12:35

I am interested in doing the same to libbitcoin-system if it makes sense

12:36

I am keen on a master cmake file pulling in all its dependancies without needing to manually download and link things

(edited)

evoskuil12:37

We've had people create packages before, but nobody has made the effort to produce the generating code necessary for them to be properly maintained. So they've all been dropped due to lack of support.

(edited)

Edward Boggis-Rolfe12:38

as the secp256k library should be static then so will this. I need to make it compile on windows and various mobile os's as well

evoskuil12:38

We have a generated installer that pulls down and builds all libraries, for each repo.

(edited)

Edward Boggis-Rolfe12:41

Cool used it, nice, however I am keen on using something that is not specific to your set of tools but is generic to all c++ code base, so with the minimum of effort you could link in libbitcoin into doom for instance with only 5 mins of configuration. Another solution that has traction is conan.

evoskuil12:42

Packaging would be great, but in order for it to last it must be generated and added to CI.

(edited)

Edward Boggis-Rolfe12:42

This is still work in progress I should have it working on linux by the end of the day: https://github.com/edwardbr/libbitcoin-system

12:43

do a compare against your repo to see what I have done, once it works on mac, windows and android it might be useful for people who want c++ packaging

12:45

Once it gets past my initial integration tests I will be chucking it on my on CD system

neillm12:50

@Edward Boggis-Rolfe Just FYI, most of our build artifacts are generated by this repo: https://github.com/libbitcoin/libbitcoin-build

Edward Boggis-Rolfe12:53

I will have a look. One feature with hunter is that the dependencies are defined at the top level project, so if your app has two different libraries that have dependencies against two different versions of boost, the developer can override both to one that works for both.

12:54

have a look at https://github.com/edwardbr/libbitcoin-system/blob/master/cmake/Hunter/config.cmake

neillm12:55

I don't use cmake. The installer @evoskuil mentioned is the generated install.sh in each of our repos. This has been sufficient for dependencies, as we don't do multiple boost version deps or anything odd like that

Edward Boggis-Rolfe12:56

cmake is horrible, however more recent versions have much improved, regrettably it is becoming the standard for want of anything better 😞

evoskuil13:30

We generate cmake build artifacts for each repo.

Edward Boggis-Rolfe13:36

Thanks for your feedback, as my project is well advanced I will continue with what I am using, it is easier when targetting different os's concurrently especially windows, no automake, configure etc. I may even target a few of your other repos as well when it makes sense.

13:37

If you would like a demo, I would be happy to show you

evoskuil13:38

Windows supports autotools, we just don’t yet have a non-VS build for it.

(edited)

Edward Boggis-Rolfe13:38

using cmake you do not need to

13:39

It is a meta make system

Edward Boggis-Rolfe15:52

Your api is a pleasure to use! Got the hunter package working including tests, though currently without the icu, png and qrencode components

👍 1

Edward Boggis-Rolfe17:29

I see that max_null_data_size is hard coded to 80, the USP of bitcoin SV is to have almost unlimited OP_RETURN sizes, would it be possible to have this as an optional variable so that it is possible to use larger values? Not urgent as I am planning to be hashing all my stuff initially

evoskuil17:30

I think there is a comment somewhere to expose this in config as policy.

Edward Boggis-Rolfe17:37

would it be possible to have an optional dialect config avaliable to the script class constructor? Also perhaps to the create_stealth_script function.

(edited)

17:39

alternatively a template class

evoskuil17:41

Can you explain why?

Edward Boggis-Rolfe17:42

Ideally I would be able to connect to more than one block chain implementation at a time in the same binary

evoskuil17:42

More than 256 op codes?

Edward Boggis-Rolfe17:43

I think bsv have un disabled a couple, however I am referring to the 80 character limit to the OP_RETURN which can be very large in BSV to carry images documents etc

evoskuil17:46

That would be carried via config, which is a cross-cutting concern. Hard to say without a closer look where it would be injected to enforce policy, but the script interpreter is policy unaware.

Edward Boggis-Rolfe17:48

I see this size check, which is valuable, however if I am connecting to bitcoin and bitcoin sv in the same binary it needs to be set to 80 in the first and 100k (or whatever) in the second. In that situation this field cannot be a constant

Untitled Plain Text · 231 B

bool script::is_pay_null_data_pattern(const operation::list& ops)
{
    return ops.size() == 2
        && ops[0].code() == opcode::return_
        && ops[1].is_minimal_push()
        && ops[1].data().size() <= max_null_data_size;
}

17:50

Sorry I have to leave to a c++ meetup..

17:50

afk

evoskuil17:53

That’s not invoked by the script processor.

Edward Boggis-Rolfe21:12

forgive me I am only on day 2 at looking at your library. I will look into it more

(edited)

evoskuil21:13

no worries. i'd be more concerned about deviations from consensus behavior of the Bitcoin op codes.

narodnik21:14

@Edward Boggis-Rolfe you said cmake is becoming a standard, but I've seen lots of new projects are using meson now. What do you think of it?

21:15

And do you prefer hunter over conan or vcpkg?

Tuesday, August 6, 2019

Edward Boggis-Rolfe08:10

cmake has by far the biggest following. I am using hunter because I know it and it works for me. However if you write cmake avoiding bad practice etc your code can be incorporated into most cmake package managers with very little effort. I am not a cmake expert by any sense of the word, however for every C++ library that I am using I am making a compliant cmake file so that it keeps my main project clean, and the packages reusable for other projects that I want to use in the future. So the secp256k library that I have done should work in conan with little or no effort.

08:12

The only thing that I am not sure about is the dependency management between packages in a project, I am sure that how conan does it will need a different approach to mine when I worked on your libbitcoin-system package

08:14

There are a lot of useful videos out there e.g. https://www.youtube.com/watch?v=sBP17HQAQjk

YouTube

CppCon 2018: Robert Schumacher “Don't package your libraries, write packagable libraries!”

Artem11:14

can anyone help i use example https://github.com/libbitcoin/libbitcoin-system/wiki/Altcoin-Version-Mappings#11-ethereum-eth-bip-3944-technology-examples where i can find kec256 implementation using in libbitcoin exploler? is it work good? some online keccak256 give different result

neillm11:40

@Artem Unavailable: https://github.com/libbitcoin/libbitcoin-explorer/wiki/Hash-Commands

narodnik11:50

@Edward Boggis-Rolfe thanks for the info. waiting for c++ to settle on anything so we can have some working standard build system + package manager.

11:50

the video looks great

Edward Boggis-Rolfe13:38

I agree, however things are beginning to solidify by critical mass, also C++ 20 will introduce modules will help.

Artem13:50

@neillm but how they do % echo “radar blur cabbage chef fix engine embark joy scheme fiction master release” | bx mnemonic-to-seed -p “” | bx hd-new | bx hd-private -d -i 44 | bx hd-private -d -i 60 | bx hd-private -d -i 0 | bx hd-private -i 0 | bx hd-private -i 0 | bx hd-to-ec | bx ec-to-public -u | sed ‘s/^..//’ | ./kec | sed ‘s/^.\{24\}//’

(edited)

neillm13:51

@Artem That's a sequence of shell commands. You should be able to run that on the command line.

13:52

It doesn't include the hash function you're asking about though ...

Artem13:53

Oh bad( i dont understand step from publick key to address algorithm radar blur cabbage chef fix engine embark joy scheme fiction master release ed37b3442b3d550d0fbb6f01f20aac041c245d4911e13452cac7b1676a070eda66771b71c0083b34cc57ca9c327c459a0ec3600dbaf7f238ff27626c8430a806 xprv9s21ZrQH143K2weTjKTSMXUM1qmfYo2iDQGPrzsbirKyf9Qn325C8DtapD8dwUL2PU8ciZ9hYVSL4Q9VkygWBosS8FMuX65QqxZQmBDYSEq xprv9vTgjZ7XMB6PSvXz5HeF2xw7GgqpjmTgy76jrNkUkZFfWzKAD1GMyxoMhJCKgCH3WKw3yhXSBuP4Gxmwp5V4ZQVv1wSV7ARQPacbLkwNdBp xprv9w29xFAdfTWFMXugTrP7x1X8c75gYKZFUEhGFqhbGxAwpJnWSo1QZUWPgr2XRKsdsdFPiKKaixbj1gZ2r63NS4EKUkjfii41gWKC5VU8gsh xprv9zWvV2FbEX7VzS96sZG1dZ7MY2j4iFDB62kfuwU95q22omhL7EpQGutw9GyRa9tic3uRnCVZXMLApeDrKb9qJrrUKZzG5dDX29BZwLaTmGZ xprvA1Qqm9XD31nDSB9iYUf4YN6Ycg2Yxey9ecZeDt4rsqUfTqjSoxsW65Ds49xftNDuxDaGUwk9ZtL3sUYeRCEHLjuU95shHpmLwbwM1nVxbXf xprvA2xEQ2iTe9QB22rvf5cbfpUxEBmMdvc7stEFxLhiMXmdLrwLbqugPCHRZiRfEq2puC5vTgwyFneV38hppF8oTf9aoaUv7M8u2XvnACTe6r4 b96e9ccb774cc33213cbcb2c69d3cdae17b0fe4888a1ccd343cbd1a17fd98b18 0405b7d0996e99c4a49e6c3b83288f4740d53662839eab1d97d14660696944b8bbe24fabdd03888410ace3fa4c5a809e398f036f7b99d04f82a012dca95701d103 05b7d0996e99c4a49e6c3b83288f4740d53662839eab1d97d14660696944b8bbe24fabdd03888410ace3fa4c5a809e398f036f7b99d04f82a012dca95701d103 0AB3387A148B3C4B18C333FCAC39B311DCEB2A4B2F5D8461C1CDAF756F4F7AE9 AC39B311DCEB2A4B2F5D8461C1CDAF756F4F7AE9

neillm13:53

@Artem: @skaht may be able to help

Artem13:54

we got xpub : 0405b7d0996e99c4a49e6c3b83288f4740d53662839eab1d97d14660696944b8bbe24fabdd03888410ace3fa4c5a809e398f036f7b99d04f82a012dca95701d103 erasing 2 symbols 05b7d0996e99c4a49e6c3b83288f4740d53662839eab1d97d14660696944b8bbe24fabdd03888410ace3fa4c5a809e398f036f7b99d04f82a012dca95701d103 Address: 0AB3387A148B3C4B18C333FCAC39B311DCEB2A4B2F5D8461C1CDAF756F4F7AE9

13:54

i dont understand steps between xpub and address

neillm13:56

Hrm, looks like @skaht added the ./kec command in there, which is an unknown. He'd have to chime in, I'm not as familiar with his alt mappings

Artem13:56

./kec is keccak256 algorithm

neillm13:56

Sure, but it's not a part of libbitcoin

Artem13:56

but there is something more

13:56

yeah

13:56

its eth shit

13:57

and some more steps skipped

13:57

inside it i guess

neillm13:58

You mean the command line he documented doesn't reproduce the result shown?

Artem13:58

yeah sure

13:58

its not just keccak256 hash

13:59

also i dont understand there is this util from ./kec

(edited)

neillm14:00

Ok, hopefully @skaht can shed some light

Katrina Swales20:32

Hello

neillm20:32

Hello

Katrina Swales20:37

I am trying to compile libbitcoin-system using a copy of secp256k1 that was installed to a custom prefix, but it seems that libbitcoin-system's configure can't find it. I am using travis and tryign to get them both to compile to a cached directory so I don't have to compile theese every time)

neillm20:39

Is it possible to use install.sh from libbitcoin-system into a custom prefix and instead use that dir as the cached one?

20:39

That way, all deps would already be available

Katrina Swales20:41

That makes much more sense, and less stupid then I was doing it,. thankyou

neillm21:02

No problem, good luck

Wednesday, August 7, 2019

narodnik01:50

@Katrina Swales it works for me. You have to set the environment variable PKG_CONFIG_PATH so the configure step can find the path to the custom installed secp256k1

👍 1

01:50

I set these 2 variables in ~/.bashrc export PKG_CONFIG_PATH=/home/user/usr/lib/pkgconfig export LD_LIBRARY_PATH=/home/user/usr/lib

01:51

also you need the switch --enable-module-recovery (or similar, check ./configure --help output of secp256k1) so like ./configure --prefix=/path/to/prefix_secp --enable-module-recovery

👍 1

Edward Boggis-Rolfe10:03

Hi if I have a txid is there an easy way in libbitcoin of getting the merkle proof for it?

neillm11:41

@Edward Boggis-Rolfe Not directly, but in v4/master, you can use fetch-header and fetch-block-hashes and determine this in a manner similar to this: https://learnmeabitcoin.com/glossary/merkle-root

narodnik11:54

@Edward Boggis-Rolfe yes generate_merkle_root function

neillm11:54

^ Heh, sorry, was assuming over the network.

narodnik11:54

$ grep -r generate_merkle_root include/
include/bitcoin/system/chain/block.hpp:    hash_digest generate_merkle_root(bool witness=false) const;

Edward Boggis-Rolfe11:56

Cool! thanks! Still learning a lot!

Katrina Swales14:19

And I am getting weird error from install.sh

neillm14:30

@Katrina Swales Can you paste/post it?

Katrina Swales14:30

make[1]: Leaving directory '/home/travis/build/nekosune/data/tmp/build-libbitcoin-system/secp256k1'
********************** Local nekosune/data **********************
/home/travis/build/nekosune/data/tmp/install.sh: line 148: ./autogen.sh: No such file or directory
The command "sudo ${TRAVIS_BUILD_DIR}/tmp/install.sh --prefix=${TRAVIS_BUILD_DIR}/usr/ --build-boost" failed and exited with 1 during .

neillm14:33

Looks like some kind of path confusion. Either the repos weren't checked out properly, or somehow it's running from a dir it's not expecting to be running from

14:33

Also, sudo should not be used. Is that required for something on your end?

Katrina Swales14:34

I was following the https://github.com/libbitcoin/libbitcoin-system#debianubuntu version, it sdeemed to imply the install script is what would download everything else

14:34

found the problem

neillm14:34

I also recommend instead this link using the install.sh: https://github.com/libbitcoin/libbitcoin-system#building-icu-zlib-png-qrencode-andor-boost

Katrina Swales14:35

I did

neillm14:35

Of course, if you don't need/want those deps, don't use them

Katrina Swales14:35

yah

14:35

I foudn the problem though

neillm14:35

Ah, ok. I just meant in terms of the install.sh line

(edited)

Katrina Swales14:35

install.sh has special casing for running on travis

14:35

they seem to assume if it is running on travis, it is their end trying to build it

14:36

https://github.com/libbitcoin/libbitcoin-system/blob/master/install.sh#L772

neillm14:36

Hrm, ok, so you could unset the $TRAVIS var?

Katrina Swales14:37

I don't know if travis would let me

14:37

or if it wouldnt break further stuff later

neillm14:37

Maybe TRAVIS= install.sh -- ..

14:38

Or maybe just change the build_from_travis call for libbitcoin-system to build_from_github then run it like you have it?

14:38

https://github.com/libbitcoin/libbitcoin-system/blob/master/install.sh#L796

Katrina Swales14:38

probably will have to, which is annoying

neillm14:39

Agreed. The travis env var is probably the better way to go in this case

Katrina Swales14:39

will play about with it, Worried about future scripts that dont think they own travis , but need to know if running on travis

neillm14:40

Sure, I don't mean to unset it globally -- just for the call to install.sh

Katrina Swales14:40

or I could be midsunderstanding bash again -_-

neillm14:40

So even TRAVIS=false install.sh --... should work

Katrina Swales14:40

doing it on same line only unsets for that line, doesnt it

neillm14:41

Yes, and in this case, for the install.sh where we want it unset, no?

Katrina Swales14:41

yeah

14:41

Trying now

👍 1

14:42

Sorry if I seemed grumpy about the install script btw

neillm14:43

No problem, it takes some getting used to

Katrina Swales14:50

TRAVIS command not found .,..

14:50

bashj is annoying

Edward Boggis-Rolfe15:00

Hi if I have ICU do I need boost-locale? I have excluded boost-locale in my cmake as it tends to wreck havoc in packaging systems as it changes the behaviour of various boost libraries and they are not aware of it. libbitcoin-system compiles fine without it.

neillm15:01

@Katrina Swales That's odd, env vars like that should work, at least at the command line. I may be missing something about Travis environments, but I thought they worked similarly

15:02

@Edward Boggis-Rolfe Yes. Or rather, we build boost-locale with ICU support if requested

Edward Boggis-Rolfe15:03

hmm I will need to check the mnemonics to see if I have broken anything

neillm15:03

@Edward Boggis-Rolfe If you're just using english, it may appear that everything is ok, but proper normalization requires the ICU support

Edward Boggis-Rolfe15:04

I have ICU compiled into libbitcoin-system but not boost-locale, compiles fine, concerned that I may have broken something without knowing it

15:05

hunter_config(libbitcoin-system URL https://github.com/edwardbr/libbitcoin-system/archive/master.tar.gz SHA1 d1de11c54e1b387ee74809cfa00e4b35331a002b CMAKE_ARGS ANDROID_STL=${ANDROID_STL} ANDROID_ABI=${ANDROID_ABI} ANDROID_PLATFORM=${ANDROID_PLATFORM} CMAKE_ANDROID_ARCH=${CMAKE_ANDROID_ARCH} with-icu=YES )

15:05

using the icu flag

neillm15:06

Try to run the unit tests, if they fail, it's probably not built properly. libbitcoin-system should be statically linked with boost-locale, which should be statically linked with ICU when requested

15:06

Other linkages are supported of course, but are sometimes trickier to determine if something is going wrong

Katrina Swales15:22

it worked! now I just have to make my travis script recognize if there is a cached dir and only run install if there isnt 😄

neillm15:22

@Katrina Swales That's great! What was the final env var issue?

Katrina Swales15:23

honestly, I don't know, for now I am just using an edited install.sh. I have spent too much time on this when I need to move on to working on the program I am working on, I can spend time doing it propoerly once I have sometrhing working

neillm15:23

Gotcha, ok. Well glad you have something going for now

Katrina Swales15:26

I think checking if cachedir/include directory exists should give a good clue if the cache exists and was used

neillm15:35

@Edward Boggis-Rolfe If you happen to get explorer built with your libbitcoin-system lib, you could run through some of these examples to make sure it's built properly: https://github.com/libbitcoin/libbitcoin-explorer/wiki/bx-mnemonic-to-seed#example-10

Edward Boggis-Rolfe15:37

I may have to thanks

Edward Boggis-Rolfe19:56

compiling with clang8 I am getting a few warnings e.g. warning: destructor called on non-final libbitcoin::explorer::commands::address_decode' that has virtual functions but non-virtual destructor

(edited)

narodnik20:17

@evoskuil the ec_scalar uint64_t constructor was removed... what is the recommended way to convert primitive unsigned integers to EC scalar values now? what is the problem with the constructor. would it be better if it was

template <typename UnsignedInteger> explicit ec_scalar(UnsignedInteger value) { ... }

20:19

or a free function called

value_to_scalar()

(but the constructor is better)

evoskuil20:39

If we want implicit convertibility it should be understood what is the canonical mapping. This implies a downcast operator, so not sure what the assumption would be on loss.

narodnik21:41

There is a canonical encoding which is standard (the openssl one for how values are embedded into ec_secret)

21:43

you can see this by for example setting the bit one and repeatedly adding to itself. if you set the lowest bit (at byte 0) then it will overflow and become x = n - 1, x + 1 mod n -> 0 mod n

21:43

or maybe I misunderstood your question? I would make it an explicit constructor.

evoskuil21:48

How is it assigned to 64 bit?

21:49

Is it essentially a uint265_t?

21:50

Is not using a 64 bit seed very unsafe?

narodnik21:59

it's not a seed, it's encoding a value