February 2018

Hey, looks like I’ve been missing messages in this section. I’ll try and catch up.

Hey @evoskuil... how are you! Thanks!

@Ty the litecoin consensus change pertains to the “time warp attack” - https://bitcointalk.org/index.php?topic=114751.0

meetups about crypto? or something else?

@Ty We can add in the time warp patch as a disabled-by-default hard fork.

@fpelliccioni Bitcoin meetups, with some “blockchain” discussion mixed in.

I think with this line you are enabling boost iostrems to use zlib... although I'm not sure if this is implies performance penalty:

Of what?

the usage of ICU with boost, is it neccesary?

When ICU is required, yes. But that is conditional (not enabled unless you set —with-icu).

oh, nice!

So boost may link ZLib if it finds it, but we don’t require it for iostreams. We set the path so that we get consistent linkage in the case where we have linked ZLib for qrcode.

I didn't see that problem in our platforms... I will check it...

You may not notice it.

I think boost build tells

It would not cause a problem unless you built —with-zlib (for qrcode) and boost linked another version of zlib (because you hadn’t set the path to match).

but I can check if Boost takes a "system" zlib

Sure, it can’t be disabled.

I will check with recent versions... thanks!

But should succeed if there is not zlib (given we don’t use it in iostreams).

Disabling zlib and bzip2 works with Boost 1.66.0. No need for the workaround with that version.

Good info, but as long as we support lower versions it’s easier to just apply the patch generally.

Guys, are the libbitcoin unit tests running OK with GCC (Linux&GCC or MinGW) ? I think I found an error in the libpng unit test. I will send the PR. But I want to confirm with you...

they appeared well to me, will try again today

All unit tests are passing, history is in Travis. MinGW is not yet supported.

the png test is confirmed working here

greetings

@fpelliccioni good find on the png bug. I'm re-testing https://github.com/libbitcoin/libbitcoin/issues/715 and will report back, as I suspect it's solved by the fix

I hope so! Thank you!

yeah, that did it

Nice!

agreed, good find!

😉

there are many combinations that are possible. travis has a good subset, but all configurations would really slow down the CI process

it is tue

at least for now, i think it's up to us to try to stay on top of this. eventually, it would be good to get something additional going. perhaps more travis integrations, not sure yet

@pmienk ah, nice, thanks

Guys, what is the Compact Blocks status? If you tell me what is missing I could work on it.

@evoskuil: Regarding the settings from constants.hpp: Do you want us to create a new settings-class similar to the ones that exist for libbitcoin-protocol, network, etc ? ( see https://github.com/libbitcoin/libbitcoin-protocol/blob/master/include/bitcoin/protocol/settings.hpp )

That will be necessary and you are welcome to do so.

Ok, great. Thanks for the swift answer

Compact blocks is behind the current large chunk of v4 work in my repo, which when merged will connect up with @pmienk work in master to maintain the tx pool index. Once those are complete it will make sense to move forward with cb, but prob not before. One exception would be the math to compute short ids (siphash).

Thanks

@neillm I can't recall, though if I'd put it there it was likely speed of the travis job.

@pmienk ok, that sounds about right, thanks

Roger is now serving instant coffee at roger.coffe: bx fetch-height tcp://roger.coffee:9091 bx fetch-height tcp://roger.coffee:9081 ‘8=mvk8!F=+>$k&-.65NCPp}4e4j)<N3wk{+N!*hM’ 😉

Nice!

@lclc 👍

Let’s get it added to community servers when you can.

Sorry, What is Roger?

Apparently a barista.

Nice

@fpelliccioni Roger Ver, obsessed with buying coffee on-chain 😉

heheheheh

I setup the bitnodes email notification now for ‘monitoring’ but I don’t trust it and it only tells me if they can connect to my node or not What are you guys using to monitor your libbitcoin-server ?

@lclc i'm not using anything fancy at the moment. in the past (running v2), i had a script that would restart the nodes that went down, but that's a bad idea for libbitcoin since corruption is more or less guaranteed on non-graceful stop. i also didn't need it to send me mails because i was unable to investigate before having access, at which point, i can just see that it had stopped

Ok. I start / stop & autostart on failure with systemd (assuming disk corruption gets catched by having the lock file). Seems to work fine so far. But I hope to later integrate something like Prometheus etc to monitor and log to know more about the usage of the server 🙂

ah, ok. for status while running, i just top or htop for the moment.

I hope to not seen any power outages as my server runs in a hosted datacenter

then there's little to worry about on that end :-)

Maybe I should put a warning in the USER_AGENT so people are aware that they are not just trusting me but also the datacenter operator regarding IP logging etc

i think that's a known risk of using any externally public system, no?

Also true. You never know anyway

I just tell people I will steal their money.

well, there's some benefit to having a readily available master node or two up, just not for production uses

I’m just referring to community servers.

right, wasn't sure by the naming if it was one ;-)

Running master is helpful in early identification of regressions not caught by unit testing.

or just testing new features

speaking of which, mainnet1/mainnet2 were lost due to that rebuild zmq regression, but are nearly done syncing and should be back online in the next couple of days

It would be good at this point to look into load balancing/failover for the public servers.

agreed

Address subscription is stateful, and uses the query endpoint, which is a bit problematic.

what if the settings took a list of servers and operations automatically try/retry until one works

Bx settings?

sure

That won’t help zmq callers.

true, they're on their own, or can do the same thing :-P

This is something you do on the server end to increase uptime.

right, but won't work across servers on different networks

Sure it can, as long as they are reachable.

ah, ok haven't tried it. i've use haproxy before for balancing between servers on the same network

Of course if the proxy is down...

heh. if it can truly do across the internet, could run it on some 'always on' cloud node

BS doesn’t really need load balancing, it’s too fast. It needs failover.

haproxy does both, so i don't think it would hurt

With an http endpoint you can probably configure a get-height call and trigger DNS failover from that.

right. sounds good, but didn't expect it would failover that quickly

Can be very low with low TTL

i'm good with trying that. i don't know if that last statement is actually true ;-)

If they do they are broken.

most software is

“must not” is correct

i remember being bit by this: https://bugzilla.redhat.com/show_bug.cgi?id=656014 (not on redhat, just ncsd issues)

Like I said, broken :).

like i said, most software is :-)

Not using DNS failover because there may be buggy clients, therefore creating a more complex dependency and potentially buggy implementation would not be my first choice.

fair enough. but if dns failover doesn't resolve the issue, users are in the same spot (all those users with broken dns caches)

Presumably the tool that monitors the endpoints would remove the failing endpoint from DNS.

or do failover how they do it now: update bx settings

Given this technique is used commercially I don’t think that platform issues would be very common on current environments.

we'll find out soon :-)

BX and client can certainly support an internal round-robin based on config and timeout. But the more general solution would be preferable, and much faster (BX is stateless).

agreed

Ideally you want to subscribe to the heartbeat endpoint, that way you get a push notification, with timing based on config.

true for something that's generally running longer than bx

Right, wallets should use the heartbeat but a dns failover monitor could use either.

@evoskuil what is the last version of Compact Blocks? v2, right?

I’m sorry, I don’t understand your question.

Libbitcoin doesn't implement bip152, that is slated for v4.

You said that and I was confused by v4. You meant Libbitcoin v4 I thought Bip152 v4.

bips don't have versions.

well, Bip152 specifies CB version1 and CB version2

Guys a couple of questions: Libbitcoin now supports up to protocol version 70002, right? With the support of CompactBlocks the protocol to be supported is going to be 70014 (or 70015). Does this imply that protocols between 70002 and 70014 must be supported? (70011:bip111, 70012:bip130 70013:bip133) Or can they be omitted?

@fpelliccioni anything can be omitted. there's no obligation for a peer to do much of anything

These are all properly supported in v3.5. BIP111 is supported by not setting the NODE_BLOOM service flag.

All network protocols up to and including 70013 are fully implemented: https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin/message/messages.hpp

Thanks guys!

I was confused because I found bitcoin/network/protocols/protocol_version_70002.hpp but I did not find the next ones (70011, 70012 and 70013)

See comments in the protocol classes.

let me see

Do you mean Construct a version protocol instance using configured minimums.?

No, I mean that the comments explain why there isn’t a versioned class for every protocol extension.

Thanks Eric! Good idea to share it in the place with most libbitcoin-knowledge 😄

@evoskuil thanks for your email reply, I am excited to work on Libbitcoin!

This is the nature of c++ templates. The array_slice (generalization of data_slice) is a template. The compiler only requires that the template parameter satisfy all of its compiled uses.

Ah got it! So this single argument constructor will be used in an implicit conversion: template <typename Container> array_slice(const Container& container);

👍

Though I'm not sure how explicit would apply to a template parameter, since the parameter is inherently implicit.

I think that my initial answer (implicit conversion) is incorrect. The data_slice is a typedef of array_slice<uint8_t>. It's constructor is a template method, so the type passed defines the type of the parameter (as opposed to being converted to it in the case of implicit conversion).

Hmm. So lets say I declare "data_chunk myChunk", fill it with several bytes of entropy and use it as argument in "bitcoin_short_hash(data_slice data);" When that hash function is called, and the input is a data_chunk instead of data_slice, the implicit conversion happens, using the single argument constructor with template <typename Container> The template for array_slice will take "data_chunk" as "Container type". And so the array_slice constructor will be: array_slice(const data_chunk& container) for this implicit conversion.

Yes, that's correct (my initial response). there is a constructed instance of array_slice<uint8_t>(const data_chunk& container) passed to bitcoin_short_hash(...).

And to be clear, you have both implicit conversion (to array_slice/data_slice) and template definition (defining Container) being used to pass the parameter to bitcoin_short_hash(data_slice value).

👍 perfect - great learning, thank you!

NP, good questions.

master has been updated with latest v4 work in progress

nice work on sync progress, particularly with bn!

@evoskuil and Libbitcoin maintainer team, I have drafted a intro chapter on working with byte containers in Libbitcoin: https://github.com/jachiang/LibbitcoinDocumentation/blob/master/02_WorkingWithBinaryData.md Shall I push directly to the wiki so change history are captured there? Comments & guidance welcome! I will move on to EC types/operators next ...

@evoskuil @neillm where could I potentially start to learn more about the block memory-less Libbitcoin daemon? Such as, how it is even possible to operate without tx-mem and block-mem pool? Thx in advance.

@James C. @evoskuil has all of the details, but the idea is that instead of storing those in physical memory, they are placed in the store just like 'valid' transactions and blocks. it does consume more space, but simplifies the architecture

@neillm ah super interesting! So more memory allocated to other syncing processes... at the cost of slower access to mem/tx pool stored on SSD. Strong chain refers to the mined chain?

currently mined chain, yes, although it's susceptible to re-orgs of course

👍 typo corrected. I will check in a couple hours and make any adjustments. thx neill!

search "short_hash secondHash = bitcoin_short_hash(myArray); //ok" <-- myArray should be myChunk

@evoskuil: 👍 testnet syncs quickly via bs. didn't get an exact timing, but it was < 20 minutes

Sorry, just saw this. slack is so lame with notifications.

I'd recommend making example code conform to style: https://github.com/libbitcoin/libbitcoin/wiki/Style-Guide

The images are very nice looking, but are not searchable or diffable, and very hard to edit (especially for other contributors). In the case where you are presenting a textual idea (such as the existing examples) I would stick to formatted text. I have some experience with this in the bx wiki:

These are out of date for the recent/v4 electrum additions.

And for array: byte_array<4u> myArray{ { 0, 1, 2, 3} }; vs. byte_array<4u> myArray = {0, 1, 2, 3}; (the initializer syntax is different for vector)

I recommend avoiding numbered titles. Renumbering will get hard and it will drift.

@James C. 👍

👍