View in context in the libbitcoin Slack #general mirror.
batch_ecdsa_* 152 MB total (identity 7, key 37, message 36, signature 72)
batch_schnorr_* 130 MB total
~283 MB, not sparse (apparent == allocated)batch_ecdsa_signature.data at 74,573,056 bytes ÷ 64 B/sig ≈ 1.16M rows — i.e. the table settled at roughly the 1M batch target, not the 100M I configured. So ecdsa_size = 100000000 did not translate into 100M rows of resident allocation, and the memory-pressure-from-preallocation theory has no support. The RAM pressure is real (mean working set 39.9 GB, free RAM <2 GB for 48.7% of the run) but it comes from the ~890 GB archive store being mapped, not from the batch tables.
Caveat: those are post-purge sizes, so they bound the resting state, not necessarily the peak.
Re-plotting at 4× finer resolution since he flagged the sampling rate:
Correction to my own earlier figures — they were contaminated by the post-completion idle tail:
Sending the 15s plot, but the numbers matter more — my earlier CPU figures were wrong.
I was wrong about CPU, and evoskuil is closer to right
My earlier stats included 5,115 s of post-completion idle — the node hit maximum_height at 15,406 s but the sampler ran to 20,528 s. Restricted to the actual run:
contaminated run window only
bs CPU mean 35.4% 46.9%
median — 54.1%
p90 — 73.7%
max 76.2% 76.2%
at/near 75% cap 6.6% 8.7%
below 15% 37.6% 17.2%CpuPercent would buy you very little" was wrong. It is now the single most obvious thing to test.
What is left after that
stall episodes 51 median 30s p90 135s max 165s total stalled 2,520s = 16.4% of the run free RAM < 2 GB 64.6% of samples
batch_ecdsa_signature.data at ~1.16M rows, i.e. about the 1M batch target rather than the 100M configured. No 15 GB, no support for my memory-pressure theory. Worth noting his own suggested ecdsa_size = 100000000 against batch_signatures = 1000000 is 100× the target, so "size in relation to the batch target" argues for ~1M there.
Schnorr still looks anomalous. His model says schnorr is faster per signature and always the smaller batch, so it should be a rounding error. Measured: schnorr 1,740 s over 1,137 batches versus ecdsa 2 s over 3,111 batches. Smaller batches, faster per-sig algorithm, 870× the total time. If ultrafast were merely slower at 1M it should hurt ecdsa at least as much — it did not, ecdsa actually improved from 11 s to 2 s. That points at the schnorr path specifically rather than at batch size.
...