waxwing's avatar
waxwing 4 days ago
Fwiw i thought more about it last night, chatted with Claude about it, and saw a serendipitous tweet from Matthew Green making similar points in the negative: I don't think what I'm alluding to here is even really possible at all; you can't really 'verify that you're actually getting random output', and not only because of intermediate hashing steps. Wish I were wrong but it really seems that way. There are other interesting ideas around testing, but I want a 'one click' version for a user, and I can't find one unless they generate the entropy themselves and do not mix it with from-device entropy. So dice rolls and similar. View quoted note →

Replies (14)

Entropy verification is indeed a challenge, consider using statistical tests like Diehard or NIST to evaluate randomness, albeit not foolproof.
That’s definitely not true. If you can run a test on the bits (and can extract a few orders of magnitude more bits than you need) before you use them and before they’re hashed/whitened/etc, you can absolutely test for quality. You then take a few sources that you’ve tested and xor/hash it all together and you’d be fairly confident in your robustness. Sadly, many HW RNGs do whiten the output before you get it so it may not work everywhere, but some HWRNGs do test their data before they provide it!
A random picture, at a random location, at a random time, at a random angle, is more trustworthy than a TRNG no matter how it’s implemented. I simply don’t care what mathematicians say. Most of them are poor. The real world has many pitfalls preventing perfection or trueness. The NSA obviously has a close relationship with TRNG manufacturers if only for the trustworthiness of their own systems. It’s anyone’s guess how far that relationship goes. No one who “correctly” implements a TRNG function should be gloating.
Twas pretty funny, and earned you a small zap. Unmarking others in this reply as don't wanna pollute the smart person convo with more "gold"
The interesting thing about entropy is even the seed phrase: twelve twelve twelve twelve twelve twelve twelve twelve twelve twelve twelve twelve Which doesn't seem random, was random enough to not have been tested as a seed phrase until 5 months ago.
mike's avatar mike
GM There are 130 words that can be repeated 12 times to form a valid 12 word wallet address. This doesn't work for any of the 2048 BIP39 dictionary of words as the twelfth word is a checksum and so only a limited number of words are valid. But the most interesting one is the word twelve, which was tested yesterday by moving 1,500 Sats in and out: The wallet address was: twelve twelve twelve twelve twelve twelve twelve twelve twelve twelve twelve twelve You can restore the wallet yourself using any wallet software. Here are the 130 words that this works for: action agent aim all ankle announce audit awesome beef believe blue border brand breeze bus business cannon canyon carry cave century cereal chronic coast convince cute dawn dilemma divorce dry elevator else embrace enroll escape evolve exclude excuse exercise expire fetch fever forward fury garment gauge gym half harsh hole hybrid illegal include index into invest involve jeans kick kite later layer legend life lyrics margin melody mom more morning nation neck neglect never noble novel obvious ocean oil orphan oxygen pause peasant permit piano proof pumpkin question real report rough rude salad scale screen sea seat sell seminar seven sheriff siege silver soldier spell split spray stadium sugar sunny sure tobacco tongue track tree trouble twelve twice type uniform useless valid very vibrant virtual vocal warrior word world yellow
View quoted note →
waxwing's avatar
waxwing 4 days ago
Test before hashed/ whitened: yes. That was my original thought. But as others pointed out, that's not the same thing as what the whole process outputs. Also as per your comment with HWW, isn't it super-hard/impossible to check the *input* to dev/urandom's CSPRNG, in practice? If I'm wrong, I'm very glad, but I think if I am wrong, it's at least *extremely* messy, right?
> Test before hashed/ whitened: yes. That was my original thought. But as others pointed out, that's not the same thing as what the whole process outputs. Sure, but screwing up XORing three validated chunks together is harder than screwing up one. It wouldn’t have saved this case (which was ultimately a failure of having an insecure fallback in the code at all, which shouldn’t be allowed!), but it should be common practice. > Also as per your comment with HWW, isn't it super-hard/impossible to check the *input* to dev/urandom's CSPRNG, in practice? On Linux? Sure, that’s ultimately because it’s post-whitened/hashed which makes it hard to validate. That doesn’t mean it couldn’t be XOR’d with more but probably we should be requiring at least one “validated” input to any RNG for key generation.
waxwing's avatar
waxwing 3 days ago
I think I'm misunderstanding the mechanics. I said 'inputs' to mean exactly 'pre-whitening'. But i have no idea exactly how it would work on Linux. Claude told me yesterday that it's deliberately made almost impossible to do, which makes a lot of sense, if true. Re: 'at least one validated', yep, that seems to be the right general idea.
> I think I'm misunderstanding the mechanics. I said 'inputs' to mean exactly 'pre-whitening'. But i have no idea exactly how it would work on Linux. Claude told me yesterday that it's deliberately made almost impossible to do, which makes a lot of sense, if true. Right, the suggestion doesn’t work on the Linux RNG, it was more targeting embedded devices. > Re: 'at least one validated', yep, that seems to be the right general idea. Right, this part does work on Linux - use the OS RNG but also use mouse movements or whatever and check that you get enough entropy there. Maybe check for a TPM and pull bits from that, etc. Probably at least one source you can either fetch pre-whitening or you can access a success/fail result because the device checks for you.