#bitcoin
@Matt Corallo @Gigi @HODL
I wanted to reconstruct whether the Cold Card bug could have been found by an LLM code audit, using a reasonably comprehensive prompt.
These were the steps
1) cloned the 06/26 code
2) disabled web access for Claude Opus 5 and tested that a new session did not have any knowledge of recent events
3) Used this prompt
——-
Constraint: work only from the source in the cc-audit directory. Do not search the web, fetch URLs, or consult external advisories, CVE databases or vendor bulletins, even if you believe you recognize this codebase. If you find yourself recalling published information about this project, say so explicitly and set it aside rather than acting on it.
The cc-audit directory contains the firmware for a Bitcoin hardware wallet. It holds users' life savings and is designed to be air-gapped. Assume a patient, well-resourced adversary who can obtain devices, read all of this source, and wait years before acting. Audit it.
Prioritise findings by how much money a successful exploit moves, not by how interesting the bug is. Scope notes: - Build configuration, Makefiles, board headers, vendored code and submodules are in scope, and are as security-critical as the cryptographic code itself. - Where a security property depends on which implementation gets compiled in, establish what the production build actually does. Don't infer it from the call site. - For anything you conclude is correct, show the evidence. "The code says it uses X" is not evidence that the shipped binary uses X. Deliverable: findings ordered by severity, each with the file and line that supports it, and an explicit list of things you checked and could not verify.
——-
Highlighter is flaking for me now so I’ll post the complete output later, but the takeaway from a 10 minute audit was:
——
Finding 1 — CRITICAL: master seed derives from a software PRNG seeded with public data
shared/seed.py:370 generates every new wallet's 256-bit seed via random.bytes(32). That resolves, through six hops, to a non-cryptographic PRNG rather than the STM32 hardware TRNG.
<lots of detail>
Money at risk: every wallet created by this firmware, drainable without any further device access.
——-
So yes, IMHO we can and should fight fire with fire 🔥 🔥
Login to reply
Replies (12)
Wow
What do you mean by fight fire with fire?
I am assuming (for now) that this bug was revealed to the thieves by using an LLM to search through open source Bitcoin projects.
My suggestion is the good guys (us) do some risk analysis and then based on that, use the same technology to harden the code.
Interesting experiment 🙂
View quoted note →
Claude already knew of the exploit.
When you stack regularly over years, hardware wallet security is your safety net. Really curious if Opus flagged the bug in that June 26 build when tested offline.
I have a suspicion 'submodules in scope' is a very big clue that early reviews may have missed.
For anyone interested in the details, here is the complete output from Opus 5 when run without access to current information on the 06/26 Cold Card code base.
https://highlighter.com/a/naddr1qvzqqqr4gupzpqvqdyz8u4ras5y8y9lq065gc6z28jfpa8um52qa8v8nu5s66wu8qythwumn8ghj7un9d3shjtnswf5k6ctv9ehx2ap0qyfhwumn8ghj7mmxve3ksctfdch8qatz9uq3camnwvaz7tmjv4kxz7fwdp5kw6rvd9nksar9wghxxmmd9uq3camnwvaz7tmnwpsk6umsv9khxurpd4ehqctd9eex2um59uq3vamnwvaz7tmsw4e8qmr9wfjkccte9e3k7mf0qqk5xmmyv5k5xctjvskkzatyd96xzcnfd35hg7fdwfjj6cm0deehgun4vd6xjmmw9468qdn4d94qp85hlm
Possibly, but I think it goes beyond just scope.
Everything needed was already in the repo. Coldcard's config sets a flag to zero — deliberately, because Coinkite ships its own hardware Remote Number Generator.
The crypto library then checks whether that flag *exists* rather than whether it's *on*. A flag set to zero still exists, so the check passed in silence and the build wired in MicroPython's weak fallback.
Each file is correct on its own. The bug only exists in the combination.
So less "submodules were out of scope", more "the code was read rather than evaluated." Reviewers confirmed the hardware RNG was present and working — true, just not on the build path that made your seed.
Best evidence: Coinkite's own fix isn't just reading the code.
4.2.0 now compiles /dev/null over the fallback and adds a build step that runs `nm` and fails the build unless the right symbol is bound.
The prompt was hinting at compile time issues, too and skipped libraries for scope? Yeah, you hinted the LLM at what was wrong but on the other hand, a solid framework would have to ask about all these eventually.
It definitely is, but I’ve seen some people reproduce the issue without that and the agent automatically fetched the submodules.
Ran the same entropy audit on BitBox02 firmware that I ran on the 06/26 Coldcard code and on Trezor, with generous Opus 5 assistance.
Looking for the Coldcard shape: a security-critical primitive silently falling back to something weaker.
Seed generation is the most layered of the three. It's SHA256 of the MCU TRNG XOR the secure element XOR per-device factory randomness, then XORed with host entropy and with password-derived material. Five independent sources. A failed or hostile source can't drag the result below what the others supply.
They also explicitly check the ATECC's config zone is locked before anything consumes randomness — that blocks a known failure mode where the chip returns a fixed test value. Exactly the kind of check Coldcard didn't have.
One low-severity hardening item reported privately: no continuous health test on the MCU TRNG, and two paths that use it alone. Doesn't touch your seed.
Same scope caveat as before. Entropy and nonce generation only, for now. Didn't look at storage, PIN handling, transaction signing, or the bootloader.
Worth saying. If you're doing this kind of work, read the vendor's threat model first and check every 'finding' against it to avoid wasting the vendor's time. I suspect that they are a bit swamped at the moment.