Finally designing payer proofs, thanks for prompt @Matt Corallo! This proves that the invoice was requested by you, and paid, in a nice standard format. Fields which aren't needed are omitted, but you can still verify the signature. Was a design aim for BOLT12, but now time to bring it into the world...

Delving Bitcoin

LN Summit 2024 Notes & Summary/Commentary

Hi y’all, ~3 weeks ago, 30+ Lightning developers and researchers gathered in Tokyo, Japan for three days to discuss a number of matters relate...

Tokyo Lightning Summit summary...

Thursday was an amazingly productive #CLN day (took a break from child wrangling during school holidays to go into my coworking space). I've been grinding askrene's min-cost-flow solver against real data: my new toy is a stress test that asks for a route from my node, then asks again with "maxfee" 1msat less than the returned fee. This makes it prioritize fees more. I repeat this until failure, for each of the 100 most-channels nodes in my gossip snapshot. This has been great for tuning the various parameters available. In particular, our linearization approximation for basefee (we need a linear function, which basefee breaks, so we approximate) was all wrong. Also, our mixing function (how much to weight fees vs probabilities) was both *complex* and *suboptimal*, so after a few tests I decided to simply multiply the probability factor by 8, which makes them comparable in practice! (This may change: we really want to compare the medians of each, to determine the factor, but 8 is simple, and reasonable for now). Our algorithm would first run ignoring fees, and if that comes in under maxfee, just return. In practice this is silly: it would sometimes choose the more expensive of two identical paths! So now we start with a 1% fee weighting, to at least have *some* bias. All this testing on real data is giving me more confidence than I ever had about our previous efforts, but there's still more to do before next month's release...

As so often, well put by Greg Maxwell: "The whole point of bitcoin is to exist outside of any centralized control. Satoshi was clearly passionate about this. So much in our lives is mediated through centralized authorities that people seem at times desperate to hand control to someone. The things in our lives that aren't centrally controlled, like -- say-- the English language just get taken for granted. How could Bitcoin achieve its purpose with some founder around? The technology was designed so that the identity of Satoshi is irrelevant, but a lot of people have a hard time understanding that." https://www.reddit.com/r/Bitcoin/comments/1fzi657/why_not_peter_todd/lrc6ok0/

So, npub.cash and Amethyst: do I really put my nsec into the npub.cash site: that seems like an anti-pattern? One day I'll de-custodize those sats!

So I compromised and volunteered to talk at OP_NEXT next month, remotely. Now I need to find time to polish up the Script Restoration implementation and draft BIP, and write a talk, while at the same time we're approaching the next CLN release which *needs* all the pay improvements I've been working on. When I overcommit like this, it's usually my family that ends up short-changed. Yet none of this can be delegated, and both are important, so I suspect that the 28.11 release of CLN may be delayed. I keep wishing one of my kernel colleagues would want to work on Lightning: a veteran C coder with taste would be a good hire for Blockstream right now!

I've been heads-down working on xpay. We've only a month to the next release, so pressure is building. Lagrang3 has been giving great feedback on the askrene APIs (the core: getroutes is now about 16x faster than it was!), as he adapts renepay, and I've been writing xpay as clean-slate. We're now pretty happy with it, and it's been a delight to collaborate. I've also checked a snapshot of mainnet into master, for more accurate testing, using my new topology compressor (500k!). It's school holidays here so I've had less time than usual to code, but I'm still cautiously optimistic we will make the next release, though as an experimental option...

"Hand sanitizer smells sad." My 8yo said this, and they're right. Damn.

Another Lightning Spec Summit complete: this time Tokyo. The good-natured competition among the teams is a steady source of delight, for these are all brilliant people striving to bring something wonderful into the world. They are also respectful and appreciative of each others' achievements: kudos from people you respect has always been a key motivation for working on Open Source projects ❤️ I tried, and failed, to avoid committing to implementing or collaborating on more things. Many have partial implementations already, fortunately. In no particular order: 1. A "marketplace" bolt12 feature for a centralized marketplace to arrange a fee for payment. 2. Completing my draft implementation of the simplified update mechanism (particularly, reestablishment mechanics!). 3. "lnp" payment proof format. 4. Rebase recurrence proposal for BOLT12. 5. Implement v1.5 gossip (once Elle revises it again!) 6. Split keys for comms, messages and payments in node announcements. 7. Proxy support for BOLT 8 handshake. This doesn't count the tasks of reviewing what others are working on, which our team is not leading but are very interested in: bolt12 payment notifications and contact information sharing being two at the top of my mind. It was a productive three days. And, yes, a pile of fun!

BOLT 12 is merged into the spec To applause at the Lightning Spec Summit dinner! ❤️

Last three weeks I've been working on improving payments. Critical to this are five things: 1. Better payment routing, a-la min-cost-flow. 2. Better low-level payment injection API 3. Simpler, more reliable implementation. 4. Better simulation and realistic testing. 5. Better gossip maintenance. For the first, I took the heart off renepay to make "askrene" the payment oracle, which provides "getroutes" for other plugins to use. A primitive version shipped in the current release, but it's already getting better as I start to use it. Today I finished "injectpaymentonion" which takes incoming payments the same way we deal with incoming HTLCs, unifying things that were previously difficult, like self-pay and bolt12 blinded paths that begin at our own node. Tying these together is a new plug-in called "xpay". This calls getroutes then injectpaymentonion, then iterates as results come back, feeding back information to askrene as we learn more about the network. To figure out how well this actually performs, I've got a new format for representing topology, with minimal information (no node names, just channel capacities, details and topology). A simple tool compresses this from a node's gossip store (and decompress back for use). The result is small enough that we can include graph snapshots on our repository for testing. Then I have written a fake "channeld" which simulates payments through this graph (it knows all the secret keys). This needs to be enhanced with simulating capacity of the channels (deterministically, based on a seed value) and realistic network delays, but it already does MPP timeouts and checks fees and cltv delays. This should allow me to test this system's performance in a way that mirrors what people actually see, and also measure it against what we have now. It may uncover bugs, but even better it will let me know where to optimize (min cost flow can definitely be a CPU hog!). And @Alex is working on ideas for more aggressive gossip gathering: if we're missing part of the map we cannot expect to have reliable payments!

I'm working on a new pay plugin called xpay (working title). It's designed to use the new (very WIP) askrene plugin, which provides routes. Actually, most of the work is there, fixing and enhancing that. Especially diagnosis: *why* did it give a weird/no route? Common issues are: there's a good channel but it's offline, or doesn't have capacity, or you don't have enough capacity at all, or the recipient doesn't. We've had issues where gossip is missing, too: detecting when that happens is an open research question, but we'll start probing more aggressively. There have been many complaints about pay in the wild, and the code has grown quite complex. I've heard good things about Boltz's mpay, which is far simpler, so I'm inspired to go in that direction. Moving the routing (which *is* complex!) out is the first step. To be honest, I should have taken a direct interest in this earlier. I considered pay a simple problem, or at least not one I needed to concern myself with. But better late than never!

Started working my way though the classic "How to Win Friends and Influence People" with my 10-yo. I should have re-read this years ago! It's a classic for a reason. All self-help books have to spend 50% of their time convincing you it's worth reading, and the remainder is the actual content. This is why the title is so off-putting to many, BUT the book itself is a gentle admonition on the requirements for nuanced empathy in human interaction, with a very reasoned approach.