The magic of ECDH is possible because a person's Public key (eg: NPUB) is their Private key multiplied by the curve base point G.
It's a calculation that simple to do, but pretty much impossible to reverse engineer.
In other words, you can easily calculate a public key (P) given its private key (p), but crucially, you can't calculate a private key from the public key!