It is all about who do you trust. (Somewhat simplified) In the end you only need the unspent UTXOs and some reassurance that your list of those has all of them and not missing any. How you got there only matters if you want to index historical data, but you are not a tax authority to care about all that. Polkadot has a so-called warp sync mode, which achieves something like that. The method exists since Parity first wrote their own Ethereum node. Which became OpenEthereum.