What's the purpose of silent payments just to expose your entire transaction history to a server? You can do the scan locally. We already have wallets that do it this way. It's the better design.