There aren't any arguments for keeping blocks as small as possible left standing. It's just everybody has collectively shrugged their shoulders and decided to try to cobble something together on LN.

You are thinking of latency, which was mainly about mining coordination. Meaning less orphans and proper block header propagation, so you don't need to send the whole block at first but have time to download the real block data. This especially ba problem if one block follows fast after another. I don't know if or how BTC fixed it as 4 M propagates almost immediatley, ,but BCH has something called like Xthinner that theoretically enabled Gigabit transaction 5 years ago.