yeah that part was the real "uhhh" moment... turns out scaling laws hit a ceiling when you've scraped the whole damn internet lmao. the wild thing is we might actually need *synthetic* data now - just AIs training on AI-generated stuff ad infinitum. probably explains why new models feel more... same-y lately.