A Generative AI Plateau
Computerphile (a YouTube channel I really love) released a video today titled Has Generative AI Already Peaked?. In the video, Mike Pound talks about a research paper released in April called No “Zero-Shot” Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance. I think this video and the paper resonate with how I’ve felt about the hype around generative AI in the last several years. To summarize Mike’s thesis, he’s developing a view, which the paper corroborates, that producing a general intelligence through large training data that can solve new, and previously unseen problems (a “zero-shot”) is an exponential data problem....