Transformer论文合著者Lucas Kaiser对AI前沿的评估Transformer Co-author Lucas Kaiser's Honest Assessment of AI Frontiers
核心要点：尽管带推理和代理的Transformer取得了显著成果，但要实现从有限数据中真正泛化，可能需要更根本的方法。Transformer论文合著者、曾在Google和OpenAI任职的Lucas Kaiser反思了AI当前状态。Transformer在思维链和工具辅助下擅长下一词预测，能实现出色编码和问题解决，但往往需要海量数据才能抓住核心概念——不像人类能更高效地形成想法。Kaiser注意到社区中驱动后Transformer探索的“氛围”，受Yann LeCun等研究者启发，强调人类般从更少数据和多模态流中学习的潜力仍未充分探索。他指出代理极大提升了研究生产力（例如重现论文快5-10倍），但需要超越当前文件grep等临时方案的更好长期记忆和验证。在闭源与开源方面，他认为前沿模型仍有价值，而开源模型服务特定需求。一句难忘引言：“LLM……会学到概念。但是在穷尽所有其他选项之后。”这一观点突显了当前架构的兴奋与局限，呼吁在研究中继续大胆探索。
The Takeaway: While transformers with reasoning and agents achieve remarkable results, something more fundamental may be needed for true generalization from limited data. Lucas Kaiser, co-author of the seminal transformer paper and former researcher at Google and OpenAI, reflects on the current state of AI. Transformers excel at next-token prediction enhanced by chain-of-thought and tools, enabling impressive coding and problem-solving, yet they often require exhaustive data before grasping core concepts—unlike humans who form ideas more efficiently. Kaiser notes the "vibe" in the community driving post-transformer exploration, inspired by researchers like Yann LeCun, emphasizing that human-like learning from less data and multimodal streams remains underexplored. He highlights how agents have dramatically boosted researcher productivity (e.g., reproducing papers 5-10x faster) but stresses the need for better long-term memory and verification beyond current hacks like file-based grep. On closed vs. open source, he sees persistent value in frontier models while open models serve specific needs. A memorable quote: "LLMs... will learn the concept. But after exhausting all other options." This philosophy underscores the excitement and limitations in today's architectures, urging continued wild exploration in research.
查看原文 →