Engram 创始人谈记忆与持续学习：将公司上下文烘焙到模型权重中Engram Founders on Memory and Continual Learning: Baking Company Context into Model Weights
核心要点：AI 模型的真正实用性来自于通过持续训练将新的、不断演化的组织上下文深度烘焙到权重中，而不是仅依赖外部上下文窗口或 RAG。Engram 联合创始人 Dan Biderman 和 Jessy Lin 认为，虽然前沿实验室专注于一个巨大的通用模型，但未来在于为团队和个人定制的模型，这些模型能持续学习私有或定制的上下文。他们使用适配器微调和各种训练信号处理来自 Notion 等工具的工作空间数据，创建能直观理解公司特定工作流的模型。“我们认为工具使用和上下文工程会发挥作用，但目前未被充分利用的工具是使用相同训练流程……应用到每种领域。” 这能大幅减少推理 token（高达 100 倍），并弥合前沿模型在新特定任务上的差距。他们设想一个拥有众多专门化神经接口连接数据平面的世界。
The Takeaway: True usefulness of AI models comes from baking new, evolving organizational context deeply into weights via continual training, rather than relying solely on external context windows or RAG. Engram co-founders Dan Biderman and Jessy Lin argue that while frontier labs focus on one giant general model, the future involves personalized models for teams and individuals that continuously learn private or bespoke contexts. Their approach uses adapter fine-tuning and various training signals on workspace data from tools like Notion to create models that understand company-specific workflows intuitively. "We think these kinds of things, like tool use, context engineering will play a part, but I think an under leveraged tool these days is using the same training pipeline... applying that to every kind of domain." This reduces inference tokens dramatically (up to 100x) and bridges gaps where frontier models lag on fresh, specific tasks. They envision a world of many specialized neural interfaces to data planes.
查看原文 →