如何构建长时程自主 AI Agent：Basis 联合创始人 Mitch Troyanovsky 的深度分享How to Build Long-Horizon AI Agents: Insights from Basis Co-founder Mitch Troyanovsky
The Takeaway：真正可靠的长时程 Agent 不能只靠 outcome evals，必须用 behavior specs 做过程监督，并精心设计 Agent 的 ontology 世界。
Mitch Troyanovsky 是 Basis 联合创始人。这家独角兽公司的 Agent 能自主运行数小时甚至数天，完成端到端报税等复杂会计工作。他指出，LLM 默认只有巨大工作记忆、没有真正的短/长期记忆，一旦任务超出上下文窗口就会出现 coherence 崩溃。解决之道不是单纯堆更多 outcome 测试（即使 100 个 evals 全过也不可信），而是向人类组织学习：把“好过程”写成 markdown 形式的 behavior specs，既作为人类对齐标准，也作为 judge 的 rubric。这些 specs 不直接喂给 Agent，而是用来评估轨迹是否表现出正确行为（例如必须引用 IRS 原始来源而非 Wikipedia）。
他强调："English is more precious because the English affects the performance. The code does not affect the performance." 上下文就是 runtime 训练数据，必须像对待代码库一样对待文档与 ontology。Coding Agent 之所以先成功，部分因为 runtime 可验证且文本原生；真实世界任务则面临数据稀缺、反馈环极长、好坏标准主观等障碍。Basis 与 Braintrust 开源了 behavior specs 标准与示例 judge，希望成为行业通用语言。未来关闭改进闭环、甚至对行为遵守做 RL，都建立在这些高质量信号之上。技术护城河终将被 bitter lesson 吞没，真正持久的是业务嵌入与对专业过程的深刻理解。
Mitch Troyanovsky 是 Basis 联合创始人。这家独角兽公司的 Agent 能自主运行数小时甚至数天，完成端到端报税等复杂会计工作。他指出，LLM 默认只有巨大工作记忆、没有真正的短/长期记忆，一旦任务超出上下文窗口就会出现 coherence 崩溃。解决之道不是单纯堆更多 outcome 测试（即使 100 个 evals 全过也不可信），而是向人类组织学习：把“好过程”写成 markdown 形式的 behavior specs，既作为人类对齐标准，也作为 judge 的 rubric。这些 specs 不直接喂给 Agent，而是用来评估轨迹是否表现出正确行为（例如必须引用 IRS 原始来源而非 Wikipedia）。
他强调："English is more precious because the English affects the performance. The code does not affect the performance." 上下文就是 runtime 训练数据，必须像对待代码库一样对待文档与 ontology。Coding Agent 之所以先成功，部分因为 runtime 可验证且文本原生；真实世界任务则面临数据稀缺、反馈环极长、好坏标准主观等障碍。Basis 与 Braintrust 开源了 behavior specs 标准与示例 judge，希望成为行业通用语言。未来关闭改进闭环、甚至对行为遵守做 RL，都建立在这些高质量信号之上。技术护城河终将被 bitter lesson 吞没，真正持久的是业务嵌入与对专业过程的深刻理解。
The Takeaway: Reliable long-horizon agents cannot rely on outcome evals alone; they need process supervision through behavior specs and carefully designed ontologies that give the agent a coherent world to inhabit.
Mitch Troyanovsky, co-founder of Basis, builds agents that already run autonomously for hours or days and complete end-to-end tax returns. He argues that LLMs have large working memory but essentially no short- or long-term memory by default. Once a task exceeds the context window, coherence collapses. Passing 100 outcome evals still does not guarantee generalization to production. The solution is to borrow from how humans organize non-deterministic work: write the desired process into markdown “behavior specs” that serve both as human alignment documents and as rubrics for judges. These specs are deliberately not shown to the agent; they are used to grade trajectories (e.g., “always cite the primary IRS source, never Wikipedia”).
A memorable line: “the English is more precious because the English affects the performance. The code does not affect the performance.” Context is runtime training data and must be treated with the same rigor as a codebase. Coding agents succeeded first partly because results are verifiable at runtime and the medium is pure text; real-world domains suffer from scarce data, multi-thousand-step trajectories, and subjective notions of quality. Basis open-sourced the behavior-spec format and example judges with Braintrust, aiming for an industry standard. Closing the self-improvement loop and eventually doing RL on behavior adherence both depend on these high-quality process signals. Technical moats will eventually be swallowed by the bitter lesson; durable advantage comes from deep process knowledge and business embedding.
查看原文 →查看原文 →
Mitch Troyanovsky, co-founder of Basis, builds agents that already run autonomously for hours or days and complete end-to-end tax returns. He argues that LLMs have large working memory but essentially no short- or long-term memory by default. Once a task exceeds the context window, coherence collapses. Passing 100 outcome evals still does not guarantee generalization to production. The solution is to borrow from how humans organize non-deterministic work: write the desired process into markdown “behavior specs” that serve both as human alignment documents and as rubrics for judges. These specs are deliberately not shown to the agent; they are used to grade trajectories (e.g., “always cite the primary IRS source, never Wikipedia”).
A memorable line: “the English is more precious because the English affects the performance. The code does not affect the performance.” Context is runtime training data and must be treated with the same rigor as a codebase. Coding agents succeeded first partly because results are verifiable at runtime and the medium is pure text; real-world domains suffer from scarce data, multi-thousand-step trajectories, and subjective notions of quality. Basis open-sourced the behavior-spec format and example judges with Braintrust, aiming for an industry standard. Closing the self-improvement loop and eventually doing RL on behavior adherence both depend on these high-quality process signals. Technical moats will eventually be swallowed by the bitter lesson; durable advantage comes from deep process knowledge and business embedding.