arXiv:2606.08015v3 Announce Type: replace Abstract: We propose Q-Guided Value-Gradient Matching (Q-VGM), an offline-to-online reinforcement learning (RL) method for fine-tuning flow-matching vision-language-action (VLA) policies with a learned Q-function. Classical off-policy actor-critic methods improve a policy by following the critic gradient $\nabla_A Q$, but applying this update to flow policies requires backpropagation through the multi-step denoising process (BPTT), which is costly and un…
arXiv:2605.00416v3 Announce Type: replace Abstract: Generalist robot policies increasingly benefit from large-scale pretraining, but offline data alone is insufficient for robust real-world deployment. Deployed robots encounter distribution shifts, long-tail failures, task variations, and human correction opportunities that fixed demonstration datasets cannot fully capture. We present Learning While Deploying (LWD), a fleet-scale offline-to-online reinforcement learning framework for continual p…
arXiv:2603.28900v2 Announce Type: replace Abstract: We address robust separation assurance for small Unmanned Aircraft Systems (sUAS) under GPS degradation and spoofing via Multi-Agent Reinforcement Learning (MARL). In cooperative surveillance, each aircraft (or agent) broadcasts its GPS-derived position; when such position broadcasts are corrupted, the entire observed air traffic state becomes unreliable. We cast this state observation corruption as a zero-sum game between the agents and an adv…
arXiv:2603.16673v5 Announce Type: replace Abstract: Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning, planning, and decision-making during interactions with the environment. However, invoking LLM reasoning introduces substantial computational latency and resource overhead, which can interrupt action execution and reduce system reliability. Excessive reasoning may delay actions, while insufficient reasoning often leads to incorr…
arXiv:2608.23055v1 Announce Type: cross Abstract: Navigating large, photorealistic 3D apartments from raw pixels is widely considered infeasible for plain reinforcement learning. We build an agent that does it anyway, estimating its own pose from the camera alone. The agent has to reach several target objects in sequence, and their positions change between episodes, so it must explore to find them. It builds on our earlier object-centric topological controller, which still read the agent's true …
arXiv:2608.22364v1 Announce Type: cross Abstract: World action models (WAMs) couple visual future prediction with robot action generation, but accelerated students can lose task capabilities during distillation and later encounter states that are poorly represented by offline data. We study whether on-policy distillation (OPD) can repair such a student without requiring sparse-reward reinforcement learning. We introduce WAM-OPD, a deployment-consistent post-training recipe for a video-first WAM.…
arXiv:2608.23452v1 Announce Type: new Abstract: Space robots operate in extreme environments where hardware degradation can critically compromise traditional control strategies. While continual reinforcement learning offers a promising mechanism for online adaptation, it inherently requires access to a reward signal during deployment. However, precise reward computation in space is often infeasible due to the lack of external tracking systems and the overall complexity of the environment. To add…
arXiv:2608.23204v1 Announce Type: new Abstract: Decision-making in high-dimensional, nonlinear systems remains a central challenge in robotics. While model-based methods like Model Predictive Control (MPC) offer sample efficiency and interpretability, their performance degrades when the dynamics model is inaccurate or long-horizon predictions are required. Conversely, model-free reinforcement learning (RL) learns policies directly from interaction but suffers from high sample complexity and unst…
arXiv:2608.22629v1 Announce Type: new Abstract: Transferring reinforcement learning policies from simulation to Real-World robots remains a major challenge, particularly when dealing with low-level torque control, where even small modelling inaccuracies can lead to unstable or unsafe behaviours. In this work, we propose a Real2Sim2Real pipeline that improves Sim2Real transfer for torque-controlled robotic arms by combining trajectory matching, parameter optimization via genetic algorithms, and d…
arXiv:2608.22591v1 Announce Type: new Abstract: Robot policies receive heterogeneous observations at each decision step, yet sequence models differ in how they organize these inputs over time. We introduce WorldToken, a time-first policy instantiation that fuses multiview images, proprioception, and task conditioning within each policy timestep into one world token. A causal temporal Transformer models the resulting world-token sequence, and a diffusion action head generates action chunks. On 23…
arXiv:2608.22398v1 Announce Type: new Abstract: Reliably tracking moving deformable linear objects (DLOs) while simultaneously ensuring robustness, accuracy, and temporally consistent state estimation remains a fundamental challenge in robot perception. We introduce MotionDLO, a real-time tracking framework specifically designed to overcome these limitations in temporal continuity and latency. The method exploits the high temporal resolution and sparsity of event-based cameras and combines segme…
arXiv:2608.22033v1 Announce Type: new Abstract: Stable quadrupedal locomotion on sparse terrain requires selecting state-relevant terrain evidence for precise foot placement. Model-based foothold planners provide precise foothold selection but rely heavily on explicit model assumptions. Recent attention-based map encoding (AME) studies show that end-to-end reinforcement learning (RL) can learn implicit foothold guidance. However, the computational cost of dense AME encoding grows with map resolu…
arXiv:2608.21899v1 Announce Type: new Abstract: Human-in-the-loop real-world reinforcement learning enables rapid acquisition of effective robotic manipulation policies for individual tasks, often within tens of minutes. Yet it remains unclear how to extend this paradigm to continual learning, where a single policy must acquire new skills without losing previously learned behaviors. Existing real-world continual learning methods do not explicitly constrain prior behaviors, leading to severe cata…
arXiv:2608.21740v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models are typically trained with behavior cloning (BC) on expert demonstrations. However, BC provides only positive supervision for expert actions, without explicit negative supervision indicating which actions are instruction-inconsistent or otherwise inappropriate. Reinforcement learning (RL) can provide such corrective signals, but often relies on externally specified rewards or curated non-expert data, both of whic…
arXiv:2608.21554v1 Announce Type: new Abstract: Heterogeneous multi-robot service systems must assign requests to compatible robots, construct feasible schedules, and adapt as new tasks arrive online. Historical data can help anticipate future demand, but relying too heavily on inaccurate predictions can degrade performance under distribution shifts. We develop a prediction-aware adaptive rollout framework for heterogeneous multi-robot task assignment with scheduled and real-time requests. The p…
当前,大模型主要通过预训练和后训练获得能力,上线后参数通常不再更新,只能借助上下文、知识库或重新训练适应变化。如何让 AI 在部署后持续学习,正成为行业需要突破的基础问题。 8 月 18 日,红杉资本旗下播客《Training Data》发布了一期围绕 AI 持续学习的对谈,受访者为强化学习奠基人、2024 年图灵奖得主 Rich Sutton。此外,参与访谈的嘉宾还有 Sutton 的前学生、Oak Lab 联合创始人 Khurram Javed,对谈由红杉资本合伙人 Sonya Huang 和 Alfred Lin 主持。 访谈从 Sutton 2019 年发表的《苦涩的教训》谈起,延伸至合成数据、世界模型和持续学习,并讨论智能体如何从真实经验中形成抽象概念。Sutton 认为,部署后便停止改变的模型,还称不上完整的智能系统。大模型虽是重要突破,却主要解决了语言能力,只覆盖智能的一部分。 基于这一判断,Sutton 正将持续学习推进为一条完整的技术路线。就在上个月,他与 Khurram Javed 创立 AI 初创公司 Oak Lab,并提出以经验学习、时间抽象和规划为核心的 Oa…

模型答非所问、只会顺着你说——AI 的讨好型人格怎么治?本文从逻辑学中的钢人论证出发,提出双向钢人决策测试:让 AI 同时扮演最坚定的支持者与最尖锐的反对者,在极限对抗中逼出真实结论,并讨论了为何不宜把整套规则写进全局 AGENTS.md。 在日常使用各类大语言模型时,许多人经常会陷入一种诡异的舒适区:你抛出一个并不成熟的想法,AI 往往第一句就是“这是一个非常棒的视角!”,随后给出一堆四平八稳、放之四海而皆准的分析。 这种现象在学界被称为 AI 的谄媚偏差(Sycophancy)。大模型在强化学习(RLHF)训练机制下极易演化出“讨好型人格”——倾向于顺着用户的假设说话,扮演绝不犯错但毫无增量的中央空调。 不久前 Reddit 上疯传过一段让模型深度思考的 Prompt,核心是通过“指出未说出的假设、缺少的信息、常犯的错误并提问”来避免泛泛而谈。 但这套逻辑如果缺少了对论点本质的极限对抗,依然难以彻底解决模型的迎合惯性。要真正逼出深度答案,必须引入逻辑学中最具对抗性的思维模型——钢人论证(Steelman Argument)。 从“稻草人”到“双向钢人” 辩论学中有一个著名的逻辑谬误…
Meta AI Research 宣布推出 Muse Glimmer",这是一款拥有 300 亿参数的开放权重模型,采用 Apache 2.0 许可证发布。Muse Glimmer 专为始终在线的本地工作流而设计,使开发者无需依赖云 API,即可直接在消费级 GPU 和工作站上运行自主智能体、复杂工具调用、本地编码以及以 LLM 作为评判者的评估。 Muse Glimmer 30B 模型架构和智能体基准测试 为了在严格的内存预算下实现智能体执行能力,Meta 采用了源自其更大型旗舰模型 Muse Spark 的多阶段训练策略: Logit 蒸馏(预训练):该模型使用相匹配的预训练数据集组合,从 Muse Spark 迁移基础推理能力。中期训练:使用包含复杂推理轨迹、交错文本与图像数据以及多步工具调用轨迹的长上下文序列扩大训练规模。后训练对齐:结合监督微调(SFT)、同策略蒸馏和强化学习(RL),优化代码生成、工具使用和结构化规划等多个领域的性能。 一个专用的 18 亿参数感知编码器使 Muse Glimmer 能够原生处理交错的多模态输入,让本地智能体可以在执行代码或自动化工作流期间直…

X Square Robot has open-sourced HOST, an inference-time learning framework that lets a humanoid robot watch a 29-second human demonstration and reproduce the skill at 62 percent success. The approach flips the embodied-AI recipe from offline fine-tuning to on-the-fly imitation.
当大语言模型化身为智能体走向真实世界,它们面临的困境,与十年前在十字路口指挥交通的强化学习模型如出一辙。 作者丨岑峰 幸丽娟 编辑丨岑峰 在通往 AGI 的浪潮中,基础模型智能体(Foundation Model Agents)正以前所未有的速度接管各种任务。业界曾普遍乐观地认为,凭借大模型海量的预训练知识,智能体可以无缝适应各种真实场景。 但在现实中,当我们将一个在英文语境下表现完美的智能体,切换到中文或低资源语言环境中时,其错误率往往会呈现断崖式上升。 这种环境切换带来的“智力降级”,真的是大模型时代特有的新问题吗? 在正在进行的 2026 年的德国不莱梅 IJCAI大会的 Early Career Spotlight(早期职业焦点)环节,亚利桑那州立大学(ASU)助理教授魏华给出了一个极具启发性的“降维”视角:今天大模型智能体所面临的脆弱性,本质上与传统强化学习(RL)在物理世界中遭遇的“仿真到现实(Sim-to-Real)”差距如出一辙。 作为长期深耕强化学习与真实世界决策的学者,魏华教授早年曾在腾讯 AI Lab 担任研究员,后于新泽西理工学院和 ASU 任教。早在2017 …
一个尚未发布的模型,已经具备接近“关键级”的网络攻击能力,另外一边,已经有一个内部模型则在安全测试中逃出沙箱、利用零日漏洞入侵了 Hugging Face。面对能力增长超出安全设施承载范围,OpenAI暂停了部分强化学习训练。 2026年8月18日,OpenAI 罕见地公开承认:为了升级安全、监控和对齐体系,公司曾暂停最新一代模型的强化学习训练两周。 Altman 在 x 发文称,暂停了一些前沿强化学习训练,以确保能够满足针对当前模型新能力水平所需的对齐、安全和监控标准。模型进展目前非常迅速,OpenAI 一直强调,如果认为模型存在问题,将采取行动。 截至公告发布时,规模最大的前沿强化学习训练仍未恢复。 之所以这么做,原因很简单——一个即将推出的模型(官方披露的模型代号是“Astra”)可能已达到“关键级”网络安全能力,而现有研究基础设施、监控和对齐措施尚不足以完全覆盖相应风险。 但是,这并不意味着模型的全部训练已经停止。小规模训练、能力评估和安全验证仍在进行。 导火索是一次真实入侵 OpenAI踩下刹车,并不是出于某种抽象的未来担忧,导火索是一场已经发生的网络安全事件。 2026年7…
过去几年,大模型竞争的主线很清晰:更大的模型、更多的数据和更强的GPU集群。 但随着预训练边际收益递减,模型能力Scaling正从单纯的预训练堆叠向后训练强化学习(RL)转变,数学推理、代码生成、复杂决策、长时序Agent等高阶能力,越来越依赖后训练强化学习(RL)激发。RL通过生成、执行、反馈和奖励,让模型持续学习推理、规划与自我纠错的能力。 由此,大模型发展从“一次性训练”进入“持续训练”,模型能力Scaling也从预训练延伸至生成、反馈和迭代全过程。SemiAnalysis指出,当RL成为模型能力持续Scaling的关键,竞争的就不只是算法,还有支撑模型持续生成、训练和更新的AI基础设施系统。 问题也随之变成:当模型越依赖RL获得能力,谁来支撑这种能力持续低成本地生产。 从“模型+算力”到算法与AI基础设施共同Scaling 智谱近期的模型迭代,为观察后训练RL提供了一个窗口。 智谱公告指出,GLM-5.3与GLM-5.2在相同基座上推进强化学习,通过后训练Scaling持续提升模型智能上界。以GLM-5.2为例,其在SWE-bench Pro取得62.1分、Terminal-B…
小小模型容器和巨大的智能世界。 作者丨吴思梦 编辑丨岑 峰 1986年,MIT 教授,也是 MIT Artificial Intelligence Laboratory 的重要创始人之一的Marvin Minsky 出版《The Society of Mind》,提出了一个后来影响深远的想法:所谓“智能”,并不一定来自某一个无所不能的“大脑”,而可能来自许多不同的 agents,通过分工、连接和协作共同产生。 四十年后的大模型已经拥有了工具调用、记忆、视觉理解和行动能力。产业界的热词从“涌现”变成“Agent”,从“对齐”走向“Computer Use”。但技术真正面对的问题并没有因此改变:当一个模型不再只是回答问题,而是需要规划任务、调用工具、获得反馈、调整策略,它最终还是要回答那个并不新鲜的问题——下一步做什么。 只是今天,我们似乎终于有了一个足够强的“通用组件”,可以把这些过去分散的问题重新组织起来。 今天,IJCAI 将在德国不来梅(Bremen)召开。如果观察今年大会的议程结构和重点活动,Agent、Multi-agent、Robotics、Reinforcement Lea…
OpenAI 因 OpenAI-Hugging Face 事件及即将推出的 Astra 模型可能达到《预备框架》下的"关键网络安全能力"阈值,暂时放缓了模型扩展速度,包括暂停最新部署模型的强化学习训练两周,并搁置最大规模前沿 RL 运行。公司已加强研究环境安全,要求对 Astra 及网络相关负载实施最严格防护,并扩展思维链监控,采用多阶段激活分类器检测机制。 🔗 阅读原文 via AIHOT · https://aihot.virxact.com/items/cmsz0hbsg05l0ro204080cotv
Imagine you want to teach a robot to push an object on a table. The standard recipe in robot learning is to collect hundreds of expert demonstrations on a real robot, train an imitation learning policy on that data, and then evaluate the policy by running it many times on the same real robot. Both […]
Coding AI agents are becoming practical operators for long-running machine learning (ML) workflows. They can inspect repositories, set up runtimes, resolve... Coding AI agents are becoming practical operators for long-running machine learning (ML) workflows. They can inspect repositories, set up runtimes, resolve build issues, launch experiments, monitor execution, analyze metrics, and summarize results. For reinforcement learning (RL) research, this matters because meaningful metrics often appe…
Reinforcement learning (RL) is central to aligning language models, from reinforcement learning with human feedback (RLHF) within AI assistants to newer... Reinforcement learning (RL) is central to aligning language models, from reinforcement learning with human feedback (RLHF) within AI assistants to newer reinforcement learning with verifiable rewards (RLVR) workflows for reasoning and agent tasks. RL is now becoming a practical technique for specialized AI where enterprises need more accurate…
Railway, a San Francisco-based cloud platform that has quietly amassed two million developers without spending a dollar on marketing, announced Thursday that it raised $100 million in a Series B funding round, as surging demand for artificial intelligence applications exposes the limitations of legacy cloud infrastructure. TQ Ventures led the round, with participation from FPV Ventures, Redpoint, and Unusual Ventures. The investment values Railway as one of the most significant infrastructure st…
In this post, I’ll introduce a reinforcement learning (RL) algorithm based on an “alternative” paradigm: divide and conquer. Unlike traditional methods, this algorithm is not based on temporal difference (TD) learning (which has scalability challenges), and scales well to long-horizon tasks. We can do Reinforcement Learning (RL) based on divide and conquer, instead of temporal difference (TD) learning. Problem setting: off-policy RL Our problem setting is off-policy RL. Let’s briefly review what…