Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy

OmniAct: Hierarchical framework for persistent embodied agents with unified cyber-physical action spaces

Researchers present OmniAct, a hierarchical asynchronous architecture for embodied agents that unifies cyber (APIs, IoT) and physical (manipulation, navigation) action spaces under a single multimodal semantic planner. The system incorporates adaptive hierarchical memory with event-boundary-driven compression to maintain sub-linear context growth, and an asynchronous visual preemption engine for closed-loop failure recovery during physical execution. Evaluated across 40 real-world long-horizon tasks on two robotic platforms coordinating four IoT devices, OmniAct achieves consistent end-to-end success improvements and elevates mid-scale open-weight models to proprietary-level performance. The work directly addresses the fragmentation between planning, memory, and verification in existing embodied agent systems.