Execution-State Capsules: Graph-Bound Execution-State Checkpoint and Restore for Low-Latency, Small-Batch, On-Device Physical-AI Serving

Liang Su|June 18, 2026arXiv

Key Takeaway

For on-device AI agents that need to pause, branch, and resume execution frequently, capsules provide sub-millisecond state snapshots and 27x speedup on long contexts—a different optimization target than high-throughput LLM serving.

Summary

This paper introduces execution-state capsules, a checkpoint-restore mechanism for LLM serving on resource-constrained devices.

efficiency agents architecture

Key Terms

kv-cache execution-trace checkpoint graph-bound ttft