You can train effective multi-agent orchestrators without human labels by learning from the artifacts agents produce—making it practical to build and improve systems that coordinate specialized LLM agents.
This paper introduces OrchRM, a self-supervised method for training orchestrators that coordinate multiple AI agents without human feedback. Instead of expensive rollouts, it learns from intermediate results of agent executions to build a reward model that guides which agent to use when.