Training user simulators to be indistinguishable from real users (via Turing-test-style rewards) works better than training them to match specific ground-truth responses, enabling more realistic evaluation of conversational AI systems.
This paper proposes Turing-RL, a new method for training AI models to simulate human users in conversations. Instead of teaching models to match specific human responses word-for-word, it uses a judge (another LLM) to score how realistic and indistinguishable the simulated responses are from real human behavior.