Post-training of multimodal large-scale language models (MLLMs) for building interactive agents holds promise across areas such as computer use, web navigation, and robotics. The main challenge in scaling such post-training is the lack of high-quality downstream agent task datasets containing diverse, executable, and verifiable tasks. Existing approaches for task generation rely heavily on requiring human annotations or limited downstream environment information from the MLLM, which are either costly or poorly scalable due to the generation of tasks with limited scope. To address this, we introduce AutoPlay, a scalable pipeline for task generation that explicitly explores the interactive environment to discover possible interactions and current state information, and synthesizes tasks based on the environment. AutoPlay works in two stages. (i) In the exploration phase, the MLLM Explorer agent systematically uncovers the state and capabilities of the new environment. (ii) In the task generation phase, the task generator utilizes the exploration trajectory and a set of task guideline prompts as a context to synthesize diverse, executable, and verifiable tasks. We show that AutoPlay generates 20,000 tasks for 20 Android applications and 10,000 tasks for 13 Ubuntu applications to train agents for mobile and computers. AutoPlay-generated tasks use MLLM task executors and verifiers to enable large-scale task demonstration synthesis without human annotation. This data allows us to train an MLLM-based UI agent that improves success rates by up to 20.0% in mobile usage scenarios and 10.9% in computer usage scenarios. Furthermore, combining autoplay-generated tasks with MLLM verifier-based rewards enables scaling reinforcement learning training of the UI agent, yielding an additional 5.7% profit. coverage. These results establish AutoPlay as a scalable approach for competent post-training MLLM agents that reduces dependence on human annotations.
- ** Work I did while at Apple
