TL;DR
- New algorithm: Alibaba’s Qwen team unveiled FIPO, a reinforcement learning algorithm that weights each token by its downstream impact, rather than distributing rewards equally.
- Improved performance: FIPO more than doubles the inference chain length and improves AIME 2024 accuracy from 50% to 56%, comparable to OpenAI’s o1-mini.
- Our commitment to open source: The Qwen research team plans to release the entire FIPO training system built on the verl framework as open source.
- Internal tension: This open source research release stands in contrast to Alibaba’s product division, which recently shut down Qwen3.5-Omni and moved away from its open source tradition.
- Limitations: This evaluation only covers mathematical inference on one dataset, and it remains untested whether the gains extend to code or other domains.
Alibaba’s Qwen research team announced FIPO, a reinforcement learning algorithm that tackles stubborn upper bounds in AI inference. Standard training rewards all tokens equally and gives logical turning points the same weight as commas. FIPO breaks through this barrier by assigning each token a reward proportional to its downstream impact, more than doubling the length of the inference chain in mathematical benchmarks.
FIPO addresses key limitations of current RL training for inference models. Standard approaches such as Group Relative Policy Optimization (GRPO) allocate a uniform reward across a sequence of tokens. The researchers argue that this blatant credit assignment causes the inference chain to grow to a certain length and then plateau, imposing a performance ceiling that prevents the model from developing deeper problem-solving strategies.
On the AIME 2024 math benchmark, FIPO’s accuracy improved from 50% to 56%, reaching a maximum of 58%. According to the FIPO paper, Chiyu Ma et al. The Qwen team said it will open source the entire training system.
The combination of improved performance and full transparency positions FIPO as a potential accelerator for the open source AI community. Many of the competing RL advances in inference are still locked in their own walls, leaving independent researchers with no choice but to reverse engineer the technology from published papers rather than directly building working code.
How FIPO works
Rather than uniformly distributing a single reward signal, FIPO uses future-KL divergence to change the weighting of tokens across all positions following a certain point. Tokens that start productive inference chains receive more rewards, while tokens that lead the model to a dead end are penalized. Dense token-level advantage signals distinguish important inference steps from routine fillers.
“We argue that this coarse-grained credit allocation fails to distinguish between important logical pivots and trivial tokens, imposing a performance ceiling.”
Chiyu Ma et al., Alibaba Qwen research team (via arXiv)
An important design choice is that FIPO accomplishes this without using a separate value model. Proximity policy optimization (PPO) techniques require an auxiliary value network pre-trained on long thought chain data, which introduces external knowledge and makes it difficult to distinguish whether performance gains are due to the algorithm or pre-trained helpers. Because its value model incorporates synthetic inference data, external information seeps into the training loop, making results difficult to interpret and reproduce.
In contrast, FIPO completely avoids both confounding factors and knowledge leakage by deriving token-level signals directly from the probability distribution of the policy model itself.
Removing the auxiliary value model also has practical benefits. Training a separate value network approximately doubles the GPU memory required during RL fine-tuning, and pretraining on synthetic data adds an additional full training run to the pipeline. FIPO’s single-model architecture positions itself as a more accessible option for research groups that lack the computing budgets of large corporate laboratories.
Furthermore, the simplicity of this architecture narrows the gap between what well-funded labs can explore and what independent researchers can reproduce, potentially allowing a wider range of teams to experiment with token-level unit allocation for inference.
FIPO includes a discount factor so that nearby tokens are weighted more heavily than distant tokens, and also includes an extremum filter to remove tokens that cause the model to vary significantly between training steps. The team applied the Dense Advantage formulation to Qwen2.5-32B-Base, a model with no prior exposure to synthetic thought chain data.
Additionally, they trained only on the public DAPO dataset. It is a large-scale, fully open source reinforcement learning system that uses the same Qwen2.5-32B base model to achieve 50 points on AIME 2024 and serves as an improvement over the baseline FIPO. Without an extremum filter, training becomes unstable around step 70, and the response length drops off rapidly. This shows how sensitive dense token-level rewards are to policy drift.
Benchmark results and inference depth
The contrast becomes sharper when you infer the length of the chain. While the model trained on DAPO is stuck at around 4,000 tokens, FIPO is over 10,000 tokens. According to the paper, this increase was not caused by outliers. The entire distribution of answer lengths has shifted upward, suggesting that the algorithm is systematically encouraging deeper inferences rather than producing occasionally longer outputs.
According to the paper’s benchmark data, on AIME 2024, FIPO outperforms DeepSeek-R1-Zero-Math-32B (about 47%) and is comparable to OpenAI’s o1-mini (about 56%). On this basis, FIPO shows further improvement on AIME 2025, with scores improving from 38% to 43% as reported in the paper, confirming that the results hold on to more difficult problem sets.
A FIPO-trained model progresses through four different phases. First, the model generates a shallow planning template containing an illusory answer. Phase 2 develops a linear inference chain that stops at the first answer, where the DAPO-trained model plateaus.
However, Phase 3 involves voluntary self-verification using multiple approaches. By phase 4, the model exhibits systematic multi-pass validation with complete derived recalculations.
Achieving self-verification with reinforcement learning alone without using synthetic chain-of-thought data is noteworthy because it is similar to the inference time scaling strategies seen in OpenAI’s o series and DeepSeek’s R1 inference models. Sufficiently granular credit allocation allows us to derive these behaviors from the base model without prior inference instructions, providing developers with an alternative to expensive distillation pipelines that rely on access to more powerful proprietary supervised models.
Consequently, the emergence of self-verification without explicit instruction suggests that token-level reward signals can unleash inferential behavior that is unattainable with coarser techniques, regardless of how long training continues under uniform reward distribution.
Alibaba context and open source tensions
FIPO comes at a turbulent time for Alibaba’s AI division. In 2026, the company lost three senior executives, including Lin Junyang, Qwen’s head of technology. CEO Eddie Wu responded by merging the AI business into a new division of Alibaba Token Hub and putting him directly in charge.
Wu also announced that he would form a task force dedicated to stabilizing AI research results after his departure, suggesting that despite Alibaba’s organizational upheaval, it still considers its model development pipeline a top priority for the company.
Days before FIPO was released, Alibaba broke away from its open source tradition and released Qwen3.5-Omni as a closed source model. As stated in the paper, the FIPO team’s efforts to open source a training system built on the verl framework has led to visible divisions. While Alibaba’s product side is becoming proprietary, its research side continues to openly share its methods. The Qwen model attracts over 290,000 developers worldwide with over 113,000 community variations on Hugging Face, a community built primarily on open access.
Meanwhile, Alibaba has committed more than $53 billion to AI infrastructure, and more than 80% of open positions are currently AI-related, up from 60% a year ago.
Since introducing Qwen2.5 Max in January 2025, the company has rapidly expanded its AI portfolio.
Most recently, Qwen 3.5 arrived in February 2026, and Alibaba claimed it outperformed GPT-5.2, Claude Opus 4.5, and Gemini 3 Pro in several benchmarks.
Limitations and prospects
Despite these results, the paper’s evaluation is only for mathematical reasoning models that were trained on a single dataset and tested only on the base model without thought chain pre-training. Whether this advantage carries over to code, symbolic logic, or other areas of reasoning remains an open question. As the inference chain becomes longer, the computational cost of inference also increases, but that tradeoff is not quantified in the paper.
Additionally, the previous Qwen math benchmark came under scrutiny. Alibaba’s Qwen 2.5 faced fraud allegations over tainted benchmark data in July 2025, a reminder that self-reported results require independent verification.
However, pure reinforcement learning still requires less instruction to the model than direct instruction from a more powerful supervised model, suggesting that FIPO may complement rather than replace distillation-based approaches. Further improvements could be obtained by combining token-level credit allocation with knowledge distillation, but this paper leaves this exploration as future work.
“Our results suggest that establishing a dense advantage formulation is an important path to evolving ORM-based algorithms to maximize the inference potential of the base model.”
Chiyu Ma et al., Alibaba Qwen research team (via arXiv)
The main RL techniques for inferring models are held exclusively by OpenAI and DeepSeek. For the thousands of independent researchers and small labs currently locked out of proprietary training methods, FIPO’s promised open source release will provide a complete and reproducible pipeline for token-level credit allocation. Whether Alibaba delivers on its promise will determine whether this paper remains an academic contribution or becomes the foundation for a new wave of open inference model development.
