AI guides

Axion 1.0 opens an agent-training pipeline using two-stage SFT and RL for scientific tool use

By 机器之心1 viewsSource: 机器之心
Axion 1.0 scientific-agent training stages and results

Researchers from Duke, ByteDance and other institutions present Axion 1.0, using supervised fine-tuning for reasoning and information acquisition followed by reinforcement learning. The paper reports mathematics, science and information-search results and releases training materials.

An agent that searches for information, runs code and completes multistep scientific-research tasks—how many training steps does it take to build one?

Mid-training, supervised fine-tuning, preference optimization, on-policy distillation, reinforcement learning: as agents gain capabilities, training pipelines grow longer. Each added stage requires researchers to prepare data, coordinate new objectives and tune another set of training parameters.

Researchers from Duke University, ByteDance, the University of Maryland, College Park and the University at Buffalo introduce Axion 1.0. Its pipeline contains only two stages of supervised fine-tuning (SFT) followed by reinforcement learning (RL), without additional mid-training, direct preference optimization (DPO) or on-policy distillation (OPD).

With this compact pipeline, Axion 1.0 demonstrates strong scientific reasoning and tool use. The paper reports 59.0% for Axion 1.0-122B on the full Humanity's Last Exam (HLE), with access to certain websites restricted, exceeding GPT-6; it achieves 66.7% on FrontierScience Research. The 35B version scores 62.2% on information-seeking benchmark SEAL-0.

More importantly, the team has fully open-sourced its data construction, generation, filtering, verification, prompts and training code, making the entire training methodology available to the community. The straightforward three-stage process and transparent open-source implementation make agent foundation-model training less opaque and let more people train powerful agents of their own.

Article illustration

  • Paper: https://arxiv.org/abs/2611.03233
  • Project: https://axion1-0.github.io
  • Code: https://github.com/bytedance/axion1_0
  • Data: https://huggingface.co/datasets/Axion1-0/axion-training-data
  • Model: https://huggingface.co/Axion1-0/Axion-1.0-35B

Article illustration

Method

Axion 1.0's key is constructing high-quality data for the different capabilities agents need. It divides training tasks into two categories: reasoning and calculation from complete information supplied in the question, and tasks that require tools to obtain missing external information before reasoning from evidence. The two SFT stages address these categories in turn, first improving reasoning and then teaching information acquisition and use.

Step one: make already-solvable problems more challenging

The first stage uses questions from mathematics, physics, chemistry, biology, computer science and other fields to train the model’s multistep reasoning and calculation capabilities. However, the base model can already answer many publicly available questions correctly. How can continuing to train on these questions provide more room to learn? Axion 1.0’s approach is to targets difficulty increases according to the base model's performance.

Article illustration

The researchers first harden seed questions by making the original answer an intermediate quantity in a new problem, adding a dependency. The base model then attempts them. Incorrectly answered questions enter a candidate pool; correctly answered questions and successful solutions go back to the generator for further difficulty increases.

For example, a seed asking for the sum from one to ten can replace fixed numbers with parameters and request a general expression, or expand the range to make enumeration impractical. Other rewrites add reasoning dependent on earlier results, introduce case-specific conditions, or combine these strategies.

After every rewrite, reference answers are revalidated; under-specified or ambiguous questions are removed. Correct solution traces form approximately 10,800 first-stage SFT samples. Python is allowed, but search and web access are not, so the model first learns reasoning and calculation within the question itself.

Step two: require the model to seek answers outside itself

Beyond internal reasoning, when existing knowledge cannot answer a question, an agent must identify missing information, decide where to find it and use the findings in reasoning. These capabilities drive second-stage task construction.

Starting from public structured data, programs filter records, select a unique entity and calculate the answer. A language model then turns that process into a natural-language question. Reference answers are fixed during question construction, making later correctness checks straightforward.

Article illustration

The paper gives an intuitive example: among ringed planets in the Solar System, identify the least massive and calculate its average density. The model must identify Uranus, find its mass and radius, then calculate the result. One question links entity identification, fact retrieval and numerical computation.

During trajectory collection, the model receives neither the source database tables nor reference answers; it must obtain information through search, web access and Python. Questions answerable without retrieval are also filtered out. Programs check unique entity selection, complete parameters and whether natural-language rewrites omit conditions or leak answers. The second stage retains about 7,500 correct tool-interaction trajectories. Both SFT stages keep the full interaction context but compute loss only on assistant-generated reasoning, tool calls and answers.

Step three: keep learning through real tool interaction

After demonstrations of reasoning and tool use, the main 35B and 122B models undergo asynchronous GRPO reinforcement learning, generating their own solution traces using search, web access, scholarly retrieval and Python.

The central RL design is a staged reward mechanism. This implementation, in turn, depends on the earlier design of the data construction process. Generating questions programmatically can provide the final answer while also recording the entities and numerical values that should be obtained during the solution. The model therefore receives the full reward when its final answer is correct. If it does not answer correctly, verified intermediate progress can still receive partial rewards, while a penalty applies when no final answer is submitted.

Intermediate progress also has ordered dependencies: identify the entity, obtain parameters, then calculate. Later steps count only when prerequisites pass verification. Entity identification requiring external evidence must be supported by tool observations, so correct content appearing incidentally in retrieval results does not count as completed reasoning.

Experimental results

Axion 1.0's two main models are based on Qwen3.6-35B-A3B and Qwen3.5-122B-A10B. The team examines scientific reasoning, information seeking and tool use across six evaluations.

Main results

Article illustration

  • HLE: Axion 122B scores 59.0%, above the paper's listed GPT-6 Astra (57.2%) and GPT-5.5 (52.2%), and close to Claude Opus 5 (63.6%) and Claude Fable 5 (63.8%).
  • FrontierScience Research: Axion 122B scores 66.7%, exceeding GPT-5.6 Sol (63.3%) and GPT-5.5 (26.7%).
  • FrontierScience Olympiad: the 122B and 35B versions score 87.0% and 86.0% respectively, both above GPT-5.6 Sol (79.0%) and Gemini 3.1 Pro (85.0%).
  • SEAL-0: the 35B version scores 62.2%, above GPT-5.6 Sol (53.2%), Gemini 3.1 Pro (53.2%) and Agents-A1 (56.4%).
  • HiPhO: the 122B version scores 46.2%, above GPT-5.6 Sol (42.9%) and GPT-5.5 (43.3%).

Effects of each training stage

Article illustration

For the 35B model, first-stage SFT raises FrontierScience Olympiad from 71.0% to 83.0%, reflecting stronger scientific reasoning. Second-stage tool-use training raises BioMysteryBench from 17.6% to 35.3%, and FrontierScience Research from 48.3% to 58.3%; RL further raises SEAL-0 from 53.2% to 62.2%. Successive checkpoints show capability developing from reasoning through information acquisition to multistep tool interaction.

Different base models

Article illustration

The same two-stage SFT method is applied to Qwen3.5-35B-A3B and Qwen3.6-35B-A3B. Before training, their HLE-130 scores differ by 9.2 percentage points; afterward, the gap narrows to 0.8 percentage points, while both reach 35.3% on BioMysteryBench. This suggests the method works across bases and helps a weaker starting model substantially close its performance gap.

Model scale

Article illustration

The method is also applied to 4B and 9B models. With two-stage SFT alone, they achieve 53.1% and 63.1% on HLE-130, and 81.0% and 83.0% on FrontierScience Olympiad respectively. The 9B model also scores 55.0% on FrontierScience Research, showing the constructed data are effective for smaller models too.

Returning to the opening question, how simple can training a strong agent be? Axion 1.0 demonstrates a viable route comprising only two-stage SFT and RL, supported by designed and verified learning tasks: use existing model solutions to guide harder questions, construct public-data questions requiring external evidence, and continue optimizing with rewards from tool interaction.

The team has released data-construction, training and testing code, providing a concrete starting point for researchers seeking to reproduce and improve agent training.

Author team

The authors are from Duke University, ByteDance, the University of Maryland, College Park and the University at Buffalo.

This article comes from Synced. Author: Synced.

After reading

Join the AINRK community

Read how to join the community and connect with people working in AI.

Community information (opens in a new tab)