Parason Read paper

NeurIPS 2026 submission

Parason

Revealing subtask and trial parallelism in LLM reasoning.

Turn long sequential traces into executable parallel programs, then train the model to balance correctness, search, and wall-clock latency.

Read the paper Explore the idea
Comparison of subtask parallelism, where every branch is required, and trial parallelism, where alternative attempts search for a successful result.
AND: collect every subtask OR: explore competing trials
1.7x average acceleration
65.5% trial share on HLE, DeepSeek-V4
4x lower budget for matched accuracy
98.8% parallel trigger ratio at best setting

The missing taxonomy

Not every parallel branch means the same thing.

Parason separates reasoning that divides known work from reasoning that explores uncertainty. The distinction changes how branches are trained, executed, and merged.

OR branches

Trial parallelism

Launch competing attempts when the right route is unknown. One useful path can unlock the answer, while failed attempts still inform the final synthesis.

  • Unknown solution path
  • Alternative hypotheses
  • Exploration is merged into context
Find the correct key
Idea 1 Key 21415 failed
Idea 2 Key 27182 failed
Idea 3 Key 16180 failed
Idea 4 Key 42424 failed
Idea 5 Key 12345 works
OR: one successful route

What the traces reveal

Trial parallelism grows with difficulty.

Across every model studied on Humanity's Last Exam, more than half of the parallelizable steps are speculative trials. Hard reasoning is not only about decomposition. It is also about trying, rejecting, and refining uncertain paths.

Training and systems co-design

From a sequential trace to an executable parallel program.

The same structured contract connects data curation, reinforcement learning, and inference-time dispatch.

  1. 01

    Reveal

    Label existing reasoning stages as required subtasks or speculative trials.

  2. 02

    Structure

    Encode branches with a context-free grammar that stays valid and engine-parseable.

  3. 03

    Optimize

    Use PA-GRPO to reward correctness, shorter critical paths, and useful branch types.

  4. 04

    Execute

    Dispatch each branch as a tool call, run workers concurrently, and merge their outputs.

parallel_trace.parason CFG valid 
<Parallel>
  <Outlines>
    <Trial>factorization</Trial>
    <Trial>modular arithmetic</Trial>
    <Trial>constructive search</Trial>
  </Outlines>
  <Thread>Try factorization.</Thread>
  <Thread>Test modular arithmetic.</Thread>
  <Thread>Run constructive search.</Thread>
</Parallel>

Runtime contract

Minimal engine changes.

XGrammar enforces valid tags. SGLang decodes normally until a parallel region appears, dispatches independent workers, then returns each thread to the main trajectory.

DecodeParseDispatchMerge

Live run

Same token rate, half the wall clock.

A single AIME24 problem decoded twice at an identical 749.6 tok/s. The sequential trace spends every token on one critical path; Parason spawns its threads at </Outlines> and only pays for the longest one.

AIME24 problem 1, answer 204. Both runs share one wall clock at the same decoding rate; the parallel run merges at 5.2s while the sequential run is still decoding at 9.8s.
Sequential tokens 7,496 one decoder, every token on the critical path
Parallel critical path 3,896 longest thread, 6,775 tokens of total work
Wall clock 1.9x 5.2s to merge versus 9.8s to finish

Results

More accuracy when latency is tight.

Parason moves useful work into parallel branches, so the longest generation path grows more slowly than the total reasoning trace.

Model AIME24 Accuracy (%)
B=2,048 B=8,192 B=16,384 B=24,576
SFT only 16.8 41.8 66.8 70.2
PA-GRPO (βsubtask = 0.025) 16.7−0.10 45.2+3.40 68.0+1.20 73.2+3.00
PA-GRPO (βsubtask = 0.050) 26.8+10.00 60.0+18.20 72.2+5.40 74.8+4.60
PA-GRPO (βsubtask = 0.100) 34.7+17.90 60.3+18.50 74.4+7.60 75.7+5.50
AIME24 accuracy under longest-path token budgets. Subscripts show absolute accuracy-point gains relative to SFT only at the same budget.

Critical-path savings

Harder problems expose more parallel work.

Acceleration stays stable as problems get harder, while the absolute number of tokens moved off the critical path increases. Measured wall-clock speedups are 1.62x on easy, 1.38x on medium, and 1.47x on hard problems.

Easy
1.70x 8.8k saved
Medium
1.74x 15.6k saved
Hard
1.73x 21.3k saved
total generated tokens longest path

Benchmark coverage

Competitive accuracy with an 8B model.

Best reported PA-GRPO results across the evaluated settings.

AIME2478.2
AIME2570.6
AMC97.5
Math50094.6
Best average84.7

Read the full paper

Parason: Revealing Subtask- and Trial Parallelism in LLM Reasoning

Open PDF