Embedding-space reasoning forest

Forest-of-Thought Reasoning

Forest-of-Thought (FoT) scales reasoning by running several thought trees, activating promising branches sparsely, correcting weak branches, and using consensus over the forest. In ToricGT the same idea is moved from prompt strings into embedding space: each branch is a graph-valued hidden trajectory, each edge is a graph edit or continuous displacement, and branch rewards are tied to BPB delta, verifier score, topological stability, and toric fan margins.

FoT arXiv:2412.09078 · upstream code · ToricBLM continuation

FoT Reference Diagram

Forest-of-Thought paper diagram

Embedding-Space Adaptation

Click or keyboard-select a vertex to reveal its local thought tree and branch metrics. The SVG is a lightweight projection; the training controller works in the original embedding coordinates.

spawn tree branchspawn exploratory branchexpand promising branchself-correction branchindependent tree expansionconsensus edgeconsensus edgeminority but diverse vote prompt root; initializes the embedding-space forestroottree 0 branch; high fan margin, medium BPB deltat0atree 0 branch; exploratory graph editt0btree 1 branch; retrieved analogy and topological stabilityt1atree 1 branch; correction brancht1btree 2 branch; diversity-preserving candidatet2aforest consensus; candidate selected by BPB-delta rewardvote
Click a vertex to reveal the local thought tree, branch role, and training signals for that node.

State

Each state is s=(G,H,Σ,μ,M): a reasoning graph, hidden table, toric/fan chart, moment coordinate, and memory pointer.

Action

Actions add/merge thought nodes, move inside a normal cone, cross a wall, retrieve an analogy, or flatten a graph output for BPB scoring.

Reward

Reward combines byte likelihood improvement with structural diversity, GUDHI persistent-homology stability, and toric/BGG certificate consistency.

Training Equation

LFoT = (log Z + Σt log pF(st+1|st) − log R(x) − Σt log pB(st|st+1))2, with R(x)=exp((ΔBPB + α·diversity + β·fan_margin − γ·certificate_error)/τ).