All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs

TL;DR

Multiple structurally distinct circuits can perform the same LLM task — each one sparse, faithful, and complete, yet sharing almost no edges with the others. This directly contradicts the Functional Anisotropy Hypothesis, the largely implicit assumption in circuit and sheaf discovery (CSD) that a task is implemented by a unique or near-unique internal mechanism. We introduce Overlap-Aware Sheaf Repulsion (OASR) to systematically uncover these competing circuits, and show that the phenomenon holds across major CSD methods (ACDC, EAP, Edge Pruning, DiscoGP) and tasks (IOI, BLiMP, AGA/ANA/DNA, Docstring).

How to Cite

@inproceedings{chen2026allcircuits,
  title     = {All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for {LLM}s},
  author    = {Chen, Xi and Jin, Mingyu and Niu, Jingcheng and Yin, Yutong and Zhao, Jinman and Guo, Bangwei and Metaxas, Dimitris N. and Wang, Zhaoran and Yue, Yutao and Penn, Gerald},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)},
  year      = {2026},
}