Publications

Xin Tong, Thi Thu Uyen Hoang, Xue-Xin Wei, Michael Hahn (2026). The Bayesian Origin of the Probability Weighting Function in Human Representation of Probabilities. NeurIPS 2026.

Preprint

Michael Hahn, Entang Wang, Xue-Xin Wei (2026). Identifiability of Bayesian Models of Perception. Proceedings of the National Academy of Sciences (PNAS).

DOI Paper

Andy Yang, Blerta Veseli, Corentin Barloy, Michaël Cadilhac, Andreas Krebs, Charles Paperman, Howard Straubing, Michael Hahn (2026). Algebraic Decomposition Theory for Transformer Length Generalization. COLM.

Paper

Blanka Kövér, Alexandra Butoi, Anej Svete, Michael Hahn, Ryan Cotterell (2026). Understanding the Parameter Space Geometry of Transformers Encoding Boolean Functions. ICML.

Preprint

Entang Wang, Yiwei Wang, Aleksandra Bakalova, Michael Hahn (2026). How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning. ICML.

Preprint

Oliver Kraus, Yash Sarrof, Yuekun Yao, Alexander Koller, Michael Hahn (2026). Barriers to Universal Reasoning with Transformers (and How to Overcome them). COLM.

Preprint Paper

Yash Sarrof, Yupei Du, Katharina Stein, Alexander Koller, Sylvie Thiébaux, Michael Hahn (2026). On the Ability of Transformers to Verify Plans. ICML.

Preprint Code

Xinting Huang, Aleksandra Bakalova, Satwik Bhattamishra, William Merrill, Michael Hahn (2026). Discovering Interpretable Algorithms by Decompiling Transformers to RASP. ICML.

Preprint Code

Mei-Yen Chen, Thi Thu Uyen Hoang, Michael Hahn, M. Saquib Sarfraz (2026). When Does LoRA Reuse Work? Theoretical Limits and Mechanisms for Recycling LoRAs Without Data Access. TMLR.

Paper Preprint

Mark Rofin, Jalal Naghiyev, Michael Hahn (2026). Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors. ICLR.

Paper Code

Hongjian Jiang, Michael Hahn, Georg Zetzsche, Anthony Widjaja Lin (2026). Softmax Transformers are Turing-Complete. ICLR.

Preprint Paper

Xinting Huang, Michael Hahn (2026). Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning. ICLR.

Preprint Paper Code

Michael Rizvi-Martel, Satwik Bhattamishra, Neil Rathi, Guillaume Rabusseau, Michael Hahn (2026). Benefits and Limitations of Communication in Multi-Agent Reasoning. ICLR.

Preprint Paper Code

Satwik Bhattamishra, Michael Hahn, Varun Kanade (2026). Automata Learning and Identification of the Support of Language Models. ICLR.

Paper

Yuekun Yao, Yupei Du, Dawei Zhu, Michael Hahn, Alexander Koller (2025). Language models can learn implicit multi-hop reasoning, but only if they have lots of training data. EMNLP.

Preprint Paper Code

Mayank Jobanputra, Yana Veitsman, Yash Sarrof, Alexandra Bakalova, Vera Demberg, Ellie Pavlick, Michael Hahn (2025). Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities . NeurIPS 2025.

Preprint Conference Paper Code

Aleksandra Bakalova, Yana Veitsman, Xinting Huang, Michael Hahn (2025). Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B. COLM.

Preprint Paper Code

Richard Futrell, Michael Hahn (2025). Linguistic Structure from a Bottleneck on Sequential Information Processing. Nature Human Behavior.

Paper Preprint Code

Alireza Amiri, Xinting Huang, Mark Rofin, Michael Hahn (2025). Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers. ICML.

Preprint Conference Code Formalization

Xinting Huang, Andy Yang, Satwik Bhattamishra, Yash Sarrof, Andreas Krebs, Hattie Zhou, Preetum Nakkiran, Michael Hahn (2025). A Formal Framework for Understanding Length Generalization in Transformers. ICLR 2025.

Preprint Paper Code

Kate McCurdy, Michael Hahn (2024). Lossy Context Surprisal Predicts Task-Dependent Patterns in Relative Clause Processing. Proceedings of the 28th Conference on Computational Natural Language Learning (CoNLL 2024).

Kate McCurdy, Paul Soulos, Paul Smolensky, Roland Fernandez, Jianfeng Gao (2024). Toward Compositional Behavior in Neural Models: A Survey of Current Views. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024).

Michael Hahn, Mark Rofin (2024). Why are Sensitive Functions Hard for Transformers?. Proceedings of the 2024 Annual Conference of the Association for Computational Linguistics (ACL). 🏆 Best Paper Award.

Cite Preprint

Yash Sarrof, Yana Veitsman, Michael Hahn (2024). The Expressive Capacity of State Space Models: A Formal Language Perspective. Annual Conference on Neural Information Processing Systems (NeurIPS 2024).

Preprint Paper Talk

Satwik Bhattamishra, Michael Hahn, Phil Blunsom, Varun Kanade (2024). Separations in the Representational Capabilities of Transformers and Recurrent Architectures. Annual Conference on Neural Information Processing Systems (NeurIPS 2024).

Preprint

Siyu Tao, Lucia Donatelli, Michael Hahn (2024). More frequent verbs are associated with more diverse valency frames: Efficient language design at the lexicon-grammar interface. Proceedings of the 2024 Annual Conference of the Association for Computational Linguistics (ACL)..

Xinting Huang, Madhur Panwar, Navin Goyal, Michael Hahn (2024). InversionView: A General-Purpose Method for Reading Information from Neural Activations. Annual Conference on Neural Information Processing Systems (NeurIPS 2024)
Previously: Mechanistic Interpretability Workshop at ICML 2024 (oral) 🏆 Awarded Second Place Prize.

Preprint

Michael Hahn, Xue-Xin Wei (2024). A unifying theory explains seemingly contradictory biases in perceptual estimation. Nature Neuroscience.

Cite Paper Preprint Code OSF News

Michael Hahn, Navin Goyal (2023). A theory of emergent in-context learning as implicit structure induction. arXiv Preprint.

Cite Preprint

Thomas Hikaru Clark, Clara Meister, Tiago Pimentel, Michael Hahn, Ryan Cotterell, Richard Futrell, Roger Levy (2023). A Cross-Linguistic Pressure for Uniform Information Density in Word Order. Transactions of the Association for Computational Linguistics.

Cite DOI Paper

Michael Hahn, Frank Keller (2023). Modeling task effects in human reading with neural network-based attention. Cognition.

Cite Preprint Paper Code

Michael Hahn, Richard Futrell, Roger Levy, Edward Gibson (2022). A resource-rational model of human processing of recursive linguistic structure. Proceedings of the National Academy of Sciences of the United States of America.

Cite DOI Paper Code OSF Zenodo (Model) Zenodo (Predictions)

Michael Hahn, Yang Xu (2022). Crosslinguistic word order variation reflects evolutionary pressures of dependency and information locality. Proceedings of the National Academy of Sciences of the United States of America.

Cite DOI Paper Preprint Code

Michael Hahn, Rebecca Mathew, Judith Degen (2022). Morpheme ordering across languages reflects optimization for processing efficiency. Open Mind: Discoveries in Cognitive Science.

Cite DOI Paper Code

Songpeng Yan, Michael Hahn, Frank Keller (2022). Modeling fixation behavior in reading with character-level neural attention. Proceedings of the 44th Annual Meeting of the Cognitive Science Society (CogSci).

Cite Paper

Neil Rathi, Michael Hahn, Richard Futrell (2022). Explaining patterns of fusion in morphological paradigms using the memory--surprisal tradeoff. Proceedings of the 44th Annual Meeting of the Cognitive Science Society (CogSci).

Cite Paper

Richard Futrell, Michael Hahn (2022). Information theory as a bridge between language function and language form. Frontiers in Communication.

Cite DOI Paper

Michael Hahn, Judith Degen, Richard Futrell (2021). Modeling word and morpheme order in natural language as an efficient tradeoff of memory and surprisal. Psychological Review.

Cite DOI Paper Preprint Code

Michael Hahn, Dan Jurafsky, Richard Futrell (2021). Sensitivity as a complexity measure for sequence classification tasks. Transactions of the Association for Computational Linguistics.

Cite Paper Preprint Code

Neil Rathi, Michael Hahn, Richard Futrell (2021). An Information-Theoretic Characterization of Morphological Fusion. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.

Cite Paper Code

Michael Hahn, Dan Jurafsky, Richard Futrell (2020). Universals of word order reflect optimization of grammars for efficient communication. Proceedings of the National Academy of Sciences of the United States of America.

Cite Paper Code

Michael Hahn (2020). Theoretical limitations of self-attention in neural sequence models. Transactions of the Association for Computational Linguistics.

Cite Paper Preprint Formalization Supplement

John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher Manning (2020). RNNs can generate bounded hierarchical languages with optimal memory. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020).

Cite Paper Preprint Code

Michael Hahn, Marco Baroni (2019). Tabula nearly rasa: Probing the linguistic knowledge of character-level neural language models trained on unsegmented text. Transactions of the Association for Computational Linguistics.

Cite Paper Preprint Code

Michael Hahn, Richard Futrell (2019). Estimating predictive rate-distortion curves via neural variational inference . Entropy.

Cite Paper Code

Michael Hahn, Frank Keller, Yonatan Bisk, Yonatan Belinkov (2019). Character-based surprisal as a model of human reading in the presence of errors. Proceedings of the 41st Annual Meeting of the Cognitive Science Society (CogSci).

Cite Paper Preprint Code

Michael Hahn, Andreas Krebs, Howard Straubing (2018). Wreath Products of Distributive Forest Algebras. Logic in Computer Science (LICS).

Cite Paper Preprint Formalization

Michael Hahn, Judith Degen, Noah Goodman, Dan Jurafsky, Richard Futrell (2018). An information-theoretic explanation of adjective ordering preferences. Proceedings of the 40th Annual Meeting of the Cognitive Science Society (CogSci).

Cite Paper Code

Michael Hahn, Frank Keller (2016). Modeling human reading with neural attention. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.

Cite Paper Code

Michael Hahn, Andreas Krebs, Klaus-Jörn Lange, Michael Ludwig (2015). Visibly Counter Languages and the Structure of NC1. Proceedings of Mathematical Foundations of Computer Science (MFCS) 2015.

Cite Paper

Michael Hahn, Frank Richter (2015). Henkin Semantics for Reasoning with Natural Language. Journal of Language Modeling.

Cite Paper Code

Michael Hahn (2014). Predication and NP Structure in an Omnipredicative Language: The Case of Khoekhoe. Proceedings of the 21st International Conference on Head-Driven Phrase Structure Grammar.

Cite Paper

Michael Hahn, Detmar Meurers (2014). On deriving semantic representations from dependencies: A practical approach for evaluating meaning in learner corpora. Dependency Theory.

Cite

Michael Hahn (2013). Word Order Variation in Khoekhoe. Proceedings of the 20th International Conference on Head-Driven Phrase Structure Grammar.

Cite Paper

Niels Ott, Ramon Ziai, Michael Hahn, Detmar Meurers (2013). CoMeT: Integrating different levels of linguistic modeling for meaning assessment. Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval).

Cite Paper

Michael Hahn, Detmar Meurers (2012). Evaluating the Meaning of Answers to Reading Comprehension Questions: A Semantics-Based Approach. Proceedings of the 7th Workshop on Innovative Use of NLP for Building Educational Applications (BEA7).

Cite Paper

Michael Hahn (2012). Arabic Relativization Patterns: A Unified HPSG Analysis. Proceedings of the 19th International Conference on Head-Driven Phrase Structure Grammar.

Cite Paper

Michael Hahn, Detmar Meurers (2011). On deriving semantic representations from dependencies: A practical approach for evaluating meaning in learner corpora. Proceedings of the Int. Conference on Dependency Linguistics (Depling 2011).

Cite Paper

Michael Hahn (2011). Null Conjuncts and Bound Pronouns in Arabic. The Proceedings of the 18th International Conference on Head-Driven Phrase Structure Grammar.

Cite