Publications

Blanka Kövér, Alexandra Butoi, Anej Svete, Michael Hahn, Ryan Cotterell (2026). Understanding the Parameter Space Geometry of Transformers Encoding Boolean Functions. ICML.

Entang Wang, Yiwei Wang, Aleksandra Bakalova, Michael Hahn (2026). How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning. ICML.

Oliver Kraus, Yash Sarrof, Yuekun Yao, Alexander Koller, Michael Hahn (2026). Barriers to Universal Reasoning With Transformers (And How to Overcome Them). arxiv.

Yash Sarrof, Yupei Du, Katharina Stein, Alexander Koller, Sylvie Thiébaux, Michael Hahn (2026). On the Ability of Transformers to Verify Plans. ICML.

Xinting Huang, Aleksandra Bakalova, Satwik Bhattamishra, William Merrill, Michael Hahn (2026). Discovering Interpretable Algorithms by Decompiling Transformers to RASP. ICML.

Mei-Yen Chen, Thi Thu Uyen Hoang, Michael Hahn, M. Saquib Sarfraz (2026). When Does LoRA Reuse Work? Theoretical Limits and Mechanisms for Recycling LoRAs Without Data Access. TMLR.

Mark Rofin, Jalal Naghiyev, Michael Hahn (2026). Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors. ICLR.

Hongjian Jiang, Michael Hahn, Georg Zetzsche, Anthony Widjaja Lin (2026). Softmax Transformers are Turing-Complete. ICLR.

Xinting Huang, Michael Hahn (2026). Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning. ICLR.

Michael Rizvi-Martel, Satwik Bhattamishra, Neil Rathi, Guillaume Rabusseau, Michael Hahn (2026). Benefits and Limitations of Communication in Multi-Agent Reasoning. ICLR.

Satwik Bhattamishra, Michael Hahn, Varun Kanade (2026). Automata Learning and Identification of the Support of Language Models. ICLR.

Yuekun Yao, Yupei Du, Dawei Zhu, Michael Hahn, Alexander Koller (2025). Language models can learn implicit multi-hop reasoning, but only if they have lots of training data. EMNLP.

Preprint Paper Code

Mayank Jobanputra, Yana Veitsman, Yash Sarrof, Alexandra Bakalova, Vera Demberg, Ellie Pavlick, Michael Hahn (2025). Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities . NeurIPS 2025.

Preprint Conference Paper Code

Aleksandra Bakalova, Yana Veitsman, Xinting Huang, Michael Hahn (2025). Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B. COLM.

Preprint Paper Code

Richard Futrell, Michael Hahn (2025). Linguistic Structure from a Bottleneck on Sequential Information Processing. Nature Human Behavior.

Alireza Amiri, Xinting Huang, Mark Rofin, Michael Hahn (2025). Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers. ICML.

Preprint Conference Code Formalization

Xinting Huang, Andy Yang, Satwik Bhattamishra, Yash Sarrof, Andreas Krebs, Hattie Zhou, Preetum Nakkiran, Michael Hahn (2025). A Formal Framework for Understanding Length Generalization in Transformers. ICLR 2025.

Preprint Paper Code

Kate McCurdy, Michael Hahn (2024). Lossy Context Surprisal Predicts Task-Dependent Patterns in Relative Clause Processing. Proceedings of the 28th Conference on Computational Natural Language Learning (CoNLL 2024).

Kate McCurdy, Paul Soulos, Paul Smolensky, Roland Fernandez, Jianfeng Gao (2024). Toward Compositional Behavior in Neural Models: A Survey of Current Views. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024).

Michael Hahn, Mark Rofin (2024). Why are Sensitive Functions Hard for Transformers?. Proceedings of the 2024 Annual Conference of the Association for Computational Linguistics (ACL). 🏆 Best Paper Award.

Yash Sarrof, Yana Veitsman, Michael Hahn (2024). The Expressive Capacity of State Space Models: A Formal Language Perspective. Annual Conference on Neural Information Processing Systems (NeurIPS 2024).

Satwik Bhattamishra, Michael Hahn, Phil Blunsom, Varun Kanade (2024). Separations in the Representational Capabilities of Transformers and Recurrent Architectures. Annual Conference on Neural Information Processing Systems (NeurIPS 2024).

Siyu Tao, Lucia Donatelli, Michael Hahn (2024). More frequent verbs are associated with more diverse valency frames: Efficient language design at the lexicon-grammar interface. Proceedings of the 2024 Annual Conference of the Association for Computational Linguistics (ACL)..

Xinting Huang, Madhur Panwar, Navin Goyal, Michael Hahn (2024). InversionView: A General-Purpose Method for Reading Information from Neural Activations. Annual Conference on Neural Information Processing Systems (NeurIPS 2024)
Previously: Mechanistic Interpretability Workshop at ICML 2024 (oral) 🏆 Awarded Second Place Prize.

Michael Hahn, Xue-Xin Wei (2024). A unifying theory explains seemingly contradictory biases in perceptual estimation. Nature Neuroscience.

Cite URL Preprint Code OSF News

Michael Hahn, Navin Goyal (2023). A theory of emergent in-context learning as implicit structure induction. arXiv Preprint.

Thomas Hikaru Clark, Clara Meister, Tiago Pimentel, Michael Hahn, Ryan Cotterell, Richard Futrell, Roger Levy (2023). A Cross-Linguistic Pressure for Uniform Information Density in Word Order. Transactions of the Association for Computational Linguistics.

Michael Hahn, Frank Keller (2023). Modeling task effects in human reading with neural network-based attention. Cognition.

Michael Hahn, Richard Futrell, Roger Levy, Edward Gibson (2022). A resource-rational model of human processing of recursive linguistic structure. Proceedings of the National Academy of Sciences of the United States of America.

Cite DOI URL Code OSF Zenodo (Model) Zenodo (Predictions)

Michael Hahn, Yang Xu (2022). Crosslinguistic word order variation reflects evolutionary pressures of dependency and information locality. Proceedings of the National Academy of Sciences of the United States of America.

Cite DOI URL Code

Michael Hahn, Rebecca Mathew, Judith Degen (2022). Morpheme ordering across languages reflects optimization for processing efficiency. Open Mind: Discoveries in Cognitive Science.

Cite DOI URL Code

Songpeng Yan, Michael Hahn, Frank Keller (2022). Modeling fixation behavior in reading with character-level neural attention. Proceedings of the 44th Annual Meeting of the Cognitive Science Society (CogSci).

Neil Rathi, Michael Hahn, Richard Futrell (2022). Explaining patterns of fusion in morphological paradigms using the memory--surprisal tradeoff. Proceedings of the 44th Annual Meeting of the Cognitive Science Society (CogSci).

Richard Futrell, Michael Hahn (2022). Information theory as a bridge between language function and language form. Frontiers in Communication.

Michael Hahn, Judith Degen, Richard Futrell (2021). Modeling word and morpheme order in natural language as an efficient tradeoff of memory and surprisal. Psychological Review.

PDF Cite DOI Code

Michael Hahn, Dan Jurafsky, Richard Futrell (2021). Sensitivity as a complexity measure for sequence classification tasks. Transactions of the Association for Computational Linguistics.

Neil Rathi, Michael Hahn, Richard Futrell (2021). An Information-Theoretic Characterization of Morphological Fusion. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.

Michael Hahn, Dan Jurafsky, Richard Futrell (2020). Universals of word order reflect optimization of grammars for efficient communication. Proceedings of the National Academy of Sciences of the United States of America.

Michael Hahn (2020). Theoretical limitations of self-attention in neural sequence models. Transactions of the Association for Computational Linguistics.

PDF Cite Preprint Formalization Supplement

John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, Christopher Manning (2020). RNNs can generate bounded hierarchical languages with optimal memory. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020).

Michael Hahn, Marco Baroni (2019). Tabula nearly rasa: Probing the linguistic knowledge of character-level neural language models trained on unsegmented text. Transactions of the Association for Computational Linguistics.

Michael Hahn, Richard Futrell (2019). Estimating predictive rate-distortion curves via neural variational inference . Entropy.

Michael Hahn, Frank Keller, Yonatan Bisk, Yonatan Belinkov (2019). Character-based surprisal as a model of human reading in the presence of errors. Proceedings of the 41st Annual Meeting of the Cognitive Science Society (CogSci).

Michael Hahn, Andreas Krebs, Howard Straubing (2018). Wreath Products of Distributive Forest Algebras. Logic in Computer Science (LICS).

PDF Cite Formalization

Michael Hahn, Judith Degen, Noah Goodman, Dan Jurafsky, Richard Futrell (2018). An information-theoretic explanation of adjective ordering preferences. Proceedings of the 40th Annual Meeting of the Cognitive Science Society (CogSci).

Michael Hahn, Frank Keller (2016). Modeling human reading with neural attention. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.

Michael Hahn, Andreas Krebs, Klaus-Jörn Lange, Michael Ludwig (2015). Visibly Counter Languages and the Structure of NC1. Proceedings of Mathematical Foundations of Computer Science (MFCS) 2015.

Michael Hahn, Frank Richter (2015). Henkin Semantics for Reasoning with Natural Language. Journal of Language Modeling.

Michael Hahn (2014). Predication and NP Structure in an Omnipredicative Language: The Case of Khoekhoe. Proceedings of the 21st International Conference on Head-Driven Phrase Structure Grammar.

Michael Hahn, Detmar Meurers (2014). On deriving semantic representations from dependencies: A practical approach for evaluating meaning in learner corpora. Dependency Theory.

Michael Hahn (2013). Word Order Variation in Khoekhoe. Proceedings of the 20th International Conference on Head-Driven Phrase Structure Grammar.

Niels Ott, Ramon Ziai, Michael Hahn, Detmar Meurers (2013). CoMeT: Integrating different levels of linguistic modeling for meaning assessment. Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval).

Michael Hahn, Detmar Meurers (2012). Evaluating the Meaning of Answers to Reading Comprehension Questions: A Semantics-Based Approach. Proceedings of the 7th Workshop on Innovative Use of NLP for Building Educational Applications (BEA7).

Michael Hahn (2012). Arabic Relativization Patterns: A Unified HPSG Analysis. Proceedings of the 19th International Conference on Head-Driven Phrase Structure Grammar.

Michael Hahn, Detmar Meurers (2011). On deriving semantic representations from dependencies: A practical approach for evaluating meaning in learner corpora. Proceedings of the Int. Conference on Dependency Linguistics (Depling 2011).

Michael Hahn (2011). Null Conjuncts and Bound Pronouns in Arabic. The Proceedings of the 18th International Conference on Head-Driven Phrase Structure Grammar.