We are a group of researchers investigating machine learning, language, and the human mind. We are based at Saarland University and Saarland Informatics Campus, Germany.

Our research studies the following topics:

• Abilities of Language Models, and how they are learned: We investigate the abilities and learning biases of transformer models and other architectures, using both theoretical and empirical methods (NeurIPS 2025, ICLR 2025, ICML 2025, ACL 2024, NeurIPS 2024, TACL 2020).

• Interpreting and Analyzing Language Models: We develop methods for mechanistically reverse-engineering the inner workings of language models (COLM 2025, NeurIPS 2024, TACL 2019).

We also do research on Computational Cognition and Neuroscience (PNAS 2022a, PNAS 2020, PNAS 2022b, Nature Neuroscience 2024).

If you’re interested in joining, see here for more.

05/2026 Five papers accepted at ICML 2026, on decompiling Transformers to programs, planning with Transformers, learning attention with queries, in-context leaning, and (un)learnable functions for Transformers.

01/2026 Five papers accepted at ICLR 2026, on unsupervised subspace discovery, seemingly useless features, theory of CoT, automata extraction, and multi-agent systems.

09/2025 Our work on the impact of pretraining on Transformers’ abilities has been accepted at NeurIPS 2025. We again organize the ELLIS pre-NeurIPS poster session at Saarbrücken, with 20 NeurIPS papers from across campus.

05/2025 Our work on lower bounds for chain-of-thought reasoning in transformers has been accepted at ICML 2025.

01/2025 Our work on provable guarantees for length generalization in transformers has been accepted at ICLR 2025.

09/2024 Three papers accepted at NeurIPS 2024, on mechanistic interpretability, state-space models, and separations between architectures. Our lab organized the ELLIS pre-NeurIPS poster session at Saarbrücken, with 17 NeurIPS posters from across campus.

08/2024 Our work on Why are Sensitive Functions Hard for Transformers? received a Best Paper Award at ACL 2024.

07/2024 Our work on InversionView for reading out information from neural activations received the Second Prize at ICML 2024 Mechanistic Interpretability Workshop.

02/2024 Our work on the mathematics underlying perceptual biases has appeared in Nature Neuroscience.

12/2022 MIT News has an article about our PNAS paper on recursion

Every semester, we organize a range of graduate-level seminars where we read current research papers. We also teach some undergraduate classes.

Summer 2026 Lecture Theory of Machine Learning for Language Models (Michael Hahn and Yash Sarrof).
Summer 2026 Seminar Neural Networks in Brains and Computers (Michael Hahn).
See teaching page for past semesters.

See here for information on how to join our lab.