Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities

Mayank Jobanputra, Yana Veitsman, Yash Sarrof, Alexandra Bakalova, Vera Demberg, Ellie Pavlick, Michael Hahn
May, 2025
Preprint Conference Paper Code
Type
Conference paper
Publication
NeurIPS 2025