lacoco-lab

Compositionality in Language and Computation

Course Description: Compositionality — roughly, the ability to correctly process wholes given the ability to correctly process their parts — is a core property of human cognition and especially natural language, where it enables ``infinite use of finite means’’ as known linguistic elements combine to produce novel words and sentences. Recent advances in Natural Language Processing have raised new questions in this domain: are modern artificial neural networks capable of compositional generalization — and for that matter, how capable are humans? This blockseminar briefly reviews foundational and recent work on the core scientific question of compositionality.

If you want to take this class, please register in CMS.

Course Management System: CMS

Instructors: Kate McCurdy. For any questions, please contact me by email: kmccurdy@lst.uni-saarland.de

Time (block seminar): 1-4 pm Monday, Wednesday, and Friday; September 9-13, 2024.

In addition, there will be an introductory lecture + coordination session 3-5 pm Monday June 24, in the Aquarium, 3rd floor in C7 4.

Room: Aquarium, 3rd floor in C7 4

Format and requirements

This is a block seminar course.

Every student will give a 30-minute presentation.

Students that do not present on a given day are expected to prepare a two-page high-level overview which summarizes the day’s assigned reading and explains how the papers relate to each other. The summary should conclude with a question for discussion. These summaries will be submitted by CMS at the start of each classroom session.

Syllabus

Date Theme Reading Presenter/s
2024-09-09 Defining compositionality Herbelot 2020 Bao Di
  Compositionality in ANNs Baroni 2019 Joel Joachim Schnubel
  … in emergent languages Chaabouni et al, 2020 Maximilian Jones Schmidt
2024-09-11 Benchmarking compositionality Kim and Linzen 2020 Amanda Silina
  Compositional representations McCoy et al, 2019 Ansh Dawda
  More representations Lepori et al., 2023 Sundam Adnan Soomro
  Data structures Akyurek and Andreas 2023 Denys Pyshchai
2024-09-13 Comparing to humans Lake and Baroni 2023 Daria Solovieva
    Kumar et al, 2023 Verma Abhishek
    Galke et al, 2024 Anamika Sreeja Sadanandan

Evaluation

For students taking the seminar for 4 credits:

Presentation: 50%
Reading summaries: 50%

For students taking the seminar for 7 credits:

Presentation: 25%
Reading summaries: 25%
Final report: 50%

Presentations

Given time limitations, presentations will be strictly kept to 30 minutes each, followed by a break and then a general discussion covering all of the papers. The presentation should focus on high-level points from the readings, such as the main argument and evidence for and against key claims under consideration.

Term Papers

You will write a 6 page report (ACL format) on one of the two following topics:

  1. Is compositionality a significant concern for modern artificial neural networks (ANNs), or can we consider this a solved problem? Give reasons for or against one of these perspectives, and motivate your points by citing relevant literature. You should address points considered in class, such as various interventions which have been shown to enhance ANNs’ compositional processing, and evidence for and against compositionality in human behavior.
  2. Select an exisiting compositionality benchmark (e.g. SCAN, COGS, SLOG, CFQ) and evaluate at least one proposed approach to improve compositional generalization (e.g. data augmentation, auxiliary tasks/fine-tuning, specialized model architecture) against a standard model baseline. Write up your findings in a technical report.

The report should be uploaded via CMS. The due date will be one month following our final in-person session, i.e. Oct. 13.

Contact

Please contact Kate (kmccurdy@lst.uni-saarland.de) or Michael (mhahn@lst.uni-saarland.de) for any questions.

Accommodations

If you need any accommodations due to a disability or chronic illness, please either contact Michael at mhahn@lst.uni-saarland.de or the Equal Opportunities and Diversity Management Unit of the university.

Optional: additional background reading