Introduction to Large Language Models

Announcements

• First class on October 14!
• The lecture on Tuesday 28.10.2025 will be swapped with the practice session on Thursday, (23.10.2025), and both will take place in their usual rooms. So, we’ll have two lectures this week and two practice sessions next week.
  • 23.10.2025: 16:15 - 17:55, Room: S103/221 → Lecture
  • 28.10.2025: 13:30 - 15:10, Room: S306/051 → Practice Session
• The lecture on Tuesday 18.11.2025 will be swapped with the practice session on Thursday, (13.11.2025).

Table of Contents

Course Information

• Instructors:
  • Lectures: Frank Niu;
  • Practice Sessions:
    • Hassan Soliman;
    • Hovhannes Tamoyan.

• Moodle: https://moodle.informatik.tu-darmstadt.de/course/view.php?id=1875

When and Where

Assignments

Schedule & Class Materials

• Textbook for the course.

  • [J&M] Jurafsky and Martin, Speech and Language Processing, 3rd edition (2025 draft).
    • Textbook Website.
    • Some chapters are not yet available for the third edition. In this case, we will use the corresponding chapters from the 2nd edition (available online here).

  This book is available online for free, so the physical copy is optional.

  • [Kracht] Markus Kracht, Introduction to Linguistics.
    • We will use the first few chapters of this book to cover some basic linguistics concepts.

Week 1. Introduction

• Date: 14.10.2025

• Readings:

  • [J&M, 2nd Edition] Chapter 1: Introduction.

• Slides:

  • Lecture Slides: L1 - Introduction.
  • Deep learning refresher: Google Colab.
  • Recording: https://youtu.be/OXNJrklI3II.

• Practice Sessions:

  • Date: 16.10.2025
  • Materials:
    • Google Colab: Introduction (Python and ML Foundations) and its solutions.
    • Recordings: https://drive.google.com/file/d/1T9Lx69gjOpULRlKLKxlBwOL294hqWrGe.

Week 2. CL/NLP Basics

• Date: 21.10.2025
• Readings:
  • [Kracht] Chapter 1: Introduction
  • [J&M] Chapter 2: Words and Tokens
• Slides:
  • Lecture Slides: L2 - CL/NLP Basics.
  • Recording: https://youtu.be/7BZaGM3RoBQ.
• Practice Sessions:
  • Date: 28.10.2025
  • Materials:
    • Colab: Practice Session 2 - CL/NLP Basics.
    • Slides: Practice Session 2 - CL/NLP Basics.
    • Recording: https://drive.google.com/file/d/1DsAH7dmbn9gB08I1ZAOWMz2cL-_UsFwn/view.

Week 3. Word Representation I: Basics

• Date: 23.10.2025
• Readings:
  • [J&M] Section 11.1: Information Retrieval
  • [J&M] Chapter 3: N-gram Language Models
• Slides:
  • Lecture Slides: Word Representation I.
  • Recording: https://youtu.be/KUj9vnmmX1o.
• Practice Sessions:
  • Date: 30.10.2025
  • Materials:
    • Colab: Practice Session 3 - Word Representation I.
    • Slides: Practice Session 3 - Word Representation I.
    • Recording: https://drive.google.com/file/d/1o9J-0s_4SvXOzRPEadmpDFtoRBKEFdDb/view.

Week 4. Word Representation I: Basics (continued)

• Date: 04.11.2025
• Readings:
  • [J&M] Chapter 5: Embeddings
• Slides:
  • Lecture Slides: Word Representation I
• Practice Sessions:
  • Date: 23.10.2025
  • Materials:
    • Slides: Practice Session 4 - Word Representation I (continued)
    • Recording: https://drive.google.com/file/d/1WfYcpbVdHCNvRQMZ3ej28qSMRUnNOuWQ/view.

Week 5. Word Representation II: Statistical Methods and “Language Models”

• Date: 11.11.2025
• Readings:
  • [J&M] Chapter 5: Embeddings
  • [J&M] Chapter 7: Large Language Models
• Slides:
  • Lecture Slides: Word Representation II
• Practice Sessions:
  • Date: 23.10.2025
  • Materials: TBA

Week 6. Transformers

• Date: 13.11.2025
• Readings:
  • [J&M] Chapter 8: Transformers
  • [J&M] Chapter 10: Masked Language Models
• Slides:
  • Lecture Slides: Transformers
• Practice Sessions:
  • Date: 23.10.2025
  • Materials: TBA

Week 7. Natural Language Understanding

• Date: 25.11.2025
• Readings:
  • [J&M] Chapter 20: Information Extraction: Relations, Events, and Time
• Slides:
  • Lecture Slides: TBA
• Practice Sessions:
  • Date: 23.10.2025
  • Materials: TBA

Week 8. Post-Training

• Date: 02.12.2025
• Readings:
  • [J&M] Section 9.1-3: Post-training: Instruction Tuning and Alignment
• Slides:
  • Lecture Slides: TBA
• Practice Sessions:
  • Date: 23.10.2025
  • Materials: TBA

Week 9. Prompting Techniques

• Date: 09.12.2025
• Readings:
  • [J&M] Section 9.4: Test-time Compute
• Slides:
  • Lecture Slides: TBA
• Practice Sessions:
  • Date: 23.10.2025
  • Materials:
    • TBA

Week 10. Interpretability

• Date: 16.12.2025

• Readings:

• Slides:

  • Lecture Slides: TBA

• Practice Sessions:

  • Date: 23.10.2025
  • Materials: TBA

🎅 Winter Break 🎄

Week 11. Information Retrieval

• Date: 13.01.2026
• Readings:
  • [J&M] Chapter 11: Information Retrieval and Retrieval-Augmented Generation
• Slides:
  • Lecture Slides: TBA
• Practice Sessions:
  • Date: 23.10.2025
  • Materials: TBA

Week 12. Question Answering and RAG

• Date: 20.01.2026
• Readings:
  • [J&M] Chapter 11: Information Retrieval and Retrieval-Augmented Generation
• Slides:
  • Lecture Slides: TBA
• Practice Sessions:
  • Date: 23.10.2025
  • Materials: TBA

Week 13. Quantisation

• Date: 27.01.2026
• Readings:
  • [J&M 2nd Edition] Sections 23.2,5: Question Answering
  • [J&M] Sections 11.3-6: Retrieval-Augmented Generation
• Slides:
  • Lecture Slides: TBA

Week 14. Career in NLP & Exam Preparation

• Date: 03.02.2026
• Slides:
  • Lecture Slides: TBA

Final Exam Information

More details to be announced.

Resources

Corrections

• L2 Slides Page 14: the example sentence should be “Every student takes a course.”
  That sentence has two different interpretations:
  1. For every student, there exists a (potentially different) course that the student takes.
     Every student takes a course. Frank took three courses.
  2. There is a single course that every student takes.
     Every student takes a course. The course is Intro2LLM.

Instructions

• Homework submit steps.

Recordings

YouTube Playlist