Deep NLP

3 episodes · follows chapter 22 of the textbook

Semantic Role Labeling and Meaning Representations

Who did what to whom. The module builds semantic roles from Fillmore's case grammar to PropBank's numbered arguments and FrameNet's frames, shows with a linking table why "subject" is not a role (diathesis alternations), measures selectional preferences as a KL divergence, and turns SRL into predicate-aware BIO tagging with a NumPy tagger and a constrained Viterbi decoder. It ends at meaning itself: first-order logic with an interpreter over a finite model, and lambda calculus with capture-avoiding beta reduction to compose sentence meanings.

Progress — loading…

Objectives

Language

  • Explain thematic roles, proto-roles, PropBank rolesets and FrameNet frames, and when each is the right tool
  • Show how diathesis alternations decouple grammatical function from semantic role
  • Frame SRL as per-predicate sequence labeling, and explain why the predicate indicator is non-negotiable
  • Read a sentence's meaning as a first-order formula, including quantifier scope ambiguity

Math & data structures

  • KL divergence as selectional preference strength (Resnik), with bits as the unit
  • Sparse log-linear models as embedding-bag sums; gradients by scatter-add; AdaGrad
  • Viterbi as constrained search over a BIO grammar; beam search when the state needs a set
  • Propositional logic, predicates and quantifiers (Newstead Ch. 1); model theory as an interpreter
  • Lambda calculus: free/bound variables, capture-avoiding substitution, normal-order reduction, Church numerals

Code

  • Linking tables and Resnik strength/association with a pseudoword evaluation and a prior baseline
  • A from-scratch BIO SRL tagger with gradcheck, loss decomposition, constrained decoding and span F1
  • Ablation experiments with seeds (indicator, constraints, voice) and a shortcut analysis
  • A FOL evaluator verified by property-based testing; a beta reducer and a Montague-style lexicon
  • A PyTorch EmbeddingBag port and a BiLSTM SRL tagger with an indicator embedding (GPU tier)

Episodes

  1. Semantic Roles, PropBank, and FrameNet19 min · 32 cues

    Thematic roles, alternations, PropBank, FrameNet, selectional restrictions.

  2. Semantic Role Labeling Algorithms18 min · 30 cues

    SRL as sequence labeling with predicates; neural SRL; predicate decomposition.

  3. Logical Semantics: Models, Quantifiers, and Lambda18 min · 34 cues

    First-order logic, lambda calculus (dev: closures), compositional semantics, model theory as an interpreter.

Exercises

ExerciseTierLadderTime
Diathesis alternations as a linking table, and Resnik's selectional preferences browserPredict · Scratch50 min
A predicate-aware BIO semantic role labeler from scratch browserScratch · Harden · Compose90 min
Break the SRL tagger: indicator, constraints, and voice as controlled experiments browserBreak it · Probe60 min
Model theory as an interpreter: evaluate first-order logic over a finite world browserPredict · Scratch · Compose45 min
Beta reduction with capture-avoiding substitution, and composing sentence meanings browserScratch · Harden · Compose75 min
Port the SRL tagger to PyTorch (EmbeddingBag) and train a BiLSTM with a predicate indicator gpuPort · Harden60 min

Labs

  • Semantic Roles, PropBank, and FrameNet browser C22.01 lab: semantic roles you can count. Linking tables (diathesis alternations made visible), a FrameNet-style frame across verbs and nouns, and Resnik's selectional preference strength and association, evaluated with pseudowords against a prior-only control. Data: nlpcore.semantics.srl.toy_propbank, a SYNTHETIC PropBank-style corpus written for this course. Exercise: C22.ex01-roles-and-preferences. · 50 min
  • Semantic Role Labeling Algorithms browser C22.02 lab: a BIO semantic role labeler with every object visible. Pipeline: toy PropBank sentence + predicate index -> features (with the predicate indicator) -> feature ids -> sparse softmax tagger (embedding-bag sum) -> constrained Viterbi -> labeled spans -> CoNLL-style F1. Training is logged through nlpcore.track.RunLogger (loss/main, loss/aux/l2, loss/total, grad_norm/global). Data: nlpcore.semantics.srl.toy_propbank (SYNTHETIC, course-made). Exercises: C22.ex02, C22.ex03. · 75 min
  • Logical Semantics: Models, Quantifiers, and Lambda browser C22.03 lab: first-order logic with an interpreter, and lambda calculus that composes sentence meanings. Everything is inspectable: the world's tables, truth tables, evaluation traces with atom counts, the cost of nested quantifiers, every beta-reduction step (including the alpha-rename that prevents capture), and both scope readings. World: nlpcore.semantics.fol.restaurant_world() (course-made). Exercises: C22.ex04, C22.ex05. · 60 min

Library contributions

Files in nlpcore this module teaches you to write.

Sources

  1. Collin F. Baker, Charles J. Fillmore & John B. Lowe, The Berkeley FrameNet Project, COLING-ACL 1998 · link
  2. Alonzo Church, An Unsolvable Problem of Elementary Number Theory, American Journal of Mathematics 58(2):345-363, 1936
  3. Massimiliano Ciaramita & Mark Johnson, Explaining away ambiguity: Learning verb selectional preference with Bayesian networks, arXiv cs/0008020 · arXiv:cs/0008020
  4. Carreras & Marquez, Introduction to the CoNLL-2004 and CoNLL-2005 Shared Tasks: Semantic Role Labeling
  5. CoNLL-2012 shared task data (OntoNotes v5.0) with PropBank SRL frames · link
  6. nlpcore.semantics.srl.toy_propbank: synthetic PropBank-style corpus written for this course (CC0)
  7. Donald Davidson, The Logical Form of Action Sentences, 1967 (event variables); neo-Davidsonian roles (Parsons 1990)
  8. David Dowty, Thematic Proto-Roles and Argument Selection, Language 67(3), 1991
  9. Charles J. Fillmore, The Case for Case, in E. Bach & R. T. Harms (eds.), Universals in Linguistic Theory, Holt, Rinehart and Winston, 1968
  10. Gildea & Jurafsky (2002) trained on FrameNet annotations
  11. Principle of compositionality, usually credited to Gottlob Frege
  12. Gale, Church & Yarowsky 1992; Schuetze 1992 (pseudowords for evaluating disambiguation)unverified
  13. Barwise & Cooper 1981, Generalized Quantifiers and Natural Language ('most' is not first-order definable)
  14. Daniel Gildea & Daniel Jurafsky, Automatic Labeling of Semantic Roles, Computational Linguistics 28(3), 2002 · link
  15. Luheng He, Kenton Lee, Mike Lewis & Luke Zettlemoyer, Deep Semantic Role Labeling: What Works and What's Next, ACL 2017 · link
  16. Beth Levin, English Verb Classes and Alternations, University of Chicago Press, 1993
  17. Richard Montague, The Proper Treatment of Quantification in Ordinary English, in Hintikka, Moravcsik & Suppes (eds.), Approaches to Natural Language, Reidel, 1973 · link
  18. Clive Newstead, An Infinite Descent into Pure Mathematics, Ch. 1 Logical structure (1.1 Propositional logic, 1.2 Variables and quantifiers, 1.3 Logical equivalence) · §Def 1.1.2-1.1.37 (connectives, intro/elim rules), Def 1.2.1-1.2.26 (free/bound, predicates, domain of discourse, forall, exists, exists!), Thm 1.2.33 (exists-forall implies forall-exists), Thm 1.3.6 (truth tables decide propositional equivalence), Thm 1.3.24, 1.3.28 (de Morgan for operators and quantifiers)
  19. NomBank (Meyers et al. 2004), as described in SLP3 §22.4
  20. Martha Palmer, Daniel Gildea & Paul Kingsbury, The Proposition Bank: An Annotated Corpus of Semantic Roles, Computational Linguistics 31(1), 2005 · link
  21. Panini's grammar of Sanskrit (karaka relations), as credited in SLP3's Historical Notes for Ch. 22
  22. Philip Resnik, Selectional constraints: an information-theoretic model and its computational realization, Cognition 61, 1996
  23. Roger C. Schank, Conceptual Information Processing, North-Holland, 1975 (conceptual dependency primitives such as ATRANS, PTRANS, MTRANS, INGEST)
  24. Peng Shi & Jimmy Lin, Simple BERT Models for Relation Extraction and Semantic Role Labeling, arXiv 2019 · arXiv:1904.05255
  25. Jurafsky & Martin, SLP3 (Aug 2026), Appendix H: Logical Representations of Sentence Meaning · link · §H.1, H.2, H.3, H.3.1, H.3.2, H.3.3, H.3.4, H.3.5, H.4, H.5, H.6
  26. Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft (Aug 19, 2026), Ch. 22 Semantic Role Labeling · link · §22.1, 22.2, 22.3, 22.4, 22.5, 22.6, 22.6.1, 22.6.2, 22.6.3, 22.7, 22.7.1, 22.7.2, 22.8, 22.9, Historical Notes
  27. Alan Turing, On Computable Numbers, with an Application to the Entscheidungsproblem, 1936