Deep NLP

2 episodes · follows chapter 25 of the textbook

Discourse Coherence

Coherence from relations and from entities. RST trees of nuclei and satellites built by a shift-reduce stack machine with a post-order oracle and scored by RST-Parseval; PDTB connectives classified into a 4-class, 16-type sense hierarchy, where context features fix the connectives a most-frequent-sense table cannot. Then centering transitions, the Barzilay & Lapata entity grid, a pairwise shuffle ranker with named losses, baselines and a role-collapse ablation over seeds, and representation-learning and global coherence.

Progress — loading…

Objectives

Language

  • Explain coherence relations (RST nuclearity; PDTB explicit/implicit connectives and senses)
  • Explain centering (Cf, Cp, Cb, transitions, rules) and entity-based coherence; global coherence

Math & data structures

  • Recursive trees, Catalan counts, stack/queue transition systems and their counting invariants
  • Feature conjunctions in a linear softmax; hierarchical label truncation
  • Sparse grids and sliding-window histograms; pairwise logistic ranking; tie-aware discrimination accuracy

Code

  • Oracle, replay with trace and RST-Parseval; a connective sense classifier beating most-frequent-sense
  • Entity grid, shuffle discrimination and an ablation experiment; a PyTorch local coherence discriminator

Episodes

  1. Coherence Relations and Discourse Parsing17 min · 31 cues

    RST, PDTB, discourse parsing.

  2. Centering, Entity Grids, and Global Coherence16 min · 32 cues

    Centering theory, entity grid, representation learning for coherence, global coherence.

Exercises

ExerciseTierLadderTime
RST trees as data: an oracle, a shift-reduce stack machine, and RST-Parseval browserPredict · Scratch60 min
Explicit discourse connectives: most-frequent-sense baseline vs a context-feature softmax over the PDTB hierarchy browserScratch · Harden · Compose75 min
The entity grid from scratch, shuffle discrimination, and a syntax ablation (does grammatical role matter?) browserScratch · Probe · Break it90 min
Port: a local coherence discriminator (Xu et al. 2019 style) in PyTorch, scored by the same shuffle test gpuPort90 min

Labs

Library contributions

Files in nlpcore this module teaches you to write.

Sources

  1. R. Barzilay, M. Lapata, Modeling Local Coherence: An Entity-Based Approach, Computational Linguistics 34(1), 2008 · link
  2. S. E. Brennan, M. W. Friedman, C. J. Pollard, A Centering Approach to Pronouns, ACL 1987
  3. B. J. Grosz, A. K. Joshi, S. Weinstein, Providing a Unified Account of Definite Noun Phrases in Discourse, ACL 1983 · link
  4. B. J. Grosz, A. K. Joshi, S. Weinstein, Centering: A Framework for Modeling the Local Coherence of Discourse, Computational Linguistics 21(2):203-225, 1995 · link
  5. J. R. Hobbs, Coherence and Coreference, Cognitive Science 3(1):67-90, 1979 · link
  6. D. Iter et al., Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models, ACL 2020unverified
  7. J. Kun, A Programmer's Introduction to Mathematics
  8. Z. Lin, H. T. Ng, M.-Y. Kan, A PDTB-styled end-to-end discourse parser, Natural Language Engineering 20(2):151-184, 2014 · arXiv:1011.0835
  9. W. C. Mann, S. A. Thompson, Rhetorical Structure Theory: Toward a functional theory of text organization, Text 8(3):243-281, 1988 · link
  10. D. Marcu, A. Echihabi, An Unsupervised Approach to Recognizing Discourse Relations, ACL 2002
  11. E. Pitler, A. Nenkova, Using Syntax to Disambiguate Explicit Discourse Connectives in Text, ACL-IJCNLP 2009 (short)
  12. R. Prasad, N. Dinesh, A. Lee, E. Miltsakaki, L. Robaldo, A. Joshi, B. Webber, The Penn Discourse TreeBank 2.0, LREC 2008 · link
  13. Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft (Aug 19, 2026), Ch. 25 Discourse Coherence · link · §25.1, 25.1.1, 25.1.2, 25.2, 25.3, 25.3.1, 25.3.2, 25.3.3, 25.4, 25.5, Historical Notes
  14. P. Xu, H. Saghir, J. S. Kang, T. Long, A. J. Bose, Y. Cao, J. C. K. Cheung, A Cross-Domain Transferable Neural Coherence Model, ACL 2019
  15. N. Yu, M. Zhang, G. Fu, Transition-based Neural RST Parsing with Implicit Syntax Features, COLING 2018, pp. 559-570 · link