Deep NLP

2 episodes · follows chapter 21 of the textbook

Information Extraction: Relations, Events, and Time

Turning text into structured facts. Relation extraction is taught three ways on a synthetic world of fictional entities: Hearst patterns, a Mintz-style feature classifier scored with TACRED-style micro F1, and distant supervision with its label noise measured exactly; entity markers and LLM extraction (with schema and grounding checks) close the first episode. The second builds a rule-based temporal expression recognizer and ISO/TimeML normalizer anchored to the document creation time, Reichenbach tense and Vendler aspect, Allen's interval algebra with a brute-force composition table and path consistency, and MUC-style template filling.

Progress — loading…

Objectives

Language

  • Frame relation extraction as entity-pair classification; know TACRED-style evaluation
  • Explain pattern-based, supervised, distantly supervised, neural and LLM extractors and their failure modes
  • Recognize and normalize temporal expressions; distinguish absolute, deictic and anaphoric anchoring
  • Describe TimeML/TimeBank, tense (Reichenbach) and aspect (Vendler), and template filling (MUC)

Math & data structures

  • Micro P/R/F1 excluding a negative class; one-sided label noise and its effect on P vs R
  • Calendar arithmetic with clamping; ISO weeks
  • Allen's 13 relations, composition by enumeration, path consistency as a fixpoint

Code

  • Hearst extractor, relation features, TACRED-style scorer, distant-labeling experiment
  • Temporal recognizer/normalizer with 70+ tests; Allen relations; timeline plots; template filler
  • PyTorch entity-marker classifier matched to a NumPy reference

Episodes

  1. Relation Extraction19 min · 40 cues

    Patterns, supervised, distant supervision, LLM extraction, evaluation.

  2. Events, Time, and Template Filling20 min · 40 cues

    Event extraction, temporal expressions and normalization, aspect, temporal datasets, templates.

Exercises

ExerciseTierLadderTime
Hearst patterns and a feature-based relation classifier, scored TACRED-style browserPredict · Scratch60 min
Break it: distant supervision, measured label noise, and at-least-once aggregation browserPredict · Break it · Probe a claim60 min
Temporal expression normalization against a document date, plus Allen's interval relations browserPredict · Scratch · Harden · Compose75 min
Port: an entity-marker relation classifier in PyTorch, matched to a NumPy reference gpuPort · Harden75 min

Labs

  • Relation Extraction browser C21.01 lab: relation extraction with patterns, features and distant supervision, all inspectable. Exercises: C21.ex01-relation-extractor (patterns, features, scoring), C21.ex02-distant-supervision (the noise experiment), C21.ex04-entity-marker-torch (the neural port). The corpus is synthetic and FICTIONAL on purpose. · 70 min
  • Events, Time, and Template Filling browser C21.02 lab: temporal expressions, normalization against a document date, Allen relations, timelines and template filling. Exercise: C21.ex03-temporal-normalizer. · 60 min

Library contributions

Files in nlpcore this module teaches you to write.

Sources

  1. Allen, J. F. (1983). Maintaining knowledge about temporal intervals. Communications of the ACM 26(11):832-843 · link
  2. Baldini Soares, FitzGerald, Ling & Kwiatkowski (2019). Matching the Blanks: Distributional Similarity for Relation Learning · arXiv:1906.03158
  3. Fader, Soderland & Etzioni (2011). Identifying Relations for Open Information Extraction. EMNLP 2011 · link
  4. Hobbs, Appelt et al. FASTUS: A Cascaded Finite-State Transducer for Extracting Information from Natural-Language Text (SRI International) · link · arXiv:cmp-lg/9705013
  5. Hearst, M. A. (1992). Automatic Acquisition of Hyponyms from Large Text Corpora. COLING-92, Nantes · link
  6. Mintz, Bills, Snow & Jurafsky (2009). Distant supervision for relation extraction without labeled data. ACL-IJCNLP 2009, pp. 1003-1011 · link
  7. Message Understanding Conferences MUC-1 (1987) to MUC-7 (1997), DARPA · link
  8. Pustejovsky et al. (2003). The TIMEBANK Corpus. Corpus Linguistics 2003, Lancaster, pp. 647-656 · link
  9. Pustejovsky, Castaño, Ingria, Saurí, Gaizauskas, Setzer & Katz (2003). TimeML: Robust Specification of Event and Temporal Expressions in Text · link
  10. Reichenbach, H. (1947). Elements of Symbolic Logic, §51 'The Tenses of Verbs'. New York: Macmillan
  11. Riedel, Yao & McCallum (2010). Modeling relations and their mentions without labeled text. ECML PKDD 2010
  12. Riloff & Jones (1999), Learning Dictionaries for Information Extraction by Multi-Level Bootstrapping (RlogF pattern confidence), as presented in SLP3 §21.2.3
  13. Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft (Aug 19, 2026), Ch. 21 Information Extraction: Relations, Events, and Time · link · §21.1, 21.2, 21.2.1, 21.2.2, 21.2.3, 21.2.4, 21.2.5, 21.2.6, 21.3, 21.4, 21.4.1, 21.5, 21.6, 21.7, 21.7.1, 21.7.2, 21.7.3, 21.8, 21.8.1, 21.8.2, 21.9
  14. Pustejovsky et al. (2006). TimeBank 1.2, LDC2006T08 · link
  15. Vendler, Z. (1957). Verbs and Times. The Philosophical Review 66(2):143-160
  16. Zhang, Zhong, Chen, Angeli & Manning (2017). Position-aware Attention and Supervised Data Improve Slot Filling (TACRED). EMNLP 2017, pp. 35-45 · link