Deep NLP

2 episodes · follows chapter 23 of the textbook

Lexicons for Sentiment, Affect, and Connotation

Words carry feelings, and a lexicon is a dict that writes them down. The module defines emotion (Ekman's basic emotions, Plutchik's wheel, Osgood's valence-arousal-dominance), scores text with lexicons while handling negation, intensifiers and coverage, and asks where the numbers come from: rating scales vs best-worst scaling, replicated in a simulation with a known ground truth. It then induces lexicons from a handful of seeds, by projecting onto an embedding axis and by label propagation on a word graph (SentProp), shows that sentiment is domain-specific, breaks induction three ways, and closes with connotation frames: the feelings verbs imply about their arguments.

Progress — loading…

Objectives

Language

  • Distinguish emotion models (categorical: Ekman, Plutchik; dimensional: valence/arousal/dominance) and their lexicons
  • Use a lexicon responsibly: negation scope, intensifiers, coverage, abstention
  • Explain why best-worst scaling is robust to annotator scale use, and what split-half reliability does and does not measure
  • Induce domain-specific lexicons from seeds and explain why the same word flips polarity across domains
  • Read connotation frames as typed relations between writer, agent and theme

Math & data structures

  • Projection onto a unit direction; difference-of-means axes; cosine decomposition
  • Spearman rank correlation, split-half reliability, ROC AUC as a pairwise probability
  • kNN graphs, symmetric normalization, random walk with restart as a contraction, harmonic (iterated averaging) propagation
  • Log-odds with an informative Dirichlet prior; scaled likelihood across rating categories

Code

  • A lexicon scorer with clause/window negation and shift vs flip, evaluated per phenomenon
  • A BWS design, counting scorer and a seeded simulation study against rating scales
  • Axis induction and SentProp from scratch on PPMI-SVD embeddings (Embeddings pipeline); a domain-specificity probe
  • Three break-it experiments (negation contexts, beta to 1, one seed pair) with fixes
  • A PyTorch port with a supervised axis and logged loss decomposition (GPU tier)

Episodes

  1. Emotion, Affect, and Lexicons18 min · 30 cues

    Defining emotion, available lexicons, human annotation (best-worst scaling).

  2. Inducing Lexicons and Connotation Frames19 min · 31 cues

    Semi-supervised induction from embedding axes, supervised sentiment, connotation frames.

Exercises

ExerciseTierLadderTime
Lexicon scoring with negation scope, intensifiers, and emotion profiles browserPredict · Scratch · Harden45 min
Best-worst scaling vs rating scales: a simulated replication with seeds browserScratch · Probe60 min
A sentiment axis from seed words: projection onto a unit direction browserPredict · Scratch · Compose45 min
Label propagation on a word graph (SentProp) and a domain-specificity probe browserScratch · Probe · Compose75 min
Break lexicon induction: negation, beta to 1, and one seed pair browserBreak it · Probe60 min
Port lexicon induction to PyTorch and learn a supervised sentiment axis gpuPort40 min

Labs

  • Emotion, Affect, and Lexicons browser C23.01 lab: lexicon scoring you can audit, and best-worst scaling vs rating scales in a simulation. Part A scores C04's synthetic reviews with course-made toy lexicons (negation scope, flip vs shift, intensifiers, coverage, emotion profiles). Part B simulates annotation with a known ground truth and measures reliability AND validity. Exercises: C23.ex01-lexicon-scorer, C23.ex02-bws-vs-ratings. · 55 min
  • Inducing Lexicons and Connotation Frames browser C23.02 lab: induce sentiment lexicons from seeds, watch 'soft' change sides, and break induction on purpose. Synthetic two-domain corpora (bedding, sports; nlpcore.lexicons.induce.toy_domain_corpus, course-made) -> PPMI + SVD vectors (C05 pipeline) -> (1) axis projection, (2) SentProp random walk / harmonic propagation, (3) supervised curves from rated reviews; then connotation frames. Exercises: C23.ex03, ex04, ex05 (ex06 on GPU). · 75 min

Library contributions

Files in nlpcore this module teaches you to write.

Sources

  1. Jisun An, Haewoon Kwak & Yong-Yeol Ahn, SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentiment, ACL 2018 · arXiv:1806.05521
  2. Logistic Regression and Text Classification sentiment_mini.tsv (synthetic, 900 reviews with phenomenon tags)
  3. nlpcore.lexicons.affect TOY_* lexicons and nlpcore.lexicons.induce toy corpora: written for this course (CC0)
  4. Sanjiv Das & Mike Chen, Yahoo! for Amazon: Extracting market sentiment from stock message boards, 2001 (NOT_ negation marking)
  5. Paul Ekman, Basic Emotions, in Dalgleish & Power (eds.), Handbook of Cognition and Emotion, 1999
  6. Field, A. & Tsvetkov, Y. (2019). Entity-Centric Contextual Affective Analysis. ACL 2019, as presented in SLP3 §23.8
  7. William L. Hamilton, Kevin Clark, Jure Leskovec & Dan Jurafsky, Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora, EMNLP 2016 · arXiv:1606.02820
  8. Vasileios Hatzivassiloglou & Kathleen McKeown, Predicting the Semantic Orientation of Adjectives, ACL 1997 · link
  9. Svetlana Kiritchenko & Saif M. Mohammad, Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation, ACL 2017 · arXiv:1712.01765
  10. Jordan Louviere, best-worst scaling (1991), as reported in Kiritchenko & Mohammad 2017
  11. Saif M. Mohammad & Peter D. Turney, Crowdsourcing a Word-Emotion Association Lexicon (NRC Emotion Lexicon), Computational Intelligence 2013 · arXiv:1308.6297
  12. Saif M. Mohammad, Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words, ACL 2018 · link
  13. Burt L. Monroe, Michael P. Colaresi & Kevin M. Quinn, Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict, Political Analysis 16(4):372-403, 2008
  14. Charles E. Osgood, George J. Suci & Percy H. Tannenbaum, The Measurement of Meaning, University of Illinois Press, 1957
  15. Robert Plutchik, Emotion: A Psychoevolutionary Synthesis, 1980 (wheel of eight basic emotions in opposing pairs)
  16. Christopher Potts, On the negativity of negation, SALT 20, 2011 (scaled likelihood of words across review ratings)
  17. Hannah Rashkin, Sameer Singh & Yejin Choi, Connotation Frames: A Data-Driven Investigation, ACL 2016 · link · arXiv:1506.02739
  18. Sascha Rothe, Sebastian Ebert & Hinrich Schuetze, Ultradense Word Embeddings by Orthogonal Transformation (Densifier), NAACL 2016 · arXiv:1602.07572
  19. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology 39
  20. Maarten Sap, Marcella Cindy Prasettio, Ari Holtzman, Hannah Rashkin & Yejin Choi, Connotation Frames of Power and Agency in Modern Films, EMNLP 2017 · link
  21. Scherer (2000), typology of affective states, as presented in SLP3 §23.1, Fig. 23.1
  22. Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft (Aug 19, 2026), Ch. 23 Lexicons for Sentiment, Affect, and Connotation · link · §23.1, 23.2, 23.3, 23.4, 23.4.1, 23.4.2, 23.4.3, 23.5, 23.5.1, 23.6, 23.7, 23.8, 23.9, 23.10, Historical Notes
  23. Maite Taboada, Julian Brooke, Milan Tofiloski, Kimberly Voll & Manfred Stede, Lexicon-Based Methods for Sentiment Analysis, Computational Linguistics 37(2), 2011 · link
  24. Peter D. Turney & Michael L. Littman, Measuring Praise and Criticism: Inference of Semantic Orientation from Association, ACM TOIS 21(4), 2003 · arXiv:cs/0309034
  25. Xiaojin Zhu & Zoubin Ghahramani, Learning from labeled and unlabeled data with label propagation, CMU tech report, 2002unverified