Deep NLP

15 episodes

AI Safety: Auditing Data and Models, Evaluations, Red Teaming, Steering, and Oversight

How the people who audit AI systems actually work, rebuilt small enough to run in a browser or on one 16 GB GPU. The Safety Landscape: Threat Models and Who Does What-Auditing Models with Interpretability cover the threat models and the ecosystem of labs, evaluators and government institutes; dataset audits (documentation, memorisation and canary exposure, membership inference, data attribution); capability evaluation with METR's time-horizon method; frontier safety frameworks and eval-gated decisions; red teaming measured with confidence intervals; and auditing models with interpretability. Steering Models: Activation Addition, Representation Engineering, and the Refusal Direction-Building an Audit Pipeline End to End on 16 GB cover steering, Procrustes alignment of representation spaces, scalable oversight and AI control, honesty and deception evaluations, and an end-to-end audit. AI Law and Governance: What Is in Force, What Is Coming-Explainability: Methods, Faithfulness, and What Regulators Mean by It cover law, governance and explainability; AI and Civil Rights: Protected Domains and Safe, Lawful Deployment-Measuring Fairness and Auditing for Bias cover civil rights, fairness measurement and bias audits. Framing is defensive and educational throughout: no hazardous technical content and no working attack strings.

Progress — loading…

Objectives

Language

  • Name the main threat models (misuse, misalignment, accidents, structural risks) and say which method in this module produces evidence about each
  • Explain how third-party evaluators and government institutes feed findings back to frontier labs (pre-deployment access, system cards, safety frameworks)
  • Audit a dataset: documentation (datasheets, data cards), memorisation (canaries, extraction), membership inference and its LLM caveats, and data attribution
  • Read an evaluation claim critically: elicitation, sandbagging, contamination, eval awareness, and what a time-horizon number does and does not mean
  • Design and measure a red-team campaign and a defense, and audit a model for a hidden behaviour with behavioural and white-box tools

Math & data structures

  • Ranks, log2 and exposure in bits; ROC curves, AUC as a probability, TPR at low FPR
  • First-order Taylor expansions; gradient dot products; the Hessian and the influence-function derivation; Newton's method
  • Logistic regression on log task length, the 50% and 80% horizons, hierarchical bootstrap, log-linear trends and doubling times
  • Standard error, Wilson intervals, clustered and paired bootstraps, pass@k
  • Decision rules under uncertainty (rule-in / rule-out with confidence bounds), sample-size planning
  • Sparse dictionaries (SAEs), relative decoder norms for model diffing

Code

  • nlpcore.safety.data_audit: canary insertion, exact and fitted exposure, LOSS/reference/zlib membership attacks with ROC
  • nlpcore.safety.attribution: gradient-dot, TracIn, influence functions, leave-one-out ground truth
  • nlpcore.safety.evals: per-item logged harness with bootstrap and clustered CIs
  • nlpcore.safety.time_horizon: METR-style horizon fit, hierarchical bootstrap, trend and doubling time
  • nlpcore.safety.redteam: automated red-team loop vs toy targets, ASR with CIs, defense trade-off curves

Episodes

  1. The Safety Landscape: Threat Models and Who Does What17 min · 27 cues

    What can go wrong (misuse, misalignment, accidents, structural risks); the ecosystem: frontier-lab safety teams, third-party evaluators and research orgs (METR, Apollo Research, Transluce, Redwood Research, UK AI Security Institute, US CAISI, Epoch AI) and how their findings feed back into labs (pre-deployment evals, system cards, frontier safety frameworks / RSPs); how a researcher enters the field.

  2. Auditing Datasets: Documentation, Memorization, Membership, and Attribution17 min · 28 cues

    Datasheets for Datasets, Data Cards, data provenance and licensing audits; auditing for PII, toxicity and harmful content (pipeline level); memorization and training-data extraction (Carlini et al.); membership inference; data attribution (influence functions, TRAK); contamination audits (link Contamination: When Your Test Set Is in Your Training Set).

  3. Evaluating Capabilities Like an Evaluator: Benchmarks, Elicitation, and METR's Time Horizons16 min · 28 cues

    Eval design and validity; elicitation (scaffolding, best-of-n) vs underclaiming; sandbagging; METR's task-completion time-horizon method (logistic fit of success vs human task length, 50% horizon, trend over time) derived and reproduced on synthetic data; inspect-style eval harnesses (UK AISI Inspect).

  4. Dangerous-Capability Evaluations and Frontier Safety Frameworks15 min · 29 cues

    Capability thresholds, RSPs / preparedness frameworks, autonomy and cyber evals at a conceptual level, uplift studies, eval-gated deployment decisions; what system cards report; limits of evals (unknown unknowns, eval awareness). No hazardous technical detail.

  5. Red Teaming: Measuring Robustness and the Cost of Defenses15 min · 29 cues

    Red-teaming methodology and taxonomies; automated red teaming with LMs (Perez et al. 2022); adversarial optimization against models in general terms (e.g., GCG, Zou et al. 2023) and transferability; prompt injection in agents; many-shot effects; defenses (input/output classifiers, constitutional classifiers, adversarial training) and how to measure robustness (attack success rate with CIs). Defensive framing; no working jailbreak strings.

  6. Auditing Models with Interpretability16 min · 29 cues

    Auditing games for hidden objectives (Marks et al. 2025); SAEs and feature search for auditing; model diffing and crosscoders; behavioral vs white-box audits; investigator agents and tools (e.g., Transluce's work on automated investigation and monitoring); limits and false confidence.

  7. Steering Models: Activation Addition, Representation Engineering, and the Refusal Direction19 min · 30 cues

    Steering vectors (Turner et al.), contrastive activation addition (Rimsky et al.), representation engineering (Zou et al. 2023), the single refusal direction finding (Arditi et al. 2024), concept erasure (LEACE) — as tools for understanding and control, with their failure modes and dual-use considerations.

  8. Procrustes Alignment: Comparing and Aligning Representation Spaces19 min · 31 cues

    Orthogonal Procrustes problem derived from zero (minimize ||WA − B|| over orthogonal W; solution via SVD of B Aᵀ — Schönemann 1966); cross-lingual embedding alignment (Mikolov 2013; Smith et al. 2017; Conneau et al. 2018 MUSE); aligning embeddings across time (Hamilton et al. 2016); comparing models' representations (CKA, SVCCA) and transferring probes/steering vectors between models or checkpoints; why orthogonality (preserves distances) and when it fails.

  9. Scalable Oversight and AI Control17 min · 32 cues

    Debate, weak-to-strong generalization (Burns et al. 2023), AI control protocols (trusted monitoring, untrusted monitoring, Greenblatt et al. 2023), chain-of-thought monitoring and its fragility, reward-hacking detection; how control evaluations are scored.

  10. Deception, Sycophancy, Scheming, and Honesty Evaluations16 min · 30 cues

    Sycophancy measurement, in-context scheming evaluations (Apollo Research), alignment-faking findings (Greenblatt et al. 2024), sleeper agents (Hubinger et al. 2024), honesty and calibration metrics, eval awareness; what these studies do and do not show.

  11. Building an Audit Pipeline End to End on 16 GB15 min · 29 cues

    From threat model to report: dataset audit, capability eval, red-team, interpretability audit, statistics with CIs (link Designing an Experiment II: Variance, Significance, and Scaling Curves), documentation, responsible disclosure, and how findings are communicated to labs; career paths and how to contribute.

  12. AI Law and Governance: What Is in Force, What Is Coming19 min · 31 cues

    Status as of the build date, verified from primary sources: EU AI Act (risk tiers, prohibited practices, general-purpose AI model obligations and systemic-risk duties, the GPAI Code of Practice, high-risk system requirements, application timeline, and any amendments or delays such as the Digital Omnibus proposals); US federal policy (executive orders issued/revoked, AI Action Plan, NIST AI RMF and AI 800-series guidance, CAISI) and state laws (e.g., California SB 53, Colorado AI Act, New York RAISE Act, others enacted or pending); UK (AISI, sector-regulator approach); China (generative AI measures, labeling rules); international (Council of Europe Framework Convention, G7 Hiroshima process, OECD); standards (ISO/IEC 42001, ISO/IEC 23894). Map each obligation to the technical artifacts an ML researcher produces (evals, documentation, incident reporting, transparency, watermarking/labeling). Not legal advice; date-stamped; flags what is proposed vs enacted.

  13. Explainability: Methods, Faithfulness, and What Regulators Mean by It20 min · 32 cues

    Feature attribution from zero: gradients x input, Integrated Gradients (axioms derived), LIME, SHAP (Shapley values derived from cooperative game theory), attention-as-explanation caveats; counterfactual and contrastive explanations; concept-based methods (TCAV) and their link to probes/SAEs; evaluating faithfulness (deletion/insertion curves, ROAR, sanity checks — Adebayo et al. 2018); explaining LLM outputs (citations, chain-of-thought faithfulness research); regulatory notions: GDPR Art. 22 and the 'right to explanation' debate, EU AI Act transparency and explanation-related provisions, adverse-action notices in US credit law as a concrete precedent.

  14. AI and Civil Rights: Protected Domains and Safe, Lawful Deployment19 min · 31 cues

    Anti-discrimination law as it applies to automated decisions, as a dated snapshot verified from primary sources (not legal advice): protected characteristics; disparate treatment vs disparate impact; domains and their laws — employment (US Title VII, ADA, ADEA, EEOC positions; NYC Local Law 144 bias audits; state laws on AI in hiring), credit (ECOA/Regulation B adverse-action notices, CFPB positions), housing (Fair Housing Act), insurance (e.g., Colorado SB21-169), healthcare (ACA Section 1557 rule on patient-care decision-support tools), education, public benefits, criminal justice (risk assessment controversy, COMPAS/ProPublica); EU: AI Act Annex III high-risk uses and fundamental-rights impact assessments, GDPR Art. 22, non-discrimination directives; how enforcement priorities have shifted (verify current federal posture). Real cases and incidents (verified) and what went wrong; safe deployment practice: purpose limitation, human oversight with real authority, notice and explanation, appeal and recourse, accessibility, documentation, ongoing monitoring, less-discriminatory-alternative search, when NOT to automate.

  15. Measuring Fairness and Auditing for Bias18 min · 29 cues

    Fairness metrics derived from the confusion matrix: demographic parity, the four-fifths (80%) rule, equal opportunity, equalized odds, predictive parity, calibration within groups; impossibility results (Kleinberg, Mullainathan & Raghavan 2016; Chouldechova 2017) derived with a tiny numeric example; proxies and redundant encoding; intersectionality (Buolamwini & Gebru 2018); LLM bias audits: counterfactual name/attribute swaps, BBQ, WinoBias (link Coreference Resolution and Entity Linking), audit studies of LLM resume ranking (verify sources); statistical rigor (CIs, multiple comparisons, small-group variance); mitigation (pre-, in-, post-processing; threshold adjustment and its legal limits); documenting an audit.

Exercises

ExerciseTierLadderTime
Read a system card like an auditor: map sections to threat models, methods and evaluators; find the gaps browserPredict · Scratch · design45 min
Plant a canary, measure its exposure, and watch memorisation grow with repetition browserPredict · Scratch · Compose · Probe a claim60 min
Membership inference from scratch, and the distribution-shift trap browserPredict · Scratch · Break it · Probe a claim60 min
Data attribution on real sentences: gradient dot products, TracIn and influence functions vs leave-one-out browserPredict · Scratch · Compose · Probe a claim75 min
A mini eval harness: per-item logs, reproducible sampling, clustered error bars browserPredict · Scratch · Harden · Compose60 min
Reproduce METR's time-horizon method on synthetic agents: fit, bootstrap, trend, elicitation, sandbagging browserPredict · Scratch · Compose · Probe a claim90 min
Design an eval plan and an eval-gated decision rule for a harmless toy capability browserPredict · Scratch · design60 min
An automated red-team loop against a toy target: ASR with intervals and the defense trade-off curve browserPredict · Scratch · Compose75 min
Break it: a defense that scores 0% ASR on a random split, and the evaluation that tells the truth browserPredict · Break it · Probe a claim60 min
Plant a trigger in a tiny model, then find it: behavioural scan, direct token effects, SAE feature search, data search browserPredict · Scratch · Probe a claim90 min
Model diffing: what did fine-tuning add? Diffed audits vs a crosscoder-style dictionary browserPredict · Scratch · Probe a claim75 min
Canary exposure in a tiny GPT you train yourself (PyTorch port of S.ex02) gpuPort · Probe a claim60 min
Steering vectors from scratch: difference of means, addition, ablation, and the erasure leak browserPredict · Scratch · Break it · Compose60 min
Steer your own GPT: capture activations with hooks, add a vector, sweep behaviour against perplexity gpuPort · Harden · Probe a claim90 min
Orthogonal Procrustes from the SVD, CSLS retrieval, linear CKA, and carrying a probe across models browserPredict · Scratch · Compose · Probe a claim75 min
Simulate an AI-control protocol: audit budgets, a weak monitor, a red team that picks its best attack rate browserScratch · Harden · Probe a claim70 min
Calibration from scratch (ECE, Brier, temperature) and a sycophancy pushback eval with a control prompt browserPredict · Scratch · Break it70 min
One audit, end to end: compose the module's tools into findings, a report and compliance artifacts browserPredict · Compose · Harden90 min
A dated compliance matrix, incident clocks, and a model card from your run logs browserPredict · Scratch · Harden · Compose60 min
Attribution from the axioms: gradient x input, Integrated Gradients, exact and sampled Shapley, reason codes browserPredict · Scratch · Compose60 min
LIME from scratch, and probing the claim that the Shapley kernel turns LIME into Shapley values browserPredict · Harden · Probe a claim50 min
Is the explanation faithful? Deletion/insertion curves and the cascading randomization sanity check browserPredict · Scratch · Break it · Probe a claim60 min
Every fairness metric from four counters: confusion matrices by group, the four-fifths rule, odds, parity and calibration browserPredict · Scratch · Compose50 min
The impossibility result, live: check the conservation law, then equalize false-positive rates and watch predictive parity break browserPredict · Scratch · Break it · Probe a claim50 min
A counterfactual name-swap audit harness with paired CIs and Holm correction, and a probe of name-blinding browserScratch · Harden · Probe a claim60 min
Search for a less discriminatory alternative: a model family x group-blind thresholds under an accuracy constraint browserPredict · Scratch · Probe a claim50 min
Compose a deployment review for an LLM resume screener: audit numbers, obligations matrix, monitoring plan, generated document browserScratch · Harden · Compose60 min

Labs

Library contributions

Files in nlpcore this module teaches you to write.

Sources

  1. HHS OCR final rule, Nondiscrimination in Health Programs and Activities, 89 FR (6 May 2024), 45 CFR 92.210; effective 5 Jul 2024; 92.210(b)-(c) from 1 May 2025 · link
  2. ADA Title I, 42 U.S.C. 12112(b)(5)-(7) · link
  3. Adebayo, Gilmer, Muelly, Goodfellow, Hardt & Kim, Sanity Checks for Saliency Maps, 2018 · arXiv:1810.03292
  4. AI Safety Institute article (UK renamed AI Security Institute Feb 2025; US AISI renamed CAISI Jun 2025; UK founded Nov 2023) · link
  5. UK AISI & US AISI, Pre-Deployment Evaluation of Anthropic's Upgraded Claude 3.5 Sonnet (19 Nov 2024) · link
  6. An, Huang, Lin & Tai, 'Measuring gender and racial biases in large language models: Intersectional evidence from automated resume evaluation', PNAS Nexus 4(3) pgaf089 (2025); preprint arXiv 2403.15281 · link · arXiv:2403.15281
  7. Anil et al., Many-shot Jailbreaking, NeurIPS 2024 (Anthropic, Apr 2024) · link
  8. Anthropic, Responsible Scaling Policy: version history (1.0 Sept 2023; 2.0 Oct 2024; 2.1 Mar 2025; 2.2 May 2025; 3.0 Feb 24 2026; 3.1-3.4 Apr-Jul 2026) · link
  9. Anthropic, Responsible Scaling Policy v3 announcement (Feb 24, 2026) · link
  10. Apollo Research, About page · link
  11. Arditi, Obeso, Syed, Paleka, Panickssery, Gurnee & Nanda, Refusal in Language Models Is Mediated by a Single Direction, NeurIPS 2024 · link · arXiv:2406.11717
  12. Armstrong, Liu, MacNeil & Metaxa, 'The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring', arXiv 2405.04412 · link · arXiv:2405.04412
  13. Baker et al., Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation (OpenAI, Mar 2025) · link · arXiv:2503.11926
  14. Bao et al., 'It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks', NeurIPS 2021 Datasets and Benchmarks, arXiv 2106.05498 · link · arXiv:2106.05498
  15. Belrose, Schneider-Joseph, Ravfogel, Cotterell, Raff & Biderman, LEACE: Perfect linear concept erasure in closed form, NeurIPS 2023 · link · arXiv:2306.03819
  16. Black, Koepke, Kim, Barocas & Hsu, 'Less Discriminatory Algorithms', Georgetown Law Journal 113(1), Oct 2024 · link
  17. Bricken et al., Towards Monosemanticity: Decomposing Language Models With Dictionary Learning (Transformer Circuits, 2023) · linkunverified
  18. Brier, Verification of Forecasts Expressed in Terms of Probability, Monthly Weather Review (1950) · link
  19. Buolamwini & Gebru, 'Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification', FAT* 2018, PMLR 81:1-15 · link
  20. Burns et al., Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision (OpenAI, Dec 2023) · link · arXiv:2312.09390
  21. Cal. Bus. & Prof. Code 22757.13 (critical safety incidents: 15 days to OES; 24 hours if imminent risk) · link
  22. California Civil Rights Council automated-decision system regulations, approved 27 Jun 2025, effective 1 Oct 2025 · link
  23. California SB 53 (Transparency in Frontier AI Act), signed 29 Sep 2025; effective 1 Jan 2026 (Hunton summary of the signed bill) · link
  24. Mayer Brown, California enacts SB 53 (thresholds 10^26 ops, $500M; transparency reports; penalties) · link
  25. Carlini, Liu, Erlingsson, Kos & Song, The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks, USENIX Security 2019 · arXiv:1802.08232
  26. Carlini, Tramèr, Wallace, Jagielski, Herbert-Voss, Lee, Roberts, Brown, Song, Erlingsson, Oprea & Raffel, Extracting Training Data from Large Language Models, USENIX Security 2021 · arXiv:2012.07805
  27. Carlini, Ippolito, Jagielski, Lee, Tramèr & Zhang, Quantifying Memorization Across Neural Language Models (ICLR 2023) · arXiv:2202.07646
  28. Castro, Gomez & Tejada, Polynomial calculation of the Shapley value based on sampling, Computers & OR 2009unverified
  29. CFPB withdrawn guidance list: Circulars 2022-03 and 2023-03 withdrawn on 12 May 2025 · link
  30. Chen et al., Evaluating Large Language Models Trained on Code (Codex), 2021, unbiased pass@k estimator · arXiv:2107.03374
  31. Chouldechova, 'Fair prediction with disparate impact: A study of bias in recidivism prediction instruments', arXiv 1610.07524 (Big Data, 2017) · link · arXiv:1610.07524
  32. CJEU, Case C-203/22, Dun & Bradstreet Austria, judgment of 27 Feb 2025 (ECLI:EU:C:2025:117) · link
  33. CJEU, Case C-634/21, SCHUFA Holding (Scoring), judgment of 7 Dec 2023 · link
  34. Clark & Amodei, Faulty reward functions in the wild, OpenAI blog (21 Dec 2016) · link
  35. CAC et al., Measures for Labeling AI-Generated Synthetic Content (promulgated 7 Mar 2025, effective 1 Sep 2025), China Law Translate translation · link
  36. Colorado SB21-169 and amended Regulation 10-1-1 (effective 15 Oct 2025), Colorado Division of Insurance · link
  37. Colorado SB 26-189 (Automated Decision-Making Technology), enrolled text; applies to consequential decisions on or after 1 Jan 2027 · link
  38. CBS Colorado: Governor Polis signed SB 26-189 on 14 May 2026 · link
  39. Council of Europe, Framework Convention on AI (CETS 225) opened for signature, 5 Sep 2024; entry into force after 5 ratifications incl. 3 member states · link
  40. Netherlands Treaty Database, CETS 225 parties: only the EU's ratification (15 May 2026) listed · link
  41. Conneau, Lample, Ranzato, Denoyer & Jégou, Word Translation Without Parallel Data, ICLR 2018 (MUSE) · link · arXiv:1710.04087
  42. DHS final rule rescinding Title VI disparate-impact provisions (6 CFR 21; 44 CFR 7), publication 22 Jun 2026 (FR doc. 2026-12399) · link
  43. Ding, Hardt, Miller & Schmidt, 'Retiring Adult: New Datasets for Fair Machine Learning', arXiv 2108.04884 (NeurIPS 2021) · link · arXiv:2108.04884
  44. DOJ final rule rescinding Title VI disparate-impact provisions (28 CFR 42.104), issued 9 Dec 2025, 90 FR 57141 · link
  45. Duan, Suri, Mireshghallah, Min, Shi, Zettlemoyer, Tsvetkov, Choi, Evans & Hajishirzi, Do Membership Inference Attacks Work on Large Language Models? (COLM 2024) · arXiv:2402.07841
  46. K&L Gates, 'The Changing Landscape of AI Federal Guidance for Employers...', 31 Jan 2025: EEOC AI guidance removed 27 Jan 2025 · link
  47. Reports of an internal EEOC memo closing disparate-impact-only charges by 30 Sep 2025 (Bloomberg Law; Littler summary, 25 Sep 2025) · link
  48. EEOC press release, 'iTutorGroup to Pay $365,000 to Settle EEOC Discriminatory Hiring Suit', 11 Sep 2023; EEOC v. iTutorGroup, No. 1:22-cv-02565 · link
  49. Executive Order 14281, 'Restoring Equality of Opportunity and Meritocracy', signed 23 Apr 2025, 90 FR 17537 (28 Apr 2025) · link
  50. Executive Order 14365, 'Ensuring a National Policy Framework for Artificial Intelligence', 11 Dec 2025, 90 FR 58499 · link
  51. Epoch AI, About page · link
  52. Regulation (EU) 2024/1689 (Artificial Intelligence Act), OJ L 12.7.2024; consolidated version 27 July 2026 · link · §Art. 4, Art. 5, Art. 50, Art. 51(2), Art. 53, Art. 55, Art. 86, Art. 101, Art. 111(3)
  53. Regulation (EU) 2024/1689 (AI Act), Annex III · link
  54. AI Act Service Desk, Article 101: Fines for providers of GPAI models (3% or EUR 15M) · link
  55. AI Act Art. 111(3) (mirror of the official text): GPAI models placed before 2 Aug 2025 comply by 2 Aug 2027 · link
  56. Regulation (EU) 2024/1689, Article 27 (fundamental rights impact assessment) · link
  57. Regulation (EU) 2024/1689, Article 5(1)(f): prohibition of AI systems that infer emotions from biometric data in the workplace and education (medical/safety exceptions); Chapter II applies from 2 Feb 2025 (Art. 113(a)) · link
  58. AI Act Service Desk, Article 51: Classification of GPAI models with systemic risk (10^25 FLOP presumption) · link
  59. AI Act Service Desk, Article 53: Obligations for providers of general-purpose AI models · link
  60. AI Act Service Desk, Article 55: Obligations of providers of GPAI models with systemic risk (page notes Omnibus amendment pending display) · link
  61. AI Act Service Desk, Article 86: Right to explanation of individual decision-making · link
  62. Commission proposal COM(2021) 206 of 21 April 2021 (Council document ST 8115/2021) · link
  63. European Commission, AI Act Service Desk: Timeline for the implementation of the EU AI Act (reflects the Digital Omnibus on AI) · link
  64. European Commission FAQ, Transparency obligations under Article 50 (applies from 2 Aug 2026; 50(2) grace to 2 Dec 2026) · link
  65. European Commission, Guidelines on transparency obligations for providers and deployers of AI systems, 20 July 2026 · link
  66. European Commission, Code of Practice on marking and labelling AI-generated content, published 10 June 2026 · link
  67. Regulation (EU) 2026/1744 of 8 July 2026 (Digital Omnibus on AI), amending Regulation (EU) 2024/1689; entry into force 27 July 2026 · link
  68. European Commission, General-Purpose AI Code of Practice (published 10 July 2025); signatory list (21 + xAI Safety and Security chapter only) · link
  69. European Commission news, 'General-Purpose AI Code of Practice now available', 10 July 2025 · link
  70. European Commission, template for the public summary of training content of GPAI models, 24 July 2025 · link
  71. Ganguli et al., Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (Anthropic, 2022) · arXiv:2209.07858
  72. Google DeepMind, Strengthening our Frontier Safety Framework (FSF 3.0, Sept 22, 2025; updated to 3.1 on Apr 17, 2026) · link
  73. Regulation (EU) 2016/679 (GDPR), Art. 22, Art. 15(1)(h), Recital 71 · link
  74. Regulation (EU) 2016/679 (GDPR), Article 22 · link
  75. Gebru, Morgenstern, Vecchione, Vaughan, Wallach, Daumé III & Crawford, Datasheets for Datasets (arXiv 2018; Communications of the ACM 2021) · arXiv:1803.09010
  76. Goodman & Flaxman, European Union regulations on algorithmic decision-making and a 'right to explanation', 2016 · arXiv:1606.08813
  77. OpenAI, GPT-4 System Card (March 2023), section on ARC's evaluation of power-seeking and autonomous replication; summarised by Ars Technica (Mar 2023) · link
  78. Greenblatt, Shlegeris, Sachan & Roger, AI Control: Improving Safety Despite Intentional Subversion, ICML 2024 · link · arXiv:2312.06942
  79. Greenblatt et al., Alignment faking in large language models (Dec 2024) · link · arXiv:2412.14093
  80. Greshake, Abdelnabi, Mishra, Endres, Holz & Fritz, Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · arXiv:2302.12173
  81. Griggs v. Duke Power Co., 401 U.S. 424 (decided 8 Mar 1971) · link
  82. Grosse, Bae, Anil et al., Studying Large Language Model Generalization with Influence Functions (Anthropic, 2023) · arXiv:2308.03296
  83. Guo, Pleiss, Sun & Weinberger, On Calibration of Modern Neural Networks, ICML 2017 · link · arXiv:1706.04599
  84. Hamilton, Leskovec & Jurafsky, Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change, ACL 2016 · link · arXiv:1605.09096
  85. Hardt, Price & Srebro, 'Equality of Opportunity in Supervised Learning', NeurIPS 2016, arXiv 1610.02413 · link · arXiv:1610.02413
  86. HHS final rule rescinding Title VI disparate-impact provisions (45 CFR 80), published and effective 24 Jul 2026 (FR doc. 2026-15000) · link
  87. Hooker, Erhan, Kindermans & Kim, A Benchmark for Interpretability Methods in Deep Neural Networks (ROAR) · arXiv:1806.10758
  88. Hubinger et al., Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training (Jan 2024) · link · arXiv:2401.05566
  89. HUD proposed rule, 'HUD's Implementation of the Fair Housing Act's Disparate Impact Standard', 91 FR 1475 (14 Jan 2026), acknowledging Inclusive Communities (2015); supplemental 91 FR 51416 (10 Aug 2026), comments to 9 Oct 2026 · link
  90. Illinois HB 3773 (amends Illinois Human Rights Act; AI in employment), effective 1 Jan 2026 · link
  91. Texas Dept. of Housing & Community Affairs v. Inclusive Communities Project, 576 U.S. 519 (2015), as acknowledged in HUD's 2026 proposed rule preamble · link
  92. UK AI Security Institute, Inspect: an open-source framework for LLM evaluations (released 10 May 2024) · link
  93. Irving, Christiano & Amodei, AI safety via debate (2018) · link · arXiv:1805.00899
  94. ISO/IEC 23894:2023 Artificial intelligence: guidance on risk management · link
  95. ISO/IEC 42001:2023 Artificial intelligence: management system · link
  96. Jain & Wallace, Attention is not Explanation, NAACL 2019 · arXiv:1902.10186
  97. Kadavath et al., Language Models (Mostly) Know What They Know (2022) · link · arXiv:2207.05221
  98. Kaplan et al., Scaling Laws for Neural Language Models, 2020 (C approx 6NBS) · arXiv:2001.08361
  99. Kim et al., Interpretability Beyond Feature Attribution: Testing with Concept Activation Vectors (TCAV), ICML 2018 · arXiv:1711.11279
  100. Kleinberg, Mullainathan & Raghavan, 'Inherent Trade-Offs in the Fair Determination of Risk Scores', arXiv 1609.05807 (ITCS 2017) · link · arXiv:1609.05807
  101. Koh & Liang, Understanding Black-box Predictions via Influence Functions, ICML 2017 · arXiv:1703.04730
  102. Korbak et al. (41 authors), Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety (Jul 2025) · link · arXiv:2507.11473
  103. Kornblith, Norouzi, Lee & Hinton, Similarity of Neural Network Representations Revisited, ICML 2019 · link · arXiv:1905.00414
  104. Kwa, West, Becker, Deng, Garcia et al. (METR), Measuring AI Ability to Complete Long Tasks (2025) · arXiv:2503.14499
  105. Lanham et al., Measuring Faithfulness in Chain-of-Thought Reasoning, 2023 · arXiv:2307.13702
  106. Li, Chowdhury, Johnson, Hashimoto, Liang, Schwettmann & Steinhardt, Eliciting Language Model Behaviors with Investigator Agents, ICML 2025 · arXiv:2502.01236
  107. Lindsey, Templeton, Marcus, Conerly, Batson & Olah, Sparse Crosscoders for Cross-Layer Features and Model Diffing (Transformer Circuits Thread, Oct 25, 2024) · link
  108. Lundberg & Lee, A Unified Approach to Interpreting Model Predictions (SHAP, Kernel SHAP), NeurIPS 2017 · arXiv:1705.07874
  109. Marks, Treutlein, Bricken, Lindsey, Marcus et al., Auditing Language Models for Hidden Objectives (Anthropic, 2025) · arXiv:2503.10965
  110. MATS program site (12-week research program, Berkeley and London) · link
  111. Meinke, Schoen, Scheurer, Balesni, Shah & Hobbhahn, Frontier Models are Capable of In-context Scheming (Apollo Research) · link · arXiv:2412.04984
  112. METR (Model Evaluation & Threat Research), Wikipedia article · link
  113. Mikolov, Le & Sutskever, Exploiting Similarities among Languages for Machine Translation (2013) · link · arXiv:1309.4168
  114. Mitchell et al., Model Cards for Model Reporting, FAT* 2019 · link · arXiv:1810.03993
  115. Mobley v. Workday, Inc., No. 3:23-cv-00770 (N.D. Cal.): order 12 Jul 2024; ADEA collective conditionally certified 16 May 2025 · link
  116. Isaac Newton, Opticks, 4th ed. (1730), public domain; 1,073 sentences bundled as curriculum/modules/S/data/opticks_sentences.txt
  117. National Fair Housing Alliance v. CFPB, No. 1:26-cv-01820 (D.D.C., filed 27 May 2026); rule effective 21 Jul 2026, not stayed; summary-judgment schedule into 2027 · link
  118. NIST AI Risk Management Framework 1.0 (January 2023) · link
  119. NIST, Artificial Intelligence update to VCAT, 27 March 2026 (AI RMF revision planned; AI 800-2 draft, comments closed 31 March) · link
  120. Northpointe (equivant), 'COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity' (response to ProPublica, 2016) · link
  121. nostalgebraist, interpreting GPT: the logit lens (LessWrong, 2020) · linkunverified
  122. Wiley, 'New York Finalizes RAISE Act ... Takes Effect January 1, 2027' (S8828 chapter amendment signed 27 Mar 2026) · link
  123. Governor Hochul signs RAISE Act (19 Dec 2025): safety protocols, 72-hour incident reporting, oversight office in DFS · link
  124. NYC Local Law 144 of 2021 (Admin. Code 20-870 to 20-874): bias audit within one year, notice >= ten business days, penalties · link
  125. NYC DCWP final rules, 6 RCNY 5-300 et seq. (impact ratio, scoring rate, independent auditor definitions) · link
  126. NY DFS Apple Card investigation report (March 2021): ~400,000 applications reviewed; no fair-lending violation found; customer-service and transparency deficiencies · linkunverified
  127. Office of the NY State Comptroller, 'Enforcement of Local Law 144 - Automated Employment Decision Tools', 2 Dec 2025 · link
  128. OpenAI, OpenAI o1 System Card (Dec 2024) · arXiv:2412.16720
  129. Obermeyer, Powers, Vogeli & Mullainathan, 'Dissecting racial bias in an algorithm used to manage the health of populations', Science 366(6464):447-453, 25 Oct 2019 · link
  130. OECD Hiroshima AI Process Reporting Framework (launched Feb 2025) · link
  131. OPM interim final rule removing UGESP references (FR doc. 2026-15586, publication 31 Jul 2026), citing DOJ OLC opinion 'Constitutionality of Disparate-Impact Liability Under Title VII' (9 Jun 2026) · link
  132. OpenAI, Updating our Preparedness Framework (Preparedness Framework version 2, April 15, 2025) · link
  133. Newton, Opticks (1730), public domain; bundled at curriculum/modules/Embeddings/data/opticks.txt · link
  134. Park, Georgiev, Ilyas, Leclerc & Madry, TRAK: Attributing Model Behavior at Scale (ICML 2023) · arXiv:2303.14186
  135. Parrish et al., 'BBQ: A Hand-Built Bias Benchmark for Question Answering', Findings of ACL 2022, pp. 2086-2105, arXiv 2110.08193 · link · arXiv:2110.08193
  136. Perez, Huang, Song, Cai, Ring, Aslanides, Glaese, McAleese & Irving, Red Teaming Language Models with Language Models, EMNLP 2022 · arXiv:2202.03286
  137. Stata Multivariate Manual, procrustes (history: Hurley & Cattell 1962 applied the name; the myth) · link
  138. Larson, Mattu, Kirchner & Angwin, 'How We Analyzed the COMPAS Recidivism Algorithm', ProPublica, 23 May 2016 · link
  139. ProPublica compas-analysis repository (Broward County data and notebooks) · linkunverified
  140. Pruthi, Liu, Kale & Sundararajan, Estimating Training Data Influence by Tracing Gradient Descent (TracIn), NeurIPS 2020 · arXiv:2002.08484
  141. Pushkarna, Zaldivar & Kjartansson, Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI, FAccT 2022 · arXiv:2204.01075
  142. Raghu, Gilmer, Yosinski & Sohl-Dickstein, SVCCA, NeurIPS 2017 · link · arXiv:1706.05806
  143. Redwood Research profile (Longterm Wiki), and Greenblatt, Shlegeris, Sachan & Roger, AI Control: Improving Safety Despite Intentional Subversion (arXiv 2312.06942) · link
  144. Regulation B, 12 CFR 1002.9 and Official Interpretation 9(b)(2) (version effective 21 Jul 2026) · link
  145. CFPB final rule, Equal Credit Opportunity Act (Regulation B), RIN 3170-AB54, FR doc. 2026-07804 (publication 22 Apr 2026; effective 90 days later) · link
  146. Numbers measured in this repository: nlpcore.safety.* and Read a system card like an auditor: map sections to threat models, methods and evaluators; find the gaps-Model diffing: what did fine-tuning add? Diffed audits vs a crosscoder-style dictionary solutions (seeds stated in each lab) · link
  147. Dastin, J. 'Amazon scraps secret AI recruiting tool that showed bias against women', Reuters, 10 Oct 2018 · link
  148. Ribeiro, Singh & Guestrin, 'Why Should I Trust You?': Explaining the Predictions of Any Classifier (LIME), KDD 2016 · arXiv:1602.04938
  149. Rimsky (Panickssery), Gabrieli, Schulz, Tong, Hubinger & Turner, Steering Llama 2 via Contrastive Activation Addition, ACL 2024 · link · arXiv:2312.06681
  150. Rozado, 'Gender and Positional Biases in LLM-Based Hiring Decisions: Evidence from Comparative CV/Resume Evaluations', arXiv 2505.17049 (May 2025) · link · arXiv:2505.17049
  151. Schönemann, A generalized solution of the orthogonal Procrustes problem, Psychometrika 31(1):1-10 (1966) · link
  152. L. S. Shapley, A Value for n-Person Games, in Contributions to the Theory of Games II, 1953
  153. Sharma, Tong et al., Towards Understanding Sycophancy in Language Models, ICLR 2024 · link · arXiv:2310.13548
  154. Sharma et al., Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming (Anthropic, 2025) · arXiv:2501.18837
  155. Shokri, Stronati, Song & Shmatikov, Membership Inference Attacks against Machine Learning Models, IEEE S&P 2017 · arXiv:1610.05820
  156. Shrikumar et al., Not Just a Black Box: Learning Important Features Through Propagating Activation Differences, 2016 (gradient x input) · arXiv:1605.01713unverified
  157. Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft (Aug 19, 2026), Ch. 1: 1.6.1 Pretraining, 1.9 Evaluating Large Language Models, 1.10 Safety and Alignment · link · §1.6.1, 1.9.1, 1.10
  158. Jurafsky & Martin, SLP3 draft (Aug 19, 2026), Ch. 10 Interpretability (10.3 The Logit Lens, 10.4 Sparse Auto-Encoders, 10.5 Causal Interpretability Techniques) · link · §10.3, 10.4, 10.5
  159. Smith, Turban, Hamblin & Hammerla, Offline bilingual word vectors, orthogonal transformations and the inverted softmax, ICLR 2017 · link · arXiv:1702.03859
  160. Transformer News, 'Claude Sonnet 4.5 knows when it's being tested' (Sept 30, 2025), summarising the Claude Sonnet 4.5 System Card · link
  161. South German Credit (Groemping 2019 correction), UCI ML Repository, donated 28 Nov 2019; CC BY 4.0; original coding errors (personal status/sex) · link
  162. Sundararajan, Taly & Yan, Axiomatic Attribution for Deep Networks (Integrated Gradients), ICML 2017 · arXiv:1703.01365
  163. 42 U.S.C. 2000e-2(a), (k) (disparate impact burden of proof, alternative employment practice) · link
  164. 42 U.S.C. 2000e-2(l) (prohibition of discriminatory use of test scores), as quoted in the DOJ/SG Ricci v. DeStefano amicus brief · link
  165. Transluce, Introducing Docent (Mar 2025) · link
  166. Jacob Steinhardt & Sarah Schwettmann, Introducing Transluce: A Letter from the Founders (Oct 23, 2024) · link
  167. Transluce, Monitor: An AI-Driven Observability Interface (Oct 22, 2024) · link
  168. Turner, Thiergart, Leech, Udell, Vazquez, Mini & MacDiarmid, Activation Addition: Steering Language Models Without Optimization (v1, Aug 2023; later retitled Steering Language Models With Activation Engineering) · link · arXiv:2308.10248
  169. Turpin, Michael, Perez & Bowman, Language Models Don't Always Say What They Think, 2023 (up to 36% accuracy drop, 13 BBH tasks) · arXiv:2305.04388
  170. Becker & Kohavi, Adult (Census Income), UCI ML Repository, donated 30 Apr 1996; 48,842 instances; extracted from the 1994 Census database; license CC BY 4.0 · link
  171. Hofmann, Statlog (German Credit Data), UCI ML Repository, donated 16 Nov 1994; 1,000 instances; license CC BY 4.0 · link
  172. Uniform Guidelines on Employee Selection Procedures, 29 CFR 1607.4(D) (four-fifths rule of thumb and caveats) · link
  173. AI Security Institute (UK), Wikipedia; GOV.UK press release 'AI Safety Institute releases new AI safety evaluations platform' (10 May 2024) · link
  174. UK government press release, 14 Feb 2025: AI Safety Institute renamed AI Security Institute · link
  175. Lewis Silkin, 14 Sep 2026: no dedicated UK AI bill introduced · linkunverified
  176. America's AI Action Plan, 23 July 2025 (summary: Akin Gump tracker; NIST VCAT deck confirms the AI RMF revision directive) · link
  177. Nextgov, 'Commerce rebrands its AI Safety Institute' (June 4, 2025) · link
  178. House Document 119-28 listing EO 14110 as revoked by EO 14148 of 20 January 2025 · link
  179. Executive Order 14179, Removing Barriers to American Leadership in AI, signed 23 Jan 2025, 90 FR (31 Jan 2025) · link
  180. Executive Order 14365, Ensuring a National Policy Framework for AI, signed 11 Dec 2025, 90 FR 58499 (16 Dec 2025) · link
  181. 12 CFR 1002.9 (Regulation B) notifications; eCFR up to date as of 23 Oct 2026 · link
  182. van der Weij, Hofstätter, Jaffe, Brown & Ward, AI Sandbagging: Language Models can Strategically Underperform on Evaluations (2024) · arXiv:2406.07358
  183. Wachter, Mittelstadt & Russell, Counterfactual Explanations without Opening the Black Box, 2017 · arXiv:1711.00399
  184. Wachter, Mittelstadt & Floridi, Why a Right to Explanation of Automated Decision-Making Does Not Exist in the GDPR, International Data Privacy Law 7(2), 2017
  185. Wiegreffe & Pinter, Attention is not not Explanation, EMNLP 2019 · arXiv:1908.04626
  186. Wilson & Caliskan, 'Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval', AIES 2024, arXiv 2407.20371 · link · arXiv:2407.20371
  187. Xing, Wang, Liu & Lin, Normalized Word Embedding and Orthogonal Transform for Bilingual Word Translation, NAACL 2015 · link
  188. Zhao, Wang, Yatskar, Ordonez & Chang, 'Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods', NAACL 2018, arXiv 1804.06876 · link · arXiv:1804.06876
  189. Zou, Wang, Carlini, Nasr, Kolter & Fredrikson, Universal and Transferable Adversarial Attacks on Aligned Language Models (2023) · arXiv:2307.15043
  190. Zou et al., Representation Engineering: A Top-Down Approach to AI Transparency (2023) · link · arXiv:2310.01405