Speech and Language Processing - Dan Jurafsky and James H. Martin

Teaches core NLP concepts through rigorous mathematical frameworks including probability theory, information theory, and formal linguistics. Uses pedagogical progression from foundational concepts (tokenization, morphology) through advanced topics (parsing, semantics) with worked examples, equations, and theoretical proofs embedded throughout. The curriculum integrates linguistic theory with computational implementations, establishing the mathematical foundations required for understanding modern NLP systems.

Organizes NLP knowledge in a deliberate pedagogical sequence starting with character and word-level processing (tokenization, morphology, part-of-speech tagging), progressing through syntactic analysis (parsing, grammar formalisms), and culminating in semantic understanding (word meaning, semantic role labeling, discourse). Each chapter builds on previous concepts with explicit prerequisites, allowing learners to understand how lower-level linguistic phenomena compose into higher-level meaning representations.

Presents NLP algorithms in pseudocode form with explicit time and space complexity analysis, allowing readers to understand both the conceptual approach and implementation considerations. Covers algorithms for tokenization, POS tagging, parsing, semantic role labeling, and other core NLP tasks with detailed walkthroughs of how algorithms process example inputs. Includes discussion of algorithm trade-offs (e.g., exact vs. approximate parsing, greedy vs. optimal solutions) and practical considerations for implementation.

Teaches probabilistic approaches to NLP including Markov models, hidden Markov models, Bayesian inference, and statistical language modeling. Explains how to formulate NLP problems as probabilistic inference tasks, estimate model parameters from data, and evaluate model performance using information-theoretic measures. Covers both generative and discriminative models with detailed derivations of how probability distributions are used to solve NLP problems like tagging, parsing, and language modeling.

Covers formal grammar theory including context-free grammars, dependency grammars, and grammar formalisms used in NLP (PCFG, TAG, CCG). Explains parsing algorithms including CYK, Earley, and shift-reduce parsing with detailed complexity analysis and worked examples. Discusses the relationship between linguistic theory (generative grammar, dependency theory) and computational parsing approaches, including how to evaluate parser performance and handle ambiguity in natural language.

Teaches approaches to representing and computing meaning in NLP including word sense disambiguation, semantic role labeling, and compositional semantics. Covers formal semantic frameworks (first-order logic, lambda calculus) and how they apply to natural language understanding. Explains how to represent relationships between words (synonymy, hypernymy, meronymy) and how to compose word meanings into sentence meanings, including discussion of semantic phenomena like negation, quantification, and presupposition.

Teaches techniques for extracting structured information from unstructured text including named entity recognition, relation extraction, and event extraction. Covers both rule-based and statistical approaches to information extraction, including pattern matching, sequence labeling, and relation classification. Explains how to design extraction systems for specific domains, handle ambiguity in extraction tasks, and evaluate extraction performance using precision, recall, and F-measure metrics.

Covers discourse structure analysis including coherence relations, discourse segmentation, and coreference resolution. Explains how discourse phenomena (anaphora, ellipsis, discourse markers) affect language understanding and how to model discourse structure computationally. Discusses pragmatic phenomena including speech acts, implicature, and presupposition, and how these affect interpretation of natural language utterances in context.

Teaches rigorous experimental methodology for NLP including proper train/test/validation splitting, cross-validation, statistical significance testing, and evaluation metrics appropriate for different NLP tasks. Covers how to design controlled experiments, avoid common pitfalls (data leakage, overfitting, multiple comparison problems), and report results reproducibly. Includes discussion of evaluation metrics for classification (precision, recall, F-measure), ranking (NDCG, MAP), and generation tasks (BLEU, ROUGE, METEOR).

Teaches corpus-based approaches to NLP including corpus design, annotation schemes, inter-annotator agreement measurement, and corpus analysis techniques. Covers how to create and use annotated corpora for training and evaluating NLP systems, including discussion of annotation guidelines, quality control, and handling disagreement between annotators. Explains how corpus statistics inform linguistic understanding and how to avoid biases in corpus construction.