structured llm application architecture curriculum
Teaches systematic decomposition of full-stack LLM systems into discrete architectural layers (data pipelines, model selection, prompt engineering, retrieval, evaluation). Uses case-study-driven pedagogy with real production patterns including RAG systems, fine-tuning workflows, and deployment strategies. Covers the complete lifecycle from prototyping to monitoring in production environments.
Unique: Integrates perspectives from multiple FSDL faculty (Chip Huyen, Josh Tobin, et al.) across data engineering, model selection, and deployment — not a single-vendor curriculum. Emphasizes practical trade-offs (latency vs accuracy, cost vs quality) rather than theoretical optimization.
vs alternatives: Broader architectural scope than vendor-specific courses (e.g., OpenAI's cookbook) or academic ML courses, with explicit focus on production constraints like cost, latency, and monitoring.
hands-on rag system design and implementation
Teaches retrieval-augmented generation patterns including vector database selection, embedding model evaluation, prompt augmentation with retrieved context, and ranking strategies. Labs involve building end-to-end RAG pipelines using frameworks like LangChain, integrating with vector stores (Pinecone, Weaviate, Chroma), and evaluating retrieval quality with metrics like NDCG and MRR.
Unique: Emphasizes the full RAG pipeline including embedding model selection, vector database trade-offs, and ranking strategies — not just 'add a vector store.' Includes practical guidance on when RAG is insufficient and fine-tuning is needed.
vs alternatives: More comprehensive than LangChain's documentation alone; includes evaluation frameworks and trade-off analysis that vendor docs don't cover.
llm fine-tuning strategy and implementation
Covers when to fine-tune vs prompt-engineer vs use RAG, including cost-benefit analysis, data preparation workflows, and training on open-source models (Llama, Mistral) and commercial APIs (OpenAI fine-tuning). Labs involve preparing datasets, training on cloud GPUs, and evaluating fine-tuned models against baselines using metrics like BLEU, ROUGE, and task-specific accuracy.
Unique: Provides decision framework for fine-tuning vs alternatives (prompt engineering, RAG, model selection) with explicit cost-benefit analysis — not just 'how to fine-tune' but 'when to fine-tune.' Covers both open-source and commercial fine-tuning paths.
vs alternatives: More strategic than Hugging Face fine-tuning docs; includes ROI analysis and trade-off guidance that helps teams avoid expensive fine-tuning mistakes.
llm evaluation and benchmarking framework design
Teaches systematic evaluation of LLM outputs using automated metrics (BLEU, ROUGE, METEOR, BERTScore), task-specific metrics (accuracy, F1, NDCG), and human evaluation protocols. Covers designing evaluation datasets, building evaluation pipelines, and interpreting results to guide model selection and fine-tuning decisions. Includes frameworks like HELM and LM Evaluation Harness.
Unique: Integrates automated metrics, task-specific metrics, and human evaluation into a unified framework — not just 'use BLEU' but 'choose metrics based on your task and budget.' Emphasizes the gap between automated metrics and human judgment.
vs alternatives: More practical than academic benchmarking papers; includes guidance on designing evaluation datasets and interpreting results for product decisions.
prompt engineering and in-context learning optimization
Teaches systematic prompt design including chain-of-thought prompting, few-shot learning, prompt templates, and iterative refinement. Covers techniques like role-based prompting, structured output formatting, and prompt injection mitigation. Labs involve building prompt evaluation pipelines and comparing prompt variants using automated metrics and human feedback.
Unique: Emphasizes systematic prompt evaluation and iteration rather than ad-hoc trial-and-error — includes frameworks for comparing prompt variants and measuring improvement. Covers both general techniques (chain-of-thought) and domain-specific patterns.
vs alternatives: More structured than OpenAI's prompt engineering guide; includes evaluation frameworks and trade-off analysis for choosing between prompt engineering, few-shot learning, and fine-tuning.
llm deployment and serving infrastructure
Covers deploying LLM applications to production including containerization (Docker), orchestration (Kubernetes), API serving frameworks (FastAPI, Flask), and monitoring. Teaches cost optimization strategies (batching, caching, model quantization), latency optimization (inference optimization, distillation), and reliability patterns (fallbacks, retry logic, circuit breakers). Labs involve deploying models to cloud platforms (AWS, GCP, Azure).
Unique: Covers the full deployment pipeline from containerization to monitoring, with explicit focus on LLM-specific challenges (cost optimization, latency, reliability). Includes cost-benefit analysis for different serving strategies (API vs self-hosted vs hybrid).
vs alternatives: More comprehensive than cloud provider docs; includes trade-off analysis and patterns for handling LLM-specific failure modes (hallucinations, latency variability).
llm application architecture patterns and design decisions
Teaches architectural patterns for LLM applications including agent architectures, multi-step reasoning pipelines, tool-use integration, and state management. Covers design decisions like when to use agents vs pipelines, how to structure context windows, and managing dependencies between LLM calls. Uses frameworks like LangChain and AutoGPT as case studies.
Unique: Provides systematic framework for choosing between agent architectures, pipelines, and hybrid approaches — not just 'use an agent' but 'when agents are appropriate and what trade-offs they involve.' Includes case studies of real systems.
vs alternatives: More strategic than framework documentation; includes architectural trade-offs and decision frameworks that help teams avoid over-engineering or under-engineering LLM systems.
data preparation and curation for llm tasks
Teaches data collection, cleaning, annotation, and augmentation strategies for LLM fine-tuning and evaluation. Covers handling data quality issues (duplicates, noise, bias), designing annotation guidelines, and using crowdsourcing platforms. Includes techniques like data augmentation, synthetic data generation, and active learning for efficient labeling.
Unique: Emphasizes data quality and curation as critical to LLM performance — not just 'collect data' but 'design annotation guidelines, manage crowdsourcing, and measure quality.' Includes techniques for efficient labeling (active learning, synthetic data).
vs alternatives: More practical than academic data annotation papers; includes guidance on crowdsourcing platforms, cost estimation, and quality control.
+2 more capabilities