storm

Generates research questions through simulated conversations between a Wikipedia writer and topic expert LLM agents, where questions are grounded in perspective discovery from similar existing articles rather than direct prompting. The system surveys related Wikipedia articles to extract diverse viewpoints, then uses these perspectives to guide the question-asking process, ensuring comprehensive topic coverage from multiple angles. This two-agent conversational approach with perspective injection produces more structured and comprehensive research directions than naive question generation.

Generates multi-level article outlines (sections, subsections, key points) using collected research references, where each outline node is anchored to specific retrieved sources. The system structures the outline hierarchically to match Wikipedia article conventions, then maps each outline element to supporting citations from the knowledge curation phase. This enables the subsequent writing stage to generate text with proper in-line citations by maintaining explicit outline-to-source mappings throughout the generation pipeline.

Orchestrates the complete STORM pipeline (knowledge curation → outline generation → article writing → polishing) for batch processing of multiple topics, implemented through STORMWikiRunner that manages state, error handling, and progress tracking across pipeline stages. The system executes each stage sequentially for each topic, maintaining intermediate results and enabling resumption from failure points. This orchestration layer abstracts pipeline complexity and enables users to generate article collections without managing individual stage invocations.

Uses sentence encoders (embeddings) to compute semantic similarity between research questions and existing article content, enabling the system to discover relevant perspectives from similar articles without explicit keyword matching. The encoder system converts text to dense vector representations, enabling efficient similarity search across large article collections. This semantic approach discovers perspectives that keyword-based methods would miss, improving the diversity and relevance of research questions.

Generates full-length Wikipedia-style articles (2000+ words) by consuming hierarchical outlines and mapped citations, producing text with inline citations that reference specific retrieved sources. The system uses the outline structure to guide section-by-section generation, maintaining citation context from the outline-to-source mappings to ensure every claim references a specific source. This multi-stage approach (outline → section generation → citation insertion) produces coherent long-form content with proper attribution without requiring additional source retrieval during writing.

Integrates with internet search APIs (Bing, Google, or custom) to retrieve relevant sources for research questions, implementing a retrieval module that handles query expansion, result ranking, and content extraction. The system executes search queries derived from research questions, collects results with metadata (URLs, snippets, relevance scores), and extracts full-text content from retrieved pages. This retrieval layer feeds the knowledge curation phase with grounded source material, enabling all downstream stages to operate on internet-sourced information.

Builds and maintains a hierarchical knowledge base (mind map) that organizes collected information into a dynamic concept structure, implemented as the KnowledgeBase class that stores information as nested concepts with relationships. The system continuously reorganizes information as new sources are added, maintaining a shared conceptual space that reduces cognitive load during knowledge curation. This knowledge base serves as the source of truth for outline generation and article writing, enabling both automated and human-collaborative workflows to reference a consistent information structure.

Implements a three-agent collaborative discourse protocol (Co-STORM) where human users, LLM expert agents, and a moderator agent participate in structured knowledge curation conversations. The moderator agent generates thought-provoking questions inspired by retrieved information not yet discussed, expert agents answer questions grounded in external sources and raise follow-up questions, and human users can observe passively or actively steer the conversation. The system maintains conversation history and the shared knowledge base, enabling the moderator to track discussed vs. undiscussed information and guide the discourse toward comprehensive coverage.

Provides a unified language model interface (lm.py module) that abstracts multiple LLM providers (OpenAI, Anthropic, Ollama, local models) behind a common API, enabling seamless provider switching without pipeline code changes. The system handles provider-specific details (API authentication, request formatting, response parsing, token counting) and exposes standardized methods for completion, chat, and function calling. This abstraction layer enables users to swap providers based on cost, latency, or capability requirements without modifying the knowledge curation or article generation logic.

Implements an optional polishing phase that refines generated articles through iterative LLM-based fact-checking and improvement, verifying claims against source material and improving clarity/coherence. The system re-examines article sections against their source citations, identifies unsupported claims or contradictions, and generates refined versions. This post-generation refinement improves article quality without requiring additional source retrieval, leveraging the citation mappings from earlier phases to validate factual accuracy.

Provides a web-based frontend (Streamlit demo) that enables non-technical users to run STORM and Co-STORM pipelines through an interactive UI, handling topic input, progress visualization, and result display. The interface abstracts pipeline complexity, manages LLM configuration, and presents results in readable formats (formatted articles, conversation transcripts, knowledge base visualizations). This frontend enables researchers and content creators to use STORM without writing code, lowering the barrier to entry for knowledge curation workflows.

Constructs structured InformationTable objects that organize collected research data (sources, snippets, metadata) into queryable tables with schema-aware operations, enabling downstream stages to access information programmatically. The system extracts and structures information from retrieved sources, maintaining relationships between sources, concepts, and claims. This structured representation enables outline generation and article writing to query information efficiently without re-parsing raw source text.