Semgrep CLI

Semgrep-core (OCaml engine) performs AST-based pattern matching against user-defined or curated rules to identify security vulnerabilities, code anti-patterns, and compliance violations. The engine parses source code into language-specific abstract syntax trees using tree-sitter and custom parsers, then matches patterns expressed in Semgrep's domain-specific language (YAML-based rule syntax) against the AST structure. This approach enables structural matching rather than regex-based detection, reducing false positives and enabling cross-language consistency.

Semgrep's taint analysis engine (available in Pro Engine) tracks data flow across function boundaries to detect vulnerability chains where untrusted input reaches a dangerous sink. The system constructs a dataflow graph by analyzing variable assignments, function parameters, return values, and object field mutations across the codebase. It identifies sources (user input, external data), sinks (SQL queries, command execution, file writes), and sanitizers (validation functions) to determine if tainted data can reach dangerous operations without proper sanitization.

Semgrep includes language-specific parsers (built on tree-sitter and custom OCaml implementations) for 30+ programming languages. Each parser converts source code into an AST that the pattern matching engine can analyze. The system implements graceful error handling where parse errors in individual files do not stop the scan; instead, errors are logged and scanning continues on other files. This enables Semgrep to scan heterogeneous codebases with mixed languages and syntax variations without failing on unparseable code.

Semgrep includes an MCP server implementation that exposes scanning capabilities to AI models and LLM-based tools. The MCP server allows AI assistants to invoke Semgrep scans, retrieve findings, and analyze code patterns programmatically. This enables integration with AI-powered code review tools, automated remediation assistants, and LLM-based security analysis workflows. The server implements standard MCP protocols for tool invocation and result streaming.

Semgrep's OCaml engine tracks token positions and source locations during AST parsing and pattern matching, enabling precise reporting of finding locations (file, line, column, character offset). The system maintains a mapping between AST nodes and their source positions, allowing findings to be reported with exact character ranges. This enables IDE integration, inline code comments, and precise highlighting in web interfaces. The position tracking is implemented at the parser level and maintained through the entire analysis pipeline.

Semgrep provides a YAML-based domain-specific language (DSL) for expressing code patterns that work across multiple programming languages. Rules are defined in YAML with pattern syntax that abstracts away language-specific details (e.g., a pattern for 'function call' works identically in Python, JavaScript, and Go). The pysemgrep CLI parses rule files, validates syntax, and passes compiled rules to semgrep-core for matching. Users can write custom rules targeting their codebase, organization standards, or specific vulnerability patterns without modifying the core engine.

The `semgrep ci` command integrates Semgrep into CI/CD workflows by scanning code, uploading findings to semgrep.dev, comparing against baseline scans, and enforcing organization-wide policies. The Python CLI (pysemgrep) orchestrates the workflow: it authenticates to Semgrep App using API tokens, fetches organization-specific rules and policies, runs the OCaml scanning engine, and reports results. The system can block CI builds based on policy rules (e.g., 'fail if critical vulnerabilities detected'), automatically triage findings based on organization rules, and track finding status across commits.

Semgrep supports incremental scanning mode where it compares current scan results against a baseline (previous commit or main branch) to report only new or changed findings. The Python CLI manages baseline storage and comparison logic: it fetches the previous scan's JSON output, compares rule matches by file path and line number, and reports only findings that are new, moved, or changed in severity. This reduces noise in CI/CD by surfacing only actionable changes rather than all findings in the codebase.

Semgrep includes built-in rules for detecting hardcoded secrets (API keys, passwords, tokens, private keys) using pattern matching combined with entropy analysis and semantic validation. The system matches common secret patterns (e.g., 'aws_access_key_id = ...', 'password: ...') and validates candidates using entropy scoring and format-specific checks (e.g., verifying AWS key format, checking if a string is a valid JWT). This reduces false positives compared to simple regex matching by confirming that detected patterns actually look like valid secrets.

Semgrep AppSec Platform includes supply chain scanning that detects vulnerable dependencies and determines if the vulnerability is actually reachable from application code. The system scans dependency manifests (package.json, requirements.txt, go.mod, pom.xml, etc.), identifies known vulnerable versions, and uses taint analysis to determine if the vulnerable function is actually called from application code. This reduces alert fatigue by filtering out vulnerabilities in unused dependencies or unreachable code paths.

Semgrep outputs findings in multiple formats to integrate with various CI/CD tools and reporting systems. The Python CLI supports JSON (for programmatic processing), SARIF (for GitHub Code Scanning, GitLab SAST, Azure DevOps), CSV (for spreadsheet analysis), and human-readable text. The output formatting layer (in pysemgrep) transforms the OCaml engine's internal finding representation into the requested format, including metadata like rule ID, severity, CWE, and remediation guidance.

Semgrep's configuration resolver (pysemgrep) discovers and loads rules from multiple sources: local .semgrep.yml files, Semgrep Registry (curated rules), organization policies (from semgrep.dev), and command-line arguments. The resolver implements a precedence system where local rules override registry rules, and explicit CLI arguments override all defaults. It validates rule syntax, checks for conflicts, and reports errors if rules cannot be loaded. This enables flexible rule management from ad-hoc local testing to organization-wide policy enforcement.

Semgrep optimizes scanning performance through parallel file processing and result caching. The OCaml engine processes multiple files concurrently using worker threads, and the Python CLI implements caching of parse trees and rule compilation results. For large codebases, Semgrep can scan thousands of files in seconds by distributing work across CPU cores. The system also supports incremental scanning where only changed files are re-scanned, further reducing overhead in CI/CD workflows.