playwright-based google search execution with anti-bot evasion
Executes real Google searches using Playwright browser automation while implementing multiple anti-detection strategies (user-agent rotation, viewport randomization, request throttling, browser state persistence) to bypass Google's anti-scraping mechanisms. The core googleSearch() function in src/search.ts orchestrates browser navigation, DOM waiting, and result extraction without relying on external SERP APIs, enabling unlimited searches without rate limits or API quotas.
Unique: Combines Playwright's headless browser automation with stateful browser persistence (saving/restoring cookies and session state) to minimize CAPTCHA triggers, unlike stateless SERP API calls. Implements multi-layered anti-detection (user-agent rotation, viewport randomization, request throttling) at the browser level rather than HTTP header manipulation alone.
vs alternatives: Eliminates SERP API costs and rate limits (SerpAPI charges $0.005-0.02 per search) while providing real-time results; slower than cached APIs but faster than manual browser interaction and suitable for agents requiring fresh data.
mcp server integration for ai assistant search access
Wraps the core googleSearch() function as a Model Context Protocol (MCP) server using the MCP SDK, enabling AI assistants like Claude to invoke Google searches via standardized tool-calling interface. The mcp-server.ts component manages McpServer instance, StdioServerTransport for stdio communication, and a global persistent Playwright browser to serve multiple search requests from a single AI session without browser restart overhead.
Unique: Implements MCP server using stdio transport with persistent global Playwright browser, avoiding browser restart overhead per request. Registers search as a native MCP tool with schema-based parameter validation, enabling seamless integration into Claude's tool-calling pipeline without custom wrapper code.
vs alternatives: Provides native MCP integration (vs. requiring custom API wrappers or HTTP servers) and maintains persistent browser state across multiple AI assistant requests, reducing latency compared to stateless SERP API integrations.
command-line interface with configurable search parameters
Exposes search functionality via CLI using the commander package (src/index.ts) with options for result limit, timeout, headless mode toggle, browser state file path, and HTML extraction modes. Parses command-line arguments and invokes the core googleSearch() function with validated parameters, supporting both structured JSON output and raw HTML retrieval for downstream processing.
Unique: Uses commander package for declarative CLI argument parsing with built-in help/version generation. Supports both structured JSON output (for programmatic consumption) and raw HTML extraction (--get-html, --save-html), enabling flexible integration into shell pipelines and scripts.
vs alternatives: Simpler than writing custom Node.js scripts while more flexible than web-based search tools; enables shell integration without HTTP server overhead.
browser state persistence for captcha mitigation
Saves and restores Playwright browser state (cookies, localStorage, sessionStorage) to a JSON file (default ./browser-state.json) between search invocations. This stateful approach preserves Google's session context and reduces CAPTCHA triggers by maintaining browser identity across multiple searches, unlike stateless HTTP clients that appear as fresh visitors to Google on each request.
Unique: Implements stateful browser persistence at the Playwright level (saving/restoring browser context) rather than HTTP-level cookie management. Preserves full browser state including localStorage and sessionStorage, maintaining Google's session context more effectively than header-based cookie jars.
vs alternatives: More effective CAPTCHA mitigation than stateless SERP APIs or simple cookie rotation; trades state file management complexity for sustained search access without manual intervention.
structured result extraction with title, link, snippet fields
Parses Google search result DOM using Playwright's page.locator() and evaluate() methods to extract structured data (title, link, snippet) from each result element. Returns SearchResponse JSON array with typed fields, enabling downstream processing without regex parsing or HTML string manipulation. Extraction logic handles Google's dynamic DOM structure and adapts to layout variations.
Unique: Uses Playwright's page.locator() and evaluate() for DOM-aware extraction rather than regex or HTML parsing libraries. Returns typed SearchResponse objects with validated fields, enabling type-safe downstream processing in TypeScript/Node.js applications.
vs alternatives: More robust than regex-based extraction (handles DOM variations) and more maintainable than brittle CSS selector chains; provides structured output suitable for LLM context vs. raw HTML strings.
raw html retrieval and screenshot capture for custom analysis
Provides --get-html flag to return raw HTML string of search results page and --save-html flag to capture and save full page screenshot/HTML to disk. Enables custom parsing, archival, or visual debugging workflows where structured extraction is insufficient. Playwright's page.content() and page.screenshot() methods handle full-page capture including dynamic content.
Unique: Offers dual output modes: structured extraction (SearchResponse) for programmatic use and raw HTML/screenshots for custom analysis. Playwright's page.content() captures dynamic content after JavaScript execution, unlike static HTML fetching.
vs alternatives: More flexible than structured-only extraction; enables custom parsing for edge cases (knowledge panels, ads, featured snippets) while maintaining option for clean structured output.
configurable timeout and headless mode control
Exposes --timeout <milliseconds> (default 60000) and --no-headless CLI options to control Playwright browser behavior. Timeout parameter sets page navigation and element waiting limits; --no-headless disables headless mode to show visible browser window for debugging. Enables developers to tune performance vs. reliability and visually inspect search execution.
Unique: Exposes Playwright's timeout and headless mode as CLI flags, enabling non-developers to adjust behavior without code changes. --no-headless provides visual debugging capability absent in most SERP APIs.
vs alternatives: More flexible than fixed-timeout SERP APIs; enables visual debugging vs. blind API calls and supports network-specific tuning.
pino-based structured logging for debugging and monitoring
Implements logging via Pino logger (src/logger.ts) with structured JSON output, enabling developers to track search execution flow, anti-bot detection events, and errors. Logs include timestamps, log levels, and contextual data suitable for parsing by log aggregation systems (ELK, Datadog, CloudWatch). Supports configurable log levels for production vs. development environments.
Unique: Uses Pino for structured JSON logging with minimal overhead, enabling log aggregation and analysis. Logs include search-specific context (query, result count, anti-bot events) suitable for monitoring search health.
vs alternatives: Structured JSON logging (vs. unstructured console.log) enables automated parsing and alerting; Pino's performance is optimized for high-volume logging.
+2 more capabilities