Turndown Based Semantic Html To Markdown Conversion With Github Flavored Markdown Support

via “intelligent markdown generation from rendered html with semantic structure preservation”

AI-optimized web crawler — clean markdown extraction, JS rendering, structured output for RAG.

Unique: Implements multi-strategy markdown generation via ContentScrapingStrategy pattern, allowing pluggable backends (BeautifulSoup, Firecrawl, Jina) with configurable content filters that preserve semantic hierarchy while removing boilerplate. Includes specialized handling for tables, code blocks, and lists with markdown-specific formatting rules.

vs others: Produces cleaner markdown than generic HTML-to-markdown converters by applying domain-specific filters for web boilerplate; preserves semantic structure better than simple regex-based approaches; supports multiple extraction backends for flexibility.