Mdream

lastmod 2025-09-25created 2025-09-251 edits

Convert any site to clean Markdown and llms.txt

About

• A tool that converts websites into Markdown and llms.txt formats quickly and lightly, helping generate LLM context that AI can search
• Traditional HTML→Markdown converters are not made for LLMs, so they use many tokens and are slow
• The mdream core is an LLM-optimized HTML→Markdown converter that produces optimized output with about 50% fewer tokens and can convert 1.4MB of HTML in about 50ms
  • A 5kB gzipped library with no external dependencies
• The @mdream/crawl package crawls an entire site to generate llms.txt, llms-full.txt, and individual Markdown files, and can be combined with AI tools such as Claude Code
• Runs in many environments, including CLI, Docker, GitHub Actions, Vite, and Nuxt
• The plugin system hooks into the pipeline to add content filtering, node transformation, custom behavior, and more
  • extractionPlugin: extracts specific elements using CSS selectors, for use in data analysis
  • filterPlugin: includes/excludes elements by CSS selector or tag ID
  • frontmatterPlugin: generates YAML frontmatter from HTML head information (title, meta, etc.)
  • isolateMainPlugin:<main>

    extracts only the main content, using the element or the header~footer range

  • tailwindPlugin: converts Tailwind CSS classes into Markdown formatting (bold, italic, etc.)
  • readabilityPlugin: scores content readability and extracts content (experimental)

See also

• markdown
• llms.txt

Favorite site

• Github - harlan-zw/mdream
• mdream - Convert any site to clean Markdown and llms.txt | GeekNews