adbar/trafilatura

• highreadme#1

  Emphasize corpus building and large-scale data collection in the README introduction

  原因：

  当前

  Trafilatura is a cutting-edge **Python package and command-line tool** designed to **gather text on the Web and simplify the process of turning raw HTML into structured, meaningful data**. It includes all necessary discovery and text processing components to perform **web crawling, downloads, scraping, and extraction** of main texts, metadata and comments.

  复制粘贴的修复

  复制

  Trafilatura is a cutting-edge **Python package and command-line tool** designed to **gather text on the Web, build large text corpora, and simplify the process of turning raw HTML into structured, meaningful data**. It includes all necessary discovery and text processing components to perform **web crawling, downloads, scraping, and extraction** of main texts, metadata and comments.

  待办完成忽略
• mediumabout#2

  Refine the repository description to highlight corpus building

  原因：

  当前

  Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

  复制粘贴的修复

  复制

  Python & Command-line tool for large-scale web data collection and text corpus building: Crawling, scraping, extraction of main content and metadata, output as CSV, JSON, HTML, MD, TXT, XML.

  待办完成忽略
• lowtopics#3

  Add more specific topics related to large-scale data collection and corpus creation

  原因：

  当前

  article-extractor, corpus-builder, corpus-tools, crawler, html-to-markdown, html2text, llm, news-aggregator, news-crawler, nlp, rag, readability, rss-feed, scraping, tei, text-cleaning, text-extraction, text-mining, text-preprocessing, web-scraping

  复制粘贴的修复

  复制

  article-extractor, corpus-builder, corpus-tools, crawler, html-to-markdown, html2text, llm, news-aggregator, news-crawler, nlp, rag, readability, rss-feed, scraping, tei, text-cleaning, text-extraction, text-mining, text-preprocessing, web-scraping, data-collection, web-data-extraction, corpus-creation

  待办完成忽略

本次扫描解析到的品类 GEO 通道：google/gemini-2.5-flash, deepseek/deepseek-v4-flash

• 品类问题

  How to programmatically extract clean main content and metadata from arbitrary web pages for analysis?

  你：第 7 位
  AI 推荐顺序：

  1. Beautiful Soup 4
  2. Requests
  3. httpx
  4. Readability.js
  5. python-readability
  6. readability-lxml
  7. Trafilatura ← 你
  8. Newspaper3k
  9. Scrapy
  10. Goose3
  11. Playwright
  12. Selenium
  查看 AI 完整回答
• 品类问题

  What Python library can help me build a text corpus by scraping web articles?

  你：未被推荐
  AI 推荐顺序：

  1. Beautiful Soup 4 (crummy/BeautifulSoup)
  2. Scrapy (scrapy/scrapy)
  3. requests (psf/requests)
  4. lxml (lxml/lxml)
  5. Selenium (SeleniumHQ/selenium)
  6. newspaper3k (codelucas/newspaper)

  AI 推荐了 6 个替代方案，却始终没点名 adbar/trafilatura。这就是要补上的差距。

  查看 AI 完整回答

• Metadata completeness

  pass

• README presence

  pass

• Compared to common alternatives in this category, what is the core differentiator of adbar/trafilatura?

  pass
  ✓AI 明确点名了 adbar/trafilatura

  AI 的回答可能信誓旦旦却是错的。请按事实核对：技术栈、目标人群、差异化点是不是和你实际的对得上？

• If a team adopts adbar/trafilatura in production, what risks or prerequisites should they evaluate first?

  pass
  ✓AI 明确点名了 adbar/trafilatura

  AI 的回答可能信誓旦旦却是错的。请按事实核对：技术栈、目标人群、差异化点是不是和你实际的对得上？

• In one sentence, what problem does the repo adbar/trafilatura solve, and who is the primary audience?

  pass
  ✓AI 明确点名了 adbar/trafilatura

  AI 的回答可能信誓旦旦却是错的。请按事实核对：技术栈、目标人群、差异化点是不是和你实际的对得上？