dkm/typo3-robots-guard
Centrally manages robots.txt, the robots head meta tag and XML sitemap inclusion for any TYPO3 site (single-site or multisite), with environment-aware safety logic that guards production against accidental crawl-blocking while keeping staging/dev fully disallowed.
时间：2026-06-23 12:17
monteduro/laravel-datafast-aicrawl
Server-side AI crawler & bot traffic tracking for Laravel, reporting to DataFast. Unofficial PHP port of @datafast/ai-crawl.
时间：2026-06-18 07:40
anakin/sdk
Official PHP SDK for the Anakin web-scraping API. Scrape, crawl, search, and run Wire actions with internal job polling.
时间：2026-04-28 18:37
kreuzberg-dev/kreuzcrawl
High-performance web crawling engine
时间：2026-04-20 07:34
mage2kishan/module-advanced-seo
Panth Advanced SEO — enterprise-grade SEO suite for Magento 2: SEO dashboard, meta templates, SEO rules engine, bulk meta editor, custom canonicals, filter URL rewrites, filter page meta, hreflang, product feeds, missing meta report, SEO audit & crawl results, structured data (JSON-LD), breadcrumbs,
时间：2026-04-14 12:20
moinframe/kirby-linkcheck
Kirby CMS plugin to crawl a website and check for broken links. From the Kirby Panel or via the CLI.
时间：2026-02-17 19:02
terminal42/escargot
A web crawler or spider library based on Symfony components
时间：2026-01-04 15:28
patryk-samulewicz/crawl-toolkit
Universal package for BrightData web scraping and crawling. And making keyword research easier.
时间：2025-05-30 15:57
jpcaparas/laravel-firecrawl
Interface with the Firecrawl API using Laravel
时间：2025-05-01 05:46
utrodus/malaysia-public-holiday
A PHP library to crawl Malaysia public holidays with filtering and caching.
时间：2025-04-19 17:00
v-sing/querylist-phantomjs
QueryList Plugin: Use PhantomJS to crawl Javascript dynamically rendered pages.(headless WebKit ) 使用PhantomJS采集JavaScript动态渲染的页面
时间：2024-04-25 03:45
tjm/web-crawler
Crawl websites, finding internal links and getting content.
时间：2024-03-18 20:33
ixnode/php-web-crawler
PHP Web Crawler - This PHP class allows you to crawl recursively a given html page (or a given html file) and collect some data from it.
时间：2024-02-24 14:14
giauphan/crawl-blog-data
This powerful web scraping tool is designed to gather data from blogs and websites with ease, providing you with valuable insights and information.
时间：2024-01-16 11:18
nerbiz/crawl-shield
Prevent search engine crawlers from indexing your Laravel dev/test sites
时间：2024-01-15 08:39