marcvanh/laravel-bot-block
A custom middleware package for Laravel. Temporarily blocks crawlers scanning for vulnerabilities.
时间：2026-01-04 07:40
vdb/php-spider
A configurable and extensible PHP web spider
时间：2026-01-04 07:23
nmure/crawler-detect-bundle
A Symfony bundle for the Crawler-Detect library (detects bots/crawlers/spiders via the user agent)
时间：2026-01-04 07:11
tomverran/robots-txt-checker
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a page
时间：2026-01-04 06:07
bugbuster/contao-botdetection-bundle
Contao bundle helper class to detect search engines, bots, spiders, crawlers ...
时间：2026-01-04 05:43
revolution/salvager
Tiny WebCrawler for Laravel using Playwright
时间：2026-01-04 04:57
vlados/laravel-blade-crawler-detect
This is my package laravel-blade-crawler-detect
时间：2026-01-04 03:02
bitandblack/document-crawler
Extract different parts of an HTML or XML document.
时间：2025-11-27 21:18
felipemateus/metatags
A Laravel package to fetch webpage metadata ( Open Graph | Twitter | Facebook | Article ) ect...
时间：2025-11-02 12:24
anassrojea/laracrawler
Laravel sitemap generator and crawler package for SEO optimization. Supports multilingual sites, image/video indexing, link validation, priority scoring, and advanced crawling automation.
时间：2025-09-23 23:36
fkrzski/robots-txt
A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience
时间：2024-12-16 20:41
survos/crawler-bundle
Provides a way to create tests that crawl a site
时间：2022-07-22 13:03
zrashwani/arachnid
A crawler to find all unique internal pages on a given website
时间：2017-02-24 21:28