anassrojea/laracrawler
Laravel sitemap generator and crawler package for SEO optimization. Supports multilingual sites, image/video indexing, link validation, priority scoring, and advanced crawling automation.
时间：2025-09-23 23:36
tfd/laravel-well-known-traffic-advice
A Laravel package that provides a standardized .well-known/traffic-advice endpoint for communicating server load, maintenance windows, and access recommendations to automated clients, crawlers, and partner services.
时间：2025-07-02 12:34
jpcaparas/laravel-firecrawl
Interface with the Firecrawl API using Laravel
时间：2025-05-01 05:46
fkrzski/robots-txt
A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience
时间：2024-12-16 20:41
survos/crawler-bundle
Provides a way to create tests that crawl a site
时间：2022-07-22 13:03
foremtehan/dom-crawler
Eases DOM navigation for HTML and XML documents
时间：2022-05-07 18:12
luka-dev/headless-task-server-php
Helper for sending requests to luka-dev/headless-task-server
时间：2022-04-10 20:01
repat/spatie-crawler-redis
Redis CrawlQueue for spatie/crawler
时间：2019-09-05 08:34
jaeger/querylist-puppeteer
QueryList Plugin: Use Puppeteer to crawl Javascript dynamically rendered pages.(Headless Chrome ) 使用Puppeteer采集JavaScript动态渲染的页面
时间：2019-01-15 10:20
nitsan/ns-twitter
Display fully customizable, responsive, and search engine crawlable Twitter feeds on your TYPO3 website. The TYPO3 Twitter Extension easily integrates to display Tweets that perfectly match your site's appearance and offers many customization options. Easily embed Twitter Timelines, Feeds, Follow b
时间：2019-01-11 10:08
zrashwani/arachnid
A crawler to find all unique internal pages on a given website
时间：2017-02-24 21:28
cyber-duck/silverstripe-seo
A SilverStripe module to optimise the Meta, crawling, indexing, and sharing of your website content
时间：2016-08-26 11:19
blackfire/player
A powerful web crawler and web scraper with Blackfire support
时间：2015-12-18 06:38