定制 quickscraper/sdk 二次开发

按需修改功能、优化性能、对接业务系统,提供一站式技术支持

邮箱:yvsm@zunyunkeji.com | QQ:316430983 | 微信:yvsm316

quickscraper/sdk

最新稳定版本:3.4

Composer 安装命令:

composer require quickscraper/sdk

包简介

https://quickscraper.co

README 文档

README

Get Free Access (Free Forever)

Please Feel free to create Issue for any help !

Installation

composer require quickscraper/sdk

Basic usage

<?php
require_once './vendor/autoload.php';
$sdk =  new QuickScraper\Main\QuickScraperClass();
$sdk->setAccessToken('YOUR_ACCESS_TOKEN');
$html = $sdk->getHtml('https://mylocation.org');
print_r(json_decode($html));
?>

Rendering Javascript

<?php
require_once './vendor/autoload.php';

$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/ip', array(
    'render'=> 'true'
  )
);
print_r(json_decode($response));

Custom Headers

<?php
require_once './vendor/autoload.php';

$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/headers', array(
      //This field is added to set cookis,authorizarion token etc from request header.
    "isKeepHeaders" => 'true',//This value will be true when we want to use header
    'customRequestHeaders' => array('X-My-Custom-Header'=> 'QS-APP')//Supports values like `'Accept-Encoding': 'gzip,deflate,compress'`
  )
);
print_r(json_decode($response));

Geographic Location

<?php
require_once './vendor/autoload.php';

$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/ip', array(
    'country_code'=> 'us'//Add your Geotargeting country which you are selecting from request page,for example, for Canada value like `CA`,for China value like `CN`
));
print_r(json_decode($response));

Premium Residential/Mobile Proxy Pools

<?php
require_once './vendor/autoload.php';

$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/ip', array(
    'premium'=> true //Up or premium proxies will be picked
));
print_r(json_decode($response));

Account Information

<?php
require_once './vendor/autoload.php';

$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->account();
print_r(json_decode($response));

Device Type

<?php
require_once './vendor/autoload.php';
$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/ip', array(
    'device_type'=> 'mobile'//Add value `mobile` if the device which you are using is mobile and no need to provide this option if want to scrap site as desktop view
));
print_r(json_decode($response));

Parser Addon

<?php
require_once './vendor/autoload.php';
$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/ip', array(
    'parserSubscriptionId'=> 'YOUR_PARSER_SUBSCRIPTION_ID'//Get the parser subscription id from request page when you select parser and paste here, for example value like '4a0360ea-042a-555e-b214-e3054a400f2a'
));
print_r(json_decode($response));

Webhook Addon

<?php
require_once './vendor/autoload.php';
$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/ip', array(
    'webhookRequestId' => 'YOUR_WEBHOOK_REQUEST_ID'//Get the webhook id from dashboard and paste here, for example value like `2233-34jkjsd-324jkds-3243kh`
));
print_r(json_decode($response));

Submit form data

<?php
require_once './vendor/autoload.php';
$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/ip', array(
    'webhookRequestId' => 'YOUR_WEBHOOK_REQUEST_ID',
    //This field is added when we want to scrap data after login or sign up.
    'formData' => array(
        'formSelector' => 'FORM_SELECTOR',//Supports values like `'form[class="unique_class"]'`
        'submitButtonSelector' => 'SUBMIT_BUTTON_SELECTOR',//Supports values like `'button[class="unique_class"]'`
        'formSelectorScript' => 'FORM_SELECTOR_SCRIPT',//Supports values like `document.getElementById("demo")`
        'submitButtonSelectorScript' => 'SUBMIT_BUTTON_SELECTOR_SCRIPT',//Supports values like `document.
        'formFields' => array(
            array(
                'value' => 'VALUE_TO_PASS_IN_FORM',//Add value of form input fields
                'selectorScript' => 'SELECT_INPUT',//Supports values like `'input[class="unique_class"]','span[class="unique_class"]'`
                'isScript' => 'BOOLEAN'//If your selector has JavaScript code then make this true
            )
        ),
    ),
    'isKeepFormDataSelection' => true
));

print_r(json_decode($response));

Add Dynamic Input with Parser

<?php
require_once './vendor/autoload.php';
$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('http://httpbin.org/ip', array(
    'parserSubscriptionId' => 'YOUR_PARSER_SUBSCRIPTION_ID',
    //These dynamic inputs field is required when some input fields are required to get data of page
    'dynamicInputs' => array(
		array(
			'name' => 'YOUR_INPUT_NAME',//Name stands for input name field like `Username`,`password`,`limit` etc
			"value" => 'YOUR_VALUE' //Value of name field like `abc`,`123456789`,`25`
		)
	),
));
print_r(json_decode($response));

Add Actions with Parser

<?php
require_once './vendor/autoload.php';
$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('https://www.hilton.com/en/locations/india/?WT.mc_id=zLADA0IN1MB2PSH3GGL4INTCRB5dkt6MULTIBR7en_&epid!_&ebuy!&&&&&gad_source=1&gclid=CjwKCAiAvJarBhA1EiwAGgZl0HWfZEaLc_HWhRFRxTT3gs8pwQvFZKtLPNMj3szkjCbFzlef4ox_XxoCGDYQAvD_BwE&gclsrc=aw.ds', array(
    'parserSubscriptionId' => 'YOUR_PARSER_SUBSCRIPTION_ID',
    'customSelectors' => array(
		array(
			'name' => 'YOUR_VALUE',//Name stands for action name and it supports only 2 actions like `waitForElement` and `click`
			'selectorScript' => 'YOUR_SELECTOR',//Supports values like `'a[href="google.com"]', 'span[class="unique_class"]', document.getElementById("demo")`
            'options' => null,//This options support timeout feature only like `{"timeout":3000}`
			'isScript'=> false //If your selector has JavaScript code then make this true
		),
        'keepSelection' => true,
	),
));
print_r(json_decode($response));

Scroll To Bottom Of The Page

<?php
require_once './vendor/autoload.php';
$quickScraperClient = new QuickScraper\Main\QuickScraperClass();
$quickScraperClient->setAccessToken('YOUR_ACCESS_TOKEN');
$response = $quickScraperClient->getHtml('YOUR_REQUEST_URL', array(
    'isScroll' => true,
    'scrollTimeout' => 1000, // Consider as milliseconds, wait to page to scrap
    
));
print_r(json_decode($response));

Do you need an expert?

Are you finding a developer for your world-class product? If yes, please contact here. Originally by QuickScraper Developers - app@quickscraper.co.

quickscraper/sdk 适用场景与选型建议

quickscraper/sdk 是一款 基于 PHP 开发的 Composer 扩展包,目前已累计 51 次下载、GitHub Stars 达 1, 最近一次更新时间为 2022 年 07 月 30 日, 在 PHP 生态内属于活跃度较高的组件。

它主要适用于以下技术方向: 「scraping」 「quickscraper」 等业务场景。在实际项目中,围绕这些方向常见需要落地的问题包括:接口对接、性能调优、并发安全、与既有框架(Laravel / ThinkPHP / Yii / Webman 等)的兼容适配,以及生产环境的日志埋点与稳定性保障。

我们在过去多个企业项目中使用过 quickscraper/sdk 或与其功能相近的方案,如果你在选型或落地过程中遇到问题,例如 版本兼容、二次改造、私有化封装、与内部系统对接、生产 BUG 排查,欢迎联系我们协助评估。

围绕 quickscraper/sdk 我们能提供哪些服务?
定制开发 / 二次开发

基于 quickscraper/sdk 在你已有业务上做功能扩展、字段裁剪、UI 适配、与内部账号 / 权限 / 日志系统的深度对接。

BUG 修复 & 性能优化

线上偶发问题、内存泄漏、慢查询、并发异常等排查修复;针对高流量场景做缓存、队列、索引层面的调优。

项目外包 & 长期维护

承接完整的项目从需求 → 设计 → 开发 → 上线 → 长期运维;也可按月提供技术保姆服务。

yvsm@zunyunkeji.com QQ:316430983 微信:yvsm316 西安尊云信息科技 · 专注 PHP / Go / 分布式系统研发

统计信息

  • 总下载量: 51
  • 月度下载量: 0
  • 日度下载量: 0
  • 收藏数: 1
  • 点击次数: 3
  • 依赖项目数: 0
  • 推荐数: 0

GitHub 信息

  • Stars: 1
  • Watchers: 0
  • Forks: 0
  • 开发语言: PHP

其他信息

  • 授权协议: MIT
  • 更新时间: 2022-07-30