Scraper API

Scraper API刮板API

The scraper API is a proxy API for web crawling. They provide APIs that can encode proxies, browsers and verification codes, so you can get data from any web page through simple API. You can access more than 20 million residential IPs pool. They provide unlimited bandwidth and only count the number of API requests you make.

Fast And Powerful Proxy API

  • More than 20 million residential IP in the pool
  • Simple dashboard to manage usage and billing
  • Covering more than 12 countries/regions in the world
  • Free trial with 1000 requests
  • Seven day refund policy
  • 24 / 7 support and excellent customer service
  • Rotating and sticky IP sessions
  • Simple setting
  • Customize browser headers
  • Advanced proxy pool
  • Automatically extract data from popular websites

代理API

The scraper API provides proxy services designed specifically for web crawling. With more than 20 million residential IP addresses in 12 countries and software that can handle JavaScript rendering and solve captcha, you can quickly complete large grab jobs without worrying about being blocked by any server.

The scraper APIs are very simple to use, and they provide unlimited bandwidth. proxies rotate automatically, but users can choose to maintain sessions as needed. All you need to do is use the API with the URL you want to grab, which will return the original HTML data. With the scraper API, you just focus on parsing the data, and the rest is handled by them.


商业刮板API计划

How Does Scraper API Work?

  • The scraper API will automatically rotate IP addresses from 20 million residential IP pools and automatically retry failed requests.
  • The advanced software of the scraper API will solve all code verification.
  • It can be used to grab pages rendered in JavaScript – just set “render = true” in the API call, which will render the page using a headless browser and return the data to you.
  • You can set a custom header, so the website will not be able to distinguish your activities from normal browsing activities.
  • They have servers in more than 12 different countries, and you can easily set where you want requests to come from.
  • Unlimited bandwidth – charges you based on the number of requests you need and the options you need.
  • The server speed is very fast, up to 100MB / s, and 99.9% normal operation time is guaranteed.
  • They provide professional support with short response time

Scraper API Price Plan

The scraper API has three standard pricing plans, one free plan, and you can request custom plans.

There is a 7-day refund policy. The free plan does not require you to enter any payment information, so it is completely free.

Scraper API的定价

The hobby program is $29 a month and can provide 250000 API calls for your standard agent with up to 10 concurrent requests. In the startup plan, your standard agent will receive 1000000 API calls, up to 25 concurrent requests. You can also choose to set the IP to come only from U.S. or de servers. The monthly fee is $99.

The business plan costs $249 per month and can provide you with 300000 API calls per month. You will be able to choose from 12 countries of IP origin. The plan also allows JavaScript rendering, advanced agents, and up to 50 concurrent requests.

If you need more concurrent requests or API calls every month, you can contact the sales team to organize the enterprise custom plan.

Business and enterprise customers will receive priority email support.

How To Use Scraper API?

The free program of the scraper API will provide you with 1000 requests per month and all available functions, so you can test it yourself.

When you log in to the scraper API account, the system will display a dashboard that will show how many requests you have used, how many requests remain in the current month, and the number of failed requests (not included in your request limit).

scraperapi的仪表板

Your API key and sample links will also be displayed on the home page, which you can copy and paste directly into your code.

In the left column, you will see a link to its document page, which has a clear description of each function. It contains code that can be copied and pasted directly into bash, node, python / scrapy, PHP and Ruby scripts.

By default, your IP will rotate automatically for each request – captcha is automatically resolved on the server side.

Simple Usage

The basic methods to access the API are:

http://api.scraperapi.com/?api_ key=[yourAPIkey]&url=[URL]

You can even try to access the key through your browser. The server will then return the HTML of the web page, which you can then store as a variable:

<html>

< head >

</ head>

< body >

Hello, world

</ body>

</ html>

It’s really that simple.

Rendering JavaScript

If you need to render JavaScript, just add ‘& render = true’ at the end of the API call, for example:

http://api.scraperapi.com/?api_ key=[yourAPIkey]&url=[URL]&render=true

The scraper API server will use a headless chrome instance to access the page, render it, and then return the code to you.

Custom Header

If you need to keep the original request header to keep the same user agent, cookie, etc.,just add ‘&keep_headers = true’

http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&keep_headers=true

Session

You can continue to use the same proxy for multiple purposes. The session expires 60 seconds after the last use. All you need to do is add ‘&session_number = [number]’- the number can be any integer. For example:

http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&session_number=123

Locations

You can use ‘& Country_code = [flag] ‘set the proxy to come from a specific location. Users using the startup plan and later will be able to set the agent location to the United States. Business plan customers will have access to proxies in Canada (code = ca), the UK (code = uk), Germany (code = de), France (code = fr), Spain (code = es), Brazil (code = br), Mexico (code = mx), India (code = in), Japan (code = jp), China (code = cn) and Australia (code = au). Example:

http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&country_code=us

IP Pool

Some websites may be more difficult to crawl, so the scraper API retains private internal residential and mobile IP pools for users of enterprises or enterprise plans. The charges for these IPS are 10 times the normal IP rate (that is, they will count as 10 requests instead of one), and if JavaScript rendering is required, each request will count as 25 requests. To use advanced IP, simply add ‘& Premium = true’ to your request:

http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&premium=true

Account

You can write a program to automatically monitor your account. Simply send a request to the API without any URL or any other settings, and it will return your current account status:

http://api.scraperapi.com/?api_key=[yourAPIkey]

Result example:

{

  “ concurrentRequests”:553,

  “ requestCount”:6655888,

  “ failedRequestCount”:1118,

  “ requestLimit”:10000000,

  “ concurrencyLimit”:1000

}

Scraper API ISP Test

scraperapi的IP

ISP对scraperAPI的测试

Therefore, you know they are using real residential IP.

Pros of:
  • More than 20 million residential IP addresses
  • Auto rotating IP
  • Be able to render Java scripts and solve verification codes
  • User friendly dashboard for proxy management
  • Free plan with 1000 requests per month