Serpstack Review: A JSON Rest API for Google Search Results
If you need to fetch search engine results—in particular those from Google—one way to do it is to run an actual search on google.com and try to parse the intricate HTML code that you’ll get as a result. It can be done but it’s quite an endeavour. If you want to take an easier route, why not use a dedicated SERP API that you can feed with your search term and that will return the results in a format that can easily be parsed. One such product is the Serpstack API from apilayer. It takes your search query string and parameters as input and returns the search results in JSON format. Let’s have an in-depth look at the product.
We’ll start off as we often do by discussing SERP in general. Next, we’ll explain what it is and describe a few of the different types of results that are typically available from search engines. We will also explore some reasons why one would use a SERP API. And once we’re all on the same page, we’ll have a detailed look at the Serpstack API. We’ll have a quick look at some of its main features before we dig into how to use the API. And before we conclude, we’ll explore the service’s pricing structure and available plans.
SERP In A Nutshell
First things first, SERP is an acronym for Search Engine Results Pages. These are the web pages that are served to users when they search for something online using a search engine such as Google or Bing. The user specifies their search query using specific terms and phrases that are commonly referred to as keywords and then the search engine presents them with a SERP.
Every SERP is unique. This is even true of multiple search queries performed on the same search engine using the same keywords or query text. The main reason for that is that virtually every search engine customizes the experience for their users by presenting results based on a wide range of factors that go well beyond their search terms. These factors can include the user’s physical location, his browsing history and social settings, etc. And while two SERPs may appear identical and may contain many of the same results, they will often feature subtle differences.
Search engine providers such as Google, Bing or other players are constantly evolving their services and technologies and experimenting with their results. Their goal is to offer their users a more responsive and intuitive experience. As new technologies are introduced in the search space—and they are at an amazing pace, the SERPs’ appearance change, and they are today quite different from what they once were.
Organic Results
SERPs will normally contain two types of content: organic results and paid results. Organic results are listings of web pages that appear as a result of the search engine’s algorithm. More on this in a moment. Search Engine Optimization, or SEO, is the science—or is it rather an art—of manipulating content to trick search engines into listing our website for a specific keyword or search string.
About Search Algorithms and Ranking Signals
So, back to organic results. In a nutshell, they are listings that have been indexed by the search engine based on a number of factors that are collectively referred to as “ranking signals.” For instance, the search algorithm used by Google features hundreds of ranking factors. Search engine providers tend to keep the details of their ranking system secret. So, while nobody outside of Google—and possibly not that many people inside—know precisely what the ranking signals are, some are considered to be more important than others.
Why use a SERP API?
There are several reasons why anyone would have a use for a SERP API. There’s the obvious need for periodically retrieving SERP data for your products and websites and keeping track of it over time as it is an essential part of any SEO strategy. With rankings constantly fluctuating and search keywords becoming more or less applicable and valuable in the course of time, there is only one way of staying on top of these changes and that is to use a ready-made Software-as-a-Service (SaaS) solution that implements SERP data directly into your own applications or websites using a SERP API like Serpstack.
In a nutshell, the main difference between using a browser to retrieve SERP data and using a SERP API such as Serpstack is the format of the response. The Serpstack API will return JSON data that is easily parsed by your in-house software. Using it frees you from having to extract the meaningful data from all the surrounding web formatting.
Introducing Serpstack
In one sentence, Serpstack is a real-time and accurate Google Search results API. Trusted by some of the largest brands worldwide, this JSON REST API is lightning fast and easy to use. The product is built and maintained by apilayer, a software company based in London, United Kingdom and Vienna, Austria. Originally created out of an internal need to keep track of search engine positions in an automated fashion, the product has grown to become one of the most trusted SERP APIs available on the market. As for apilayer, this is the company behind some of the most popular API and SaaS products worldwide, including Currencylayer, Invoicely and Eversign.
The Serpstack API retrieves SERP data from search engines in an automated fashion by using a proxy network and a proprietary scraping technology whenever API requests are made. The resulting SERP data is publicly available and anyone can access it. Furthermore, the Serpstack API supports nearly all the search result types returned by Google, including web results, image results, video results, news results, shopping results, sponsored ads, questions, and much more.
Built on top of apilayer cloud infrastructure, a system that some of the world’s most popular real-time API services are running on, the Serpstack API is highly scalable and flexible at any stage. It was built to offer a way of scraping Google SERP data in real-time and at any scale. Implementing the service takes just a few minutes using a simple HTTP GET URL structure, and results are returned either in JSON or CSV format.
Main Features Of Serpstack
Feature-wise, the Serpstack API leaves nothing to be desired. First and foremost, the service is highly scalable and always queueless. Apilayer’s powerful cloud infrastructure was simply built to withstand high volume API requests without the need for a queue. It has been at the core of several of the company’s products. It is also highly customizable. Since no two customers have identical needs, you can tailor your automated search queries based on a series of options, including location, language, device, and more.
Using this product, gone is the worrying about global IPs, browser clusters or CAPTCHAs. Thanks to a powerful underlying proxy network, this SERP scraping API has got you covered in pretty much every single situation. And it’s also easy on the budget. You get up to 100 monthly requests for free and, should you need more than that—and you’ll most likely do—paid plans start at only $29.99 a month. We’ll cover the product’s pricing in greater detail shortly.
Here’s an overview of some of the product’s most important features:
Simple Integration
Using the Scrapestack API could hardly be easier right from the get-go. It only takes a few minutes to create customized queries and start scraping SERP pages at any scale using custom locations, devices, languages, and more.
High Level of Security
All data streams sent to and received from the Serpstack API are encrypted using 256-bit SSL Encryption. This is the same type of security that is used by banking sites or most e-commerce platforms. You can rest assured that your data will remain protected.
Unparalleled Speed and Availability
With a near-perfect availability score, you can rest assured that the Scrapestack API will be there when you need it. And apilayer never compromises on speed. API requests sent to the API are processed in a matter of milliseconds. Of course, how fast the results come is also dependent on Google’s response times.
Choice of Output Formats
To make suing the SERP data even easier, the Serpstack easy-to-use REST API offers a choice of JSON or CSV responses. That makes it compatible with any programming language.
World-Class Support
It is often said that a product is only as good as its technical support structure. Again, this is a place where Scrapestack shines. If you have any technical questions, a trained team of specialists is there to assist you with any requests you might have. The response is quick and the service works rather well.
Extensive Documentation
Just as important as the support, the Documentation will often be your primary way of learning how to use a product. Serpstack offers extensive API documentation that is waiting to be explored. Using it will get you started within just a few minutes and make you an expert in a matter of days, not weeks or months.
Using Serpstack
Enough bragging about the product’s great features—and they truly are, let’s have a look at how you can actually use the product and the different types of requests you can make and the corresponding responses you can expect.
The Basics
To use the Serpstack API, you’ll first need to sign up for an account. This is free and as a result, you’ll be assigned a unique API access key. This is what you’ll use from now on to authenticate with the API via simple HTTP GET requests to access any of the available endpoints. So, to use the API you simply need to append your API key to the base URL using the “access_key” parameter. Here’s what a typical—albeit very basic and totally useless—request looks like:
https://api.serpstack.com/search ? access_key = YOUR_ACCESS_KEY
For added security, all paid plans allow for the use of 256-bit SSL encryption. To use it, all you need is to send HTTPS requests instead of HTTP requests. For instance, the above example with encryption would become:
https://api.serpstack.com/search ? access_key = YOUR_ACCESS_KEY
Whenever an API request fails, the Serpstack API will return a JSON object containing details about the error. For example, here’s what would be returned by the API if the maximum allowed API request limit has been reached or exceeded. Each subscription plan has a different limit to the number of requests one can make.
{
"success": false,
"error": {
"code": 104,
"type": "usage_limit_reached",
"info": "Your monthly API request volume has been reached. Please upgrade your plan."
}
}
API Request and Parameters
Unless specified otherwise, the Serpstack API will always default to delivering search results from Google. Consequently, if you want to query the API for Google search results, all you need to do is append the “query” parameter to the API’s base URL and set it to a search query of your choice. Here’s how a query using “mcdonalds” as the search term looks like with authentication and encryption:
https://api.serpstack.com/search ? access_key = YOUR_ACCESS_KEY & query = mcdonalds
As you can see, this is a rather simple syntax. There are, as you certainly would have imagined, a ton of optional parameters that can be appended to a request to further narrow down the search results. We’ll explore a few of them in a moment but a detailed list of all of them is available in the online API documentation.
API Responses
The responses you’ll get from the Serpstack API will strongly depend on your search query and the parameters you choose to use. A basic query with no filtering specified will return a ton of data. Here’s the beginning part of the response you’d get from the example query above.
{
"request": {
"success": true,
"processed_timestamp": 1566207832,
"search_url": "https://www.google.com/search?q=mcdonalds",
"total_time_taken": 1.5
},
"search_parameters": {
"engine": "google",
"query": "mcdonalds",
"type": "web",
"device": "desktop",
"google_domain": "google.com",
"hl": "en",
"gl": "us",
"page": "1",
"num": "10"
},
"search_information": {
"total_results": 759000000,
"time_taken_displayed": 0.91,
"did_you_mean": null,
"showing_results_for": null,
"query_displayed": "mcdonalds",
"detected_location": null,
"no_results_for_original_query": false
},
The purpose of the above sample is just to give you an idea of what the results may look like. I’ve only included the first 25 lines of the response which, in its entirety is over four hundred lines long.
As you may have noticed when using Google search, a typical search result page has different sections, each displaying a different type of results. You’ll have paid ads results, organic results, images, videos, etc. Furthermore, a ribbon at the top of the page will let you display specific types of results such as maps, news, shopping or books, just to name a few. Various optional parameters will allow you to specify what type of search results you’re interested in getting, making the parsing of the resulting JSON file much simpler. Let’s have a deeper look at some of the different types of results you can get.
Ad Results
Whenever there are sponsored ads for your search query—and there will most often be, the API response will come with an ads object. It contains all the ads in the order they are shown in the search result. It includes both ads that are shown at the top and at the bottom of the search result.
Organic Results
Organic search results are the main search results provided by Google, and they are likely those you’ll be particularly interested in. They are determined by a series of factors, such as web traffic, back-links, social media presence, and much more, with the exact algorithm kept secret by Google in order to keep its competitive edge. These results are parsed by the API in detail and returned as “organic_results”.
Image Results
Just as there are two ways to get image results from a Google search, there are two ways images can be returned by the API. When doing a standard web search, some images are provided by Google within the search results. These images—which are referred to as inline images—are returned by the Scrapestack API within an “inline_images” array.
"image_results": [
{
"position": 1,
"title": "McDonald's Gold Card: The True Story ...",
"width": 1024,
"height": 683,
"image_url": "https://www.rd.com/app/uploads/2017/12/the-true-story-behind-the-mysterious-mcdonalds-gold-card-673955074-Vytautas-Kielaitis-1024x683.jpg",
"type": "png",
"url": "https://rd.com",
"source": "https://rd.com"
},
{
"position": 2,
"title": "McDonald's - Wikipedia",
"width": 1200,
"height": 1051,
"image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/3/36/McDonald%27s_Golden_Arches.svg/1200px-McDonald%27s_Golden_Arches.svg.png",
"type": "png",
"url": "https://en.wikipedia.org/wiki/McDonald%27s",
"source": "https://en.wikipedia.org"
},
{
"position": 3,
"title": "Best New Global Menu Items ...",
"width": 970,
"height": 545,
"image_url": "https://assets3.thrillist.com/v1/image/2825587/size/gn-gift_guide_variable_c.jpg",
"type": "png",
"url": "https://thrillist.com/",
"source": "https://thrillist.com"
}
]
If, on the other hand, images are searched specifically, with the type parameter being set to images, images are returned as an “image_results” array. In these situations, the API returns 100 images by default.
Video Results
The exact same considerations we just went over for images are also valid for video results with the availability of both inline videos withing the global search results or dedicated video results
We could go on for pages and give you detailed information on the various types of results that the Serpstack API can return, but we think you’re getting the picture. Suffice to say that whatever type of results are returned by a typical Google search is also available within the API’s response. In addition to the types of results already described, the API will return, news results, shopping results, and local results as well as the contents of the results page’s answer box, weather box, events, top carousel, top stories, etc.
Supported Programming Environments
One of the best things about the Serpstack API is that you can easily use it from within most environments, Whatever the language you are using, as long as you can send HTTP requests, you should be good to go. All the popular web languages such as PHP or Python are supported. In fact, the API’s documentation includes examples of how to use it from PHP, Python, Nodejs, jQuery, Go, and Ruby.
Here is, for instance, the PHP code to print the total number of search results as well as each organic search result returned by the Serpstack API.
$queryString = http_build_query([
'access_key' => 'YOUR_ACCESS_KEY',
'query' => 'mcdonalds',
]);
$ch = curl_init(sprintf('%s?%s', 'https://api.serpstack.com/search', $queryString));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$json = curl_exec($ch);
curl_close($ch);
$api_result = json_decode($json, true);
echo "Total results: ", $api_result['search_information']['total_results'], PHP_EOL;
foreach ($api_result['organic_results'] as $number => $result) {
echo "{$number}. {$result['title']}", PHP_EOL;
}
And here’s some code to accomplish the same thing in Python:
import requests
params = {
'access_key': 'YOUR_ACCESS_KEY',
'query': 'mcdonalds'
}
api_result = requests.get('https://api.serpstack.com/search', params)
api_response = api_result.json()
print "Total results: ", api_response['search_information']['total_results']
for number, result in enumerate(api_response['organic_results'], start=1):
print "%s. %s" % (number, result['title'])
Pricing And Plans
The Serpstack API is available under several plans depending on your needs. The most basic plan is the Free plan. As its name implies, it is free but, although it is not time-limited like typical trial versions it is feature-limited as well as usage limited. You’ll only be able to run up to 100 searches per month.
The next three levels are full-featured and vary only by the maximum number of monthly searches you’re allowed to run. The Basic plan will give you 5000 for $29.99/month, the Business plan at $99.99/month brings it up to 20 000, and the Business Pro plan lets you run a whopping 50 000 searches for $199.99 per month. If none of these plans suit you, there’s also a variably-priced Enterprise plan that can be customized to your exact needs. If you choose yearly billing instead of monthly, you’re entitled to a 20% discount, making the service even more affordable.
Wrapping Up
In a nutshell, the Serpstack API is an easy-to-use, full-featured product that you can put to good use whenever you need to fetch SERP data from Google. Its stellar uptime and impressive response time are some of its best assets. Backed by apilayer which has brought us other excellent APIs, we’re confident that this one will deliver the results you expect. This is a product that we can only recommend.