Flexible, Highly Scalable Real-Time SERPs Scraping with Zenserp (REVIEW)
Google has a long history of giving a hard time to SEO developers attempting to scrape publicly available data off their search results pages. Indeed, SERP scraping is officially against Big G’s terms of service. However, the practice does not violate the Computer Fraud and Abuse Act (CFAA), and thus direct legal actions by Google to curtail the practice have fallen flat.
So, while scraping isn’t actually illegal or hurting anyone’s feelings in practice, developers must still contend with the roadblocks designed by Google to impede their data collection. Common headaches include Captchas and IP blocking; while cease-and-desist letters have been sent in the past, this tactic is increasingly rare.
But what if you didn’t have to deal with these nuisances at all to get your precious data? Enter Zenserp, a core offering from Austria-based SaaS Industries.
Zenserp in a nutshell
The Zenserp API is a dream tool for SEO devs looking for a turnkey solution to clear common scraping hurdles. It’s dead-simple to use and integrate, thanks SaaS Industries’ user-centric development cycle and broad language compatibility (Browser, Curl, Python, Node.js, PHP). What’s more, their voracious appetite for consumer feedback and feature requests has led to April 2019’s massive API update to include a comprehensive array of data types, including Google images, video, maps, news, shopping and trends.
Combined with competitive pricing and world-class customer service, you truly have an all-in-one solution to your SERP-scraping woes in Zenserp.
Getting started with Zenserp
So, what’s it like to actually use Zenserp? Below, we cover the process of getting started with Zenserp. Then, we take a dive into their API documentation to get a closer look at Zenserp’s robust functionality.
Dashboard overview
Assuming you’re just taking a peek at what Zenserp’s all about, you’ll click the bright green “Sign Up for Free Plan” button on their home page. Once you’ve created and verified your account, click “Login” to access your Dashboard.
The page you land on displays an overview of your most important usage information, including your API key, monthly request quota, API usage over time, plus documentation of any failed requests that may have occurred. There’s also a handy Request Builder section that directs new Zenserp users towards a wealth of useful resources; most of this is stuff we’ll cover in the Documentation section of this review, so hang tight on that.
For now, let’s take a quick look at all the sidebar options:
- Dashboard – You are here already! This is simply a quick link to return to your overview page at any time.
- Playground – Appropriately named, this page offers a fully visual (i.e., no coding required) and interactive interface for you to mess around with Zenserp’s various endpoints. Here, you can generate some sample responses, and use them to configure your API request. Don’t rely on this 100%, as there are corner cases beyond the capability of this tool. Nevertheless, we love how you have the option to export your experimentation to code (Browser, Curl, Python, Node.js, PHP) when you find something that really works.
- Documentation – Details the guts of the Zenserp API. We cover this in depth below.
- Profile/Security – Basic account configuration, including your avatar, contact information, and password management. Nothing fancy.
- Billing – Consists of three options, where you can upgrade/downgrade/cancel your subscription level, manage your payment method and VAT ID, plus a free field where you can request text be included on your Zenserp invoices for making accounting easier.
All of this is pretty standard fare for any API, but we appreciate how simply laid-out everything is. What you see is what you get, and Zenserp doesn’t attempt to bury subscription settings in obscure corners of your profile.
The Zenserp API documentation
With the basics out of the way, now we can get into the good stuff: Zenserp’s documentation. Here, you’ll see exactly what the API is capable of, and how to use it. While full API implementation into your app is beyond the scope of this review, we’ll briefly touch on each endpoint, some key parameters, and provide some example responses.
Basic functionality
Starting off the documentation page is an aptly named “Getting Started” section, which details your most basic API functions capable of interacting with all other endpoints. Let’s take a look:
- Authentication – This shows you how to use your API key for authentication. There is an example of use in the header, followed by GET and POST request examples. Easy peasy.
- /status – Allows you to request the number of remaining requests per your subscription limits. After all, why bother logging into your Zenserp Dashboard when you can just have your app tell you directly?
- /batch – Appending this endpoint allows you to make up to 1,000 searches in a single batch. You’ve got the option to request all batches at once, specific by batch ID, or to submit a new batch or merge existing ones together as needed.
- Error codes – Zenserp also has the ability to return errors as they occur, including 403 Forbidden when you’ve hit your monthly limit or used an incorrect API key, 404 when no results have been found, or 500 whenever there’s a server error. Don’t forget that failed requests also appear on your Dashboard.
The /search endpoint
The core functionality of Zenserp is here. By appending /search to your API request, you’ll receive a comprehensive array of data on basically anything you can think to collect SERPS for. Your query will return a wide array of results, including:
- Organic search results – Querying organic SERPs returns all kinds of data, including item ranking, its title, both displayed and destination URL, several types of reviews, whether the link is a Google AMP link, strings representing the page description, and plus the Local 3-Pack which contains business info.
- Paid results – You’ll also get a full overview of paid results. Info returned includes paid position (top, right, or bottom), title, the AdWords tracking URL, breadcrumbs, the visible URL as it appears on Google, page description, plus any paid product placements (including position, title, associated images, URL, and of course, price).
- Answer box/featured snippet/top story – Pure gold to any SEO. Easily see what performs best to gain these coveted spots on SERPs, then build a strategy to replicate those results.
- Related searches/did you mean – Zenserp enhances your keyword research by returning valuable data on all related searches to your query. You can also anticipate how Google responds to common errors users make when searching out your query.
Here’s a cut-down version of an example that appears in the Zenserp documentation, with “Pied Piper” as the query. You can see the different types of results that appear in order: the Wikipedia entry (1), questions related to the query (2), a website with an exact-match domain name (3), a popular article on the Pied Piper’s origins (4), plus some related videos on the topic (5). You’ll also see some related search phrases, plus the total number of results returned.
{
"query":{
"apikey":"YOUR-API-KEY",
"q":"Pied Piper",
"url":"https://google.com/search?q=Pied+Piper&hl=en&gl=us&sourceid=chrome&ie=UTF-8"
},
"organic":[
{
"position":1,
"title":"Pied Piper of Hamelin - Wikipedia",
"url":"https://en.wikipedia.org/wiki/Pied_Piper_of_Hamelin",
"destination":"https://en.wikipedia.org › wiki › Pied_Piper_of_Hamelin",
"description":"The Pied Piper of Hamelin is the titular character of a legend from the town of Hamelin (Hameln), Lower Saxony, Germany. The legend dates back to the Middle ...",
"isAmp":false
},
{
"position":2,
"questions":[
{
"question":"What is the meaning of the Pied Piper?"
},
{
"question":"What is the story of the Pied Piper?"
},
{
"question":"Is the pied piper a fairy tale?"
},
{
"question":"Is Pied Piper a real company?"
}
]
},
{
"position":3,
"title":"Pied Piper",
"url":"https://www.piedpiper.com/",
"destination":"www.piedpiper.com",
"description":"The old Pied Piper “flute player” logo was referencing a fairytale character and in no way was meant to suggest a sexual act involving either the mouth or ear of ...",
"isAmp":false
},
{
"position":4,
"title":"The Disturbing True Story of the Pied Piper of Hamelin ...",
"url":"https://www.ancient-origins.net/myths-legends/disturbing-true-story-pied-piper-hamelin-001969",
"destination":"https://www.ancient-origins.net › myths-legends › disturbing-true-story-pie...",
"description":"Aug 14, 2014 - Many are familiar with the story of the Pied Piper of Hamelin. Few realise however, that the story is based on real events, which evolved over ...",
"isAmp":false
},
{
"position":5,
"videos":[
{
"title":"Pied Piper",
"url":"https://www.youtube.com/watch?v=VEZ_Ui6d9AM",
"parsed":{
"source":"YouTube",
"date":"May 2, 2018",
"channel":"BTS - Topic"
}
},
{
"title":"The True History of The Pied Piper of Hamelin | Fairy Tales ...",
"url":"https://www.youtube.com/watch?v=obIAwAAcAAg",
"parsed":{
"source":"YouTube",
"date":"Aug 4, 2017",
"channel":"Jen Campbell"
}
},
{
"title":"The Pied Piper: a modern day fairy tale",
"url":"https://www.wvi.org/child-health-now/video/pied-piper-modern-day-fairy-tale",
"parsed":{
"source":"World Vision International",
"date":"Aug 22, 2013",
"channel":"WVIAdvocacy"
}
}
]
}
],
"related_searches":[
{
"title":"pied piper instrument",
"url":"https://google.com/search?hl=en&gl=us&q=pied+piper+instrument&sa=X&ved=2ahUKEwj-vPeI3oXnAhVJJBoKHQ6iDYkQ1QIoAHoECBMQAQ"
},
{
"title":"pied piper poem",
"url":"https://google.com/search?hl=en&gl=us&q=pied+piper+poem&sa=X&ved=2ahUKEwj-vPeI3oXnAhVJJBoKHQ6iDYkQ1QIoAXoECBMQAg"
},
{
"title":"moral lesson of pied piper story",
"url":"https://google.com/search?hl=en&gl=us&q=moral+lesson+of+pied+piper+story&sa=X&ved=2ahUKEwj-vPeI3oXnAhVJJBoKHQ6iDYkQ1QIoAnoECBMQAw"
},
{
"title":"the pied piper of hamelin summary",
"url":"https://google.com/search?hl=en&gl=us&q=the+pied+piper+of+hamelin+summary&sa=X&ved=2ahUKEwj-vPeI3oXnAhVJJBoKHQ6iDYkQ1QIoA3oECBMQBA"
},
{
"title":"pied piper of hamelin story pdf",
"url":"https://google.com/search?hl=en&gl=us&q=pied+piper+of+hamelin+story+pdf&sa=X&ved=2ahUKEwj-vPeI3oXnAhVJJBoKHQ6iDYkQ1QIoBHoECBMQBQ"
},
],
"number_of_results":34800000
}
Refining your request by search type
Beyond the default SERPs request, you can further refine your Google Search query by search type (Image, Video, Maps, Shopping, News, Shopping, etc.). Simply set the tbm (“to be matched”) parameter to any of the following options:
- isch – Returns Google Image Search results. This request returns a litany of data including position, thumbnails, source URLs, titles, and more. (For Reverse Image Search, you’ll run the image_url parameter instead of tbm=isch)
- vid – Returns Google Video Search results. You’ll get info like source domain, video URL, title, description, plus any extensions or thumbnails.
- lcl – Another goldmine of SEO data, the lcl parameter returns Google Maps Search results. You’ll see business/location name, website URLs, whether or not the result is paid-for, any stars or reviews, plus vital information like address, phone numbers, hours of operation, notable prices, and even the type of business. You also have the option to request directions to these businesses.
- nws – Need to keep up to date with current events and developments in your industry? Add the nws parameter to your query to return real-time Google News Search results. See names of articles, their links, dates published (and parsed by your API), the publisher, a brief article description, plus any available thumbnails.
- shop – Access Google Shopping Search results with the shop parameter. Let’s take a look at some sample results (heavily truncated for the sake of readability) for water heaters in Atlanta, GA:
{
"query": {
"q": "water heater",
"tbm": "shop",
"device": "desktop",
"gl": "US",
"location": "Atlanta,Georgia,United States",
"token": "03AHaCkAaxBFsJL7NT22hKMf3S8--99l5CUVg9XTmsk0PEakhoy-4NScAkAt6h995YdYgCGeA3DvJdQyh4qPdsrNihev9B1IfMWx381jmVXGNByr-z4zyswm_36ghJOn7VRtKaKTO7uGzHJdkvhZVc-OMWO37wI6rnOvXDu9KaVmnIo2ZeQVjNa7RVrJ2YB-zEbCY8iUCoqFcyGUyVEJckRy3CmJWMStwC_m-ECc6gIjNxL8ENqbFHbeqkFfWbjuximGyVtCamXO9JRv42GBs3FXw013QzCLbJR3vEzrIMXoUVUmtD2fUaeiojxE7wWeG5dsmDA5CPU0Ps3TPqHi_zCUFFG_7tWDfGxfcYXQPwXSq3Now5YEeXMpqgk0A9iNNkjAPZP0HRxkyx",
"url": "https://www.google.com/search?q=water+heater&uule=w+CAIQICIdQXRsYW50YSxHZW9yZ2lhLFVuaXRlZCBTdGF0ZXM&gl=US&tbm=shop&hl=en-US"
},
"no_results_auto_correct": "Sponsored ",
"related_searches": [],
"shopping_results": [
{
"title": "Reliance Water Heater 219809 50 gal Hybrid Electric Heat Pump",
"link": "/aclk?sa=l&ai=DChcSEwjR-LWBnPDoAhUI1WQKHSvMBFUYABAFGgJwag&sig=AOD64_2X7NgvfDv9ugCCLYqCFpiZP8Xq3g&ctype=5&q=&ved=0ahUKEwisl7GBnPDoAhWEr54KHf6zCRMQ2CkIjwM&adurl=",
"description": "About This Item We aim to show you accurate product information. Manufacturers, suppliers and others provide what you see here ...",
"price": "$59.99",
"source": "resenta",
"reviews": 0,
"thumbnail": "data:image/jpeg;base64,/9j/4AAQSk...",
"extensions": [
"Tank",
"Electric",
"50 gallons",
"Energy Star",
"Heat Pump"
],
"price_parsed": {
"currency": "USD",
"value": 59.99
}
},
{
"title": "110v 3000w Instant Electric Tankless Hot Water Heater Shower Kitchen",
"link": "/aclk?sa=l&ai=DChcSEwjR-LWBnPDoAhUI1WQKHSvMBFUYABAGGgJwag&sig=AOD64_0Iy1yAMjTocLSO7Nhczxc4RaHBqQ&ctype=5&q=&ved=0ahUKEwisl7GBnPDoAhWEr54KHf6zCRMQ2CkImwM&adurl=",
"description": "110V 3000W Instant Electric Tankless Hot Water Heater Shower Kitchen Bath Mini Specification: Brand new Material: ABS Color ...",
"price": "$39.99",
"source": "eBay - ss_487",
"reviews": 0,
"thumbnail": "data:image/jpeg;base64,/9j/4AAQSk...",
"extensions": [
"Tankless",
"Electric"
],
"price_parsed": {
"currency": "USD",
"value": 39.99
}
}
}
Note: No matter which type of SERP request you make, you still have full access to the standard set of parameters under the /search endpoint.
The /shopping Endpoint
Besides the core /search endpoint, the Zenserp API also features /shopping, which enables you to parse Google Shopping Product Pages. You’ll append the product_id parameter to your /shopping query, with the option to specify your desired search geo-location (location), country code (gl) and/or Web Interface Language (hl). You can also change which region variant of Google Search you want to use with search_engine.
The /trends Endpoint
Google Trends requests are made by appending /trends to your request (though Zenserp’s documentation incorrectly states that you should use /shopping). Your Trends results are powered by the keyword[] parameter, where you pass your keyword phrase as the argument within the []. Zenserp’s documentation has a handy reminder about which operators Trends uses (+ for “or”, – for “exclude”, space for “and”, and quotes “ “ for an exact match), which will be instrumental in your query.
You can further specify timezone (tz), category ID (cat), the timeframe for your results (hourly, daily, monthly, yearly in various subdivisions), the type of Google property to search (like, images, Flights, YouTube, Finance, etc.), the geolocation or Google location code (gl), and even the resolution within that region (essentially, setting whether you search country-wide, city-wide, regionally, etc.)
So, let’s say you wanted to see how Kanye West has been trending over the past month. You’d make the request like so:
https://app.zenserp.com/api/v1/trends?apikey=YOUR-API-KEY&keyword%5B%5D=kanye%20west&timeframe=today%201-m
The API then returns a month’s worth of Google Trends info, followed by related queries–which include, unsurprisingly, Taylor Swift and Kim Kardashian. (Below, we take a look at just the past week’s interest in Yeezy for the sake of saving space on this page.)
{
"Kanye West": {
"trends": {
"2020-04-08T00:00:00.000Z": {
"Kanye West": 26,
"isPartial": false
},
"2020-04-09T00:00:00.000Z": {
"Kanye West": 29,
"isPartial": false
},
"2020-04-10T00:00:00.000Z": {
"Kanye West": 31,
"isPartial": false
},
"2020-04-11T00:00:00.000Z": {
"Kanye West": 35,
"isPartial": false
},
"2020-04-12T00:00:00.000Z": {
"Kanye West": 38,
"isPartial": false
},
"2020-04-13T00:00:00.000Z": {
"Kanye West": 31,
"isPartial": false
},
"2020-04-14T00:00:00.000Z": {
"Kanye West": 24,
"isPartial": false
},
"2020-04-15T00:00:00.000Z": {
"Kanye West": 32,
"isPartial": false
}
},
"queries": {
"Kanye West": {
"top": [
{
"query": "taylor swift kanye west",
"value": 100
},
{
"query": "taylor swift",
"value": 94
},
{
"query": "kanye west kim kardashian",
"value": 65
},
{
"query": "kim kardashian",
"value": 61
},
{
"query": "famous kanye west",
"value": 36
},
{
"query": "famous",
"value": 35
},
{
"query": "kanye west net worth",
"value": 31
}
...
Performance
Now that we’re done with the features overview, how does Zenserp actually perform? After all, all this nice data is useless if you can’t actually access it due to Google’s stonewalling programmatic data collection efforts.
In our experience, Zenserp performed its tasks without a hiccup, no matter what we searched, and where in the world we targeted our queries. The secret weapon here is Zenserp’s incredible worldwide proxy network, which redirects your requests where they need to go while changing your IP address to fit. While Zenserp doesn’t publicize the actual numbers of their server network, it proved sufficiently robust to never throw up a single error when we made our test requests.
Also included under the hood of the Zenserp API is a sophisticated Captcha solver. Once again, we don’t have details on just how they made this work; but work it did, and consistently too.
But hey, don’t take our word for it! Head over to the Zenserp API Playground and run your own real-time tests.
What about uptime? After all, Zenserp’s clientele typically have an ongoing requirement for high-volume, up-to-date SERPs information. That’s a lot of data to process, potentially on the order of millions of requests per month depending on the size of your company. Fortunately, Zenserp is well-equipped to handle such traffic, and reports an incredible 99.9% uptime over the past 12 months.
Pricing
Zenserp’s infrastructure is robust enough to satisfy any scale of operation. They offer a wide range of subscription price points so that you only pay for exactly what you need. Whichever tier you choose, you can pay a low monthly fee, or snag a 20% discount on a yearly plan.
There are five core subscription models:
- FREE – True to its name, the Free tier costs you nothing, and doesn’t demand that you put credit card information on file. Your allotment of monthly requests is quite low at just 50 total, but it does give you plenty of room to play around with all the available SERP types. We really appreciate that, unlike most free APIs, Zenserp enables HTTPS encryption by default. Even if you’re a small startup with limited need for SERPs scraping, your activity gets the same level of security and anonymity as that of a global enterprise. You’ll have to integrate the API yourself, and all your questions will be answered over email, but overall this is a generous offer for people who know what they’re doing and want to explore Zenserp with no obligations.
- SMALL – This tier costs $29.99 per month. Truthfully, this tier is identical in utility to the FREE plan, though it sees a hundredfold increase in your quota to a maximum of 5,000 monthly requests. Once again, if you have a modest need for SERP data of any time, and the knowhow to handle your own API integrations, this represents a great value.
- MEDIUM – Zenserp’s most popular choice, the MEDIUM tier will set you back $74.99 per month while increasing your monthly quota to a generous 15,000 requests. However, this tier unlocks a host of new functionality, including batch requests and vital SEO intel like keyword search volume and keyword CPC. You also get concierge onboarding, personalized customer service, plus an SLA. Of course, you still get the ever-valuable HTTPS encryption to keep your business safe from prying eyes.
- LARGE/VERY LARGE – These two tiers include identical feature sets to the MEDIUM tier, but vastly enhance monthly capacity. The LARGE tier costs $174.99 per month, offering 40,000 requests; while VERY LARGE is priced at $379.99 for a whopping 100,000 monthly requests. Choose the subscription that best fits your company’s needs!
In addition to the standard tiers, Zenserp offers High Volume Plans, which are locked into a monthly payment scheme. Each tier includes all premium features of the MEDIUM-VERY LARGE plans, and scales your monthly quota according to how much you pay:
- PREMIUM – $599.99 and 250,000 requests per month
- PROFESSIONAL – $1,399.99 and 750,000 requests per month
- ENTERPRISE – $1,599.99 and 1,000,000 requests per month
Not enough? You can also request a customized plan. Simply reach out Zenserp’s enthusiastic support team, and they’ll work with you to develop customized API solutions to fit your needs.
Zenserp final verdict
SaaS Industries has produced an incredible offering with their Zenserp API. It’s customizable and highly scalable thanks to its robust infrastructure, yet dead-simple to integrate into your app thanks to easy parametric logic and wide coding language compatibility.
Our testing yielded consistent results without ever throwing up an error, though we suspect the service is not invulnerable to errors on the scale required by large enterprises. No matter what type of request we made, the results were always timely and meaningful. It is enormously helpful to be able to refine your SERPs request by search type, and as such it’s clear that Zenserp offers an invaluable yet affordable tool to any SEO dev.
If we had any criticism of the service, it’s that the documentation could use a touch more natural language to explain its contents. Of course, an experience developer should have no problems understanding what’s on offer, so this is likely a non-issue. Even if it is, all but the smallest payment tiers include concierge onboarding and personalized customer support, so you’ll always be able to address your needs perfectly.
All in all, Zenserp gets the Addictive Tips stamp of approval; we wholeheartedly recommend this API to anyone looking for a truly sophisticated SEO tool.
Got any questions or comments about Zenserp? Start a discussion in the comments below.
