How to get Bing Shopping Search Results in JSON
We've recently launched a new API for scraping shopping search results at SerpApi - Bing Shopping API. This API is able to get structured search results of product data from Bing Shopping. This blog post will walk through its usage.
Introduction
The most straightforward way to try out the API is through the Playground. If you don't have an account yet, register one, and you will get 100 free searches per month, which should be enough for testing.
After fetching the results, you will be able to get the result page on the left and the parsed data in JSON on the right. You can click on the elements or JSON fields, and the corresponding JSON fields or elements will be highlighted. This is a very convenient feature for quickly checking if our Bing Shopping API parses the data you want.
You can also tune the parameters in the middle of the page to get different results. We'll be covering those later.
What's scraped
Let's take the search result of jacket for example. The page can be divided into the following blocks.
For the initial launch of the API. We've supported the following:
shopping_resultsinline_ads_resultsads_resultspagination
filters, refine_this_search and other blocks will be supported soon after the initial launch.
As of the writing of this blog post, here are all possible fields that may be parsed:
{
"shopping_results": [
{
"title": "String - Title of the item result",
"link": "String - URL to the item",
"thumbnails": "Array - List of URLs to the item thumbnails",
"seller": "String - Seller's name of the item",
"price": "String - Price to the item",
"sale": "Boolean - Is the item on sale",
"price_drop": "String - The percentage of the price drop",
"old_price": "String - Non-discounted price to the item",
"installments": {
"price": "String - Displayed price, e.g. $0",
"text": "String - Instalment text, e.g. now",
"installments": "String - Instalment price to the item, e.g. $41.67/mo",
"duration": "String - Instalment duration in months, e.g. 24",
}
"rating": "Float - Rating of the item",
"reviews": "String - Number of reviews of the item",
"price_history_link": "String - Link to the price history",
"compare_sellers_link": "String - Link to comparing sellers",
"free_shipping": "Boolean - Is the item shipping for free",
"top_pick": "Boolean - Is the item on top pick",
"popular": "Boolean - Is the item popular",
"great_deal": "Boolean - Is the item on great deal",
"special_offer": "String - The content of item's special offer",
"ethical_choice": "String - The 'Good On You' rating for the item's brand"
}
],
"inline_ads_results": [
{
"title": "String - Title of the item result",
"link": "String - URL to the item",
"thumbnail": "String - URL to the item's thumbnail",
"seller": "String - Seller's name of the item",
"price": "String - Price to the item",
"block_position": "String - Position of the item on the page, one of 'top', 'middle' and 'bottom'",
"old_price": "String - Non-discounted price to the item",
"unit_price": "String - The unit price to the item",
"installments": {
"price": "String - Displayed price, e.g. $0",
"text": "String - Installment text, e.g. now",
"installments": "String - Installment price to the item, e.g. $41.67/mo",
"duration": "String - Installment duration in months, e.g. 24",
}
"rating": "Float - Rating of the item",
"reviews": "String - Number of reviews of the item",
"free_shipping": "Boolean - Is the item shipping for free",
"tag": {
"top": "String - The tag to the item on the top",
"bottom": "String - The tag to the item on the bottom",
}
}
],
"ads_results": [
{
"title": "String - Title of the item result",
"link": "String - URL to the item",
"displayed_link": "String - Displayed link to the item",
"price": "String - Price to the item",
"snippet": "String - Snippet to the item",
"extensions": "Array - List of extensions to the item result",
"sitelinks": [
{
"title": "Title to the sitelink",
"link": "URL to the sitelink",
}
]
}
]
}If you don't find the fields you want, feel free to contact us or submit an issue at SerpApi's public roadmap.
Get location-specific results
The default results come from the US market if mkt is not specified. You are able to change the location with the mkt parameter. All possible values of mkt are available from the dropdown list in the Playground.
Deal with pagination
SerpApi's Bing Shopping API provides an easy way of accessing the results of the next/previous page through the pagination and serpapi_pagination attribute. Along with data results, you will get pagination information like this:
{
...
"pagination": {
"current": 1,
"next": "https://www.bing.com/shop?q=jacket&filters=scenario%3a%2215%22+LookupKey%3a%2244a324dedd814dd7a64d17673dd792a1%22+ParentRguid%3a%2244a324dedd814dd7a64d17673dd792a1%22&originIGUID=DE26044C76414C40B8790A3EA51E3DF0&page=2&efirst=37",
"other_pages": {
"2": "https://www.bing.com/shop?q=jacket&filters=scenario%3a%2215%22+LookupKey%3a%2244a324dedd814dd7a64d17673dd792a1%22+ParentRguid%3a%2244a324dedd814dd7a64d17673dd792a1%22&originIGUID=DE26044C76414C40B8790A3EA51E3DF0&page=2&efirst=37",
"3": "https://www.bing.com/shop?q=jacket&filters=scenario%3a%2215%22+LookupKey%3a%2244a324dedd814dd7a64d17673dd792a1%22+ParentRguid%3a%2244a324dedd814dd7a64d17673dd792a1%22&originIGUID=DE26044C76414C40B8790A3EA51E3DF0&page=3&efirst=73",
"4": "https://www.bing.com/shop?q=jacket&filters=scenario%3a%2215%22+LookupKey%3a%2244a324dedd814dd7a64d17673dd792a1%22+ParentRguid%3a%2244a324dedd814dd7a64d17673dd792a1%22&originIGUID=DE26044C76414C40B8790A3EA51E3DF0&page=4&efirst=109",
"5": "https://www.bing.com/shop?q=jacket&filters=scenario%3a%2215%22+LookupKey%3a%2244a324dedd814dd7a64d17673dd792a1%22+ParentRguid%3a%2244a324dedd814dd7a64d17673dd792a1%22&originIGUID=DE26044C76414C40B8790A3EA51E3DF0&page=5&efirst=145"
}
},
"serpapi_pagination": {
"current": 1,
"next_link": "https://serpapi.com/search.json?device=desktop&efirst=37&engine=bing_shopping&page=2&q=jacket",
"next": "https://serpapi.com/search.json?device=desktop&efirst=37&engine=bing_shopping&page=2&q=jacket",
"other_pages": {
"2": "https://serpapi.com/search.json?device=desktop&efirst=37&engine=bing_shopping&page=2&q=jacket",
"3": "https://serpapi.com/search.json?device=desktop&efirst=73&engine=bing_shopping&page=3&q=jacket",
"4": "https://serpapi.com/search.json?device=desktop&efirst=109&engine=bing_shopping&page=4&q=jacket",
"5": "https://serpapi.com/search.json?device=desktop&efirst=145&engine=bing_shopping&page=5&q=jacket"
}
}
}We provide the original pagination URL to Bing Shopping through pagination, but you may find serpapi_pagination more useful, which are URLs to the SerpApi search endpoints that return JSON results corresponding to the links from pagination. With serpapi_pagination, you can get full results of the keyword by iterating serpapi_pagination.next until it disappears.