SerpApi Release Notes


History

2020-04-02
- Update app/models/search/google/parser.rb

2020-04-01
- Update app/models/search/google/parser.rb
- Update app/models/search/google/parser.rb
- Update app/models/search/google/parser.rb
- Update app/models/search/google/parser/local_results.rb

2020-03-31
- Fix places results gps for some searches
- Exclude related searches widget from images_results

2020-03-30
- Add parser

2020-03-28
- Fix lsig extraction

2020-03-27
- Fix parser

2020-03-26
- Update rich_snippet parser
- Fix and update popular_times parser
- Add question mark at the end of method name to indicate boolean

2020-03-25
- Added new selector for top stories title.
- Update parsers
- Restructure parser logic

2020-03-24
- Fix people_also_search_for parser
- Fix phone parser across apis
- Add constraints on thumbnail extraction as an option.
- Fix parameter encoding
- Rename hashes

2020-03-23
- Fix thumbnail extraction from inline top stories

2020-03-20
- Look for preceding g-tray-header to define group name.
- Add validator
- Remove json parser
- Sepparate main `organic` results parser to its own file
- Update documentation
- Changed tests for mobile top stories.
- Remove unnecessary whitespace

2020-03-19
- Fix inline top stories for mobile.
- Update parser
- Revert "Fix Google product pagination"
- Fix Google Product star rating extraction with no reviews

2020-03-18
- Update parser to support live videos
- Add hl parameter
- Finish parser

2020-03-14
- Update selectors

2020-03-13
- Fix selector
- Unit-test get_thumbnail
- Fix google_reverse_image_spec
- Fix design_alt_spec
- Fix local_results_for_products_spec

2020-03-12
- Use get_thumbnail everywhere
- Normalize Google Scholar Cite
- Normalize Google Product

2020-03-11
- Update Google Product "Product not found" selector
- Normalize Google Jobs
- Normalize Google Events
- Normalize Google
- Fix Yandex infinite loop on empty results response
- Normalize Yandex
- Normalize Yahoo!
- Normalize Bing
- Normalize Baidu

2020-03-10
- Fixed missing inline videos for mobile. Added carousel parsing for organic results for mobile. Fixed related searches for mobile.
- Fix spec/search/google/images_results/images_results_spec.rb
- Fix spec/search/yandex/yandex_organic_results_coffee_spec.rb
- Add parser for old youtube layout

2020-03-09
- Extract JS image for Google Local results thumbnails
- Fix events_results_mobile_spec tests: added selector for table layout of events
- #611: Get additional inline_top_stories from js.

2020-03-06
- Extract query_displayed from script.
- Added comment on query_displayed for spelling fix result.
- Added spelling fix for yandex search.
- Convert location_requested into location for and api request.

2020-03-05
- Fix token authentication for Guests
- Add new saved HTMLs

2020-03-04
- Fixed misspelling for multiword query.
- Fix Baidu captcha not getting detected
- Check for num parameter value and set to 21 if value is 20
- Fix 302 loop
- Disable youtube polymer

2020-03-03
- Added search_information for Yandex engine.
- Add device and location to the cypher key to make it more unique (search md5 uses user_id which is not know in token auth)
- Update parser
- Implement the changes
- Remove device from query
- Fix Ebay encoding

2020-03-02
- Fix api pagination bug with escaping location.
- Remove debugging statements
- Convert to binary encoding to store more hex chars in the same space
- Fix Yandex encoding

2020-02-28
- Fixed yandex loop issues.
- Add more parsers
- Add support for more parameters

2020-02-27
- Add video_results parser
- Add initial parser support

2020-02-26
- Fixed selectors for total_results and time_taken_displayed.
- Update sports results parser
- Update Google results_state parser

2020-02-25
- Fix spelling_fix selector.
- Fix Bing local pack gps extraction
- Fix Bing organic results thumbnail extraction
- Add and update js_image_extractor for Bing
- Revert "Revert "Fix spelling result for image searches.""
- Update Google product API results parser

2020-02-24
- Update Google product result parser
- Update google flights parser
- Update images_results.rb
- Fix Google direct answer box parsing against new design

2020-02-21
- Revert "Fix spelling result for image searches."

2020-02-20
- Fix and update Google lcl parser
- Fix and update Google local pack mobile parser
- Fixe spelling result for image searches.
- Make parameters escaping explicit.

2020-02-19
- Update sports results parser

2020-02-18
- #554: Fixed uule for reverse image search.
- #554: Fixed uule escaping for google jobs and events. Added location to events playground. Added location and uule to jobs playground. Added tests to ensure uule is encoded correctly.
- Transformed inline images tests. Fixed case for js image extraction for non-base64 images.

2020-02-14
- Added new selector for google total results.
- #198: Removed present_with_type helper. Moved yandex languages into a separate page Changed validator to compare against html elements.

2020-02-13
- Fixed yahoo local pack reviews for mobile.
- Fixed reverse images devito spec. Transformed reverse images witcher spec and made non-strict.
- Requested fixes and changes
- Changes and fixes
- Transformed yahoo organic tests.

2020-02-12
- Update parsers
- Return key as "unknown" if value empty

2020-02-11
- #198: Addressed PR comments.
- Make sure key is never blank to ensure JSON.load work in every languages
- Update parsers
- Add TextUtils.snakecase method
- Update ebay tests
- Update Google events

2020-02-10
- Remove comments.

2020-02-07
- #198: Addressed pr comments.
- Fix empty hashes by replacing parameterize method
- Update parser
- Update parser

2020-02-06
- Implement changes
- Transform test for samsung tv `tbm=shop` into # AUTO_ADD_SAVED_HTMLS
- Revert "Fix typo"
- Add documentation
- Update validator
- Restore old selectors for google jobs backwards compability.
- Update validator
- Add support for events_results_state
- Fixed google jobs parsing after updated jobs page design.

2020-02-05
- #198: removed unused code.
- #198: Added yandex search api. Added tests for yandex search parser organic search and knowledge graph. Added parser for organic search and knowledge graph. Added language and page parameters. Added playground. Added domains and languages. Added parser for inline images and inline videos. Added documentation. Added tests for inline images and inline videos. Fixed bug in organic results parsing. Fixed bug in inline images parsing. Made correction to documentation.
- Implement changes
- Extract anchor tags instead of list items and remove if link = item.at_css('a') check
- Use serpapi.com for all urls except development
- Add parser
- Remove location support from Google events
- Move extract_trending_searches_by_node
- Rename rank to position
- Fix typo
- Update extract_js_image method
- Add initial support for Google events engine API

2020-02-04
- Update app/models/search/ebay/parser/organic_results.rb
- Remove safe navigation as nil key for pagination should break anyway
- Implement the TrendingSearches parser
- Fix link and title parsing when tbs=shop
- Remove unnecessary check for existence of anchor tag in refine_by_item
- Add parser
- Fix 'serpapi_link' when in 'suggested_searches' of Google Images results when chips query is separated by spaces or '+' signs

2020-02-03
- Only use `serpapi.com` for production URLs
- Revert the location_requested backup
- Add parser

2020-01-31
- Move SearchSplitter Bing cookie logic to Validator
- Update links extraction to use get_valid_link method
- Update Ebay search API
- Add Google shopping results filters parser

2020-01-30
- Update Google product result mobile parser

2020-01-29
- Update parameters.rb
- Add price parameters
- Fix links in Google product results

2020-01-28
- Support new layout format that uses `span` instead of `cite` for inline top stories
- Update Ebay engine with proper parameter names
- Update parser.rb
- Update parser

2020-01-27
- Check if the result is sponsored
- Add raw and extracted price
- Fix related_searches parser when empty results
- Add pagination
- #160: Renamed organic_results to image_results for reverse image search.

2020-01-24
- #160: Prevent knowledge_graph image to appear null.
- #160: Removed unused headers from utils return value. Changed error message.
- Fix Google flight parser issue
- Fix typo
- Revert the check for "$$$"
- Update app/models/search/google_maps/parser/maps_search_results.rb
- Update parsers
- Add spelling fix

2020-01-23
- Addressed PR comments.
- Add comments
- Fix price parsing on restaurants and hotels
- Update organic_results to support results from foreign domains
- Update validator
- Update validator
- Add new parameters (pgn, ipg, ebay_domain)
- Add related_searches parser
- Add related_items parser
- Update get_valid_link method

2020-01-22
- #160: Fixed knowledge graph image.
- #160: Fixed knowledge graph image.
- Add results_state parser
- Add organic_results parser
- Add initial support for ebay engine

2020-01-21
- #160: Added playground. #160: Fixed storing reverse image result url.
- Prioritize `data-iurl` actual image urls over `data:image` ones
- Fix pagination
- Update parameters.rb

2020-01-20
- Update parameters.rb
- Return empty error if no results for a query
- Update validator

2020-01-17
- Fix rating extraction for Yahoo searches
- Add playground support for google scholar cite engine
- Add parser
- Add Google scholar cite engine support
- Generate google scholar cite link

2020-01-16
- #160: Added parameters, parser and validator. #160: Added extra request to get link to images. #160: Added specs. #160: Reuse google parser modules.
- Update hash names

2020-01-15
- Update menu_pages.rb
- Update knowledge_graph.rb
- Update top_stories parser
- Fix news parser
- Extrack `stick` parameter
- Update parser
- Remove top_carousel endpoint
- Add aditional info to Google's KG popular_times
- Update validator

2020-01-14
- Fix google_product validator
- Update regex

2020-01-13
- Enforce indentation cop on all .rb files

2020-01-10
- Fix product_results test by making rating a float number
- Fix rating extraction in google product results
- Add comment about music block in Baidu organic results

2020-01-09
- Exclude music block from organic search
- Fix Baidu parser
- #431: Replaced URI with Addressable. Removed unused code.
- #237: Changed default query form coffee to barista for jobs engine in playgroud. Fixed typo in query_ordered.
- #237: Added defensive guard clauses.
- #237: Addressed comments for documentation and tests.
- Update local_pack parser
- Rename PATTERNS to IMAGE_URL_PREFIXES

2020-01-08
- Disable all Rubocop cops except indentation-related ones
- Fix video_results extraction
- Fix inline_images parsing from Google SERPs
- Fix google_flight parsing
- Check source attribute prioritization of get_valid_link
- Revert changes in Bing local parser

2019-12-30
- Undo unescaped link error changes to validator.
- Fix unescaped urls raising InvalidURIError.
- Add missing saved HTMLs for sports results
- Improve get_valid_link
- Check if place is verified
- Extract gps on desktop
- Don't show `top_carousel` if `stick` parameter is used
- #237: Added parser, validator, parameters modules for jobs engine. #237: Scrape basic jobs data. #237: Added tests for jobs scraping. #237: Added chips scraping. #237: Added playground page for jobs engine. #237: Added documentation page for jobs. #237: Changed tests to use automatic test file generation.

2019-12-27
- Fix ruby style
- Don't use to_sym in Bing local results
- Update parser
- Add yahoo local pack parser
- Add device support to Yahoo

2019-12-26
- Update ads_results.rb
- Update ads_results.rb
- Update organic results for mobile support
- Update related_searches.rb
- Add device support to Yahoo
- Add tests
- Add rich_snippet parser to Yahoo organic results
- Update parser.rb

2019-12-25
- Fix inline_shopping_results_spec
- Update parsers
- Fix possible top stories exceptions
- Add yahoo related_searches parser
- Fix Google's top_stories Airbrake error
- Add Yahoo related_questions parser
- Add device support to Yahoo
- Add get_valid_link method to yahoo
- Escape url before parsing.

2019-12-24
- get_valid_link in all engines
- Update knowledge_graph.rb
- Update top_carousel parser
- Fix people_also_search_for carousel
- Add top_carousel parser
- Add profiles parser

2019-12-23
- Fix boston_red_sox_spec
- Fix shipping_results spec again
- Fix lionel messi stats spec
- Fix local_pack_for_specific_address_desktop_spec
- Fix tests in refiny_by spec
- Fix tests in refine_by

2019-12-20
- Fix Bing local pack
- #250: Fixed doc example.
- #250: Handled ads links.
- Fix thumbnail and source extraction in inline top stories
- Update sports_results.rb

2019-12-18
- #250: Added extensions. Fixed incorrect price extraction.

2019-12-17
- Re-create .rubocop_todo.yml

2019-12-16
- #250: extract_numbers -> extract_number.
- Move top_stories_inline to mobile only method

2019-12-13
- #250: fixed mobile reviews parsing.
- Add more classes for mobile
- Update top_stories parser
- Added guard clause.
- Added refine_by key fallback. Added test for Japanese search.

2019-12-12
- Add hotel pricing parser
- #260: Fixed mobile parsing.
- Fix top stories links
- Fix popular_times parser
- Revert "Revert "260 add refine by""

2019-12-11
- Revert "260 add refine by"
- Revert "Google kg popular_times results"

2019-12-10
- #250: Added mobile inline shopping results.
- #260: Convert refine by text to symbol.
- Update Google shopping_results

2019-12-09
- Add popular_times parser
- Update sports_results.rb
- Fix and improve Baidu answer box results
- Fix Google sports parser

2019-12-06
- Add "missing" and "must_include" annotations parser
- Rename review_summary to user_reviews
- Add critic_reviews and reviews_summary to Google's kg
- Update Google's get_valid_link method
- Show original description and saved HTML file path in RSpec output
- Add another selector for related_questions

2019-12-04
- #260: Addressed review comments.
- Update local_results.rb
- Fix parser, add more options to extract gps

2019-12-03
- Update bing local pack
- revert changes made to clone_with_new_parameters method
- Update Bing local pack parser

2019-12-02
- #260: Added 'Refine by' extraction.
- Add Bing local pack parser
- Update error message
- Update bing validator
- Update some ad results to include title

2019-11-30
- Fix #367

2019-11-29
- #101: Added inline images for google organic search.
- Revert "Revert "Bing related searches and inline ads""

2019-11-27
- Revert "Bing related searches and inline ads"
- Lint Ruby code on pre-commit hook
- Remove debug.
- Use extract_integer.
- Added total_results for bing search.
- Improve top and bottom ads results
- Improve Bing parsers
- Add support for inline ads
- Add related searches parser to Bing
- Made fixes after code review.
- Be more defensive when extracting video duration
- Add "unsupported device" error message
- Fix typo

2019-11-26
- Add support for mobile sports parser
- Parse favicon in Yahoo! ads results
- Extract thumbnail urls for organic search from js.
- Fix maps API parser
- Move Yahoo! parsers to the separate modules

2019-11-25
- Parse Yahoo! search results on different locales
- Test Yahoo! on different languages
- Small fix
- yahoo_domain for Yahoo!
- Knowledge graph fields added: - See photos - See outside
- Update and improve sports parser

2019-11-22
- Update parser
- Remove 'n' parameter from Yahoo! search API
- Yahoo! search parameters on playground
- Add documentation
- Update parser and add support for baseball

2019-11-21
- Add parser for formula and moto gp
- Add parser for tennis athletes
- Handle Yahoo! web search parameters
- Add parser for basketball and american football athletes
- Improve `get_valid_link` method

2019-11-20
- Update team sport and add individual athlete (soccer) parser
- Yahoo! pagination
- Update results_state_and_spelling_fix.rb
- Improve parser error handling
- Re-use `dig`
- Update maps validator and tests
- Yahoo! query_displayed and total_results
- Button in ads_results for Yahoo! Search API

2019-11-19
- Yahoo! ads results with sitelinks
- WIP sports_results parser
- Yahoo! ads results with list sitelinks
- Replace blank? with present?
- Re-using `dig` in maps parser
- Add `google_domain` support to Maps
- Test Yahoo! agains real queries

2019-11-18
- Parse Yahoo! organic results
- Improve Maps parser
- Make sure place and search results are not loaded at the same time

2019-11-17
- Fix for when html shows wrong type of results
- Separate results state and spelling fix to different file

2019-11-15
- Update and improve `Maps` parser
- Add validator test
- Add `position` to some place results and rename hash

2019-11-12
- Extract total_results on Baidu
- Improve and update parser
- Revert "Improve and update parser"
- Add ReCaptcha validator
- Improve and update parser
- Fix Google Maps URL constructor
- Update inline products screenshots in documentation
- Extract thumbnails in compact inline products

2019-11-11
- Extract compact inline products
- Extract regular inline products
- Move inline_products extraction to own module
- Extract extended inline_products
- Add support for countries cities and provinces
- Add support for hotel ads

2019-11-08
- Improve `Maps API` parser
- Rename state from `place` to `local`
- Update public JSON sample

2019-11-07
- Add pagination.previous and fix pagination.next on Baidu

2019-11-06
- Update `Maps API` parser
- Fix Google video results parsing on mobile devices
- Fix tests
- Clarify code comment for extract_js_image_by
- Unit test performance-optimized JSUtils.unescape
- Rename `place_results` hash to `local_results (tbm=lcl)

2019-11-05
- WIP update Google Scholar validator
- Decode HTML entities with regular expressions instead of loops
- Add support to Google Scholar API for `empty results state`

2019-11-04
- WIP Add Google Maps API parser
- Add support for `type` parameter
- Test saved HTML with different designs of Google Images results
- Extract original image URL from JS
- Remove unused code

2019-11-03
- Fix and improve places_results parser

2019-11-01
- WIP Initial files for `Google Maps API`
- Parse Google Images from both HTML and JS

2019-10-31
- Start images extraction
- Change spelling in the documentation

2019-10-30
- Add top_stories_link

2019-10-29
- Fix tests
- Move top_stories extraction to own file
- Parse top_stories defensively
- Support list format of top stories
- Support carousel format of top stories
- Fix mobile ads parsing without description

2019-10-28
- Add support for results state to all Google API endpoints
- Improve places reuslts parser

2019-10-25
- Exclude 'rich_snippet.bottom' for organic results with list format for sitelinks
- Parse list format for sitelinks
- Update bing related questions parser
- Add related questions parser
- Document Direct Answer Box snippet selector usage in organic_results
- Code improvements for places results parser

2019-10-24
- Fix parsing of hotel bookings
- Exclude incomplete answer_box duplicates in organic_results
- Use new gps_coordinates method

2019-10-23
- Improve reliability of geting a place type
- Enable `organic_results_state` for `tbm` results
- Add places results parser
- Rename file to `places_results` and move under proper conditional
- Use both of new and old answer box CSS selectors

2019-10-22
- Test Bing pagination with high number of results
- Handle pagination without 'other_pages'
- Fix GPS coordinates extraction in local results
- Fix 'knowledge_graph.people_also_search_for' parsing
- Update selectors to parse dictionary answer box of Google results
- WIP tbm local results parser

2019-10-21
- Move pagination-specific methods to correct modules

2019-10-19
- Fix for product results
- Fix for local pack results

2019-10-18
- Add support for editorial_reviews to knowledge_graph and update description
- Test pagination in Google Scholar results
- Small fix

2019-10-17
- Baidu integration test
- Update `local_pack` parser
- Bing pagination
- Edit `get_valid_link` method to support `data:image` links
- Update product_results parser

2019-10-16
- Add serpapi_pagination to Baidu
- Unify pagination in Google Scholar
- Unify pagination and serpapi_pagination in google's organic results
- Improve gps and valid link methods

2019-10-14
- WIP local_pack parser
- Update `get_valid_link` method
- Extract pagination to separate method
- Don't set organic_results_state when no results are found
- Rework `extract_gps_coordinates` to enable `js` extraction
- Additional case for 'Empty showing fixed spelling results' state
- Code review fixes
- Add the `get_valid_link` method
- Be sure title is returned for `product_result`

2019-10-11
- Remove TODO
- Remove method for setting organic results state
- Extract spelling fix type
- Guard agains nonexisting nodes in inline videos

2019-10-10
- Fix Airbrake errors for `product_result`
- Fix bed links and comment out `lsig` parameter
- Fix NoMethodError for inline video title parsing

2019-10-09
- Move set_organic_results_state to parser
- Test empty results in Arabic and German locale
- Fix for when labels have space in them
- Fix syntax issue
- WIP - local map for specific address
- Add support for the new `lsig` parameter
- Fix some syntax issue
- Handle inline videos without 'channel' in HTML
- Tests for quotes spell fix

2019-10-08
- Improve method arguments naming
- Empty search result states implementation
- WIP - Add support for `Mobile v2` results
- Rename to `product_result` and other small improvements

2019-10-07
- Google product results parser

2019-10-04
- Test inline videos in three different languages
- Support results_for_exact_spelling and fully_empty states

2019-10-03
- Describe challenges of parsing :video_thumbnail

2019-10-02
- Document inline videos in organic results
- Base64-encode thumbnails
- Convert indentation to spaces
- Fix Google and Google Scholar parser

2019-10-01
- Improve organic results parser
- Support inline video results

2019-09-30
- Remove Google Search specific validators from Google Scholar
- Improve organic results
- Fix syntax
- Show error if Google hasn't returned any results

2019-09-25
- Remove parameters that don't work with Google Scholar
- Add support for Google Scholar `cites` and `cluster` parameters

2019-09-24
- WIP Google Scholar organic results
- WIP Google Scholar parser

2019-09-21
- Add support for stats to Google Scholar results
- Reduce some whitespace
- Add support for related searches to Google Scholar results

2019-09-20
- Add support for Google Scholar specific parameters

2019-09-17
- WIP Add basic skeleton to support Google Scholar API

2019-09-10
- WIP Support Bing organic search in mobile

2019-09-04
- Fix #234
- Fix #228

2019-09-03
- Fix issue with new Google design

2019-08-26
- Fix Baidu issues

2019-08-22
- Fix issue with rich snippet questions not being supported and messing with direct answer

2019-08-20
- Fix issue with related searches including link with blank hrefs
- Fix data extraction for Google Shopping API
- Fix product headers extraction
- Fix product reviews, ratings, and number of reviews extraction
- Don't transform a Float product rating into an Integer
- Fix extensions not being correctly extracted
- Snakecase keys to avoid JSON parsing issue for Google Product API
- Fix issue with extracting related searches on mobile
- Fix issue on mobile with organic results and ad results getting mixed up
- Extract more rich data form organic results extensions
- Stop removing direct answer from document

2019-08-19
- Add Google News API support for people also search for blocks
- Fix issue with knowledge graph when key already exists
- Add `raw_hours` key for people relying on a string instead of an object
- Fix rating and review extraction for knowledge graph API
- Fix position issue with Google Images Results API
- Add support for extra layout variation

2019-08-15
- Fix Bing search ads
- Fix issues with stopping recreating hash dynamically

2019-08-13
- Fix Google Images pagination end infinite loop

2019-08-11
- Fix issue with knowledge graph
- Fix issues with direct answer box and regular hash
- Use regular hash instead of recursive created hash

2019-08-09
- Fix price, extension, and snippet for shopping results #210
- Actually avoid parsing parent knowledge graph section
- Ensure `reviews_from_the_web` has results
- Don't parse results from parent knowledge graph section for `see_more_about1`

2019-08-08
- Only create hash for `see_more_about` if it exist
- Avoid setting empty reviews
- Add support for `See more about` section

2019-08-07
- Fix issue with valid searches about Captcha being blocked

2019-08-06
- Don't include missing hours in local results #93

2019-08-05
- Update rating to float
- Extract number from `review_count`
- Add support for businesses' weekly schedule and reviews from the web

2019-08-02
- Add support for custom location by latitude and longitude input
- Fix naming of hours, address and description in local pack results
- Fix missing ratings in local results box #93
- Fix links and title links for Binb organic results
- WIP Rewrite final_hash and spec logic

2019-07-30
- Display JSON and HTML endpoints for Baidu search
- Allow guest to reach to their JSON and HTML endpoints via tokens

2019-07-26
- Add Google Spell Check API
- Fix commits and PR to streamline tokens

2019-07-25
- Fix typo
- Avoid parsing answer divs when they are not there
- WIP Use shorter tokens that look more user friendly

2019-07-23
- Add `location_used` to search json when using Bing #183
- Fix thumbnail and images

2019-07-19
- Fix missing parameters error with validators
- Add support for setting of precise location #181
- Early preparation, add cookie setting

2019-07-18
- Fix mobile title for desktop news
- Fix mobile search

2019-07-16
- Fix Google News API

2019-07-14
- Fix extraction of ratings from reviews

2019-07-09
- Remove conditional pagination (trouble on mobile)

2019-07-05
- Add product id to shopping results

2019-07-02
- Update knowledge graph to support parsing of business' operational status #116

2019-06-27
- Fix issue when knowledge graph answer type is missing
- Remove raw_submodules from final hash
- chore(app/*) whitespace changes and newlines at EOF

2019-06-23
- merge organic_results and video_results for desktop fix video_results parser for mobile

2019-06-21
- Fix temporary live total results empty error
- Handle edge cases of Google pagination

2019-06-19
- Fix Base64 Errors

2019-06-17
- update pagination parameters with Search#clone_with_new_parameters
- remove `then`'s from Bing knowledge graph results

2019-06-16
- convert bing knowledge graph into a hash vs. list of hashes
- add `formatted` field to parseable bing submodules
- Resolve conflicts
- Actually fix bugs

2019-06-15
- Local map bug fixes

2019-06-13
- remove unique top-level type parsers & fallback to generics
- remove unused include for bing ads results

2019-06-11
- Beta operational Baidu parsing

2019-06-10
- Fix emtpy hash being evuated as true issue #167
- Add new serpapi search links for `place_id` #167
- Rename location_id to place_id to match Google own documentation #167
- Use `api_key` instead of `user_id` to encrypt in order to allow re-generation
- Use only "Success" status instead of both "Cached" and "Success" for successful searches
- Use tokens to authentifcate users tied to one specific request
- stub out unimplemented Bing KG submodule type
- filter out JavaScript links in bing KG link parser
- add bing 'people_also_search' submodule formatter
- add written_works, timeline, and interesting_stories bing formatters
- add generic bing header submodule format to food & famous people
- use Ruby's splat operator to get head & tail of Bing submodules
- add generic format logic for Bing's knowledge graph header submodule
- add 'Organization SocialActive' to Bing Organization types

2019-06-07
- Extract `ludocid` (location_id) from local pack

2019-06-06
- Add local map support and GPS coordinates for knowledge graph results
- Support for dictionary answer box
- Add parsing support for "Related news" and "rich snippet" results
- Update parser with dynamic module loading based on required element
- [squash] start on organization-specific Bing KG fields
- add Bing 'flatmodule' knowledge graph parsing
- add Bing 'slide' submodule parser

2019-06-05
- parse multiple Bing knowledge graph lists when they exist
- replace Bing 'answers' with 'knowledge_graph'

2019-06-03
- add framework that captures typical Bing answer types

2019-06-02
- refactor bing ads pull helper fns from organic results module
- move bing organic results into their own parser module
- move bing ads results into their own parser module

2019-05-31
- Fail-safes for missing elements in local pack mobile
- Add pagination, source icon for Baidu

2019-05-29
- Add parsing of address on specific style for local pack mobile
- Extract phone number from local pack mobile
- Add support for local packs with mobile device #109
- Initial commit

2019-05-26
- Fix media parsing in `product_results`

2019-05-25
- Add fail safes for when elements don't exist or output is nil

2019-05-24
- add title field to Bing ad results
- fixup tracking_link (vs. link) field names for Bing ad sitelinks

2019-05-23
- Ensure values aren't null, actually add parsing of related_search_boxes
- Remove `log` in validator
- add basic serpapi_pagination logic
- Modify validation of successul request in product page
- Finalize renaming `product` paramter to `product_id`
- Strip quotes at start and end of review summary
- Avoid parsing unrelated sections for `related_search_boxes`

2019-05-22
- Rename`related_search_box` to `related_search_boxes`
- Add documentation page, navbar menu item, and icon for related searches
- Routine cleanup

2019-05-21
- Update `specs_results` for alternate product page
- Update `related_products_results` for alternate product page
- Fix removal of text for expand description

2019-05-20
- Update `sellers_results` to work alongside alternate product page
- Update `reviews_results` to work alongside alternate product page
- Update `product_results` to work alongside alternate product page
- Use parameters `specs` and `reviews` as booleans
- Refactor get_related_search_box

2019-05-19
- Add additional selectors for parsing results on alternate product page #105
- Check if `title` tag contains `Product not found` text
- Checking if the url contains `/shopping/product` isn't really necessary
- Use `if` `end` block if two or more conditions are used

2019-05-18
- Initial write-up for related search box #129
- Bug fixes and adjustments

2019-05-17
- add Bing-specific query parameters

2019-05-15
- Support knowledge graph type extraction for business listing

2019-05-14
- replace the link with a tracking_link url
- Use a more precise selector for knowledge graph type extractor
- Fix knowledge graph type support
- Changes per code review

2019-05-13
- add bing ad parsing logic

2019-05-12
- add in `form=QBRE` to trigger Bing ad results
- move bing organic parsing into it's own fn
- only return valid Bing organic results

2019-05-10
- Support float and integer number extraction for all locales
- Massive overhaul
- add basic Bing organic result parsing logic

2019-05-03
- Rename store/seller to name in sellers results, more adjustments to rating/reviews parsing in reviews_results

2019-05-02
- Fix stars and total reviews regex for international languages (hopefully), begin documentation for subparts of product API

2019-05-01
- Fix google_domain setting for engines "google" and "google_product", pagination fix for sellers and revies

2019-04-30
- Fix parsing of Google calculator direct answer
- Add local stores parsing, rating and reviews to integer, fix param name on Google Prod API documentation,

2019-04-28
- Add some documentation for global api, reviews results. Adjust sellers results for online and local, refine filtering/sorting inline with documentation,
- Fix reviews, adjust price extraction for multilingual
- Add filtering, sorting, pagination, full results for reviews and specs/details

2019-04-26
- Use `unless` in replace of `if` blocks
- Add feature to iterate through shopping results and fetch product page results (Not completed)
- Restructuring
- Naming, ensure elements exist before creating hash and iteration
- There are ads but needs seperate parsing
- Initial Google Product parsing integration

2019-04-24
- Support ludocid parameter (Google My Business Listing CID)
- Structuring adjustments
- Migrating to respective branch for development

2019-04-23
- Add parsing for specification details section
- Add parsing for additional review data
- Fix skeleton for other engine support
- Initial product page parsing

2019-04-18
- Fix conflicting html attribute
- Seperate search domain and search engine parameters

2019-04-05
- Support basic Bing search WIP