Web scraping stock prices in Python

Ever wondered how to keep track of stock prices without constantly refreshing web pages? With Python, we can easily scrape stock price data and have all the information at our fingertips, all in real-time. Let’s dive into how you can do this with just a few lines of code!

Scrape a Company’s Stock Data

If you need to monitor a company's stock, you can use this API: Google Finance API. It scrapes data from the Google Finance page.

Video Tutorial
If you prefer to watch a video, here it is

Step 1: Tools

We'll use the official Python library by SerpApi: serpapi-python .
That's the only tool that we need!

As a side note: You can use this library to scrape search results from other search engines, not just Google Finance.

Usually, you'll write your DIY solution using something like BeautifulSoup, Selenium, Scrapy, Requests, etc., to scrape Google Finance results. You can relax now since we perform all these heavy tasks for you. So, you don't need to worry about all the problems you might've encountered while implementing your web scraping solution.

Step 2: Setup and preparation

• Sign up for free at SerpApi. You can get 100 free searches per month.
• Get your SerpApi Api Key from your dashboard.
• Create a new .env file, and assign a new env variable with value from API_KEY above.
  SERPAPI_KEY=$YOUR_SERPAPI_KEY_HERE
• Install python-dotenv to read the .env file with
  pip install python-dotenv
• Install SerpApi's Python library with
  pip install serpapi
• Create new main.py file for the main program.

Your folder structure will look like this:

|_ .env
|_ main.py

Step 3: Write the code for scraping Google Finance results

Here is the code example for searching Tesla stock today:

import os
import serpapi

from dotenv import load_dotenv
load_dotenv()
api_key = os.getenv('SERPAPI_KEY')

client = serpapi.Client(api_key=api_key)
result = client.search(
	engine="google_finance",
	q="TSLA:NASDAQ",
	hl="en",
)

print(result)

Try to run this program with python main.py or python3 main.py from your terminal.

Let's compare the result from Google Finance page with the API response:

As you can see, we've successfully gotten the data programmatically.

Q parameter
We need the company's ticker name as value for the q parameter. You can find this by searching on Google. Feel free to change the value of the q parameter a ticker's code of a company.

Filter by time
You can filter the results by time using the window parameter. Here are the options:
1D - 1 Day(default)
5D - 5 Days
1M - 1 Month
6M - 6 Months
YTD - Year to Date
1Y - 1 Year
5Y - 5 Years
MAX - Maximum

How to get Ticker name programmatically
If you need to get the ticker name programmatically, you can use our Google Search API. The query will be "the company name" + " ticker name". Here is an example:

company_name = "Tesla"
result = client.search(
	engine="google",
	q=company_name + " ticker name",
	hl="en",
)

The ticker name will be available on the response under knowledge_graph > type.

Warning: There is no guarantee that it will work correctly for all the companies.

Interested in a no-code solution? Learn how to scrape Google finance reports using Google Sheets:

Scrape Market Trends

SerpApi also provides the API for current market trends from this page: https://www.google.com/finance/markets/indexes?hl=en. We can use Google Finance Markets API for this.

Code sample
We can use the same setup as before with a few tweaks to some values:

result = client.search(
    engine="google_finance_markets",
    trend="indexes",
    hl="en"
)

• Update engine to google_finance_markets
• add trend parameter

Here is the result:

Which represents this page:

Trend parameter
We can change the ranking factor using trend parameter. This parameter is used to retrieve different market trends. Available options:
- indexes - Market indexes
- most-active - Most active
- gainers - Gainers
- losers - Losers
- climate-leaders - Climate leaders
- cryptocurrencies - Crypto
- currencies - Currencies

I hope this is helpful! I hope it can be useful for your finance app project!