Home About Me

A Three-Step Stock Quote Scraper with Requests and re

Once you're comfortable with re and Requests, you can start collecting data for things you actually care about—stock prices, news, images, or whatever else interests you.

For this example, imagine a busy part-time stock trader who wants to keep an eye on the market during work but can't stare at a trading app all day. A small stock quote collector is a practical way to solve that.

Step 1: Load the data source

import re, requests # Load the required modules
r = requests.get('http://hq.sinajs.cn/list=s_sh000001') # Stock price of an internet-listed company
r.encoding = 'gbk'# Set the encoding to avoid garbled text
print(r.text) # Return HTML code (data)

After running it, you get a line like this:

var hq_str_s_sh000001="上证指数,3367.1194,6.3091,0.19,2824975,31372209";

The fields are the index name, current points, current price, change rate, volume (lots), and turnover (10k yuan). Once the 上证指数,... portion is isolated, it can be turned into the object or structure you need, then printed or processed.

Step 2: Parse the response and extract the part you need

s = r.text # Get the data
p = r'"(.+)"' # Set the regular expression
r = re.search(p, s).group(1) # Match and retrieve the result string
print(r)
ls = r.split(',')
print(ls)

Step 3: Format the final printout

print('''{}:{}
当前价格:{}
涨跌率:{}
成交量(手):{}
成交额(万元):{}'''.format(ls[0],ls[1],ls[2],ls[3],ls[4],ls[5])
)

# Final output:
# Shanghai Composite Index:3367.1194
# Current price:6.3091
# Change rate:0.19
# Volume (lots):2824975
# Turnover (10k yuan):31372209

With that result in hand, you can notify yourself about the index in many ways—by email, through a simple web page, by SMS, or another channel—and keep up with the market. The delivery mechanism is beyond the scope of this walkthrough.