Being able to quickly check the search results of a query can be done several ways.

If you’re in a spot, however, and your Wifi/hotspot connection is very poor and you need to quickly check what ranks on page one for a specific query, being able to check ad hoc queries via Python can be a useful trick to know.

Before you start, you will need to open terminal and install BeautifulSoup:

pip3 install urllib3 requests beautifulsoup4

The Python Code

You need to save this as a UTF-8 file in your destination folder with a .py extension.

We recommend copy and pasting the code into something like SimpleText, Sublime or Wrangler.

import sys
import urllib
import requests
from bs4 import BeautifulSoup

USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/601.3.9 (KHTML, like Gecko) Version/9.0.2 Safari/601.3.9"

USAGE = """
Usage: {0} [terms...]
Example:
    {0} football
    {0} "billy bob thornton" "celine dione" "salt agency seo"
""".format(sys.argv[0])

def get_google_url(term):
  return "https://www.google.co.uk/search?{0}".format(urllib.parse.urlencode({"q": term}))

def get_rank(term):
  print("Search Term: {}".format(term))
  r = requests.get(get_google_url(term), headers = {
    'User-Agent': USER_AGENT,
  })


  soup = BeautifulSoup(r.text, features="html.parser")
  links = soup.find(id="search").find_all("a")

  pos = 1
  print("\tPosition\tTitle\t\tURL")
  for link in links:
    title = link.find('h3')
    if title:
      print("\t{}\t\t{}\t{}{}".format(pos, "{0}...".format(title.text[:29]) if len(title.text) > 32 else title.text, '\t' * (0 if len(title.text) > 32 else 2), link.get('href')))
      pos+=1

if __name__ == '__main__':
  if len(sys.argv) > 1:
    for term in sys.argv[1:]:
      get_rank(term.strip())
  else:
    print(USAGE)

How to run the Python SERP checker

To start with you need beautifulsoup4, if you’re on a Mac you can open terminal and run the below command:

pip3 install urllib3 requests beautifulsoup4

And then to open the Python SERP checker you need to navigate via terminal to the location of the above code file. If you’ve saved this on your desktop, you need type:

cd desktop

Followed by:

python3 [whatever-you-called-your-file].py

And then to run the checker, you need to run the command:

python3 [whatever-you-called-your-file].py [SEARCH QUERY]

Single word queries are fine, but anything with spaces needs to be wrapped in “quotation” marks, otherwise the script will treat each word as a separate query.

Hit enter, and you will get your results within seconds:

Things to consider

There are a couple of things to consider when doing quick SERP lookups from your terminal using Python:

  • Excessive use will trigger your IP to get reCaptcha’d for all devices on the network, so make use of a VPN or proxy.
  • The script only returns the classic blue link results, as in the example above it only returned seven results — featured snippets and other SERP features and special content result blocks aren’t pulled through.