Google quality raters are people employed by Google to conduct manual searches and rate the results that are ranking for their queries.

The data that raters feed back to Google is used to improve algorithms to increase the relevance of the results.

To inform their decisions, quality raters use a public document called the Search Quality Evaluator Guidelines, which contains 168 pages of information. It was last updated on 5th December 2019.

The guidelines give specific factors that the raters should consider when rating the quality of search results.

What is included in the Quality Raters Guidelines?

The document is a rich source of information about what Google considers to be high-quality content, and it is essential reading for both webmasters and digital marketers.

The guidelines are split into three distinct sections:

  • Part 1: Page Quality Rating Guideline
  • Part 2: Understanding Mobile User Needs
  • Part 3: Needs Met Rating Guideline

Part 1 provides a wealth of information on identifying the purpose of a webpage, understanding the main and supplementary content, reviewing the reputation of the website creator, and how to identify high and low-quality pages.

This is relevant to digital marketers as it offers insights into what Google considers high-quality content. It also provides specific examples. In general, Part 1 is the section that gains the most attention within the SEO community, particularly in recent years, with increased attention focused on the concept of E-A-T.

However, in this article, I’m going to cover the main points of Part 2, a section that is generally talked about a lot less, but still provides valuable information about how Google tries to meet the needs of mobile users.

Part 2 asks the quality raters to consider the needs of users on their smartphones and assess the search results they are given for mobile queries.

Understanding mobile user needs

Optimising websites for mobile is crucial for success in organic search, not just due to the roll out of mobile-first indexing, but because mobile is where your customers are. The rise of mobile usage at home, in-store, and on the go, is the biggest shift in internet history and Google has confirmed that more searches take place on mobile than on desktop in many countries.

Our smartphones are our sat navs, our calendars, our newspapers, books, alarm clocks, TVs, calculators, notepads, dictionaries, address books, and our entire music collections…. The list is endless.

This creates a unique challenge for search engines, as the mobile searcher isn’t sat at their desktop simply looking for information; their intent can be a whole host of actions that don’t necessarily apply to desktop searches.

The guidelines describe queries having one or more intents, which are defined as:

  • Know query – some of which are Know Simple queries.
  • Do query – some of which are Device Action queries.
  • Website query.
  • Visit-in-person query.

Know and Know Simple Queries

Know queries are broken down into two variants; Know and Know Simple.

Know Simple queries intend to find a specific piece of information, where there is a factual answer that is described in a short paragraph or list.

These queries are generally answered in the SERP by the knowledge graph and would be more difficult to optimise content for, such as ‘when was World War 2?’ or ‘what is the capital of India?’

Simple Question and Answer example.
An example Know Simple question.

These Know Simple queries often include question words such as When, Who or What, but they don’t have to contain question words where the question words are implied by the query.

For example, ‘capital of India’ implies the question ‘what is the capital of India?’ without including the full question.

A know query example using New Delhi as the example.
An example of a know query, using the statement “capital of India” with a return answer of “New Delhi”.

Know queries are slightly more complex as they may not have a short answer. They may require a more in-depth or considered response, or they may be covering a controversial topic.

If the informational query doesn’t necessarily have a ‘correct answer’, or if the answer is likely to be an opinion that not everyone would agree on, it is classed as a Know query.

The query may also be ambiguous. An example given by the guidelines is the difference between ‘who is Graves disease named after?’ (Know Simple) and ‘Graves disease’ (Know).

It’s not clear from the query ‘Graves disease’ what the searcher is looking for — whether that be medical or historical information.

Google will answer both Know Simple and Know queries with factual answers supported by scientific consensus, even if there are people who disagree with this, such as ‘is the earth flat’?

A search query asking whether the Earth is flat.
A search query asking whether the Earth is flat.

Do queries and Device Action queries

The raters guidelines define action queries as ‘Do queries’, some of which are ‘Device Action’ queries. The intent behind a Do query is ‘to accomplish a goal or engage in an activity on a phone’.

If you have an eCommerce site, for example, you should be looking to optimise your product pages for specific Do queries related to conversion, such as ‘buy gym bag’, ‘subscribe to magazine’, or ‘order flowers online’. It may be useful to highlight Do queries as part of your keyword research to highlight this lower funnel intent and prioritise these when optimising page content.

The quality raters are asked to assess the results provided for these Do queries and rate how well the needs of the mobile user are met. As SEOs, we can optimise for Do queries by providing Google with exactly what it needs to help the user complete the action.

Some of these Do queries will specifically require the device to do the query for you, which is where voice commands come in. These device action queries generally contain clear ‘action words’ such as call (“call my mum”), navigate, get or open (“open the Facebook app”).

Website queries

Website queries have the intent of finding a specific website, whether the user types in the full website URL, a partly constructed URL, or a full URL that no longer exists. Google attempts to provide the website that the user wants, and location may be a factor in these queries.

Depending on the “locale” (the language and the geographical location), Google will select the most appropriate site for the website query.

If you search for a website in Spanish, Google will attempt to provide the Spanish version of said website in the search results. Ensuring your site is set up correctly for international targeting — both technically and with appropriate language-specific content — will help Google send users to the right part of your site.

Website queries are essentially your audience looking for your site, so it makes sense to ensure that you’ve optimised for these queries across locations.

Visit-in-Person queries

These queries apply when mobile users are using their smartphones to locate specific places, find out opening times, or discover a type of location nearby, such as ‘Indian takeaway near me’.

As with Know queries, Visit-in-Person queries may explicitly include the relevant question words within the query (Where is, Get Directions to) or they may not, and the intent is implied.

Google gives specific examples within the guidelines of queries that have both visit-in-person and non-visit-in-person intent. These are:

  • [hotels]
  • [post office]
  • [apple store]
  • [citibank]
  • [best buy]
  • [office depot]
  • [target]
  • [library]
  • [bank of america]
  • [the gap]

For these searches, a user may want to visit a physical location, they may just want to visit the website, or they may be searching for information around these topics.

If you have an eCommerce site that also has a physical location, it’s important to set up Google My Business and fill out as much relevant information about your business as you can, including Opening Hours and Address.

If you have multiple stories or locations, keeping track of all your business profiles in a centralised location is a good way of managing this process. Make it as easy as possible for Google to serve visit-in-person queries with the right information about your business.

In summary

The Raters Guidelines are a valuable source of information about what Google considers to be high-quality content, and an excellent starting point for understanding how Google categorises user-intent.

An SEO strategy should be specific to each individual business, but the theory that is included in the guidelines is essential reading for all webmasters and SEOs, irrelevant of niche or business type.