Effective ways to get your site indexed by Google

What is crawling and indexing?

Google finds new pages using a spider that crawls the World Wide Web ("crawling" literally means "crawling") and adds pages to its database, the index. This spider is called a search robot, and Google's robot has its own name - Googlebot.

Confused? Let's define the basic terms.

Crawling - the process of following hyperlinks on the Internet to find new content.

Indexing - The process of storing each web page in an extensive database.

Search robot - A programme that performs crawling.

Googlebot - Google's search robot.

When you search for something on Google, it brings up all relevant pages from its database - the index. Since there are often millions of pages that can match a query, Google's ranking algorithm does its best to sort them so that the pages that best match the query come first.

This is where it's important to realise that indexing and ranking are completely different things.

Indexing is about registering for the race, while ranking is about distributing the prizes.

You can't win the race without registering first.

How to find out if your site is in Google's index

Open the Google site and type site:your_site.com into the search box

This number roughly shows how many pages your site has in Google's index.

If you want to know the indexing status of a specific URL, use the same operator site:your_site.com/address_pages.

If there are no results, it means the page is not in the index.

It's worth noting that if you're using Google Search Console, you can use the "Coverage" report to get more accurate information about your site's indexing status. Simply go to:

Google Search Console > Indexing > Coverage

Look at the number of valid pages (with and without alerts).

If these numbers add up to be different from zero, it means that at least some of your site's pages are contained in Google's index. If both numbers are zero, then you have a serious problem because no pages on your site are indexed.

In Search Console, you can also check if a particular page is indexed. To do this, paste its URL into the URL Checker Tool. If the page is indexed, the service will display the message "URL is in Google's index". If not, it will say "URL is not in Google's index".

How to make Google index your site

Found that Google is not indexing your website or page? Try the following:

Open Google Search Console.

Go to the URL Checker Tool.

Paste the URL you want to see in Google's index into the search bar.

Wait for Google to verify the address.

Click the "Request Indexing" button.

This is useful to do when you publish a new article or page. This is how you directly tell Google that you've added something new to your site and that the search engine should take a look at it.

However, a direct request won't help solve the underlying issues that are preventing older pages from being indexed. If this is your case, go through the list below to diagnose and solve the problem.

Remove blocking terms from your robots.txt file

Remove stray noindex tags

Add the page address to the sitemap file

Remove stray canonical tags

Check if the page is orphaned

Fix internal nofollow links

Add "strong" internal links

Make sure the page is unique and has value

Remove low-quality pages (to optimise the "crowding budget")

Get quality backlinks

Indexing and ranking are different things

Having your website in Google's index doesn't mean it will get ranked and bring traffic. These are different things.

Being indexed only means that Google knows your site exists. It doesn't mean that your site will show up at the top of the issue for relevant queries.

That's where SEO comes in - the art of optimising web pages to appear at the top for specific queries.

In general terms, search engine optimisation (SEO) includes:

Identifying the topics your customers are searching for.

Creating content around those topics.

Optimising the created pages for targeted keyword phrases.

Building backlinks.

Keeping content up-to-date at all times.

There are only two possible reasons why Google is not indexing your website or web page:

Technical issues that prevent the search engine robot from doing so.

Content quality - Google considers your site or page to be of poor quality and useless to its users.

It is quite possible that both of these problems are occurring. But in my experience, technical problems are much more common. Technical problems can also be the cause of automatically creating indexable low-quality content (e.g. faceted navigation problems). This is not a good thing.
But going through the above list nine times out of ten should solve indexing problems.