How You Can Get Google To Crawl Your Website Instantly

In this post, we’re going to take a look at how you can use Google’s new indexing API to get your website’s pages and content crawled immediately rather than having to wait! 🖖

Note: The process has slightly changed since the above video was recorded – so, feel free to refer to the up-to-date written tutorial below in the meantime.

Here’s why this really is worth the extra effort. 

1. Rank faster (especially important for time-sensitive content)
2. Increase your chances of ranking above your competition
3. Keep search results up-to-date for higher quality user traffic

Thanks to Google’s Indexing API, Rank Math SEO for WordPress and this guide we’ve put together for you, it’s now possible for you to get your website to rank faster.

Using this method, you can stay one step ahead of your competition by making sure that your site is crawled and has the opportunity to rank before their site does which is extremely important when it comes to time-sensitive, competitive content.

The process of setting this up isn’t typically very easy, but if you follow this step-by-step guide, you’ll be up and running (or, I guess you could say up and ranking) in no time! So, without further ado – let’s jump right in and take a look at how to set this up.

Note: Google recommends that you use the Indexing API ONLY for Job Posting and Live Streaming websites. During our tests, we found that it works on any type of website with great results and created this plugin for testing.

If you are not sure whether you want to be using this or not, please stop reading and don’t proceed further. As Google recommends, you should not use this indexing API on money websites. Please proceed at your own risk.

We understand this is not for everyone 🙂

PROOF: Here’s How Google Responded to Our Request

In the 1st hour after publishing this article:

After a couple of hours, Google gave us a featured snippet for the search term

Get Your Website Crawled Immediately by Google Using Their NEW Indexing API

1. Download Plugin Files
2. Steps to Create an API Project
   • 2.1 Go to Google API Console
   • 2.2 Create a Service Account
   • 2.3 Add the Service Account as an owner
3. Configure the Plugin
   • 3.1 Add API Key
   • 3.2 Send first indexing request to Google
4. Most Common Errors & Their Fixes

1 Download Google Indexing API Plugin

CLICK HERE

2 Steps to Create an Indexing API Project

2.1 Go to the Google API Console and create a new project

Ensure that you’re creating a new Indexing API project which you can do automatically by clicking here. And then click continue.

If subsequent to clicking Continue, you see the following screen, then you’ve successfully created the project:

Please note: There is no need to click ‘Go to Credentials’ button. You can close this tab.

2.2 Now create a Service Account

Once you’ve created your project, create a service account by opening the service accounts page. You will first be prompted to select the API project you wish to create this service account in (the one created in the previous step).

After selecting the project you wish to create a service account for, you’ll be taken to the following page, where you simply need to click the Create Service Account button highlighted below:

On the Create service account screen, enter a name and description for the newly created service account.

Select and copy the whole Service Account ID (the one that looks like an email address) because you will need it later. Then, click on the Create button at the bottom:

Click Create and Continue to proceed to the next step, where you need to change the role to Owner and, as you might’ve guessed, click continue once again…

Once you’ve set the role to Owner as shown above, simply click continue to save that change and then click done.

You will then be able to download the file that contains your API key. To do so, simply click the three vertical dots in the Actions column, and then select Manage keys as shown below:

You will then be taken to the following page when you can click Add Key and then select the Create new key option, as shown below:

Choose the default JSON format when prompted in the overlay, and click Create:

Upon clicking Create the .json file will be automatically downloaded in your browser meaning you’ve successfully created the API key & can proceed to the next step…

2.3 Add the Service Account as an owner of your Google Search Console Property

To do this, you’ll need to register and verify your website with the Google Search Console (if you haven’t done so already) which is super easy: just follow the recommended steps to verify ownership of your property.

After verifying your property, open the Google Search Console, select your property on the left (if prompted), and then click on Settings near the bottom:

Click on Users and Permissions:

Click on the three dots next to your account, and then click on Manage property owners:

A page will open in a new tab. Scroll to the bottom and click on the Add an owner button in the Verified owners section:

Enter the Service account ID (the one you copied out earlier) in the new owner’s email address field and click Continue:

Switch back to the previous tab and refresh it. You should now see the Service account listed as a new Owner.

You can use a single Project, Service Account, and JSON API Key across multiple sites, just make sure that the Service Account is added as Owner for all the sites in the Search Console.

3 Configure the Plugin

3.1 Insert your API Key in the plugin settings

After installing the plugin, just navigate to Rank Math > Instant Indexing Settings in your WordPress admin area, and under the Google API Settings tab, paste in the contents of your JSON file, or just upload the file.

Select the Post Types where you want the Indexing API to ping Google to crawl your website as soon as a post is published, updated, or deleted and click on Save Changes.

3.2 Now it’s time to use Google’s indexing API to get any post or page on your site indexed almost instantly.

Head over to the Console tab from the top of the page. The screen should appear similar to the one below.

Enter a URL (or more) in the Instant Indexing section, and choose the Action as Google: Publish/update URL. Then, click the Send to API button.

In just a second, you should see a response from the API, similar to the one below.

You can use the Remove URL or Get URL Status actions, to remove the URL from the index or get the current status of the URL.

Instead of entering the URLs here and then indexing them, you can also do so from the WordPress Posts section. There, you should see an option to update a post, or check its status.

You can also use the bulk actions options to get multiple posts indexed or updated quickly.

And that’s it! We hope that you were able to make great use of this guide to help you get up and running with Google’s Indexing API.

So, getting Google to index your site instantly wasn’t all that complicated after all, was it? If you have any questions, comments, or suggestions – feel free to Tweet @rankmathseo. We can’t wait to hear from you. 😅Please Note: It will take a few minutes for your website to start crawling after successful submission. Learn more about the entire process here.

PS: It would be remiss not to mention that we got the idea for this post after reading this post that David Sottimano wrote about Google’s Indexing API. Another article that’s worth checking out is Tobias Willmann’s in which he thoroughly tested the indexing API with normal URLs that do not have a job posting or Livestream structured data.

PS 2: Officially, the API is for pages containing Job Posting and Livestream Structured data, however, from our internal tests and our public beta it seems that Google will crawl and index any page type regardless of the Structured data. It is an efficient way to get the page(s) crawled quickly, rather than requesting it from inside Google Search Console.

Also, Google plans to use the Indexing API instead of Sitemaps down the line.