Which pages Google supports
Google officially supports the Indexing API only for pages with JobPosting structured data, or BroadcastEvent embedded in a VideoObject (livestreams). Requests for other pages are accepted and count towards your quota, but Google may not act on them. For other pages, keep your XML sitemaps up to date.
What you need
- A Medium, Large, Agency or Enterprise plan. The free trial and the Small plan do not include the Indexing API.
- A Google Cloud project with the Web Search Indexing API enabled.
- A service account in that project, with a JSON key.
- The service account added as an Owner of the site's Search Console property. A Full or Restricted user is not enough.
The Indexing API uses its own service account, not the Google account you connect for Search Console data.
Setting it up
- In the Google Cloud console, choose or create a project and enable the
Web Search Indexing APIunderAPIs & Services→Library. - Go to
IAM & Admin→Service accountsand create a service account. It needs no roles. - Open the service account, choose
Keys→Add key→Create new key→JSON. Your browser downloads a.jsonfile. - In Search Console, open the property of the site. Go to
Settings→Users and permissions, open the menu next to your own account, chooseManage property ownersand add the service account's e-mail address (theclient_emailin the key, ending in.iam.gserviceaccount.com) as an owner. You must be a verified owner to do this. - In Search Sentinel, open the project and go to
Settings→Indexing API. Paste the JSON key or clickChoose JSON file, then clickVerify and save key. - Click
Test connection. Search Sentinel looks up one of the project's URLs at Google to check that the service account is accepted as an owner of the site.
One key for several projects
When you save a key, choose This project only or All my projects without their own key. The second option makes it your account default. A project's own key always wins. In a project shared with your team, the project's own key is used, or else the account default of the project's owner.
How the key is stored
- Google checks the key before Search Sentinel saves it: Search Sentinel signs a token request with it and only stores it if Google returns an access token.
- The key is encrypted (AES-GCM) and stored server-side. It is never shown again; the app only shows the service account's e-mail address and the Google Cloud project.
Remove keydeletes it from Search Sentinel. Delete the key in Google Cloud as well if you no longer need it.
Requesting indexing
On the Indexing API tab, choose the notification type and the URLs:
- URL_UPDATED: the page is new or its content changed. Google may crawl it again soon.
- URL_DELETED: the page was removed. Use it only for pages that return 404 or 410.
Select URLs and click Submit selected, or click Submit all for every URL in the project. Up to 200 URLs are sent per submission.
On a URL's page, Request indexing next to Recheck Now sends URL_UPDATED for that URL. Below the button you see when indexing was last requested and whether Google accepted it. Google status asks Google when it last received a notification for the URL.
The button only appears when your plan includes the Indexing API and the project has a key.
Quota
Google allows 200 publish requests per day per Google Cloud project by default. Status lookups do not count. Search Sentinel counts the publish requests of the last 24 hours for the key's Google Cloud project and shows the requests remaining. When the quota is used up, it refuses new submissions and shows when the next request becomes available. A submission larger than the remaining quota is refused as a whole, so select fewer URLs.
If Google approves a higher quota for your Cloud project, enter it in Daily publish quota when you save the key. Requests sent with the same Cloud project from outside Search Sentinel also use Google's quota, but Search Sentinel cannot count them; Google then answers with HTTP 429.
The Indexing API does not use checks from your plan.
Request history
Recent requests lists the last 25 requests of the project: when they were sent, the URL, the type, Google's HTTP status, and the notification time Google returned or its error message. Everyone with access to the project can see the history.
Errors
| Message | What to do |
|---|---|
| 403 Permission denied. Failed to verify the URL ownership. | Add the service account's e-mail as an Owner of the Search Console property that contains the URL. Check that the property covers the URL: protocol, www and subdomain must match, or use a domain property. |
| 403 Web Search Indexing API has not been used in project … or it is disabled | Enable the Web Search Indexing API in the service account's Google Cloud project and try again after a few minutes. |
| 429 Quota exceeded | The daily quota of the Google Cloud project is used up. Try again after 24 hours, or request a higher quota from Google. |
| Google rejected the service-account key: invalid_grant | The key was deleted or disabled in Google Cloud. Create a new JSON key and save it again. |
| This is not a service-account key | Upload the JSON key of a service account, not an OAuth client file or an API key. |