Hide pages from search engines
By default, all visible pages in TYPO3 are indexed by search engines. If there are pages you do not want to be searchable by others — such as website experiments or new sections under construction — you can actively block them for search engines in the backend.
Be aware that blocking search engines also applies to AU’s own internal search engine.
How to block search engines
- Go to the page properties
![[Translate to English:] Billedet viser, hvordan man kommer ind på sidens egenskaber](/fileadmin/_processed_/8/7/csm_soege1_2dc306f562.png)
2. Select the 'SEO' tab, and scroll down to the bottom.
3. Here you will find 'Robot instructions'.
'Index this page'
In this field, you can choose whether the page should be indexed. Checking 'No' informs search engines not to index the page.
'Follow this page'
Checking 'No' in this field tells search engines not to follow links on the page.
'Inherit'
The 'Inherit' checkbox appears in the two categories above and means that the settings of the main page are inherited by the selected subpage. If the main page settings are checked as 'No', the subpages will also inherit this setting.
Hidden page with subpages
- TYPO3 automatically ensures that an inactive (hidden) page is not indexed by search engines.
- However, subpages (which are not hidden themselves) will still be indexed unless you actively modify the settings for the inactive (hidden) page. Follow the instructions in the expanded element above to change the page settings.
![[Translate to English:] Billedet viser hvor henne man skal trykke for at skjule en side](/fileadmin/ingen_mappe_valgt/disable1.png)
![[Translate to English:] Billedet viser at undersiden til en skjult side ikke bliver skjulte](/fileadmin/ingen_mappe_valgt/disable2.png)
Clean up
It may also be relevant to block (disable) old and outdated pages. Google crawls several thousand AU pages each day. The number of pages indexed daily is referred to as the 'crawl budget'. Naturally, it is unfortunate if many of the pages being crawled are old archive pages, while more up-to-date pages higher in the hierarchy are overlooked by Google for several days in a row. If we 'waste' our crawl budget on archive pages in this way, it will take longer for news and other front-page content to appear on Google. Therefore, it can be relevant to block old pages.
Particularly for files in the file list
- The above applies only to pages and not to documents uploaded to the file list. If you want search engines to not find a file in the Filelist module, you need to either rename the file or completely remove it ('Disable' or 'Delete'). Simply removing the link to the file from your page is not sufficient.
- Once search engines have indexed your page, it may take some time before they remove your page from search results.