To give Semji access to your content
Why does Semji use a bot?
Semji uses a crawler, commonly called a bot (or crawler, web crawler, web spider) in order to analyze and extract the content of your website as well as the pages that rank for your keywords.
This bot, called SemjiBot, developed specifically for Semji's needs, has to be authorized to visit your website to extract its content, analyze it and show it within the platform.
If this is not possible, some of Semji's features will be restricted.
When does SemjiBot access my website?
- As a client, SemjiBot accesses your site when you import one or more pages into the platform.
- SemjiBot comes back when you request a synchronization of your content.
- SemjiBot can also come back in other cases and at different times of the week in order to improve the performance of your content.
How do I allow SemjiBot to browse my site?
The SemjiBot announces itself to the site it visits using one of the two User-Agents below:
- Mozilla/5.0 (compatible; SemjiBot/1.0; +http://semji.com)
- Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; SemjiBot/1.0; +http://semji.com) Chrome/79.0.3945.88 Safari/537.36
This information needs to be processed by your technical team, which may then explicitly allow SemjiBot.