How to create robots txt file for SEO: Hello everyone, everyone loves hacks, I am also trying to finding hacks in many things.so, today we are going to talk about an S.E.O hack (technique) that help your blog to grow faster. so, today I am going to talk about the robots.txt file (that is also called robots exclusion standard or protocol).
the robots.txt is a normal text file but this file is most important part of every website. but Sadly, most of the people didn’t know about this. Robots.txt is a way to increase your search engine visibility.
if, You do not have any technical knowledge to edit or create robots txt then continuously read this article.
so let’s start talking about robots txt file.
What you will learn here
- What is the robots txt file?
- What is the use of robots txt file in SEO
- How to generate robots txt
- setting up robots txt in blogger
What is the robots txt file for SEO?
Robots.txt: In simple words, Robots txt is a text file that is created by the webmaster to give some instruction to search engine. for example- if you want search engines did not crawl the specific page then disallow this page in robots txt. This gives instruction to search engines to did not index this result in their pages.
Let’s learn about the structure of the robots.txt file.
This is the robots.txt file of www.ctmtehnical.com. Now, let us discuss all these terms of the robots.txt file.
2. User-agent: Mediapartners-Google
By using this application, (command) we are able to hide our AdSense code from the user. With this command, we are able to save our Adsense account from competitors. If we disallow this then command then Google Adsense was not able to serve ads on our blog/websites
If we allow an URL in the robots.txt file. this means search engines able to crawl to the whole URL. In my robots.txt file, I add / (slash) this, mean search engines able to crawl all the URL.
If you want search engines did not crawl a particular URL then you add the URL in disallow section. if you are using an admin panel on your website then you must need to disallow your admin panel for your website security. The disallow URL become part of the Deep web.
in this case i add wp-admin this mean search engines did not crawl my website dashboard.
If you did not know about Deep Web and Dark Web then also read this article.
This option helps the web crawler to indicate your sitemap. By using sitemap web crawler crawls your whole site help us to index on search engines. if you did not want to add sitemap option on robots.txt then you must need to add sitemap file link on your homepage of website
How to send push notifications from the website – Blogging
Facebook F8 Features, Whatsapp hits 450 Millon – Daily updates
Top 7 Best PC games for 2GB ram without graphics card – Gaming
Mobile SEO Guide – By Backlinko
How to create robots txt file for SEO
Remember: This link works only in desktop view
How to set up robots txt in blogger
- Open your websites blogger dashboard.
- Now, click on the settings option in the dashboard sidebar.
- Now, click on the search preferences option.
- Now, search robots txt on the page.
- click on edit
setup robots txt
- paste your code and click on save button.
That’s it now you successfully setup robots txt file for blogger.
I hope you liked this article “How to create robots txt file for SEO“. If you liked this article then never forget to subscribe to our blog and share this article with every blogger. Thanks for reading and Goodbye.