How to create robots txt file for SEO - Full Guide

How to create robots txt file for SEO: Hello everyone, everyone loves hacks, I am also trying to finding hacks in many things.so, today we are going to talk about an S.E.O hack (technique) that help your blog to grow faster. so, today I am going to talk about the robots.txt file (that is also called robots exclusion standard or protocol).
How to create a robots.txt file for SEO, what is a robots txt

   the robots.txt is a normal text file but this file is most important part of every website. but Sadly, most of the people didn't know about this. Robots.txt is a way to increase your search engine visibility.
  if, You do not have any technical knowledge to edit or create robots txt then continuously read this article.
so let's start talking about robots txt file.

What you will learn here
  1. What is the robots txt file?
  2. What is the use of robots txt file in SEO
  3. How to generate robots txt
  4. setting up robots txt in blogger

What is the robots txt file?

   Robots.txt: In simple words, Robots txt is a text file that is created by the webmaster to give some instruction to search engine. for example- if you want search engines did not crawl the specific page then disallow this page in robots txt. This gives instruction to search engines to did not index this result in their pages.

Let's learn about the structure of the robots.txt file.
 This is the robots.txt file of www.ctmtehnical.com. Now, let us discuss all these terms of the robots.txt file.
User-agent: Mediapartners-Google

User-agent: *

Disallow: /search

Allow: /

Sitemap: https://www.ctmtechnical.com/sitemap.xml 
 1. User-agent: *
How to create A robots.txt file for SEO, What is robot txt By using this command we are able to hide our coding in our website source code. For example, you have a specific that you did not want anyone see so you add coding in robots txt

 2. User-agent: Mediapartners-Google
By using this application, (command) we are able to hide our AdSense code from the user. With this command, we are able to save our Adsense account from competitors. If we disallow this then command then Google Adsense was not able to serve ads on our blog/websites

3. Allow:
If we allow an url in the robots.txt file. this means search engines able to crawl to the whole url. In my robots.txt file, I add / (slash) this, mean search engines able to crawl all the url.
4. Disallow
If you want search engines did not crawl a particular url then you add the url in disallow section. if you are using an admin panel on your website then you must need to disallow your admin panel for your website security. The disallow url become part of the Deep web.
If you did not know about Deep Web and Dark Web then also read this article.

The Dark and Deep web explained

5. Sitemap
This option helps the web crawler to indicate your sitemap. By using sitemap web crawler crawls your whole site help us to index on search engines. if you did not want to add sitemap option on robots.txt then you must need to add sitemap file link on your homepage of website
Also, read
How to send push notifications from the website - Blogging
Facebook F8 Features, Whatsapp hits 450 Millon - Daily updates
Top 7 Best PC games for 2GB ram without graphics card - Gaming
Mobile SEO Guide - By Backlinko

How to create robots txt file


If you want to generate robots txt then visit small SEO tools robots txt generator.if you did not want to generate robots txt then go to the link below.
https://www.ctmtechnical.com/robots.txt
Remember : This link works only in desktop view 
  Copy all data and paste it into your robots file. Never forget to change sitemap url. Now, paste this all commands in your robots txt file.

How to set up robots txt in blogger

Warning: Do these all work very carefully this can affect your search engine ranking.
  1. open your websites blogger dashboard.
  2. now click on settings.
    How to create robots.txt file for SEO, what is robots txt, setup robots in blogger
  3. now click on search preferences
  4. Now search robots txt.
    what is the use of the robotst txt file in seo, what is robots txt, setup robots in blogger
  5. click on edit
  6. what is the use of the robotst txt file in seo, what is robots txt, setup robots in blogger
    setup robots txt
  7. paste your code and click on save button.
 That's it now you successfully setup robots txt file for blogger.
Also, read:
  I hope you liked this article "How to create robots txt file for SEO". If you liked this article then never forget to subscribe to our blog and share this article with every blogger. Thanks for reading and Goodbye.  

No comments

Powered by Blogger.