Where And How To Upload Robots txt file In WordPress Or Cpanel

Mar 30, 2017
Upload Robots txt file In WordPress

Upload Robots txt file in WordPress, now becomes little tricky after the security update, According to this update, you do not have permission to Edit File in WordPress. But thankfully, we have a Cpanel for it. I already cover the importance of WordPress robots.txt file, So now the turn is, how we can upload it.

 Upload Robots txt file In WordPress
Upload Robots txt file In WordPress

I this tutorial, I will explain both the methods are

  1. By using Plugin
  2. Manually upload to Cpanel

But first, here is a little explanation of Robots.txt File for fresher. Whenever search engine crawls your site, It first looks at the robots.txt file and then checks all the command in it and move next. This file tells search engines what they should and should not index from your site. It also may indicate the location of your XML sitemap that helps you to the faster indexing of your website. Maximum bots follow this thumb rule, expected bots that are made for the hacking purpose. You can protect your website by applying protection layer on it.

Upload robots.txt By Using Plugin

You need a plugin called Yoast SEO. This is a free plugin used for SEO purpose in WordPress. You can Alos use All in one SEO Plugin. But I recommended Yoast SEO plugin.

  • Login your WordPress as an Admin and click on SEO Tab
  • Not click on Tool and then click Check File Editor Option.
File Editor option in wordpress under yoast seo plugin
File Editor

Note: –  If File editor option is not available then switch to Method 2

  • Now you will see Robots.txt file box with your old Robots file.
  • Replace this robots.txt file with your New robots.txt file.
Copy and past robots file here in wrodpress
Copy and past file here
  • Now click on save.

After saving your robots file. Check, your file uploaded or not

  • Open your browser window and type,

Yourwebsitename.com/robots.txt For example www.hitechwork.com/robots.txt And hit the enter button.

Check Your robots.txt File in browser
Check Your File

You will see your default robots.txt file are replace with your new file. Upload Robots txt file In WordPress is done.

Upload Robots txt file through Cpanel.

In this case, you need password and username of Cpanel.

  • Login your Cpanel and then click on the File manager.
File manager option in cpanel
File manager

Now you are in the Public_html folder. Where all your content, theme and file are placed.

  • Click on New File and hit create new file button.
Create a new file in cpanel
Create a new file

Now, your blank file for Robots.txt file is ready.

  • Click on the file and then click edit on the top of your browser.
Edit file in cpanel and open
Edit file
  • When you click on edit, a new window will be open in editing Mode.
Copy and paste file here and hit save
Copy and paste file here
  • Copy and paste your robots file here and save and close the window.

Again you have to check your robots file uploaded fine or not. Open your browser window and type the following command and hit the enter button. Yourwebsitename.com/robots.txt Note: – Replace Yourwebsitename.com with your website name. Here is a Short Video Tutorail How to Upload Robots.txt file In WordPress.

https://youtu.be/dnCCKryxzsA

Make Sure Your Robots file Not Blocking Website

Sometimes we accidently block robots to crawl a website. Make sure robots.txt file not stopping search engine to crawl your website (directory of content). You need Google webmaster tool account to ensure that, your robots txt file does not stop search engine to crawl your site.

  • Login your webmaster tool Account and click on Crawl.
  • Ater that click on Fetch As Google.
Fetch As Render option in google webmaster tool
Fetch As Render
  • Now copy the URL of any post and paste in the Fetch As Google Box.
Render the page in google webmaster tool
Render the page

Note: – Your domain name already written outside the box, so delete your domain name from the URL Box.

  • Now hit the Fetch as Render Button and check the status report.
fetch Index Report in webmaster tool
Index Report

If your robots file blocks the assessing the page it will show in the report. For example, Look I block my “tag page” from assessing.

Blocked page in robots txt file
Blocked page in robots

See the fetch as google report, shows you,

  1. Page block or not.
  2. If it blocks, then which file blocking it.

Click on the robots.txt test to check which file blocking the page.

robots blocked file
robots blocked file

When you click you will see, the file which is blocking your content and highlighted automatically by google. Then make some necessary change in the file or delete it.

Recommended Post

Remember to share this post with anyone who might benefit from this information, including your Facebook friends, Twitter followers and members of your Google+ group! And also Support Us By Liking Our FacebookTwitter, and Google+ Page.

If you have any suggestion or problem about “Upload Robots txt file in WordPress or Cpanel” please feel free to comment below.

Fawad Yousuf
Follow me
10 thoughts on “Where And How To Upload Robots txt file In WordPress Or Cpanel”
  1. In this robots.txt we have done Disallow: /tag/. It will disallow the tags of the post, right? I guess we should allow tags. Because your post is discovered on search engine via tags too. What’s your thought on this?

    1. well, i do not prefer to add tag in the search engine. Because it creates internal duplicate content,
      for example, if you adding 1 tag in 5 article, then this article can be accessed from 5 different tags, I’m right. but for Google, these are 5 different pages, Because each tag can be accessed from its URL.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back To Top