Select Page

Upload Robots txt file in WordPress, now becomes little tricky after the security update, According to this update, you do not have permission to Edit File in WordPress. But thankfully, we have a Cpanel for it. I already cover the importance of WordPress robots.txt file, So now the turn is, how we can upload it.

Upload Robots txt file In WordPress

Upload Robots txt file In WordPress

I this tutorial, I will explain both the methods are

  1. By using Plugin
  2. Manually upload to Cpanel

But first, here is a little explanation of Robots.txt File for fresher. Whenever search engine crawls your site, It first looks at the robots.txt file and then checks all the command in it and move next. This file tells search engines what they should and should not index from your site. It also may indicate the location of your XML sitemap that helps you to the faster indexing of your website. Maximum bots follow this thumb rule, expected bots that are made for the hacking purpose. You can protect your website by applying protection layer on it.

Upload robots.txt By Using Plugin

You need a plugin called Yoast SEO. This is a free plugin used for SEO purpose in WordPress. You can Alos use All in one SEO Plugin. But I recommended Yoast SEO plugin.

  • Login your WordPress as an Admin and click on SEO Tab
  • Not click on Tool and then click Check File Editor Option.
File Editor option in wordpress under yoast seo plugin

File Editor

Note: –  If File editor option is not available then switch to Method 2

  • Now you will see Robots.txt file box with your old Robots file.
  • Replace this robots.txt file with your New robots.txt file.
Copy and past robots file here in wrodpress

Copy and past file here

  • Now click on save.

After saving your robots file. Check, your file uploaded or not

  • Open your browser window and type,

Yourwebsitename.com/robots.txt For example www.hitechwork.com/robots.txt And hit the enter button.

Check Your robots.txt File in browser

Check Your File

You will see your default robots.txt file are replace with your new file. Upload Robots txt file In WordPress is done.

Upload Robots txt file through Cpanel.

In this case, you need password and username of Cpanel.

  • Login your Cpanel and then click on the File manager.
File manager option in cpanel

File manager

Now you are in the Public_html folder. Where all your content, theme and file are placed.

  • Click on New File and hit create new file button.
Create a new file in cpanel

Create a new file

Now, your blank file for Robots.txt file is ready.

  • Click on the file and then click edit on the top of your browser.
Edit file in cpanel and open

Edit file

  • When you click on edit, a new window will be open in editing Mode.
Copy and paste file here and hit save

Copy and paste file here

  • Copy and paste your robots file here and save and close the window.

Again you have to check your robots file uploaded fine or not. Open your browser window and type the following command and hit the enter button. Yourwebsitename.com/robots.txt Note: – Replace Yourwebsitename.com with your website name. Here is a Short Video Tutorail How to Upload Robots.txt file In WordPress.

Make Sure Your Robots file Not Blocking Website

Sometimes we accidently block robots to crawl a website. Make sure robots.txt file not stopping search engine to crawl your website (directory of content). You need Google webmaster tool account to ensure that, your robots txt file does not stop search engine to crawl your site.

  • Login your webmaster tool Account and click on Crawl.
  • Ater that click on Fetch As Google.
Fetch As Render option in google webmaster tool

Fetch As Render

  • Now copy the URL of any post and paste in the Fetch As Google Box.
Render the page in google webmaster tool

Render the page

Note: – Your domain name already written outside the box, so delete your domain name from the URL Box.

  • Now hit the Fetch as Render Button and check the status report.
fetch Index Report in webmaster tool

Index Report

If your robots file blocks the assessing the page it will show in the report. For example, Look I block my “tag page” from assessing.

Blocked page in robots txt file

Blocked page in robots

See the fetch as google report, shows you,

  1. Page block or not.
  2. If it blocks, then which file blocking it.

Click on the robots.txt test to check which file blocking the page.

robots blocked file

robots blocked file

When you click you will see, the file which is blocking your content and highlighted automatically by google. Then make some necessary change in the file or delete it.

Recommended Post

Remember to share this post with anyone who might benefit from this information, including your Facebook friends, Twitter followers and members of your Google+ group! And also Support Us By Liking Our FacebookTwitter, and Google+ Page.

If you have any suggestion or problem about “Upload Robots txt file in WordPress or Cpanel” please feel free to comment below.

Follow me

Harvinder Singh Sharry

owner at Hitechwork
I'm Professional Blogger, SEO, and Digital marketing expert. I started my blog in 2016 with the aim to share my knowledge and experiences for the people associated with my field as well as for the general public.
Follow me
Subscribe To Our Newsletter

Subscribe To Our Newsletter

Join our mailing list to receive the latest news and updates from our team.

You have Successfully Subscribed!

Pin It on Pinterest

Shares
Share This