Must-Have htaccess Tips for you to Avoid Duplicate Content on Your Site

Optimizing a website to avoid duplicate content is a crucial issue for SEO experts

by Luigi Nori Date: 25-03-2020 htaccess apache tips


In order to be able to implement these tips it is necessary that your Apache server already has the mod_rewrite module activated. mod_rewrite and .htaccess are used together so that your site has the possibility to present semantic and public friendly URLs.

In the past it was more common to find sites that have addresses like www.mysite.com/index.php?id=1. Nowadays, this is already completely out of use. It is much more interesting to use an address like this: www.mysite.com/my-good-post, do you agree?

Well, considering that you have already made the necessary settings for the .htaccess file to work properly, I present my tips used in my day-to-day.

1. Remove slash (/) at the end of the URL

An address www.mysite.com and www.mysite.com/ are different and if they have the same content, it is duplicate content. So, to avoid this, I use the following code to always remove the bar at the end of the address.
RewriteCond    %{HTTP_HOST} !^. [NC]
RewriteRule    ^(.+[^/])/$ http://%{HTTP_HOST}/$1 [R=301,L]

2. Insert the www if you don't have one.

An address www.mysite.com and mysite.com are also different addresses and is considered as duplicate content if they display the same content. So, to avoid this I use the following code:
RewriteCond    %{HTTP_HOST} !^www.
RewriteRule    ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

3. Remove index.php, if it exists

If you access a website with www.mysite.com and www.mysite.com/index.php and both display the same content, it will also be considered as duplicate content. In this case, the problem is solved as follows:
RewriteCond    %{THE_REQUEST} ^.*/index.php
RewriteRule    ^(.*)$ / [R=301,L]

Okay, okay, okay, okay, okay. This way you guarantee that your project will always be accessed without bar at the end (/), always with www and avoid that the home is accessed via /index.php.

Here, follows the complete code of my .htaccess used in most of my projects.



  RewriteEngine on

 #redirect if exist end slash
  RewriteCond    %{HTTP_HOST} !^. [NC]
  RewriteRule    ^(.+[^/])/$ http://%{HTTP_HOST}/$1 [R=301,L]

 #redirect if does not exist www
  RewriteCond    %{HTTP_HOST} !^www.
  RewriteRule    ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

 #redirect if exist index.php
  RewriteCond    %{THE_REQUEST} ^.*/index.php
  RewriteRule    ^(.*)$ / [R=301,L]


 
by Luigi Nori Date: 25-03-2020 htaccess apache tips hits : 2927  
 
Luigi Nori

Luigi Nori

He has been working on the Internet since 1994 (practically a mummy), specializing in Web technologies makes his customers happy by juggling large scale and high availability applications, php and js frameworks, web design, data exchange, security, e-commerce, database and server administration, ethical hacking. He happily lives with @salvietta150x40, in his (little) free time he tries to tame a little wild dwarf with a passion for stars.

 
 
 

Related Posts

How To Use Varnish As A Highly Available Load Balancer On Ubuntu 20.04 With SSL

Load balancing with high availability can be tough to set up. Fortunately, Varnish HTTP Cache server provides a dead simple highly available load balancer that will also work as a…

htaccess Rules to Help Protect from SQL Injections and XSS

This list of rules by no means is a sure bet to secure your web services, but it will help in preventing script-kiddings from doing some basic browsing around. MySQL injection…

How to write real client IP address in error Log with Varnish 4 and Apache 2.4 in Ubuntu 16.04

In order to have Varnish 4 pass on the real client IP to your Apache 2.4 error log in Ubuntu 16.04 , you'll need to edit your Varnish configuration (/etc/varnish/default.vcl…

How to Configure the Mod_Security Core Ruleset in Ubuntu

ModSecurity is a Web Application Firewall, a program that can be used to inspect information as it passes through your web server, intercepting malicious requests before they are processed by your…

Install Apache, MariaDB and PHP7 on Ubuntu 16.04

Ubuntu 16.04 LTS Xenial Xerus comes with PHP7 by default so you don’t have to rely on third-party PPA to get PHP7 installed. In this tutorial, we are going to…

Setup SSL Certificate on Apache and Ubuntu 12.04

How to Create a SSL Certificate on Apache for Ubuntu 12.04   About SSL Certificates A SSL certificate is a way to encrypt a site's information and create a more secure connection. Additionally,…

Install and Configure Varnish with Apache multiple Virtual Hosts on Ubuntu 12.10

About Varnish Varnish is an HTTP accelerator and a useful tool for speeding up a server, especially during a times when there is high traffic to a site. It works by…

Install apache2 mod_security and mod_evasive on Ubuntu 12.04

This guide is intended as a relatively easy step by step guide to: Install and configure Apache2 ModSecurity and mod_evasive modules on Ubuntu 12.04 LTS server. Things have become much easier…

How To Uninstall Apache 2

This is a little tip that we can share to manage your linux box server, below you can see the consolle command to uninstall apache webserver sudo update-rc.d -f apache2 remove

Setting the Hostname (FQDN) in Linux Servers

Here we will show how to set your system's hostname and fully qualified domain name (FQDN). Your hostname should be something unique. Some people name their servers after planets, philosophers, or…

Million of visitors per day with a super cheap php mysql server using nginx and varnish

These instructions are the rather verbose, but hopefully easy enough to follow, steps to build a new Linux server using Varnish and Nginx to build a php application on a…

Big Data and Hadoop: an explanation

Our world is a potential treasure trove for data scientists and analysts who can comb through massive amounts of data for new insights, research breakthroughs, undetected fraud or other yet-to-be-discovered…

We use our own and third-party cookies to improve our services, compile statistical information and analyze your browsing habits. This allows us to personalize the content we offer and to show you advertisements related to your preferences. By clicking "Accept all" you agree to the storage of cookies on your device to improve website navigation, analyse traffic and assist our marketing activities. You can also select "System Cookies Only" to accept only the cookies required for the website to function, or you can select the cookies you wish to activate by clicking on "settings".

Accept All Only sistem cookies Configuration