1. Canonical robots.txt
2. Canonical Favicons
3. Canonical Sitemaps
4. Canonical Category, Tag & Search URLs
5. Canonical Feeds
6. Simpler Login URL

Before making changes..

The .htaccess code in this post is designed to work when placed in the web-accessible root .htaccess file of your domain. Before making any changes to this file, make a good backup and keep it on hand just in case. Working with .htaccess is nothing to be afraid of, but it’s critical to not make any mistakes in syntax, spelling, or anything that’s not a comment (#). If you forget a dot working with CSS, your design might look messed up. If you forget a dot working with .htaccess your server will return a 500 – Internal Server Error. If this happens, don’t panic, just upload your backup and everything will be fine.

1. Canonical robots.txt

Help bots and visitors find your robots.txt file no matter what. Given that the robots.txt file should always be located in the root directory, you would think that this wouldn’t be an issue. Unfortunately, bad bots and malicious scripts like to scan for robots.txt files everywhere. Fortunately, this .htaccess snippet eliminates the nonsense by directing any request for “robots.txt” to the actual file in your root directory. If you’re sick of seeing endless requests for nonexistent robots files, this code’s for you:

# CANONICAL ROBOTS.TXT
<IfModule mod_rewrite.c>
 RewriteBase /
 RewriteCond %{REQUEST_URI} !^/robots.txt$ [NC]
 RewriteCond %{REQUEST_URI} robots\.txt [NC]
 RewriteRule .* http://example.com/robots.txt [R=301,L]
</IfModule>

To use this code, replace example.com with your own domain and include in your web-accessible root directory. These directives collectively redirect all requests for any “robots.txt” file, with the exception of the actual, root robots.txt file. If you need to whitelist a similarly named file, just include the following line beneath the first RewriteCond, replacing the path and file name with your own:

RewriteCond %{REQUEST_URI} !/wordpress/robots.txt$ [NC]

Alternate Method: Instead of using Apache’s rewrite module, we can do the same thing with less code using mod_alias:

RedirectMatch 301 ^/(.*)/robots\.txt http://example.com/robots.txt

Either method is effective, but mod_alias is optimal for typical sites with only one robots.txt file. If you have multiple robots.txt files, the mod_rewrite method will enable complete granular control from the root .htaccess file.

2. Canonical Favicons

Avatars, gravatars, and favicons are a big hit with malicious scanners. Evil scripts like to traverse your directory structure with requests for commonly used images such as the ubiquitous favicon.ico. So just as with robots.txt, we can stop the madness and redirect any request for “favicon.ico” to the actual file in your root directory.

# CANONICAL FAVICONS
<IfModule mod_rewrite.c>
 RewriteEngine On
 RewriteBase /
 RewriteCond %{REQUEST_URI} !^/favicon.ico$ [NC]
 RewriteCond %{REQUEST_URI} /favicon(s)?\.?(gif|ico|jpe?g?|png)?$ [NC]
 RewriteRule (.*) http://example.com/favicon.ico [R=301,L]
</IfModule>

To use this code, replace example.com with your own domain and include in your web-accessible root directory. These directives collectively redirect all requests for any “favicon” or “favicons” with a .png, .gif, .ico, or .jpg extension. To prevent an infinite request loop, the code includes an exception for the actual, root “favicon.ico” file. If you need to whitelist a similarly named file, just include the following line beneath the first RewriteCond, replacing the path and file name with your own:

RewriteCond %{REQUEST_URI} !/images/favicons.png$ [NC]

3. Canonical Sitemaps

Just as idiotic bots can’t seem to find your robots and favicon files, so too are they clueless when it comes to finding your sitemap, even when declared in the robots.txt file. Nefarious bots will ignore your robots suggestions and hammer your site with ill-requests for nonexistent sitemaps. This malicious behavior chews up system resources and wastes bandwidth. To eliminate the waste while helping stupid bots find your sitemap, slip this into your root .htaccess:

# CANONICAL SITEMAPS
<IfModule mod_alias.c>
 RedirectMatch 301 /sitemap\.xml$ http://example.com/sitemap.xml
 RedirectMatch 301 /sitemap\.xml\.gz$ http://example.com/sitemap.xml.gz
</IfModule>

To use this code, edit each rule with your own domains and file paths. The first rule redirects all requests for your regular, uncompressed sitemap, and the second rule redirects all requests for your compressed (gzipped) sitemap. These rules are independent of each other, so feel free to remove either to suit your needs.

4. Canonical Category, Tag, and Search URLs

Out of the box, WordPress returns a 404 – Not Found for the following URLs:

• http://your-domain.tld/blog/tag/
• http://your-domain.tld/blog/search/
• http://your-domain.tld/blog/category/

These are commonly requested URLs that may be leaking valuable page rank. As explained previously, you can use the following slice of .htaccess to redirect these URLs to your home page (or anywhere else):

# CANONICAL URLs
<IfModule mod_alias.c>
 RedirectMatch 301 ^/tag/$      http://example.com/
 RedirectMatch 301 ^/search/$   http://example.com/
 RedirectMatch 301 ^/category/$ http://example.com/
</IfModule>

To use, place the previous code into your root .htaccess file and replace each example.com with the desired redirect location. Alternately, if WordPress is installed in a subdirectory, use this code instead:

# CANONICAL URLs
<IfModule mod_alias.c>
 RedirectMatch 301 ^/blog/tag/$      http://example.com/
 RedirectMatch 301 ^/blog/search/$   http://example.com/
 RedirectMatch 301 ^/blog/category/$ http://example.com/
</IfModule>

Again, edit the examples with your actual URLs. A good place to redirect any juice or traffic from these three URLs is the home page. Hopefully future versions of WordPress will handle these redirects internally, but until then, this bit of .htaccess is a simple and effective solution.

5. Canonical Feeds

WordPress generates many different feeds for your site. The default feed configuration works great, but it’s worth including in this post a couple of useful .htaccess tricks:

• Set up canonical feed types (e.g., deliver only RSS2 feeds and redirect other formats)
• Setup FeedBurner feeds and redirect associated feed requests

Each of these techniques are easily achieved with a little .htaccess. Better grab a beverage..

Set up canonical feeds

WordPress provides feeds in four different formats: Atom, RDF, RSS, and RSS2. This variety was useful in the past to accommodate different apps and devices, but these days I think it’s safe to say that most readers and devices can handle any format you throw at it. So instead of having 4x the feeds, you can consolidate your feeds into a single format:

# CANONICAL FEEDS
<IfModule mod_alias.c>
 RedirectMatch 301 /feed/(atom|rdf|rss|rss2)/?$ http://example.com/feed/
 RedirectMatch 301 /comments/feed/(atom|rdf|rss|rss2)/?$ http://example.com/comments/feed/
</IfModule>

This code will redirect requests for alternate feed formats to your canonical choice for both main-content feed and all-comments feed. To use this code, replace both of the target URLs with your own and place in the web-accessible root directory.

Redirect feeds to FeedBurner

Redirecting feeds to FeedBurner is another useful .htaccess snippet to have in your tool belt. Here are two snippets to do the job – one for main-content and another for all-comments feeds:

# REDIRECT to FEEDBURNER
<IfModule mod_rewrite.c>
 RewriteCond %{REQUEST_URI} ^/feed/ [NC]
 RewriteCond %{HTTP_USER_AGENT} !(FeedBurner|FeedValidator) [NC]
 RewriteRule .* http://feeds.feedburner.com/mainContentFeed [L,R=302]

 RewriteCond %{REQUEST_URI} ^/comments/feed/ [NC]
 RewriteCond %{HTTP_USER_AGENT} !(FeedBurner|FeedValidator) [NC]
 RewriteRule .* http://feeds.feedburner.com/allCommentsFeed [L,R=302]
</IfModule>

The first block redirects all requests for your main-content feed, and the second block handles your all-comments feed. So to use, just replace the mainContentFeed and allCommentsFeed with the URLs of your associated FeedBurner feeds. You may also redirect special category feeds by replicating the pattern with another block of code:

 RewriteCond %{REQUEST_URI} ^/category/wordpress/feed/ [NC]
 RewriteCond %{HTTP_USER_AGENT} !(FeedBurner|FeedValidator) [NC]
 RewriteRule .* http://feeds.feedburner.com/specialCategoryFeed [L,R=302]

As before, just replace the URL of your FeedBurner feed in the RewriteRule and test accordingly.

6. Simpler Login URL

Lastly, a cool .htaccess trick for a better user-login experience. A single line of code placed in your root .htaccess file is all it takes:

RewriteRule ^login$ http://example.com/wp-login.php [NC,L]

Edit the example.com with the domain/path of your WordPress installation. Now instead of typing “wp-login.php”, we just type “login” and whoop, there it is.

You could also use RedirectMatch to enable a login URL switch:

RedirectMatch 301 \@admin http://example.com/wp-admin

The @ prevents the rule from bothering anything else, and RedirectMatch picks up on the switch from anywhere in the site, so regardless of what page you’re on, you just append “@admin” to the URL and you’re there. I use this trick at http://perishable.biz if you want to see it work. Check the comments of Chris’ original post for good discussion and some more great ideas.

© 2011 Digging into WordPress | Permalink | 20 comments | Add to del.icio.us | Post tags: htaccess, performance, SEO

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

When working with WordPress, that snippet is the only htaccess code that is actually required by WordPress, and even then, it’s only necessary if you are using so-called “pretty-permalinks” on your site. Needless to say, there are probably millions of instances of these original permalink rules in place on servers and sites all across the Web. The code is so familiar that many WordPress devs can probably write it from memory. So changing things is actually kind of a big deal.

The NEW WordPress Permalink Rules

There’s really no reason to get all excited about anything. I just tend to get hyped up for anything relating to WordPress or htaccess. So mix them together and I’m all over it. For the new WordPress permalink rules, only a single line was added, and it’s a good one to have in there:

RewriteRule ^index\.php$ - [L]

So that’s the new hotness, and it works great at what it does, which we’ll get to here in a moment. For now, let’s go ahead and add that new directive to our original htaccess ruleset to get the new WordPress permalink rules:

For sites installed in root directory:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

And for sites installed in a subdirectory:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /subdirectory/
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /subdirectory/index.php [L]
</IfModule>
# END WordPress

I’m not sure exactly when the htaccess permalink rules were changed, and I’m not sure it really matters, but I’m thinking it happened sometime between versions 3.0.0 and 3.0.4. I didn’t notice it until version 3.0.4, but I am guessing that it happened at 3.0. Somebody let me know that I’m wrong here.

So what does it do?

What is the purpose of the new htaccess directive? Why change something that has worked so well for so long? Better have a good reason to go tampering, right? Yes, and as it turns out, this new rule is involved with canonicalization [1] of your WordPress URLs. I don’t think it was by any means necessary, but it’s definitely an improvement over the original code. The new directive basically strips off any dangling index.php from your WordPress permalinks.

[1] Update: As Robert points out, the new line of code is there to fix an issue with Apache’s mod_rewrite; it has nothing to do with canonicalization, which is handled automatically by WordPress.

For example, with the original permalink rules (and no other htaccess modification), the following URLs all refer to the same page:

• http://digwp.com/
• http://digwp.com/index.php
• http://www.digwp.com/index.php

Fortunately, WordPress already handles the www canonicalization, and so the new line of htaccess takes care of the other end of the URL: the index.php. This may not seem like a big deal, but can have a big impact on script performance, search-engine optimization, and user-experience.

Updating your htaccess rules

So the question now is “do I need to go back and add this new directive to my existing permalink rules?” I think the best answer is that you don’t need to do anything – your WordPress-powered sites will continue to work just fine. But if you want the additional canonicalization that the new rule provides, then you probably should stick it in there.

A couple of things to keep in mind about the new htaccess code:

• No need to update existing htaccess files unless you want to
• As with the original code, the new htaccess works in all versions of WordPress
• If you do add the new line, make sure to remove/adapt any similar/related htaccess rules

For example, as I was going through and updating all of those old sites, I had been using my own recipe for proper permalink canonicalization:

# CANONICAL WP RULES
<IfModule mod_rewrite.c>
 RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.php [NC]
 RewriteRule ^index\.php$ http://example.com/ [R=301,L]
 RewriteCond %{HTTP_HOST} ^www\.example\.com$ [NC]
 RewriteRule ^(.*)$ http://example.com/$1 [R=301,L]
</IfModule>

This code works on any site – not just WordPress installs – to clean up both ends of the URL: the www and the index.php. As I updated my htaccess files with the new WordPress permalink rules, I had to remove this custom canonicalization code to make room (functionally speaking) for the new stuff, which is a much more elegant solution.

© 2011 Digging into WordPress | Permalink | 28 comments | Add to del.icio.us | Post tags: htaccess, permalink

Monitoring PHP errors is something that all responsible WordPress administrators should be doing. In this DiW article, we’ll show you three easy ways to monitor PHP errors for WordPress. The first method is exclusive to WordPress, and the second two methods work great for any website.

Method 1: Error Logging via the WordPress configuration file

Perhaps the easiest way to implement PHP error-logging for your WordPress-powered site is to add a few simple lines of code to your wp-config.php file. The WordPress wp-config.php file may be used to specify various PHP initiation settings to modify the functionality of your PHP installation. In this method, we will take advantage of this feature by implementing basic error monitoring for your site. Here’s how to do it:

Step 1: Create a log file

Create an empty file called “php-errors.log”. This file will serve as your site’s PHP error log. Your server will need write access to this file, so make sure to set the appropriate permissions. This file may be placed in any directory, but placing it above the web-accessible root directory of your site is advisable for security reasons. Once this file is created, writable, and in place, take note of its absolute directory path and continue to the final step.

Step 2: Add the magic code

Next, open your site’s wp-config.php file (located in the root WordPress directory) and place the following code immediately above the line that says, “That's all, stop editing! Happy blogging.”:

// log php errors
@ini_set('log_errors','On'); // enable or disable php error logging (use 'On' or 'Off')
@ini_set('display_errors','Off'); // enable or disable public display of errors (use 'On' or 'Off')
@ini_set('error_log','/home/path/logs/php-errors.log'); // path to server-writable log file

Once in place, edit the third line with the absolute directory path to the php-errors.log file created in the first step. Upload to your server and call it done. All PHP errors will now be logged to your php-errors.log file, thereby enabling you to monitor and resolve errors as quickly as possible.

The other two directives in this tasty little snippet enable you to log and display PHP errors at your will. The current configuration is ideal for production sites, but you may want to enable PHP error display for development purposes. See the code comments for more information on changing these settings.

Cool tips

A couple of notes regarding this method.. First, if you place the following definition to your wp-config.php file, all WordPress debug errors will also be written to your php-errors.log file:

define('WP_DEBUG', true); // enable debugging mode

Also note that the code used for this example demonstrates a very basic implementation. You may add any number of additional @ini_set() directives to enhance and customize your error-logging system as needed.

Method 2: Error Logging via the PHP initiation file

If you have access to your site’s php.ini file, or if you are able to implement “per-directory” initiation files, this method will enable you to monitor PHP errors for any website, not just those powered by WordPress. The process is very similar to the previous method, and requires the following two steps:

Step 1: Create a log file

Create an empty file called “php-errors.log”. This file will serve as your site’s PHP error log. Your server will need write access to this file, so make sure to set the appropriate permissions. This file may be placed in any directory, but placing it above the web-accessible root directory of your site is advisable for security reasons. Once this file is created, writable, and in place, take note of its absolute directory path and continue to the final step.

Step 2: Add the magic code

Next, open your site’s php.ini file and add the following code:

;;; log php errors
display_startup_errors = false
display_errors = false
html_errors = false
log_errors = true
track_errors = true
error_log = /home/path/logs/php-errors.log
error_reporting = E_ALL | E_STRICT
log_errors_max_len = 0

As before, this code is configured for optimal error-handling for production servers. The only thing that you need to edit is the absolute directory path for your log file. During site development, you may want to change some of the variables to enable error display, startup errors, repeat errors, and so forth. For more information on these directives, and for a more in-depth guide to logging errors with PHP, check out my article at Perishable Press, Advanced PHP Error Handling via PHP.

Method 3: Error Logging via the HTAccess file

Last but not least, let’s look at how to enable PHP error-logging via HTAccess. This method works great if HTAccess files are enabled on your site, and will enable you to log PHP errors on any site, not just those powered by WordPress. The process is very similar to the previous method, and requires the following two steps:

Step 1: Create a log file

Create an empty file called “php-errors.log”. This file will serve as your site’s PHP error log. Your server will need write access to this file, so make sure to set the appropriate permissions. This file may be placed in any directory, but placing it above the web-accessible root directory of your site is advisable for security reasons. Once this file is created, writable, and in place, take note of its absolute directory path and continue to the final step.

Step 2: Add the magic code

Next, open your site’s root .htaccess file and add the following code:

# log php errors
php_flag display_startup_errors off
php_flag display_errors off
php_flag html_errors off
php_flag  log_errors on
php_value error_log /home/path/logs/php-errors.log

As before, this code is configured for optimal error-handling for production servers. The only thing that you need to edit is the absolute directory path for your log file. During site development, you may want to change some of the variables to enable error display, startup errors, repeat errors, and so forth. For more information on these directives, and for a more in-depth guide to logging PHP errors via HTAccess, check out my article at Perishable Press, Advanced PHP Error Handling via htaccess.

Troubleshooting Tips

To determine the absolute path of your log file, upload a PHP with the following code to the same directory and open it in a browser:

<h3><?php echo $_SERVER['DOCUMENT_ROOT']; ?></h3>

Depending on your server configuration, you may need to use method #1 or #2 if you are running PHP5. Certain dual-PHP setups and other configurations may require any php_value directives to be placed in a php.ini file instead of htaccess.

An easy way to test that your error-logging system is working is to trigger a few basic PHP errors. To do so, create a PHP file and add something that will trigger an error, for example:

<?php echo "error" ?>

Lastly, if you are triggering errors but nothing is being written to your log file, triple-check that the file is writable by the server. Depending on your particular server configuration, you may need to increase the permissions level of the file. If you have to run with a setting of “777” (full permissions), definitely make sure that the log file is placed above the web-accessible root directory of your site.

Wrap it up then

With one of these three PHP error-logging techniques, you have everything you need to implement easy, automatic error-logging for your website. Everything happens quietly behind the scenes, so remember to keep an eye on things by checking your log file periodically. Doing so will enable you to ensure a well-performing site and the best possible experience for your visitors.

© 2009 Digging into WordPress | Permalink | 15 comments | Add to del.icio.us | Post tags: errors, maintenance, PHP

As good as WordPress happens to be at catching and redirecting non-canonical requests, there are other types of problem URLs that need to be fixed. Specifically, for permalink-enabled WordPress sites, the following three URLs result in dead-end, 404 (Page Not Found) errors:

• http://your-domain.tld/blog/tag/
• http://your-domain.tld/blog/search/
• http://your-domain.tld/blog/category/

The pages referred to by these URLs simply do not exist by default. Although these pages may not be requested by the average visitor, they are frequently visited by search engines such as Google. Whenever Google (or a curious visitor) attempts to load these pages, it encounters a dreaded 404 error. This is not good for your visitors and statistics, and it certainly isn’t helping your site’s search engine position and page rank. Fortunately, an easy fix is available for those on Apache servers with access to either their httpd.conf or root HTAccess file.

If your site is located in the root directory of your domain, place this code into your root HTAccess file:

RedirectMatch 301 ^/tag/$      http://your-domain.com/
RedirectMatch 301 ^/search/$   http://your-domain.com/
RedirectMatch 301 ^/category/$ http://your-domain.com/

Likewise, if your site is located in a subdirectory called “blog”, use this code instead, and place it into your site’s root HTAccess file:

RedirectMatch 301 ^/blog/tag/$      http://your-domain.com/
RedirectMatch 301 ^/blog/search/$   http://your-domain.com/
RedirectMatch 301 ^/blog/category/$ http://your-domain.com/

Almost done! If you are using the first block of code (WordPress installed in site root), edit each of the three instances of “http://your-domain.com/” to reflect the location to which you would like to redirect the dead-end URLs. For example, you may want to redirect all three URLs to your home page to extract any leftover rank juice. Another option would be to redirect each URL to a similar page on your site. Anything is possible here; proceed according to your own strategy.

If you are using the second block of code, in addition to editing each instance of “http://your-domain.com/“, you will also need to edit each instance of “/blog/” to match that of your actual subdirectory.

After editing the code and uploading to your site, you should now enjoy three less dead-end, 404 errors to worry about, while enjoying the SEO benefits of a cleaner, tighter WordPress-powered blog.

© 2009 Digging into WordPress | Permalink | 12 comments | Add to del.icio.us | Post tags: 404, permalink, redirects, search