Update: This issue is FIXED in WordPress 3.3.
Here’s the really short version: I used /%postname%/ as my permalink structure on CSS-Tricks for a long time. I have lots of Pages. My site went down. I changed my permalink structure to begin with a number. Now it’s fine.
And the long version:
All the sudden one evening the server that runs this site and one of my other sites, CSS-Tricks seemed to get very slow (I noticed while trying to save a draft of a post). I tried visiting the homepage and various other parts of the site and it either took a long time or just timed out completely. Always a heart-sinking time for me, as I know very little about servers and how to troubleshoot them.
When the server responded at all, serving up just the base document was by far the slowest thing:
These sites are hosted on a MediaTemple (dv) server1. MediaTemple is known for responsive support and theoretically I’m on some kind of VIP program although I’m pretty sure I get the same level of support anybody else would. So of course I leaned on them for help. Even if the problem was outside of the scope of their support (likely since they don’t support the software you run, totally understandable) I thought if I pleaded hard enough they’d at least help me figure out the root cause.
I knew enough myself to look in Plesk (the server management software that runs on my server) in the Virtuozzo part where I can see “QoS Alerts” (Quality of Service). The biggest recurring issue was “kmemsize” (server memory), but pretty much everything was in the black.
In the end, it was ultimately me who figured it out, but Media Temple was quite helpful in that I got to speak with someone on the phone who was clearly quite knowledgeable and helped watch the server as I tried different things confirming what was working and what was not.
Diagnosing the problem
I made a quick HTML-only test page on the server and hit that. Assuming the server wasn’t in a completely dead state, that would come back fairly quickly. Then I made PHP page that just echo’d something out. That came back even faster. So it’s not just purely Apache, or server load, or PHP. Then I made PHP page that ran a fairly intense MySQL query. That came back fairly quickly as well. So it wasn’t MySQL being overloaded either. Then I made a WordPress template where all it did was get_header(). That page took forever to serve up. So I proved that the problem was related to WordPress (something particular to my WordPress setup).
The root of the problem
Turns out it was the permalink structure set up in WordPress for CSS-Tricks that was the problem. From day one, about four years now, it’s been /%postname%/. I really like how short and readable the URL’s are in that structure. A URL like http://css-tricks.com/css-sprites/ is very clear about what that page is about, as well as great SEO for the search term “CSS Sprites”. But alas, having your post structure set up like that has serious performance implications.
I use W3 Total Cache for caching, and it’s worked great for a long time. One of the things that it does is database caching. It displays the results at the end of the page in HTML comments. With the permalink structure set to /%postname%/, the comment on the end of the page looked like this:
Twenty five hundred queries on one page? That ain’t right. If I changed the permalink structure to something else, like one of the default options like /%year%/%monthnum%/%postname%/, then flushed the cache and checked what W3 Total Cache was saying, it was this:
Even that seems high but way more reasonable.
So, database queries? WTF? Why (or even does) WordPress need the database involved with permalink structures? Andrew Nacin, a WordPress core developer, was super helpful in all this. He told me that queries aren’t the problem and urged me to make that clear after publicly sharing these results. It really doesn’t make sense to me that the number of queries shoots way up under that permalink structure, but hey, I’m just reporting what I saw.
This is a very relevant part of The Codex:
For performance reasons, it is not a good idea to start your permalink structure with the category, tag, author, or postname fields. The reason is that these are text fields, and using them at the beginning of your permalink structure it takes more time for WordPress to distinguish your Post URLs from Page URLs (which always use the text “page slug” as the URL), and to compensate, WordPress stores a lot of extra information in its database (so much that sites with lots of Pages have experienced difficulties). So, it is best to have at least two path segments in your post’s permalink structure such as /%year%/%postname%/ or even /posts/%postname%/.
Emphasis mine.
So why the number of queries increased so dramatically? No idea. Does WordPress use the database to resolve URLs when you use just /%postname%/? Yes. And if you use one of their recommendations, does it remove the need to interact with the database at all? I also don’t know, but it’s certainly faster.
How did I even think of permalinks as a potential problem? I’m not sure. I just wracked my brain trying to think of things it could be, and remembered that I’ve read things here and there over the years that this has been a problem. One of those articles was on “Otto on WordPress” which delves into this…
Starting with a text-based part instead of a number-based part, that’s the problem.
The problem comes in when you try to use a non-number for the beginning of your permalink string.
When WordPress is parsing the URL to figure out what to serve up, it really wants to figure out if it’s a post or a page. Pages always have the /%postname%/ structure. You can’t really change that. So if you use any one of the word-based permalink keywords (%category%, %tag%, %postname%, or %author%), which can theoretically be anything, WordPress can’t easily tell the difference between Posts and Pages.
Fairly obvious right? Your posts structure is /%postname%/ and your pages structure is /%postname%/ – if you publish one of each and call them both “Contact”, how does it tell them apart? Thankfully WordPress is smart enough to not let you choose the same slug for both, but it still proves problematic for URL parsing.
So if you’ve chosen to begin your URL structure with one of the text-based options, WordPress triggers a flag called use_verbose_page_rules which literally creates a rule for every single page that you have. So instead of a rather concise set of rewrite rules (maybe 10) now you have a very thick set of rewrite rules (10 + # of pages you have). I have 550 pages or so on CSS-Tricks, so this is clearly problematic. Andrew Nacin guesses performance issues seep in around 50-100 pages. Not only do these new rewrite rules need to be read through on every single page request, the need to be recreated every time you create new content. Both very slow processes.
If you use anything numeric to start your permalink structure (or even a static word like “/posts/”), the rewrite rules can remain compact, and verbose rules do not need to be used.
Some might say, hey, just use the year or the year and month! That’s cool, go for it. I like that for the most part. Some people feel super strongly that having the date in the URL is vital since it gives people information about when the article was published. I don’t feel that strongly. I think that information is vital but it’s more important it’s visible on the page itself. SEO expert (not kidding, he’s the man) Joost De Valk told me dates in the URL’s can kill clickthrough rates on Search Engine Results Pages since the content can look old before they even see it.
Personally, I’ve opted to start my URL’s with /%post_id%/ then also use /%postname%/. It looks a little weird maybe, but I don’t overly mind it. Performance is more important to me.
What can WordPress (the project) do?
I’d vote that on the Settings page for permalinks, it should at least have a sentence like “It is not recommended that you begin your permalink structure with /%category%, %tag%, %postname%, or %author% for performance reasons.”
What about me?! I use /%postname%/. What should I do?
The thing to consider is how many pages you have now, and how many pages you might ever have. If you are pretty sure you’ll have very few, like under 20 pages EVER. Then whatever, I think you are probably fine with that structure. If you think you might have more than that, I think you ought to change now before it becomes a problem. Consider one of the date based structures or starting with the %post_id%.
What about SEO?
This one had me sweating. There are a lot of links out there to CSS-Tricks. I don’t want to lose traffic. That site is decent part of my income. Now all of those incoming links are wrong. That can’t be good for SEO!
Well as it turns out, it’s really not that big of a deal. There is a plugin out there that redirects all old posts to the new posts. Don’t use that! You don’t need it. WordPress automatically handles the 301 redirects from the old format to the new format. 301 redirects are what Google needs to know about your new format and update itself and retain your ranks. It’s been five days since my new transition, and even after a little snafu where I left it on a date-based structure while testing a little too long and Google picked it up, I’m right back where I was before and Google is showing my new structure just fine.
1 Media Temple (dv) Dedicated-Virtual 3.5 – Extreme w/ additional 256MB memory upgrade.
Loading ...
I did this about si months ago on all my sites and it was awesome to see how much smaller my databases got.
Chris-
I do remember reading that now about text-based permalinks. Have to take this into consideration, although I haven’t created a site that has that many pages yet.
Thanks for writing this up!
Excellent read, thanks for the information. This is pretty prudent to a current issue that I fear we might be having with out blog. I had one question. When you say:
The word “pages” – you’re talking about actual pages and NOT posts/articles, right? Or are they one in the same in the above quote?
I ask because we have about 1100+ posts, but next to no pages, yet we suffer incredibly slow load-times and such, even with the use of “W3 Total Cache”.
Thoughts?
PAGES. yes. Like literally the page type.
Hmm… So this wouldn’t apply, at all, to posts – or could it? Currently using:
If we changed that to post number, or a year or something with a number, would that improve the speed?
Still very new to WordPress (picked up your book though!) and am learning. Sorry in advance if it’s a foolish question.
Starting with category is just as bad, so yeah, start with year instead if that works for you.
Categories is actually worse than not having it there, as it introduces yet more queries.
I’m probably too late to save the commenter, but NO! Pages meant web pages, like post single pages, etc.
In all fairness Chris, you do have a lot of stuff in pages that should be in a custom post type :)
PF. I was building that content when custom post types wasn’t a glimmer in WordPress’ momma’s eye.
But yeah, true. And if you wanna help me move it I will allow this.
True, and I’m not saying WP’s permalink resolving is perfect :)
Yoast, would would you recommend from an SEO point of view? Considering that the main reason for doing it is for Google in the first place.
What to do for pages in a hierarchy as opposed to posts – or custom post types sorted by a taxonomy for that matter?
Perhaps a post over on yoast.com to cover it and how to deal with the issue within the Yoast SEO plugin.
I like what Dan said: “Perhaps a post over on yoast.com to cover it and how to deal with the issue within the Yoast SEO plugin.”
Keep up the good work Joost!
LOL.
Moving them would be optimal most likely even longer-term, right chris?
Glad to hear that it got resolved, Chris! Thanks for insight into what the problem was and how to prevent it. Keep the CSS-Tricks screencast coming as well!
Wow, had no idea that this would cause such issues. It would certainly seem prudent that a warning, or perhaps a link to an article explaining in detail the permalink options and consequences (such as this one). Still, you writing about it (and now, me and others tweeting about it) will hopefully help shed light on the issue somewhat.
It’s not an issue for me personally as most of my sites have just 3-4 pages, but it’s certainly something I’ll be aware of when creating sites for my clients.
Thanks so much for sharing your findings.
This is a great post and I have always wondered about it. I have used like you %postname% only on my blog. Lately I have noticed how sluggish things have been. Thank you for also addressing the SEO concerns. WordPress is pretty cool.
I just changed from /%postname%/ to >> /%year%/%monthnum%/%postname%/ and checked it in search, the redirect worked perfectly. Personally I like the permalink better and I may have noticed a difference in performance. This was good timing since I am working on numerous projects for clients.
Thanks for the post and all the comments.
Toni
It works from search but the old URL doesn’t seem to work (for bookmarks and such). Thoughts? Solutions?
In these sentences, are you referring to Posts rather than Pages?
I just want to make sure I’m understanding this correctly.
Also, what are the chances that WordPress will have a fix for this in a future version? This seems like a potentially huge deal that I never heard about before last weekend.
Some of your assumptions aren’t very scientific.
If I was you, I’d switch to NginX and get a real VPS (Media Temple is all marketing talk and no performance, I know from experience) and then disable your cache plugins.
I looked up your site, for such a small amount of traffic you absolutely don’t need any cache plugin. However, you SHOULD configure my.cnf correctly, MySQL has its own cache that is blazing fast.
You could easily serve 20x as many page per month on a $20 Linode account with NO cache plugins.
People should avoid cache plugins unless:
1. They absolutely understand how the cache plugin works and they’re sure it will help the situation, rather than adding more code to further complicate everything. A cache of a cache of a cache is very likely slower than just generating the content from scratch, correctly.
AND
2. They have exhausted more sensible alternatives.
AND
3. They have over 1,000,000 visitors per month
You have access to my analytics?
Various 3rd parties collect ISP data, for example, from there you can the basic idea without splitting hairs over a few 1000 visits/day. I don’t need your server logs to know you don’t need a cache plugin.
Bazinga?
BSA estimates up to 1,820,000 ad impressions per month in the sidebar of CSS-Tricks. It’s safe to assume this roughly equates to the total page impressions.
I don’t believe per month numbers are of use as an indicator when it comes to deciding if one should use a caching plugin. A server may operate well under average load while peak load – say, after a new post – could kill it. Especially if a post goes viral.
As Chris says early on in the post, he isn’t a server guy. Without appropriate tuning a self managed server can be worse than the cheapest, most fragile shared hosting avail.
Funny, I looked up digwp but css-tricks has about as much traffic.
It’s safe to assume BSA is talking about “ad impressions” not pageviews! So 8 BSA ads on a page (Yes I counted ;-) that’s only ~227,500k impressions per month. And looks like css-tricks took a big haircut in April, how recent are those BSA numbers?
Anyway, Digg is shrinking since 2007, Slashdot is shrinking since 2005, I rarely hear about “Slash-dotting” anymore. These “viral” bursts, even if they do cause downtime, it’s not like you’re losing money in that short span of time. Matt Cutts has said (on YouTube) that even a full day of downtime is not a big deal, as long as it’s not consistent. The real value of those “viral” events is the long-term authority boost that adds to your ongoing monthly numbers, then you can afford to upgrade your hardware and/or hire an expert!
Regardless, doesn’t WordPress have its own “WP_CACHE” coded into the core since 2.x? Also, consider all the extra value you get from not using a cache plugin: Your content is more fresh which helps your SEO and user experience if you’re generating dynamic content, no issues with comments not getting cached, fewer problems like the subject of this blog post, no plugin or version compatibility problems, no (accidental or intentional) backdoor security vulnerability, no “phone home to the mother ship” issues, and I’m assuming less PHP code loaded into memory which is probably a performance boost in itself.
If you really have 1 million pageviews per month and your server looks stressed, paying another $20/mo. for a separate MySQL VPS seems like a pittance compared to how much money you should be making with that much traffic. This is just my personal opinion and I’d love to hear counter arguments, but I’d rather pay an extra $20/mo. to pamper MySQL at that point to avoid the consequences of 3rd-party cache plugins.
So, if I’m hearing you correctly, you’re saying that 1) Chris doesn’t need a caching plugin for css-tricks.com and 2) he might experience some downtime, but that’s really not a problem.
Perhaps I misunderstood you.
@Travis Yes I would avoid cache plugins. Even a cache plugin needs to regenerate your pages at some point, so the underlying problem is most likely still there, but now harder to track down–and then you’ve introduced many new variables that could exacerbate the fundamental underlying problem–or even create new problems.
Also I’m not saying downtime is OK. I’m saying I don’t see traffic spikes as threatening. If you’re featured on the front page of CNN, that’s great–you might not see any downtime if your site isn’t already on the brink of disaster–and you could always make a static page for your 10 minutes of fame, and/or upgrade your hosting plan. If you’re planning for ongoing Kim Kardashian drama, forget WordPress altogether and go with a custom application ;-) But ordinary traffic spikes shouldn’t be an issue for a typical well-managed VPS hosting plan that doesn’t have some deeper issue lurking in the dark, like a haywire plugin or some MySQL configuration problem.
Are WordPress’s internal redirects actually 301, though? I know it’s redirected for a while, but when I looked in the codex I didn’t find any useful information about the kind of redirect they were. I’d love to see a citation because it’s something I’ve thought about doing but I don’t really want to run a redirection plugin or not have the 301 redirects.
As far as I can tell, yes, they are. I can’t exactly vouch for it, but I used this tool to test and that’s what it told me.
http://www.seologic.com/webmaster-tools/url-redirect.php
Thanks! :)
Just wanted to add that I gave it a shot and it’s showing as a 301.
You can also do this from a command line:
wget -I http://example.org/some-permalink
Oops. That should be:
curl -I http://example.org/some-permalink
Brain malfunction.
There’s a plugin from AskApache that is recommended in Codex which “displays the type of permalinks being used and detailed information on the internal rewrite rules used by WordPress. ” http://wordpress.org/extend/plugins/askapaches-rewriterules-viewer/
We have an entire component called Canonical that handles these kinds of redirects, and yes, they’re 301.
Just use Firebug for FF on NET tab you will see the request status.
On January 2009, Otto wrote an interesting post about this on wp-testers mailing list. He explained the process that WordPress performs when it looks for something.
Hey Chris, thanks for the info. Just a reminder that in the book (Digging Into WordPress) you did say performance issues may be an issue if you use just %postname% in the permalink (page 28).
Howdy. Let me address a few things here. Great write-up, Chris.
That’s how many queries it’ll take to generate rewrite rules for 500 pages. I guarantee there is a plugin on that site that is flushing the rewrite rules on every page. Another really bad no-no and a performance penalty whatever permalink structure you have.
This is not my area, but… that’s because the content is old. Simple as that. If the content isn’t tied to a certain time, it probably should be a page, rather than a chronologically arranged post.
In the help tab, it says: “Note that permalinks beginning with the category, tag, author or postname structure tags require more advanced server resources. Double-check your hosting details to make sure those are in place or start your permalinks with other structure tags.”
Well, and to answer a question from above, “Also, what are the chances that WordPress will have a fix for this in a future version?” — the chances are great that we’ll be removing all performance penalties for %postname% in the release of WordPress 3.3, due later this year.
I’d love to be able to return in a few months and confirm we’ve done it.
That’s great news, Andrew! Let us know yo when you guys that live in the attic at Automattic flip the switch!
Only some members on the core team get paid by Automattic; I’m not one of them. Keep in mind most of our contributors aren’t either, including many of our best ones. The code I’m eyeing for 3.3 to handle these permalink issues were written by two who work as independent WordPress consultants.
3.3 is going to have an updated rewrite system? +1 for this!
We’re working on it piece by piece. There’s a lot we can do on the existing system — improving endpoint support, better performance in some areas, smarter rule generation, etc. — that can go a long way. This is one of those things.
Thanks for a great writeup and all the hard info, Chris! I’ve seen a bunch of rounds of “%postname% is best” vs “%postname% is terrible and don’t do it” — but this is the first time I can remember seeing it with the real numbers attached. It’s striking.
Are you saying that external links to your site to, let’s say
csstricks.com/postname/
will auto redirect to the new page?
What are the pre requisites for this to work? What if your url structure has changed many times throughout the years?
If I look at my 404 logs I still see things from old permalink settings.. I found it best to manually add the rules to the .htaccess file or use the WP redirection plugin to help out…
As said above, WordPress has an entire component called Canonical to handle these kinds of redirects. While they don’t cover every rewrite rule structure (particularly when you introduce custom ones), it’ll always work for the common ones and also for %postname%. Best to just try it out to see if we support it.
Thanks Andrew,
When I do a final design on my site (HTML5) I plan on restructuring a lot, so I will test this out and see what happens.
Jim
Great post and details, Chris.
I agree that if it was a new site, a custom post type would be ideal. We’ve got to get out of the thought process that WordPress is posts and pages and utilize the amazing features we’ve been given :) Of course, you can’t help you’ve been using it for so long…
Either way, it’s nice to hear this debate will hopefully end soon.
Thanks for this, I had heard /%postname%/ caused problems but it is good to have a rule of thumb (# of pages) to manage this to.
This is good to know. I was vaguely aware that I shouldn’t use just
/%postname%/; now I know why for realsies.Personally, I think
/%post_id%/%postname%/looks pretty nice and dates are kind of ugly and unnecessary.I had the same exact problem. We had a website that was for a specific event that lasted three days. We got considerable amount of traffic during those days, so clearly we thought our server could not handle the amount of traffic. The site worked fine locally and was quite speedy. As soon as we deployed it, the site would timeout and we had the same kmemsize issue.
We changed servers, called in experts. No one could figure it out. Luckily we forgot to add the .htaccess file when moving servers and the site was running smoothly. As soon as we added it, the cpu would hit 100% and the site timed out again. So much stress, but then again learned something new.
“Luckily we forgot to add the .htaccess file when moving servers and the site was running smoothly. ”
Or you could have switched to NginX (no .htaccess)
;-)
I have create many tests, on many differnet installs and also differnet checks via query and webgrind. And my opinion ist to us th post:id for start the permalinks, the smallest unit to generate the permalink. WordPress is very faster, do you use at first a identifier and its also great for readers – she ahve an unique identifier, also in the shortlink of WP and this also great for identifier the post; but i add also the postname for seo and readers.
the result:
/%post_id%/%postname%/If you must change the permalinks, the small plugin Change Permalink Helper can help
What about a static text field like:
/blog/%category%/%postname%/
I read in the past that it was fine;
It should be fine, but it’s not. That’s another thing we hope to fix in 3.3.
I’m confused. The WP documentation states plainly:
In turn, I set my link structure up to be “/article/%postname%/” based on that info. Am I still going to have problems? Is the documentation wrong?
I have about 600 articles at the moment, and I haven’t noticed any lags…but I’d like to conquer this before it becomes a problem.
Did some more investigation on my end. I’m still using the “/article/%postname%/” link structure. I benchmarked before and after changing it on my test server. No significant difference.
I too use W3 total cache, and I took a look at the comments at the end of the page with the stats (on the production server). On the home page, I had as many as 143 queries. It’s a lot, but not too terrible considering how much I have loading on the home page. On a single post, it’s only pulling 29-40 queries (depending on the article) which I don’t consider significant.
Bottom line…I don’t think it’s so much as having a number at the head that matters, so long as it isn’t a dynamic text. As WP says in their documentation, “/article/%postname%/” should work fine.
Unless someone can prove otherwise – and also prove that the WP dev’s messed up that particular item of documentation – I’m going to continue using my structure. I think this study is worth noting, but it’s not entirely conclusive for situatons like my own.
Never realised this, thanks for the post, and thanks to Creative Prose for the tweet.
Does this also apply to custom post types with the cap. of “page”?
I have about 400 ‘pages’, but splitted up in different custom post types, 30-40 pages each.
No. Applies only to post_type=page. Rewrite rules for custom post types are customary ^slug/(.*) which means that the slug sets it off in a different set of rules.
When %postname% is used, pages and posts are on the same plane, and WordPress can’t tell them apart. That’s why we have verbose rewrite rules.
As @Nacin said, check for flush_rewrite_rules in all your plugins. Some plugins may be generating them on ‘init’. And that operation is very intensive. I learned this the hard way by profiling all plugin filters and actions just to find out what was slowing things down. This function is a killer.
And then, after removing it (if found), visit the Permalinks page to regenerate rewrite rules. That’s the best way to go. Manual labor FTW.
PS: Best practice is to prefix posts and not make them look like pages.
E.g.: /read/%postname%. They’re meant to be different. Using category in URL is also wrong. Why add a changeable element in URLs. Go for statics.
@5subliminal: Unfortunately /slug/%postname%/ (where /slug/ is something hardcoded) still triggers verbose rewrite rules. Lame, but something else we hope to tackle in 3.3. (Reference.)
But, to re-enforce it: The real problem only crops up when you have lots of pages, right?. If you have a zillion posts, but only a few pages, the effect isn’t as bad, right? At least, that was always my understanding. Glad to hear that it will be addressed in an upcoming version!
I wrote about this back in 2009 when it first came up on wp-hackers.
thanks for bringing this up chris.
with 3.3 updates to help with performance we can all breathe a sigh of collective relief.
i’m sure the serious sites (using WP as CMS) have already had to deal with huge page counts through hosting/setup/configs.
:)
@Chris: Does it make a different if i use “%postname%-%post_id%” instead of “%post_id%-%postname%”?
Big time, it’s all about how it STARTS, and making that one of the numeric options.
like you i also have /%postname%_%post_id% on thousands of pages and if i reverse it all my legacy links on the web fail. I presume I need some .htaccess code to handle the change to /%post_id%_%postname%/ so not to damage my SEO of my legacy backlinks right ?
Watching the back and forth with you and Media Temple. I’m honestly surprised that they gave you as much help as they did. From my experience (and many others), a problem like this would have ended in countless responses that third-party applications are not supported by (mt). I have dealt with kmemsize problems in the past and after upgrading to (dv) 4.0 plus adding an additional amount of RAM, the problem has essentially gone away (for now I’m sure). BTW – I read Otto’s post long ago concerning /%postname%/ permalinks and to avoid them, so I have and I guess we can rule that out as any problem I would encounter again in the future.
I just want to go back to you saying you have “VIP” support. If that is the case, it really saddens me that (mt) support would help you as much as they did, when it seems like a similar problem with another customer would have resulted in a different level of support. Yes, you may as well be paying them more a month/year, but we should all be on equal ground for support for what their support level doc spells out.
Maybe we just need to push on (mt) support more for when we need help. But these are self-managed server products that we are suppose to maintain and fix ourselves and not rely on support for third-party software we install on them.
… anyway, glad to see everything is working good for you now.
Hi Peter…
I was a GS client for about 3 years, having great support.
But later this year, one of my sites grew more than expected and was using to much gpus (extra costs), so i changed to the $50 DV (512mb), and still had many memory problems.
Had to make many fine tunes by myself, with support just giving me little tips, not the whole way, and not even doing anything.
Later, even with all those finetunes, found that was one of wordpress installations that caused that, and as usual, they didn’t help A LOT, since it’s a 3rd party, but they were pretty responsive, giving many tips, many ways to test, and even calling me when i messed everything up on my server trying to install mod_pagespeed.
Their support is the main reason i keep with them, being more expensive than many other hosts, but it’s fast, and over the scope they promote, was pretty good, and no, i am not in VIP lists, hehe =)
Out of my experience, I seem to get a faster responsive if I submit a ticket and then contact them on Twitter.
Yes, I do agree that they try to go out of their way to point you in the right direction. But at the end of the day, some server knowledge is required for the hosting they provide (beyond gs).
Thanks Chris for this excellent article. I am always looking for performance enhancement (sometimes to the point of obsessing).
While I don’t use the type of permalink structure you described (I use a date based structure), I do have clients that do some goofy things and articles like this add to my ability to help them track things down.
The take-away for me was the SEO information at the end. I had no idea that WP internally handles 301s when you change permalinks. That is great to know as I’ve made changes on sites before and I’ve obsessed about what would happen SEO-wise.
“WordPress automatically handles the 301 redirects from the old format to the new format.” meaning that WP will add the 301 to the .htaccess file for all the old url?
Nope, WordPress doesn’t need to add anything to the .htaccess file to enforce redirects. It’ll take the URL in and figure out where you were meant to end up, and then issue a 301.
Which btw, also means that if you can do the redirect from the old to the new with a simple .htaccess RedirectMatch line, if you have such skills, you should, as that’s way faster.
This means though that it’ll only work when you remove info from your URL, not when you add it :)
And if a RedirectMatch (or RewriteCond/RewriteRule etc) and all the associated regular expressions is too much a few “Redirect permanent” lines for the most popular URLs could go a long way. The format is like this:
Hi Chris,
This article is great thanks for the insight to something I had not even considered to be an issue. I have been trying to solve some performance issues on many of my sites for months. Thanks to the information here I will be doing some experimentation on those sites and if the performance comes back I will be ecstatic and definitely give the credit where it is due. Kudos for your hard work and persistence in ferreting out this issue.
I had no idea that is what WordPress did when building the pretty link based on the permalink settings but now that I am aware I will be letting my listeners know and the visitors to my site.
Thanks for your hard work
John Overall
Hi all,
great post. I’m using the Pretty Link Plugin to mask affiliate links and use always domain.tld/slug to rewrite these links and /%postname% as permalink settings.
Does this affect also performance as wordpress has to decide if it’s a post or a pretty link url rewrite?? Anyone have experience with this case??
I have to say, I read this article with HUGE interest, because I have many, many, many WP installs spread across 7 dv servers, and I see the kmem issue on a weekly basis. All of my installs use this permalink structure:
/%category%/%postname%/
(mt) support basically keeps telling me it is a memory issue. I upgraded the memory on an older dv 3.0, and have seen very little improvement. (BTW… I still think MT support overall is the best in the business)
Thanks for the great writeup! Guess what I’m trying next? ;-)
Jeffrey Friend
… So I think you’d to rewrite chapter 2.3.1 through 2.3.5 on the book…
Yes, we will absolutely update relevant information in the book next time it’s updated.
This is a big disappointment. I was planning on using WP as a CMS (~2000 static pages, 0 blog posts). What would you do in my place?
Just use a date-based permalink structure for the blog posts, even if there aren’t any. You’ll be fine.
Make the Pages Posts instead…
Chris, this is an excellent writeup. I recently switched to /%postname%/ after reading this article at Yoast.
Now you’ve got me reconsidering. I have thousands of posts, but very few actual pages. Haven’t had any issues yet, but if I start experiencing issues similar to what you witnessed, I’ll probably switch back to /%year%/%monthnum%/%day%/%postname%/.
Just so I’m clear, this only becomes an issue with large numbers of pages, correct?
PAGES, yes, no posts.
I know it seems weird, like, how does the permalink structure which only affects posts matter for # of pages. But it does. Tried to cover that as best as I could above in the article. And also read that Otto on WordPress article.
Good gosh – thanks for the warning! I have a very similar code that sounds like is going to get me into a world of pain later on…
/%category%/%postname%/
I’ve even experienced some slowness on a few sites, but shrugged it off since no one else said anything.
Going to go add in that post_id now!
I’ve read about avoiding using purely %postname% in the past but hadn’t heard of anyone having problems until your Post. Thanks for sharing the info!
Wow. Very informative article and comments! I used /%year%/%postname%/ in my photo store, built recently. Glad I did, but now it looks like, Maybe, an upcoming WP version will be able to handle /%postname%/ ? Did I read that right?
Yes. Expect %postname% to work better in 3.3.
I just launched a new site & blog and used the /blog/%postname%/ permalink structure. Since I only have 5 posts, I thought it would be safe to go and change the permalink structure. I’m using the tweetmeme plugin. When I changed the permalink to /blog/%post_id%-%postname%/ the tweetmeme icon disappeared and was replaced with a bunch of error code. However, when I changed it to /blog/%post_id%/%postname%/, everything is fine.
And thanks for this article!
I think the permalink structure has to START with numeric value and /blog/ is not numeric.
@Kelli and @Momofone:
/blog/%post_id%/%postname%/ is fine. The requirement isn’t strictly that it must start with a numeric placeholder, it’s that the first placeholder needs to be a numeric placeholder (which excludes postname and category, and the lesser used author and tag).
/blog/%postname%/ on its own should work just fine, but it doesn’t yet. I expect that for version 3.3, we’ll be tweaking the requirement so that the first segment cannot be a non-numeric placeholder. That’ll allow /blog/ or any other static string to work. (We’ll also be allowing /%postname%/ to work, as a second enhancement.)
Good to hear that 3.3 is getting the change, Andrew. Until then, I’ll just stick with the post_id stuff. I’m sure the tweetmeme error was because tweetmeme was looking for an url that no longer existed, so as long as I didn’t append the permalink with the post_id, it would be ok.
I love WordPress and you guys are great.
SEO people (that have I chats with) say not to use date numbers–as they show up in search results, such as Google. Thereby hindering proper/expedient/word-focused results in page placement.
If you use %years%, as you suggest, and your blog is four years old, then the performance problem (slowness) is only 25% as bad as before. And is still there lurking in the background and growing.
There is an issue of posts/pages sorting in WP, as you opine, when using the %posts% and %pages% in permalinks. But adding the %year% is more like a band-aid than a fix.
Why not divide more to conquer and push out the potential re-occurring slowness problem? Use %months% as the premalink predecessor, i.e., divide/sort by 12. After 10 (or more) years you can archive with a different page/tag hierarchy to get the performance back up to speed as you suggest here today by changing one’s permalinks. Yes, numbers are still being scanned by the Google bot but not “dated” as with %years%.
At some point, a very successful (huge) blog will migrate to a dedicated server and this may become a moot point, depending on the creativity of the IT people and the RAM/processor speeds.
When you remove dates, sure, you might be optimizing content for search engines, but you’re de-optimizing it for humans. The content is old. It’s less relevant. Humans need to be able to see that information. Hiding it just to bait them into clicking on it is really lame.
I think it’s only lame if you hide the date on the design of the site itself. If you want to use date based URL’s, that’s great, but being honest about the date of the content is the design’s job.
I agree.
I would say most users don’t look at the URL for the date, or at the URL in general.
But we’ve found URL structure to be huge for SEO – so being flexible to control it is key…
I was very excited about Post Types since I could break content up in the backend for client usability… But you can’t eliminate the post type in the url structure… So we are primary using pages and posts.
Hi Chris,
thanks for this interesting article!
Quick question: Does the built-in redirection work instantly or does it take some time to update all the links?
I’ve been using /%category%/%postname%/ since the beginning and wanted to try out /%post_id%/%postname%/. I’m running wordpress 3.1.3 locally and on a live site and on both it doesn’t to work. I get 404s on all the blog posts and it broke the pagination as well. I deleted the .htaccess and let wordpress write a new one, edited it manually and tried the default theme. Everything looks right, but it doesn’t work. Not even on new posts.
It happens automatically, dynamically, on the fly, however you wish to describe it. (Data isn’t touched.)
On a local install, I just:
* Changed my permalink structure to /category/postname/
* Published a post, /some-category/some-post/
* Changed my permalink structure to /category/postname/
* Tried to access /some-category/some-post/
* Was properly redirected to /1234/some-post/.
The .htaccess file doesn’t change when you change your structure, only when you switch between no permalinks and any permalinks.
You might have canonical redirection disabled. For example, you may have installed a plugin such as this one to fix some other issue you may have experienced.
Chances are you just need to make sure the rewrite rules are flushed (just heading to Settings > Permalink does thaT) and the .htaccess file is in place.
mediatemple just rocks!
best 24/7 support in the business and no, I do not work for them but have been using them for years.
Thanks for the update on permalinks…
I’ve ran into this issue a lot… Typically I design and develop for SEO companies and unfortunately using Pages is an issue.
I still use /%postname%/ due to SEO, and Post Types when possible, and my solution is to make almost everything Posts.
2 Weeks ago I had a site that had roughly 1000 pages and 400 or so blog posts. This led to the server hitting 300% (but still running :) and showing 3000 queries per page.
I moved everything to Posts, still using /%postname%/ and it’s running perfectly. Queries went down to 35 per page.
I wouldn’t, and I’m sure my SEO clients wouldn’t recommend adding a number to the url…
This is something I really think WordPress needs to address – there should be no problem adding an unlimited number of pages and posts.
As outlined in the post, it’s not a query issue. That’s caused by a rogue plugin flushing the rewrite rules on every page, which is a big no-no. (And is still slowing down your site.)
You may find that your site performs fine with that number of pages, as long as you don’t have a plugin calling flush_rewrite_rules() or $wp_rewrite->flush_rules() on every page.
We actually stay away from plugins – currently we only use Yoast SEO and W3 Cache.
How can I tell if plugin is doing this?
Just FYI
I just tested a site on a new WP install with 1000 pages and 300 posts and NO plugins using /%postname%/. Server processing is at 180% – it might not be queries but displaying page queries in my footer in the above situation shows 2400 queries.
When the permalinks are set to Default – the server is back to normal and queries go to 25.
Great post. Additional clarity around this point would also come from emphasizing Pages always have the /%postname%/ structure. You can’t really change that. . Although this should be intuitive, it’s not really clear in the Codex, particularly if you’re coming from primarily posts. This should help people engaged in using WordPress for static sites. Now I’m looking forward to 3.3.
Chris,
I’ve edited the Codex to better reflect the current state of things. This:
Now says:
Again hopefully we can address some issues in 3.3.
The reason the Codex page got updated was because Otto had this discussion on his blog a year ago, and a result of that and dozens of discussions in forums led him to update it.
Like you mentioned, many people don’t understand that the issue only comes up if your permalink struction starts with category, tag, author, or postname AND you have at least 15-20 static wordpress Pages. Like Andrew said , bigtime issues do arise when you start getting 50-100 pages.
We wrote about the whole issue here as well:
http://www.jtpratt.com/wordpress-permalinks-for-seo-and-speed
Thank y0u Chris, I’m so glad that i stumbled upon this post . will be changing my permalink structure. Thanks again.
Uh-oh. How do these rules work with custom content types? I have a custom content type on my site to help organize my comics. I have hundreds and hundreds of comic pages, and I’ll be adding more and more of them. My permalinks are set to use dates, but obviously I have no control over my custom content types permalinks.
Am I in danger of putting in another 4 years of comic work only to find the CMS I’m using crumbling under the weight of all my custom content?
Rachel: No, you’re fine. None of this affects custom content types at all.
If your site has a large number of pages (post_type=page) and an inefficient permalink structure for your posts (post_type=post), then you may run into general performance issues. But that’s it.
Sidenote, you do have control over permalinks for custom content types, but only through code. The defaults are fine for most use cases, however.
Ok, phew! I also am using a custom permalink structure for them using “comics” as the slug, ie: http://rachelthegreat.com/comics/trent-reznor-doll/
Hi, very good post.
I myself always used the only the /%postname%/ and since most of my clients have few pages I never had a problem.
But since I’ve read this, I went to test my blog with /%post_id%/%postname%/ and found that since the URL looks like “45/blahblahblah” if I just type the “45″ it loads the post.
No problem there, but if I try stuff like “46″ or “49″ it loads things like just an image of the “45″ post as a post itself. And weirdly it doesn’t work always, some other numbers give me a 404 (wich is the expected).
How’s that supposed to work?
All the stuff you put into wordpress (posts, pages, attachments, custom posts) are stored in a line in the wp_posts database table. Every line in this table is identified by a unique ID, so when you type a random ID in the browser you could come across an attachment, a post, a page, whatever, or a 404 page
But shouldn’t this be filtered by WordPress? I mean, it’s ok to load posts or pages by id, but for all the other things it shouldn’t.
And it only works with the /%post_id%/%postname%/ structure, that’s why I’m back to the only /%postname%/ structure. I don’t think it’s a good idea to let this possibility open.
I think this is not a real problem for blog. But how can i think to add a number in URL for a ecommerce, or portal or a websites with lots of post? :O
Please give me a positive answer
@Chris
This is not new but your post creates louder noise, so thank you.
@nacin
WP can not be the best CMS if this issue is not fixed.
Hi Chris,
I have a couple of sites — not super big, about 35 pp each — one with a blog and one without (just pages). I changed to /%category%/%postname%/ on both sites — the one without a blog is fine and did speed up substantially (both sites are incredibly slow — I’ve been blaming the hosting company). The other site has a “static” home page and a number of pages, then a link to the blog. The link to the blog broke when I changed the permalink structure. Does the new structure need to be /blogname/%category%/%postname%/ to maintain that link?
Thanks!
I have used WordPress as a CMS to presemt static resources, some with a quantity of Pages, some with Posts. After only 30-40 Pages the back end of a site starts to slow down. Posts seem to hold up better if there are fewer categories – does that make sense?
I’ve always used /%category%/%postname%/. When information is time-sensitive as blogs often are, prefixing with some date-related reference makes sense to me. When information will not be time-sensitive, I really don’t want to get into adding a month or year to the URI.
I’ve been thinking of trying a multi-site setup for sites that need an articles section. I could use a few blog-style categories to organize the articles, then still have a whole separate chunk of database for a separate blog. Do you think that would help with speed?
Thanks for this superb post. I just said the same thing on a similar post by Otto. I’m just amazed how many seo guys out there advice to go for either %postname% or %category%/%postname% etc… I think there are 1000s of people that don’t know this…
Well, if WordPress is likely to change it soon, I might not worry about it too much as I don’t have that many pages, mostly around 5-7. So perhaps the performance won’t be affected that much? Not sure..
Thanks again
Chris,
Thanks for posting this. Someone on a forum I was browsing recommended it and I was like “what could I possibly learn about permalinks” – well I use the /%postname%/ structure and my site runs like dogsht. I changed it as you suggested and it is visibly faster when loading pages. Thanks so much for the excellent tip you just earned another subscriber.
Thanks a lot. Don’t underestimate the effect that properly setting MySQL variables can have on query speeds.
Thanks again, Shirlyn.
What this post forgot is to tell poeple that a static text prefix is ok and do not affects perfomances, eg: /post/%postname%/ witch is nicer than numeric prefixes.
Actually it does effect performances, read Andrew Nacin comments up here
“”WordPress automatically handles the 301 redirects from the old format to the new format. “”
Please help. I’m just getting page not found when clicking old links.
any ideas?
I’ve always used the %category%/%postname% way and i’m up to 24156 posts so far on our blog and it seems to run fine. Go figure :}
That’s what I was thinking. The two sites I have that are larger – 1000+ posts – never show domain.com/post-name. Maybe that helps?
One of those larger sites has about six top level categories with short names and a load of subcategories of varying length. The other has no subcategories, but all of the 200-ish categories have short names. With the short category names, especially the short top level category names, did I stumble into something that saved me? Just guessing and wondering! I’m not a database design person.
p.s. Neither of those two sites show any sign of slowing down.
Did you read the article ?
The problem won’t affect when you got very little number of pages ( post_type = page ).
Yes, I read the article. “Pages” ( the_page ) don’t have /%category%/ or /%postname%/, do they? Did I miss something, or did the discussion of using /%category%/%postname%/ touch on posts, as in blog “Post” categories? I am really trying to understand.
Those two sites of yours with permalink set to catergory/postname don’t have this problem ( yet ) because they don’t have many number of PAGES.
Your sites will be totally okay as they have been, UNLESS you put on many PAGES ( 50 – 100 pages as Nacin said )
So just make sure you don’t write any more PAGES on your sites. ( POSTS are okay ) and you don’t have to change your permalink setting, just keep it like that.
WordPress will fix this issue in the next release ( hopefully. )
“For performance reasons, it is not a good idea to start your permalink structure with the category, tag, author, or postname fields. The reason is that these are text fields, and using them at the beginning of your permalink structure it takes more time for WordPress to distinguish your Post URLs from Page URLs (which always use the text “page slug” as the URL), and to compensate”
If this is the case then why on earth does the Custom Post Type structure insert a text element into the URL? I have CPT for reviews and my URL is now http://www.domain.com/review/slug – what do you suggest to “fix” it and do I need to?
Custom post types are not affected by the problem
I like use /%postname%/%post_id%/. This slug is very optimize for performance site :)
Are you serious ?
The whole article here is to demonstrate the issue of not using date-based numeric at the beginning of permalink structure.
Your structure ( postname/postid ) is no better than using only postname alone. Performance-wise, it’s poor.
Very nice article.
I’ve only created small and medium WordPress sites so I’ve never came across this problem but I’m sure that when time goes by I would certainly receive a phone call or two.
I’ve always used either category/postname or just postname for the exact same reason as CSS-Tricks: The pretty URL’s and the nice SEO properties.
I just changed all my permalinks on all sites (to either: year/postname or year/month/postname) and I’m glad to learn that the 301 is taken care of by WordPress and Google then can update the index in their search results.
Thank you for an very interesting article !
So Chris I just used /%post_id%-%postname%/ as our permalinks.
When someone comes now on the old /paintball-street-art.html page they get a 404 page instead of /5421-paintball-street-art/. Can that be fixed?
Thanks.
Reverted it back now so I won’t lose people coming on our website. BUT :) the question still remains.
/paintball-street-art.html isn’t a WordPress generated URL, so whatever thing is making the URL into that, that’s the thing that needs to be turned off or altered to respect the redirect.
paintball-street-art.html is a WP generated URL. %postname%.html so yes it can be whatever you want it to be. You can put whatever extension you want, like I saw at a time on some blog: %postname%.somename
http://photobase.ro/di-A6B8.png check it out Chris. That’s how it is setup right now.
Instead of http://digwp.com/2011/06/dont-use-postname/ it can as well be http://digwp.com/2011/06/dont-use-postname.digwp if you want :)
That’s cool I didn’t know that. I’m not 100% sure but I think that’s the heart of your problem. WordPress can handle most permalink changes with 301 redirects but perhaps not when altered with file extensions like that.
Then I probably have to mess with htaccess :/ Damn …
The quote above does not appear on the page you linked to. Specifically, this sentence: “So, it is best to have at least two path segments in your post’s permalink structure such as /%year%/%postname%/ or even /posts/%postname%/.” Perhaps that is your own writing and you accidentally included it as part of the quote? The ACTUAL next sentence on that page begins with “So, it is best for the first structure tag to be a numeric one…”.
So what do you think about /%post_id%/%category%/%postname%/ structure? It’s started with post id (which is best for performance) but also has category in front of the post title (best for SEO in some case). I try to take the best of both world.
I just want to confirm that WordPress 3.3 (due in November) will indeed remove all permalink performance issues related to %postname%, %category%, and the like. It’s done and currently in trunk. So look for that in the coming months.
You heard it here first, folks =)
I am really relieved to hear this as my site is getting up there with over 50 pages and this has been of great concern to me.
Sweet…I was worried I had to check & make updates to the permalink structure on many of my sites.
Thanks for everyone who works hard on the patch.
This is a really great improvement.
Very interesting article. I particularly liked to hear that WP handles changes to the permalink structure – I started using WP with the date on permalinks and have been bothered about it for months now…
I guess now I should try changing that.
Hi,
With /%post_id%/%postname%/ fixed the CSV import slow issue.
However, when I try to add a page manually. It takes about a minute to load the editor page and 1 minute to publish.
I don’t have any plugin enabled and I’m using 3.2.1 wp.
thanks
Hi, If I implement the changes to my sites so that it becomes:
/%post_id%/%postname%/
what hsppens down the line if I want to change back to just /%postname/ again?
is it going to cause a whole load of 301 commands in the database telling the site to redirect from the original url to the 2nd url (/%post_id%/%postname%/) and finally back to /%postname/
ie are there are donwsides to me trying this on my sites, and changing them back down the line… or even when wp has a new version?
thanks
shaun
Hi, If I implement the changes to my sites so that it becomes:
/%post_id%/%postname%/
what happens down the line if I want to change back to just /%postname/ again?
is it going to cause a whole load of 301 commands in the database telling the site to redirect from the original url to the 2nd url (/%post_id%/%postname%/) and finally back to /%postname/
ie are there are donwsides to me trying this on my sites, and changing them back down the line… or even when wp has a new version?
thanks
shaun
So I read the entire post but i still dont get it… According to this post, one of the first things I should do it change my id to /%postname%/http://yoast.com/articles/wordpress-seo/
If I have a small site, will it affect me? Thanks
Regards,
Sanjin
Chris,
You really need to update this post in light of WordPress 3.3.1. The advice in this post will no longer apply to WordPress as of Nov. 30th and 3.3.10. WordPress has fixed the performance issue with large sites and %postname%
Thanks so much for this post. I’ve spent two weeks looking at solutions to this problem and it was cured by your suggestion -awesome!