The whole notion of plugins is that they provide special niche functionality that not every site would need. But that is just theoretical, as most new features that have ever been added to the WordPress core began life as a plugin. If the plugin was useful to enough people, it was built in.
In my opinion there are still some out there that are musts that I feel are useful on any WordPress installation regardless of it's ultimate purpose.
And remember, support your plugins! Plugins are open source just like WordPress is, but the authors generally make little or no money off their hard work. If a plugin was very useful to you, donate to the author.
This plugin will create a Google sitemaps compliant XML-Sitemap of your WordPress blog. It supports all of the WordPress generated pages as well as custom ones. Everytime you edit or create a post, your sitemap is updated and all major search engines that support the sitemap protocol, like ASK.com, Google, MSN Search and Yahoo!, are notified about the update.
This is a super easy activate-it-and-forget-it plugin that can only help you, by making sure search engines find every last corner of your site.
The default comment notification email from WordPress is kind of fugly. It's plain text, and contains a whole bunch of links. Clean Notifications put in some very basic HTML to help the emails be much more readable and friendly.
The point of using FeedBurner is to get some statistics on how many people subscribe to your site. What point are statistics unless they are accurate? This plugin will redirect anyone trying to access your WordPress feed directly to your FeedBurner feed address. Set-it-and-forget-it.
This is just a personal one for me. The built-in WordPress media gallery is getting better over time, but I still don't like it. I don't like how it inserts a bunch of class names into the markup that I don't want and can't turn off. This plugin is dead-ugly (and ancient! 2004!) to look at, but it makes uploading images to your site and getting the code to insert them dead-easy. Check out this screencap from literally the last image I uploaded while writing this post:
There have been various services in the past to try to consolidate all your comment activity though the web-o-sphere into one place, but so far none of them have really sank their teeth in as being the go-to choice. But when I comment on a blog, and I anticipate a reply, I want to know about it. So do your commentors. This plugin makes it easy by appending a checkbox to the comment form to email the commentor when new comments are published. You'll see in in action if you comment on this post:
There is nothing more important and vital to your WordPress-powered site than the mysterious database that lives on your server. If your entire server was destroyed, but you had a recent backup of your database, you would be OK.
Among other useful features like on-demand backups and database optimization, you can have this plugin email you your database at set intervals.
There is only one setting in WordPress to display how many Posts to show on a page (Settings > Reading). What if it made sense that you only wanted to display one post at a time on your blog homepage? That would mean that your search page would also display only one post, which is dumb. This plugin allows you more fine grained control over how many Posts are displayed per type of page (search, category, homepage, etc).
I must admit I was skeptical of this one for a long time, and have recently become a believer. The #1 selling point to me is that it automatically generates an appropriate meta description tag for each Post/Page of your site, based on the content of that Post. Meta descriptions are what control what shows up as the descriptive text in Google underneath each link on SERPs, and so is greatly important when people are deciding which link to click. You can also override these settings and control each pages meta description / keywords / page title individually.
It also takes care of automatically making your page titles the best possible format, and giving your pages canonical URLs (avoids duplicate content issues).
I'm a fan of disabling the visual editor (User Setting) for editing posts. I like seeing the raw HTML that make up my posts. But doing so limits the options available to you as far as editor buttons. You get bold, italic, code and such, but you don't get the same slew of choices as with the visual editor. I've always like to create my own set of editor buttons, to include stuff like a button for wrapping code in both <pre> and <code> tags, and h3, h4, h5 tags.
The plugin allows you to create your own buttons on the fly, which could be potentially useful for any type of site.