Using transients to improve the performance of custom queries

Transients in the database
Not sure what transients are? Try reading the introduction to transients first.

My emergency introduction to transients

As I mentioned in my introduction, I learned about transients the hard way: I had built a site for a client that covers the supercomputing industry: their website has been online pretty much since the advent of the Internet, so they have a lot of content. They were previously on a proprietary, terrible CMS and wanted to make the move to WordPress.

They insisted on copying their existing layout and content and building it on top of WordPress, so I set about that with a custom theme, custom post types, custom taxonomies: the whole nine yards. Their front page, like many other news/magazine style sites has a lot of different content areas, showcasing the latest news, whitepapers, multimedia, features and podcasts from the different parts of their website, so home.php has a lot of custom WordPress queries, using WP_Query to retrieve all of the different information for display.

Everything went well in testing and on the staging site, so we went ahead and launched. After the site was online for a while and traffic kept increasing, when a new article was posted or updated the site would go down with nothing more than a blank screen to show (no error code or message: nothing). So with little to go on, I was truly perplexed by what might be causing it. After a whole lot of troubleshooting, Query Monitor highlighted that each page load was making over 400 queries to the database. 400!!! I’m amazed that the site even stayed online at all (props to WP Engine). Many of these queries (over 190) were coming from the theme, as a result of running several different WP_Queries and then additional queries to retrieve meta information for the posts returned by that query.

So needless to say, I knew that needed to change. However, we couldn’t very well just eliminate some of the WP_Queries: it was important to show that information on the front page. They just needed to be refined, by using transients to store that information temporarily, rather than searching for it every time a page was loaded.

Modifying my WP_Query

home.php was already littered with various instances of WP_Query: I just needed to “retrofit” them to use transients. Doing so really wasn’t too burdensome, especially once I got the hang of it.

Here’s an example of one of the many WP_Query instances I had in home.php:

It’s a fairly standard use of WP_Query, searching for the most recent published “spotlight” custom post type, that is not in my array of posts to not be duplicated on the front page and whose metadata for when the feature should expire has not passed. The loop follows the query, but has been omitted in my example because it is not important to how we’re going to be implementing transients here.

This is the revised query, which I will explain line by line below:

The original query (lines 2-14) exists in the exact same form in the new snippet (lines 3-15). The real difference in the code are the lines either side of the actual query.

Just before the query, in line 2, I’m doing two things: first, I’m setting the variable $its_query equal to the value of the transient of the same name, using get_transient(). Then, I’m checking to see whether it contains information. If it doesn’t, either because it is the first time this transient is being created, or because the data has expired and been deleted, it runs the WP_Query to retrieve new information to store in the transient.

Once it has run the query (lines 3-15), it saves this latest data to the its_query transient for retrieval on the next time this query is attempted (this is in line 16) using set_transient(). Note that I have set an expiration time of 4 hours (60 seconds x 60 minutes x 4 hours) – more on that later.

Realising your savings

Now we can use that data in the loop in the same way that we normally would. In effect, I have run a query once and stored that information in a transient so that over the next four hours, any time that query is attempted again, it will retrieve the information from the transient, (because of line 2) instead of rerunning the query.

If that page is loaded even once a minute (just 1440 pageviews/day), it’s the difference between running that single WP_Query six times instead of 1440 times over the course of 1 day. Multiply that by all of the WP_Queries you might have on a single page, and you’ll soon note that by using transients in this way, you’re saving a whole lot of complex queries to the database.

Keeping the data current

In our new query, we set an expiration time of four hours for the data. But what if a new post is published and we don’t want to wait four hours for it to be included in this query (particularly important for news/magazine sites, such as the one I was working on!)? Well, this expiration time is more of a fail-safe – it’s the absolute maximum time that I want that data to be there before forcibly refreshing it.

However, I can be more proactive about refreshing that data. I know that every time a new post is published, I want that transient data to be deleted, so that on the very next page load, the query is rerun and the newest post is included, so that it can be displayed on the site without delay.

To achieve this, I wrote a new function. In this instance, since the queries are in my theme files (and thus are dependent on my theme), it’s acceptable to put this function in functions.php, though I still put it in the functionality plugin to keep my code cleaner. Here’s what I did:

This very simple function hooks into the save_post hook which runs any time a post changes status (new to published, draft to published, published to trash etc.) or is edited and saved.

The function checks for whether the post that was saved is of our custom post type first (lines 4-5), since there’s no point in deleting the transient data for this custom post type, if the new post was of a different post type and won’t affect this query.

If it was our post type that was saved, line 6 uses the delete_transient() function to delete the transient from the database, forcing it to be refreshed the next time that transient is requested.

Conclusion

So there you have it. I tried to be as comprehensive as possible in showing you how I took just a single query in one of my theme files and modified it to reduce resource-intensive queries to the database. I proceeded to do this with all of the queries in my theme, and that was enough to stop the site from timing out and showing that blank page.

I strongly encourage developers to make using transients part of their normal development practice. You’ll need to learn it soon enough, so the quicker you start making this part of your practice, the better.

Want to speed up other aspects of your site in a similar way? Try speeding up your WordPress menus or making your shortcodes less intensive on your database.

10 thoughts on “Using transients to improve the performance of custom queries”

  1. Great great info. Gonna do the same in one of the heavy front page template.

  2. JC says:

    Very informative. One question – are there any disadvantage to using transients? (apart from the slight extra coding)

    1. It depends on how you use them. I can’t think of a downside to the way that I’ve implemented them in this and the two forthcoming examples, though I certainly welcome input from other developers who can point to good reasons where transients (when used correctly) might not be a good idea.

  3. jacobperl says:

    This info is great. Just FYI the actual code snippets are not displaying on your pages in Chrome or FF on OS X 10.6.8

    1. Thanks Jacob. There seems to have been a problem with my Gist oEmbed plugin, but it’s fixed now. Thanks for the alert.

  4. Michelle McGinnis says:

    Thanks for the article! Am I going crazy or are all the code examples deleted? In the view-source I see “missing gist ID”… maybe something’s moved? Thanks again-

    1. Nope, definitely not going crazy. There was an issue, but it’s fixed now. Take another look :)

      1. Michelle McGinnis says:

        And they’re back! Thanks!

  5. This is great info. I usually have WP_Query heavy homepages like you are describing. I would also be curious to hear if there is a downside. I am no SQL pro, but is there any data left in the cache or db? At this point I am going to add this into the next site I build where I know these types of calls will be massively load time saving. Thanks

    1. Well, the database will always contain two rows in wp_options for each transient: one with the data and one with the timeout value. Once a transient is requested, the timeout value is first queried. If it hasn’t yet passed, it retrieves the data. If it has passed, the transient data is fetched again and saved back to the same row, so it will always keep that data in the database, however, it will not “bloat” over time.

Leave a Reply

7ads6x98y