Last week, Roo Reynolds wrote about how we’ve been improving site search on GOV.UK. Throughout the past couple of months we’ve been analysing our search data to find out more about how people are searching the site. Here are some of the most interesting things we’ve learned.
Getting people to what they need
First of all, we looked at everything we already knew from the first 6 months since GOV.UK was released back in October 2012. About 4% of visitors use the site search, making more than 2 million searches a month.
We regularly review the most popular search terms and check how they’re performing – particularly when people refine their search or leave the site. ‘Refinements’ show how many users are changing their search term and trying again, usually to make their results more specific. Overall, the average refinement rate was 27% – we’d like to get that down so that more people are finding what they want first time.
‘Exits’ show how many users are leaving the site straight from a search results page, rather than clicking on a link within the site. The average exit rate was 15%, but this includes clicks on external links to other useful websites for information that isn’t covered by GOV.UK, such as NHS Choices and the National Careers Service. However, if searches without external links have high exits, it suggests that those results aren’t relevant enough so users are going elsewhere.
Inside Government searches
Around 20% of site searches now start from an Inside Government page (the corporate part of the site with government policy, announcements and publications). The rest come from the services and information for citizens and businesses or from the GOV.UK homepage.
A lot of these Inside Government searches are made from the main department or organisation pages. This may be partly because department websites have recently moved to GOV.UK so regular users are finding their way around the new content. But we’ve also seen in user testing that some members of the public still think in terms of the department responsible for their needs – for example, they’ll go to the Home Office or HM Passport Office to renew their passport, though these pages are aimed mainly at ‘professional’ users (those whose work involves research, the detail of policies or the workings of government).
Many of the top Inside Government search terms are general needs that are also in the overall most common searches, for example: ‘jobs’, ‘visa’, ‘passport’, ‘careers’, ‘ehic’, ‘travel advice’.
This crossover presents a challenge for site search: how do we meet the advanced needs of professional users without confusing members of the public who just want a simple answer? We can’t rely on the page they searched from to define which type of user they are; some people expect to search only within that department, but others have landed in the wrong place and need to find the general results. One of our priorities for this project was to start making search better for advanced users, without getting in the way of less experienced users.
The long tail of search
One of the first questions from the search team was: ‘how big is the long tail?’ Out of the 2 million site searches a month, about 800,000 different search terms are used.
We sliced up a month’s data by filtering on the number of searches: first the top 100, 200 and 1,000 search terms (that is, 101-200 and 201-1,000), which were used about 100 times or more, and then everything searched for more than 50 times, 20 times, and so on.
At one extreme, the word ‘jobs’ alone makes up 1.5% of all our searches, with almost 30,000 a month. The 100 most commonly used search terms (including ‘jobs’) account for 13% of all searches; the top 1,000 search terms combined make up 28% of searches.
At the other end of the scale, more than half a million different search queries were only made once in the whole month – that’s 34% of searches, but 84% of search terms.
We also found that refinement rates increase considerably as you go further down the tail. In the top 100 search terms, most of which are quite specific and well optimised, only 14% of users searched again. In the middle groups, the percentages rose through the 20s, and in the large set of more complex search queries that were only entered once, 34% of users changed their search terms and tried again.
Word count of search terms
We also analysed the length of site search terms, and found that most people use short queries of up to 5 words – half of all searches are 2 or 3 words.
These tend to be simply the name of the thing they need to find, for example: ‘passport’, ‘student finance’, ‘working tax credit’, ‘employment and support allowance’. Sometimes they include the action people want to take, for example: ‘find a job’, ‘renew child passport’, ‘book practical driving test’.
As queries get longer, they start to get more specific, for example: ‘driving licence out of date’, ‘returning to work after maternity leave’, ‘change the date of my practical test’, ‘how long does it take to get a passport’.
Some users enter very detailed queries describing their personal situation, for example: ‘I have a British passport and I had a British driving licence but I have lived in Spain for the last 10 years and now have a Spanish licence, I am returning to UK and need to get a British licence’.
We found that search exits (users leaving the site) increased with longer queries, rising steadily from 14% for 1 word up to 20% for 9 words or more. Interestingly, refinements (searching again) didn’t follow the same pattern; they rose slightly from 28% to a peak of 31% for 5-word queries, but then dropped back down to 27% for the longest queries.
Why does all this matter?
We’ve been improving the search algorithm, to make it better at matching exact phrases or words that appear close together, and to reduce irrelevant results that only match some of the words. Knowing the most common types of searches helped our developers decide where to focus their efforts to make the most effective improvements.
Now that we’ve released these changes, we’ll continue to monitor and analyse search performance to see how much difference they’ve made and to prioritise further development work.
Filed under: GDS, Performance
