GDS is committed to supporting and building platform products that make it simpler, easier and faster for other parts of government to build good digital services. This means continuing to invest in GOV.UK Notify, GOV.UK Pay, the GOV.UK Design System, and the Prototype Kit, as well as building new platforms like the new GOV.UK Forms product.

We also need to operate with a growth mindset in GDS. This means constantly evaluating the market, and making sure that our products and services are fulfilling a need and are seeing usage expand. Our growth strategy is obviously not about making profit, but we are a small team at the centre of government and we need to make sure our people and our money are focused on services that have the biggest reach and the most impact.

GOV.UK PaaS was set up in 2015, to help public sector organisations quickly and securely host their digital services without worrying about infrastructure. However, 7 years is a long time in tech and things have changed. The big cloud providers (AWS, Azure, GCP and others) have upped their game and reduced the barriers to entry for digital teams. Over the same period departments have built better and more expert in-house cloud engineering capability, and are (broadly) clustering around a Kubernetes based architecture.

GOV.UK PaaS has not seen the rapid and continued growth that we’ve seen with some of our other platform products, and is now at a point where we either invest heavily in some significant technical architecture changes, or we make the difficult decision to sunset the product. We have decided to do the latter, and GOV.UK PaaS will be decommissioned over the next 18 months.

In parallel, we are starting a piece of joint work with the Central Digital & Data Office, in partnership with Chief Technology Officers (CTOs) across government, to understand what a future central hosting offer could or should be. We don’t know what we’ll conclude, the options ranging from doing nothing, to creating a reusable set of configuration and management components (similar to the GOV.UK Design System, but for secure cloud hosting) all the way through to building a new PaaS v2 using different architecture.

Celebrating GOV.UK PaaS

We’re rightly proud of the work that’s been done on GOV.UK PaaS since its launch in 2015. The team who built and run the service are passionate, committed and quite brilliant. It was the right product at that time. Our web hosting service has enabled more than 60 departments, agencies and local authorities to support 172 digital services.

It has enjoyed uptime of 99.95%, and suffered only one major incident in its 7 years. All this while tenants deployed services more than 122 times a day, made up of 3,200 applications.

At the start of the coronavirus (COVID-19) pandemic, GOV.UK PaaS helped new services to be spun up quickly. It also enabled GOV.UK Notify to scale rapidly to handle an increase in messaging volume, enabling them to send an average 400 messages per second, with a peak of 15 million notifications in a day.

GOV.UK PaaS has been a great product, led by a fantastic team. The focus now is on supporting our tenants through the decommission process and reflecting on our future in this space.

Making our organisation a genuinely representative, inclusive place - where people feel that they belong - is a priority for me, and for my leadership team. GDS needs to keep learning how to be better, so in this blog post I will briefly explain why creating a safe, diverse and inclusive culture for staff matters, what our data show and where we are going to be focusing our efforts.

Why this matters

GDS exists to help government make brilliant public services for everyone. We do that by running major services like GOV.UK, and by building digital products that underpin digital services built by other parts of government. To achieve this, it is vital that our teams reflect the rich diversity of the society we serve. Having diverse teams with alternative perspectives means we’re less likely to fall into group-think or miss important inclusivity considerations. It’s not a “nice to have”, it needs to be part of our DNA.

By way of example, right now our biggest programme is building a new single-sign on and digital identity service that will be used by every department and agency in the UK government. We call this “One Login for Government”. Proving your identity is a hugely complex topic, and we have to be constantly thinking of how our design would work or feel to different groups.

Users will have the option to use facial recognition and matching to prove their identity, and this needs to work whatever their skin colour or facial shapes. People will be able to prove they are who they say they are by answering “knowledge-based verification” questions. This needs to work for people with a mortgage and a credit card, but also for someone who has just left prison and doesn’t have much in the way of a digital credit history.

By ensuring our teams are diverse and representative, we will have a richer, more creative, and higher-performing GDS.

What our data tells us

It is important for us to remember that the data can’t tell us much about the lived experience of staff. Diversity is not the same thing as a culture that is safe and inclusive. That said, the data are worth looking at. Here are a few interesting points.

At the end of May 2022, 50% of the GDS workforce was female, 20% were from an ethnic minority background and 10% declared a disability.

We have a good gender balance at GDS at all grades, especially compared to the wider tech sector, with the Tech Charter annual report reporting that the tech workforce in the UK has a 21% female workforce as per the 2019 Office of National Statistics (ONS) data. However, we also have a 10.4% average gender pay gap (6.6% median).

The ethnic diversity of our teams is similarly better than average, with the Civil Service dashboard reporting that 13.6% of the UK’s working population are from ethnic minorities. But again, we have an average ethnicity pay gap of 12.36% (median 6.62%).

At the end of May 2022, 10% of the Government Digital Service workforce reported that they had a disability (compared to the 21% of UK working age adults as a whole, according to National Statistics); 3% preferred not to say and 20% chose not to declare.

Currently, 10% of our workforce is lesbian, gay or bisexual (compared to 2.7% of the UK adult population as a whole as recorded in 2019, per the ONS); 10% preferred not to say and 21% chose not to declare.

Where we are focusing our efforts

Over the next 12 months, we are focusing on the following 5 areas to make GDS a fairer and more inclusive place for our people:

1. Open recruitment

To make sure our teams are diverse, we need to try harder to foster connections with diverse technology groups (like Black Women in Tech and Girls Who Code), and to make our adverts appeal to everyone.

We have been committed to using diverse recruitment panels for a long time, making sure our panels feature a mix of people who represent different genders, races, backgrounds and ethnicities. We think that’s a step in the right direction but also recognise that it can place a greater burden on some groups - who get asked to sit on lots of panels on top of their busy day jobs - so we need to strike a balance and widen the pool of potential volunteers.

We’re also introducing ‘This is Me’ biographies so interviewees get a better idea of the experience and diversity of the panel they will be speaking to in advance.

2. Inclusion training for line managers

We are ensuring that all of our people managers in GDS are trained on building an inclusive culture, becoming disability confident, and making sure that everyone is treated fairly.

3. Challenging inappropriate language and behaviours

We are training our people to be active bystanders and teaching them how to challenge behaviours that have no place in our organisation.

4. Diversity champions

All of our diversity networks will have a senior civil servant champion, and ensuring the networks prosper will be in the performance objectives of everyone in our leadership team.

We will be running workshops to build closer links between all network leads and their sponsors, to set out what it means to be a network champion and provide a space for them to share best practice.

5. Introduce assurance of pay agreement for new and existing female and ethnic minority recruits

We will be recruiting circa 200-250 people in 2022/23 - this is an excellent opportunity to address the pay gap, or a potential risk to make it worse. We also want to look at the pay gap for all existing staff.  The digital, data and technology (DDaT) allowances give us a mechanism to do that.

For public sector services, text messages are a cost-effective way to speak to people. They’re cheaper than letters and, if they’re well written, they can be more engaging than emails.

It’s important that services get the best value for money from the messages they send. To help them do this, we’ve been reading (and rewriting) their text messages.

Less is more

The cost of sending a text message depends on how long it is. Anything over 160 characters counts as more than one message. This gives us a simple - if simplistic - way to measure if a text message provides good value for money.

For example, if you cut 1 character from a 460-character message, you reduce the cost of sending it by a quarter. Small changes like this could help a service to make substantial economic savings. They might even be able to use Notify for free under our current pricing model.

To test this idea, we decided to offer some free content design support.

We contacted a small number of services and asked them to pick a long text message template. Then we worked our way through a series of questions:

1. Does the content address the user’s needs? If not, discard it.
2. Is everything in the right order?
3. Can we say the same things in fewer characters?
4. Is it clear, easy to understand and free from acronyms and jargon?
5. Does the message feel trustworthy? If the user needs to click on a link or pay money, it has to look legitimate.
6. Is there any personalisation in the template? Names, dates and locations can all affect the length of a message.

One of the services we worked with was Leicester Children’s Centres. They handle everything from early-years activities to supporting families dealing with domestic violence.

This is the template they chose:

Leicester Children’s Centres: Hello. Due to the situation regarding COVID-19, the Children, Young People and Family Centres are not able to offer their Bumps to Babies antenatal courses at the moment. However, we have uploaded our course videos online so that you are still able to access the information that we would have gone through in our classes. If you have any queries after watching, please contact your midwife at your next appointment or on [phone number]

The link to the videos is: [link]

And here’s the new version we wrote together:

Leicester Children’s Centres: Bumps to Babies antenatal classes are not running at the moment, so we’ve made them available as a series of videos. You can watch them at: [link]

If you have any questions, speak to your midwife or call [phone number].

The original was 4 text messages long. We got it down to 2. Cutting it by half will save the service up to 4,000 text messages a year.

We later found out that Leicester City Council has 6 similar services. They’re all going to start using the new template, which could save the council up to 24,000 text messages a year.

Not a bad result for a couple of hours of work.

We’ve heard from services that prioritising content design can sometimes be hard. But a well-written message can save time and money as well as making things easier for recipients.

Rewriting a few templates could mean that a service never has to pay to use Notify. Those that do pay can take the money they save and spend it on other important work.

What we’re doing next

GOV.UK Notify is a small team. We don’t have enough content designers to offer one-to-one support for 6,000 plus services. But pair writing with our users has been a great way to learn about the challenges they face, and how we can help.

Next we’re going to explore:

• designing reusable content patterns for text messages
• updating the existing guidance

The last 2 years have been tough for us all. During this period, we’re proud that the Government Digital Service (GDS) had teams working on the government’s response to the coronavirus pandemic.

The size of our multidisciplinary teams changed throughout the pandemic to respond to circumstances, policies, and user needs. As an organisation, we aim to maintain an agile approach to ensuring that we quickly adapt to the needs of the day. At this point there is no longer a dedicated coronavirus (COVID-19) team for GOV.UK, and new information is handled by our wider teams as needed.

This blog is slightly different from our usual blogs and it’s a bit of a longer read - we have so much to share. In this post we will cover the timeline of events and what GDS delivered, and talk about how we got through this challenging time. We’ll also share the big numbers the team successfully achieved.

How we worked

Before the pandemic

At the start of 2020, news reports of a new virus emerged from Wuhan, China. On 24 January 2020, we published the first coronavirus guidance page on GOV.UK.

A few weeks later, the UK recorded its first case of coronavirus. The World Health Organisation declared the novel coronavirus outbreak as a global pandemic on 11 March 2020.

GDS drew from its existing staff to form a team to respond to the rapidly developing situation, working tirelessly behind the scenes. On 20 March 2020 we launched the first bespoke coronavirus landing page at gov.uk/coronavirus.

The first lockdown

On 23 March 2020, the Prime Minister announced the first national lockdown during an address to the nation. Behind the scenes, the teams had already begun working around the clock to build products, services and content to help users understand the new rules and the help that was available.

This included creating the frontend form of the Clinically Extremely Vulnerable People Service in 2.5 days, and more in a significant collaboration between GDS, NHS, a number of government departments, local authorities and the private sector.

Only a few days later, we launched another service with other government departments, which allowed businesses to offer support. We also introduced content hubs to better organise and surface content for users looking for information about education, running a business, and workplace safety.

With similar efficiency, we also planned and built the GOV.UK/ask service in one week to submit questions for Downing Street press conferences - but which also showed us areas of public concern to respond to. In total, more than 700,000 questions have been submitted through the GOV.UK/ask service.

On 30 April 2020 we published the ‘Get a coronavirus test’ page on GOV.UK. The page, which has been through many iterations since its launch, received 135,000 views on the first day and accumulated more than 91 million views between April 2020 and April 2022.

By this point we had solidified our relationship with the COVID-19 Task Force, which was crucial for our work over the past 2 years, so that our respective skills, roles and responsibilities were understood across government. We strengthen this relationship by having regular meetings and communicating daily. We built a foundation of trust and understanding.

The second lockdown

A second national lockdown began on 5 November 2020 in response to rising cases in the UK. The national lockdown was due to end on 26 November 2020, to be replaced by local restrictions (“tiers”) across all of England.

Between 11am and 3pm, we served more than 17 million errors on GOV.UK, as the system responded to a huge spike in traffic: the postcode checker trying to look up results and failing. We had to fix this issue urgently, so we improved caching. To make the service more resilient, Collections and the Router Cache machines were also scaled up. It’s important to note that although we load tested before the launch, the traffic was much higher than expected.

This was an important lesson for our team and allowed us to improve communication processes between ourselves and colleagues in departments, coordinating ministerial statements and major announcements.

A few days later, Margaret Keenan became the first person in the world to receive the Pfizer COVID-19 jab as part of the mass vaccination programme on 8 December 2020.

The third lockdown

On Monday 4 January 2021 at 8pm, the Prime Minister announced the third national lockdown. The team worked late into the night to publish landing page updates and guidance to support the announcement as they emerged.

We experienced the busiest hour in GOV.UK history that day, with 3.2 million visits between 8pm and 9pm. It was estimated that the equivalent of 12% of the UK population visited GOV.UK on that day.

By the third lockdown, teams had been working very hard for 10 months. In that time, we made sure people took time off. For example, if people worked a late shift, they took the next morning off. We hosted virtual away days, had daily fikas. We also had 1-2-1s and checked on one another regularly. Delivery Managers also played a huge role in spotting when team members needed a break.

As one team member reflected on this period: “The team was down and some were personally affected by Covid. But we all really supported each other and looked out for one another. The team spirit was what got us through.”

Roadmap out of lockdown

On 22 February 2021, we published the ‘Roadmap out of Lockdown’. The introduction of a red, amber, and green country list for international travel was announced shortly after. We joined a cross-government communications group to ensure communications and guidance were aligned.

Plan A and the omicron variant

The Prime Minister set out the autumn and winter plan 2021, or “Plan A”, on 14 September 2021. This was in place on the condition that hospitalisations remained low. It would be replaced with a second plan, known as “Plan B”, should hospitalisations increase.

Though scientific researchers continued to make huge progress on vaccinations, news of the omicron variant of coronavirus emerged from South Africa, with the first cases detected in the UK on 27 November 2021.

The vaccine booster campaign was accelerated on 12 December 2021, and in aid of this our team launched a site-wide vaccination booster campaign.

A team member reflects on this time: “I remember feeling overwhelmed and exhausted but knowing I had to stay focused. In the coronavirus team, the impact is too big not to.”

Fortunately, omicron’s impact was milder than previous variants, with hospitalisations remaining low. Because of this, the return to Plan A was announced on 27 January 2022 and quickly updated on GOV.UK by our team.

On 16 February 2022, we launched a Minimum Viable Product (MVP) of our coronavirus travel smart answer, as a response to users finding COVID travel guidance complicated and confusing. It was regularly changing and owned by various departments as well as other governments.

We worked with experts in departments to ensure COVID travel advice was up to date and easy to access, including the Department for Transport, Department for Health and Social Care, and the Foreign, Commonwealth and Development Office.

User research and feedback

The team read and analysed user feedback on a weekly basis throughout the pandemic. Although this was an incredibly high-pressure scenario, user research on the landing page, our services and products was essential and worth the time.

The team had to conduct all user research activities remotely. We were able to apply methods like usability testing, interviewing and usability benchmarking effectively using online research and video-conferencing tools. We benefited from the work done by other user researchers and accessibility specialists on conducting remote research sessions with people with access and digital support needs. However, exploratory methods like contextual inquiry were no longer an option. Conducting research remotely did have limitations, but it allowed us to continue learning about the user experience and improving our work based on this insight.

The insights derived fed into decisions we made about design and content iterations.

Living with COVID

On 24 February 2022, we published the ‘Living with COVID’ guidance on GOV.UK. At this point, with an end in sight, the team started thinking about winding down and retiring coronavirus-related content, products and services.

We retired the following key products between September 2021 and April 2022:

• the content hubs (employment, business, and education)
• 3 smart answers (‘Find Support’, ‘Business support’, and the travel smart answer)
• 3 lookups (volunteering, local help, and test and trace support payments)

Decisions to retire were based on analytics, user feedback, wider policy changes and shifts in the landscape of the pandemic. We monitored user feedback on a weekly basis. This provided useful insight into what users were struggling to engage with and what pages were still needed. We also looked at trends and number of visits to decide if pages for retired services should be removed. So declining traffic and low numbers meant we were confident to shut pages down.

In April 2022, for instance, we retired the coronavirus site-wide banner, which was first published in March 2020. This was the first time, since September 2019, that we didn’t have the emergency banner on GOV.UK. As we began to retire services and content, we continued to work closely with the C-19 Task Force, making recommendations based on analytics and policy.

The evolution of the coronavirus landing page

So many pages had to be published and then retired, but the landing page has consistently been a key product for us: it was often the first place people visited when looking for information about COVID-19 on GOV.UK.

The page has been through numerous changes to reflect policy changes and help meet changing user needs, such as the addition of the radio buttons allowing users to choose which devolved administration they live in, or page type, changing from guidance to landing page and will be changed back to guidance soon.

Between its launch in March 2020 and its retirement in April 2020, it was viewed a whopping 123 million times.

The Big Numbers

We have already mentioned some key figures, but here are more:

• GOV.UK coronavirus content had an average of 14 million weekly views
• the ‘Roadmap out of lockdown’ has been accessed by 7 million users
• the ‘postcode checker’ accumulated 32 million unique page views
• the ‘Report a lateral flow test result’ service has been visited more than 49 million times, making it the most popular page on GOV.UK.
• ‘Clinically extremely vulnerable people service form front end’ received 48,900 registration on the first day it was published and in total 4.2 million food boxes were delivered through the service
• ‘Find support’ service gained over 1 million visits
• ‘Business support’ service gained over 3.2 million visits
• GOV.UK coronavirus content gained 1,453,380,995 unique page views from January 2020 to 10 May 2022

How we got through it

It was clear from the beginning that the team’s wellbeing had to be a major priority. To ensure that the team was as happy and healthy as they could be during such a tough period, we worked in a sustainable way, to avoid anyone suffering from burnout. Similarly, we knew that in order to effectively help others, we needed to prioritise our wellbeing and operational resilience.

Although we were building and iterating products, services and content at extreme pace throughout the pandemic, the team remained motivated and proud to be working on something so impactful.

We understood that, with weighty responsibility on the team’s shoulders, we had to prioritise our wellbeing and resilience so we could sustain our work over a long and unpredictable period.

As we were approaching the end of the pandemic period, we knew that we had to start winding down the team. We spoke to each team member and asked them about what they were interested in working on next. Working with other teams on transition dates, we moved our team members into their new teams whilst ensuring that we still had enough people to carry on with the remaining work.

Also, to help reflect on our work, the team created a GDS COVID-19 memory wall that recorded the timeline of events, services delivered and our experiences.

As of May 2022, there is no longer a dedicated coronavirus team in GOV.UK but what  remains is the connection we have, having worked on something so significant and the hope that we have helped the country during this challenging time. We are mindful that Coronavirus remains a serious illness, affecting many people, especially the vulnerable.

Our mission to build one fast, simple and secure way for people to access government services is both relatively straightforward, and hugely complex. Validating somebody’s identity so they can access services is mostly a ‘solved problem’ across government. Patterns and processes have grown up over time, and people are able to prove their identity, even though their experiences might be suboptimal, and they might face different hurdles and repetitive steps for different services. But here’s the thing: no one department or service is doing it all well.

It’s been nearly a year since I became the Single Responsible Owner of the One Login for Government programme, and it’s true when they say that time flies when you’re having fun. I can’t believe it’s been a year already! I remember arriving last September and giving my first ever external-facing presentation and now we’ve got products in beta, a packed dance card for services who want to work with us, and we are well on our way to building up the product suite that our government partners have asked for.

So what are my reflections and what have I learned? First of all, the Government Digital Service, or GDS, is a great place to be. It’s full of bright, enthusiastic people who care passionately about what we’re doing and making our single sign-on and identity-checking solution as easy to use and inclusive for users as we possibly can. There’s a great sense of purpose and real buzz in the air.

Delivering at pace

So, really, my first job was to work out what already exists and negotiate myself to a position of being able to ‘steal with pride’ the excellent work that’s gone on in other departments in this area.

As one of my key stakeholders said to me, “If you can bring the best of the best under one product set, you’re all set, you’ll be the market leader in government”. So that’s what we’re working towards… at pace!

We’ve got an initial version of our browser-based route - with a passport check and knowledge-based verification - in limited beta with our partners, the Disclosure and Barring Service. We’ve got an identity-checking app for people with driving licences in beta with HMRC for Government Gateway users, and credible plans for passports and Biometric Residence Permits to be added soon. We’re actively working on digital vouching, a face-to-face route, and new knowledge-based verification question sets that leverage government data – all of which will make identity checks more inclusive.

We’re most definitely underway in terms of delivery. We’re building the things that departments have told us they need and we’re standing on the shoulders of all the great work and research that’s already been done across government, so we’re not starting from scratch.

Understanding what success looks like

That’s the relatively straightforward bit. Working with departments to migrate their services in a way that makes sense for them and their users is the complex bit – particularly if you don’t want everyone you migrate to have to prove their identity again.

The legacy and diversity of identification methods across government means that there’s quite a lot of analysis work to be done to map existing processes to the Good Practice Guide (GPG) framework and understand the relative levels of assurance they need. There’s a trade-off between the veracity of the process and the percentage of people who will get through it, so departments have set their assurance levels in light of this.

We’ve spent a lot of time working to understand what’s really important to users and departments, and then building our roadmap and plan to focus on these key metrics. This has been really interesting because departments all have slightly different success criteria, and within their own organisations have optimised around different things based on their user groups.

One thing that does seem to be consistently important is the percentage of eligible users who make it through an identity verification route successfully. More on this in a subsequent blog post, no doubt, but working to understand different service needs and the implications of variations of numbers of users via different channels has been really helpful in determining our near-term goals.

So, we’ve made big steps forward in terms of building the things that departments have asked for, and this will remain our core focus for the next six months at least. However, in parallel, we’re starting to work through the complexities of user migration, how our account functionality needs to work, and how we can start to reuse identities verified in one place to access services in another.

We’re also thinking about how we can start to influence user behaviour when they’re on GOV.UK, so that being ‘logged in’ becomes a default behaviour, because it ‘unlocks’ other things and improves user experience.

It’s been a great first year and I’m looking forward to the next one!

We’re on a mission to build a tool that will allow anyone in government to easily create online forms.

There are nearly 8,500 document-based forms published on GOV.UK, many of which are not fully accessible. These forms can be either very difficult or impossible to fill out for users with certain disabilities. They’re also harder and take longer to process for government teams.

Our aim is to help teams to create and publish online forms that are quicker and easier to process and use. And we’ve just passed a major milestone by publishing the first form using the GOV.UK Forms platform!

Now in private beta, GOV.UK Forms is supporting a number of partners to easily create accessible, digital forms in minutes. Our users are people working in government who build and use forms, typically to collect information which enables somebody to access a government service. This is a really common need across the civil service. So it’s crucial that our users do not need to have any technical background to create online forms.

Co-designing the GOV.UK Forms platform

Building on our findings from the alpha and to get things started, the Government Digital Service (GDS) identified a number of private beta partners across government to help us co-design the GOV.UK Forms platform. Our first partner is the Redundancy Payments Service (RPS) team in the Insolvency Service.

Together, we chose a form that only required simple questions and features. This meant we could quickly build the basic functionality needed to create the form. We selected a form that is used to amend details about accrued holiday pay for a redundancy claim. This is one of six forms that can be used to amend a claim if you need to update the information you’ve already provided.

After a lot of work designing, researching, learning and building over several months, we were finally ready to run a session to build the first form. Our partners in the Insolvency Service were then able to successfully create the digital form, from start to finish, in less than 30 minutes.

This new digital form will help those who have made a redundancy claim to easily update the details about their holiday pay accrued. It will also help the Insolvency Service’s team to process the form faster and speed up payment to the claimant.

What’s next?

This is a straightforward form that doesn't use any trickier kinds of question types or features - things like marking questions as optional or allowing users to upload files or add multiple responses to a question. These are all features our private beta partners require, so they’re the next ones we’re planning to design, build and test. And the Insolvency Service will be working on digital versions of the remaining RPS forms soon too.

Why has GDS created an online form builder?

Currently, PDFs and other document types are easier to create than online, HTML forms, especially for teams without any digital specialists. However, this comes with significant drawbacks in the accessibility stakes.

Many of these forms are difficult or impossible to fill out for some users, for example those with certain visual, cognitive, mobility or neurological disabilities. And whilst there are well-designed PDF forms which are on the whole better, even these still do not meet government accessibility requirements, which are to comply with the Web Content Accessibility Guidelines 2.1 AA standard.

So we want to help colleagues by providing them with a way that allows teams to create accessible, easy-to-use and quick-to-process digital forms. We think that by providing tried and tested components and patterns - for example a confirmation page that advises an applicant what happens next after filling in a form, radio buttons to select an option, or email validation - we can help form creators to achieve this goal.

On average, a new form is added every working day and more than 400 new forms have been added in just the last year. We therefore need to act now to stop this number growing further.

Getting down to the detail

We’re really grateful for the interest in GOV.UK Forms. One of the most common questions we get is, “When can I use it?”. As we are in the early to mid stages of private beta, there is a lot of building, testing and iterating to come.

We’re aiming for at least 60% of our partners to have published a form by the end of private beta. Then, building off this learning, we aim to launch the platform for central government use in 2023.

About the platform

The platform will be web-based, with no coding required. The platform code though will be open-sourced, and you can already see the accessible forms code we use in the components section of the GOV.UK Design System.

We currently use GOV.UK Notify to send an email to the team which processes form data containing the information in the completed form. We think by sending data in a structured way that is not contained within a document-based form, like a PDF, we can help teams to save significant processing time and costs.

Looking ahead, lots of teams have asked us about how else they might use form submission data and so, in the future, we might want to look at sending aggregated data and integration with other applications.

Working across government

In private beta, we are collaborating with other government departments, such as the Maritime and Coastguard Agency, Department for Environment, Food and Rural Affairs (DEFRA), the Driver and Vehicle Standards Agency (DVSA) and HM Revenue and Customs (HMRC), to further develop the platform and to publish at least one of their forms.

We also talk regularly with the Ministry of Justice (MoJ), Foreign Commonwealth and Development Office (FCDO), and the cross-government form building community, who have their own form-building products. This is helpful as it encourages us all to share learnings as well as to focus on the different problems we are tackling. For example, our team is targeting a wider user base, including people in government with zero tech background, so we are focusing on providing GOV.UK Forms as an easy-to-use self-serve platform.

Sharing learnings

We do regular talks at civil service-focused events as well as with the international form-building and design communities. We’re always happy to discuss our work and learn about users, so please contact our team for more information.

The Government Digital Service (GDS) exists to help government make great public services for everyone. In addition to developing, maintaining and optimising our own flagship products, like GOV.UK, we work closely with other government departments (OGD) to support their service offerings, ensuring they are user-centric and fit-for-purpose.

Central to this mission is data-driven and evidence-based decision making, which in itself needs strong data foundations and high quality data sources and insights which we can share with confidence.

With that in mind, we are working on an ambitious project to join up and share digital analytics tracking across departments, to enable a single user view across services, departments, and platforms, and by extension help departments understand pain points and optimise user journeys.

Let’s take a step back to look at the problem space from the beginning.

The problem space

GDS currently has access to Google Analytics data for users who have opted into analytics tracking on GDS-managed domains, such as gov.uk and design-system.service.gov.uk. Once a user clicks on a link to a service on service.gov.uk or elsewhere, GDS loses sight of the journey, since service domains are managed wholly by the government departments that own them.

To give a sense of scale, in August 2022, users clicked on links to around 250,000 external domains/pages from GOV.UK, most of which are other government service domains with their own analytics tracking. The same is also true for other government departments - they can see their own analytics data, but not how these journeys connect to others, including GOV.UK.

Without visibility of the journey end-to-end, we are limited in:

• our understanding of user behaviour
• ability to identify and address issues and pain points
• generate insights or measure performance, such as completion rates, at a macro level

For example, we know that last year GOV.UK had more than 800,000 searches for Universal Credit and close to 100 million sessions that accessed the page “Sign in to Universal Credit”.  Unfortunately we have no visibility of what proportion completed the end-to-end Universal Credit journey successfully, at which point they may have dropped off, or how users of Universal Credit might interact with other information or services on GOV.UK.

Joining up the government’s digital analytics estate would not only have a direct impact on optimising our users’ experience of government and their ability to successfully complete tasks, but also provide government benefits through reducing failure demand and lowering average costs of administering services, whilst also supporting GDS’ and the Central Digital and Data Office’s (CDDO) broader strategies.

The background

But this isn't a new idea. There was an earlier attempt to track user journeys across government domains through the Cross-Domain Data (CDD) project. This required government departments to add in code snippets that would allow for journeys to be linked across the domains through page views.

A lot was learnt through this early iteration, and these learnings have been pivotal for what’s next. This included streamlining the data-sharing process, greater clarity in who could access the data through data-sharing agreements, and a scalable technical approach.

The Single Data Environment

This next iteration is based on creating a Single Data Environment (SDE), where there is a single analytics consent model used across all government domains, which allows consent status to remain valid across pages and domains. This in itself, will make the user experience significantly smoother, as we already know from user research that having to consent multiple times, through what users believe or expect to be a single website, is frustrating.

Furthermore, this will be coupled with a single Google Analytics (GA) property to allow all web analytics data to flow into the same property, and eventually, with a consistent tracking schema through the forthcoming migration to GA4. Unlike the CDD project, the Single Data Environment will allow both the GDS and departments to view this data, enabling the review of the wider end-to-end whole user journey through anonymised data aggregates, as well as in-depth performance reviews of departmental services and GOV.UK journeys.

Our intention is to keep the overheads on departments for this new approach as low as possible, which will consist of: (1) adding a javascript code snippet into their frontend, (2) implementing an analytics consent component, and (3) signing up to a revised and consistent data sharing agreement; all of which will be provided by GDS.

It has been important for us to maximise the learnings from CDD, to ensure the success of this major data project. So far, we have:

• conducted an initial proof of concept with the GDS Digital Identity domain to prove the methodology and data
• presented the project for comment to key stakeholder groups across all levels of government (Chief Data Officers council, Chief Technology Officers council, Privacy and Consumer Advisory Group (PCAG), Data Protection Officers (DPOs), Functional Leaders Group, Digital and Data Board)
• created a cross-government DPO working group, to have early visibility and comment of our data-sharing and privacy documentation
• created a prototype, sandbox environment, to demonstrate how the product works and what the data looks like
• onboarded early adopters (Cost of Living and Department for Education’s Find and Apprentice campaigns) to further stress-test our approach and processes

We’re keen to release early value and streamline our onboarding process with our current version of GA Universal Analytics, but also want to avoid onboarding too many services that we will then have to migrate again in the new version.

The intention is to begin our wide-scale roll-out from January 2023, which will coincide with the launch of our GA4 tracking schema and Data Protection Impact Assessment (DPIA). We also want to integrate Single Data Environment tracking within the Design System, so new services include this out of the box, and the onboarding list does not keep growing indefinitely.

GOV.UK Notify has grown dramatically over the past 6 years. Over 6,500 service teams currently use Notify and around 60,000 individuals have a Notify account.

We want to meet the needs of this large and diverse group of people. Our users provide a range of services, from passport applications to children’s services, fishing licences to business continuity. To help us do this, we changed the way we run our annual user satisfaction survey.

The survey is one of the main ways we collect quantifiable data. It gives us a good snapshot of what our users want and how they feel about Notify.

When we designed the survey in 2020, we would only send one per team, to the person who first set up the service. While this approach has benefits and made sense at the time, it also meant we weren’t always getting the full picture.

We know that people who set up a service on Notify are more likely to be decision-makers. They may not be the people who use Notify on a day-to-day basis. It therefore feels risky for us to assume that these individuals can tell us about the experiences and needs of their whole team.

So in 2022, the third year of the satisfaction survey, we changed our methodology. We sent the survey to everyone who has signed in to Notify at least once in the past year, even if this meant collecting more than one response per service.

We also included some questions about team make-up and what other GDS services those teams were using. We’ll be sharing the insights we get from these questions with colleagues in GDS who are looking at how we can better support government users, especially those with no DDaT specialisms on their teams.

The benefits of our new methodology

Changing our methodology allowed us to achieve a much larger sample size. We got nearly 2,000 responses to the latest survey - more than 5 times as many as we got last year. This means we can be more confident that the results are an accurate reflection of our user base.

It also means we had enough responses to be able to break the results down by categories such as type of organisation, job role and access needs. This makes it possible for us to identify whose needs we could be doing a better job of meeting.

Findings

84% of respondents told us they’re satisfied with Notify. That’s good, but actually a bit lower than last year’s result. It’s possible this reduction is down to the change in our methodology. If we get a lower score with a larger (and more diverse) group of users, it shows that we’re not meeting everyone’s needs.

The top reasons given for satisfaction were ease of use, simplicity of the user interface and performance.

High on the list of things users are dissatisfied with are our message templates. Some respondents find them inflexible and difficult to implement. Others said that Notify’s model of creating reusable templates rather than one-off messages does not fit their use case. We also identified a need for more data and better reporting once messages have been sent.

What we’re doing next

The responses to our annual satisfaction survey will inform what to work on over the coming year.

We’ve combined our findings with other data sources, such as:

• support ticket analysis
• usage data
• ongoing primary user research

We’re creating a quantified list of pain points and unmet needs that our users have told us we need to address. These will become part of our roadmap.

In the next few months, we’ll focus our research and design effort on:

• reviewing our email formatting options
• exploring the ability to send one-off messages
• more detailed reporting and analytics

In summer 2021, we published a blogpost outlining the Government Digital Service’s (GDS) strategy for the next three years and outlined five “missions” around which we would coordinate our delivery efforts. I also noted that the strategy was a moment in time, and we expected it to change and adapt based on what we discovered.

GDS has always had a commitment to working in the open, so a little more than 18 months on, I thought it was time to reflect on what has gone well, where we have adjusted our approach, and where we are now focusing our work. 

What has gone well in product

Overall, we have had a successful 18 months, and some of our achievements have gone far beyond what I thought possible. 

On product, we started with an ambition to take the learnings from GOV.UK Verify and build a new digital identity and single sign-on service for all of government - something we are now calling GOV.UK One Login. Looking back, we had a mountain to climb in terms of designing and building the product set, building confidence with our customers across government, and securing funding to get it all done. The team has developed from a small group to large, high-performing team who are demonstrating user-centred, agile delivery at an extraordinary pace and scale. This success is down to a combination of a deeply talented delivery expert in our Digital Identity director, Natalie Jones, a focused and united team, and strong support from HMRC, DWP, Home Office and others across government. There is a long way to go, but we are in beta, and making real progress. We’ll be blogging more about this in the next few months.

Our exceptional GOV.UK team has developed content and new design patterns to help users navigate some of the many major events of recent times. From the response to HM Queen Elizabeth II’s death, the UK’s help for Ukraine, Cost of Living, and many others, the team have diligently worked with No.10 and colleagues right across government to make the content simple and accessible to all. Behind the scenes, they have been making GOV.UK simpler to use and navigate, improving the civil service facing publishing tools, and migrating to new web hosting infrastructure.

This year we have also made important progress on our “common tools”, which we now refer to as our “digital service platforms”. The individual platform products - like GOV.UK Notify, GOV.UK Pay and the GOV.UK Design System - have gone from strength to strength, and are mature products with ambitious development roadmaps in their own rights. 

What is even more exciting is our fledgling plans to integrate these products into our newest offering, GOV.UK Forms. There are nearly 8,500 document-based forms published on GOV.UK, and most are inaccessible, put an unnecessary burden on the end user, and are expensive for government to manually process. To fix a problem of this scale, we need to go wholesale - we cannot rely on digital departments to get to every single one. Instead, we want to develop a digital “build a service” product that can be used by civil servants in operational and policy roles right across government. We’ll be talking more about this in the coming months.

What has gone well in building the GDS team

Away from delivery, we have also had a busy year of realigning the organisation to be a more focused, more inclusive, and happier place to work. There have been some considerable challenges, which I’ll get to in the next section, but overall we are in a better shape now than 18 months ago. 

We have started to reintroduce internal communities of practice at GDS to create better alignment across teams, and to provide a space for professionals to share insights and develop their skills. In some areas we have gone further and centralised a small number of professions like technical architecture and cyber security. We have also set out our locations strategy, with a commitment that in addition to our existing strong foundation in London, we expand our teams in Manchester and Bristol.

Finally, we have expanded our leadership team, bringing in talent from the private sector, from other parts of government, and by promoting talent from within GDS. We have done this with a careful eye on increasing the diversity and equity of our teams, and we are moving firmly in the right direction.

Challenges and changes

The past 18 months have not been without their challenges. I won’t go into detail here, but everything from budgets to headcount has been under considerable pressure. This has meant we have had to make prioritisation decisions to make sure our people and resources are focused on the highest value areas. In the summer, we announced that we would be sunsetting our hosting platform, GOV.UK PaaS. More recently we have made the difficult decision to close both our GDS Advisory service, referred to in my previous blogpost as “GDS Expert Services” and our in-country GDS Advisory International work.

We have also made less progress than we had hoped on our plan to develop “Joined-up services that solve whole problems and span multiple departments”. While the ambition remains central to our mission, we discovered more dependencies than we had anticipated. The biggest dependency we discovered was having the GOV.UK One Login service rolled out across more services to enable a persistent, logged in user journey. 

Looking forward

Recently, we’ve been doing some big thinking about our longer term ambitions for digital government in the UK, and about GDS’ role in making them a reality. We have come a long way in the past decade, and the user experience of most government services has vastly improved. However, where once the UK was world-leading, and positioned first in the UN’s e-government index, we have now slipped to 11th. We need to look to those countries who have now gone past us - Denmark, Estonia, Korea and others - and adapt our approach towards a more joined-up and proactive experience of government. We will be blogging more about this in the coming months.

About GDS

The Government Digital Service is a team of around 750 product managers, software engineers, designers, researchers, technical architects and other specialists who sit right at the heart of UK government in the Cabinet Office. We are based in London, Manchester, Bristol and in our home offices. While we are relatively small, our software products and services are used by more than 13 million people each week, and are relied upon by more than 1,900 public sector organisations across central and local government, NHS and schools.

GDS is here to make digital government simpler, clearer and faster for everyone. Good digital services are better for users, and cheaper for the taxpayer. 

The core of our mission is to reduce failure demand in services and keep the user journey online. We are trying to do that in three main ways:

• ensuring a clear, easy-to-navigate and welcoming front door to government
• building common platforms to help departments make better digital services
• using our unique position at the centre of government to connect the dots

And have boiled that down into the following objectives:

1. Develop the next iteration of GOV.UK as the single and trusted online destination government information and services
2. Provide a single way for people to log in to government services and prove who they are
3. Build common components to make it easier for departments to build better digital services
4. Connect data around users to create a faster, more personalised experience of government.

At GOV.UK Notify we currently have 7,000 live services. That’s 1,700 more than we had a year ago. The more users we have, the more important it is that they can do things without our intervention.

How we prioritised branding

Over the course of a year, we received 567 requests to change email branding. That’s roughly equivalent to 2 support tickets per working day.

Adding branding on GOV.UK Notify used to be a fairly manual process. Services had to fill in a form that would open a support ticket, and we would work with them to update their branding. If we already had the right logo, we could apply it straight away. If not, we would ask for the logo, text and optional background colour. 

This approach worked when Notify was small, but it didn’t scale well. It also had some other drawbacks: users had to wait to use their new branding, and they had no way to preview email branding before it was set. Once we changed it for them, a user would have to send themselves a test email to see the new branding in action.

We decided to make this process more self-service, to reduce the time and effort spent on support and to give our users a better experience.

Before

User actions and interactions with Notify support needed to set branding for a service before we made the changes.

After

Service users can now preview their new branding before setting it.

Doing the hard work to make it simple - one step at a time

This feature is a prime example of doing the hard work to make things easier for the users. 

Although making the whole feature self-service was a very large and complex piece of work, we wanted to deliver improvements as soon as possible. 

We tested some prototypes with users to refine our ideas, and drew up an action plan. We split the work into several stages, taking into consideration factors such as impact on the support team, complexity of the work, and also potential issues related to quality control for logos uploaded by users.

First, we built 2 new flows. One to let NHS services pick NHS branding by themselves, and another to let central government services pick GOV.UK branding.

Then, we created ‘branding pools’ - a set of approved branding options for each organisation. This allowed services to choose any branding from their organisation’s pool, as long as we have it on our system. This feature had the most moving parts, and it took us a while to build it all.

If we didn’t have a particular logo on our system, a service would still need to open a support ticket to ask us to add it. These are the most complex tickets to solve, because getting a logo in the right quality and aspect ratio can sometimes be quite a task. Also, the user only gets to see their new branding when we’ve already applied it. If they don’t like it, there’s more support work to do, and a longer wait for the service before they can use their new branding. 

To improve this, we built a way to let users upload their own logos and preview their new branding. This was the trickiest problem to solve. We had to maintain a balance between empowering our users and maintaining some quality control. After all, poorly-branded notifications can damage recipients’ trust in government communications.

We settled on letting users upload and use their new branding straight away, while also keeping an eye on newly-added branding options to ensure they meet quality standards. To make this easy, our app sends a daily digest of new branding options to be reviewed by our team. We’re also happy to help if users need extra support when updating their branding. 

Cross-discipline collaboration

Letting our users set their email branding by themselves required collaboration across all disciplines in our team. From designing the prototypes, testing them, and analysing the results together, to the many conversations to figure out how to solve some thornier problems along the way.

A lot of work went into interaction and content design, so that we could support our users in adding high quality branding. This includes guidance on writing helpful alt-text for logos. Our users can also preview their new branding embedded in a mock email or letter before they start using it.

We held a retro for the epic to ensure that we capture any learnings about how we work together, and bring them into our future projects. 

Conclusion

Letting our users set their own branding took a lot of time and effort, but it was definitely the right thing to do. Especially as the number of services using Notify continues to grow.

Branding requests used to account for 8% of all support tickets. Processing an average of 2 branding requests a day took up a lot of our time. Now we can use that time to answer other support tickets, or work on developing more new features.

The GOV.UK Prototype Kit is a tool for building working prototypes of new government services in just a matter of hours. Our new version (version 13) makes creating and updating prototypes easier for all users, particularly those getting started with the Kit. 

The prototypes the Kit creates are real websites, using normal HTML, CSS and JavaScript that run in standard browsers. This means you can use the prototypes you build to do realistic testing with users, on their own devices, and using any assistive technology such as screen readers. In short, it makes it easier for departments to build better digital services.

Test with prototypes and save time

Testing with prototypes before building full production services saves time and money. It lowers the risk of spending money building the wrong thing and therefore also reduces failure demand (the demand created by failing to do something correctly). Building prototypes using the Kit also takes much less time than building production ready services only to have to change them. 

We believe that user centred design is the key to government building better, more usable and more accessible services; prototyping is at the heart of this approach. Get in touch for a demonstration.

What’s new

We’ve made a new version of the GOV.UK Prototype Kit to create a better experience for users. We’ve simplified the installation and update process and created a new ‘manage your prototype’ section that makes it easier to both create new prototypes and make changes to existing ones. 

In the spirit of practising what you preach, we tested the changes with our users. They said the new features make prototyping easier and more efficient, as they help users get started more quickly and easily, without necessarily needing to find the relevant documentation. They also said that these changes make it more intuitive and easy to use, particularly for people new to the Kit or new to coding.

Find out more on our What's new page. 

What we’re doing next

Having less coding experience shouldn’t stop you from using the GOV.UK Prototype Kit. We know getting started can still be challenging for new users, so we’re prioritising building new features that continue to make it easier for new users to get started and improving our tutorials and guidance. 

We’re also making it easier for departments to customise the Kit with their own style guides through plugins. This means they won’t need to build and maintain their own versions.

How we tested version 13

We’ve tested this new version with two stages of research:

Stage 1

Pre-beta usability testing with 5 users. One hour sessions that specifically helped us to test changes to the new homepage element of v13.

Stage 2

A 4-week private beta. We invited around 20 existing users of the kit to use the new kit over 4 weeks. We asked participants to use the kit as they would usually to create and work on prototypes. We set up a dedicated Slack channel to collect feedback and offer support, and we  collected feedback more formally through a survey and interviews. 

Participants were a range of existing GOV.UK Prototype Kit users with differing experience and confidence with the Kit, and varying overall digital capability. Because of this we feel confident that the decisions we made based on the research have been influenced by a mix of perspectives, based on all of our user types.

Earlier this year, the Cabinet Office launched a public consultation to seek views on proposed changes to the law. These aim to improve how government departments use the data they hold to make it easier for people to prove their identity online.

We’ve had a lot of interest in the consultation, with many respondents asking questions and providing feedback, so we wanted to talk a bit more about what the changes will and won’t mean for you.

One of the challenges of building a way for people to prove their identity online to access government services is that it has to work for everyone.  Around 3.5 million adults in the UK don’t have any form of photo ID according to the Electoral Commission, and there are many who don't have a strong credit history on which they can answer questions to prove who they are. The proposed changes would strengthen the legal basis for using government held data to help someone prove who they are, as well as strengthening the legal basis for sharing the result of an identity check with another government department once it has been carried out, so that the user doesn’t have to do it again. This would help more people to be able to prove their identity online, simply and efficiently.

This consultation is about the government’s proposed new regulation to the Digital Economy Act, an Act passed in 2017 which already exists to strengthen public services through the better use of data.

Although the consultation is not about GOV.UK One Login, this digital platform being built by GDS, is expected to be the first application of the new legislation. We talked about it in the consultation to help bring to life how the legislation could work in the real-world. GOV.UK One Login is a fast and simple way for people to access government services, while maintaining stringent safeguards on user data and protecting against fraud. 

GOV.UK One Login is not about replacing existing offline and face-to-face routes, which we know some users need, and neither is it about the creation of an ID card. Personal data will always be protected in the system and will only be used to support identity verification. It will enable us to confirm that “yes, this is Jaz Bloggs”, making it easier for Jaz to use the government services they need, while also ensuring that fraudsters can’t get access if they’re not who they say they are.

The new legislation won’t change GOV.UK One Login’s approach to privacy. We will continue to comply with data protection legislation and all guidance published by the Information Commissioner's Office (ICO). The principles of ‘data minimisation’ are applied as set out in UK GDPR, so that only the smallest amount of data needed to prove a user’s identity is processed. GOV.UK One Login also has very strong cyber security and counter-fraud processes in place to uphold privacy, secure data and identify and rapidly respond to threats. 

The consultation 

This month will be all about celebrating those with neurodiverse superpowers; product leaders like Steve Jobs, entrepreneurs like Richard Branson, top actors like Orlando Bloom and Keira Knightley, influencers like Stacey Machelle and athletes like Simone Biles. This will be an opportunity at GDS to champion our own neurodiverse talent and find out how we can work together to leverage it and at the same time level up for everyone in the organisation.

Why we set up the Neurodiversity Network

Tony Richards, Network Chair:

With the official Neurodiversity Celebration Week in March, it is time for us to reflect on how far we have come as a network in GDS over the past year.

I was first inspired to re-evaluate my dyslexia as a positive when I went on a Civil Service Fast Stream secondment to neurodiversity employment advocacy organisation Exceptional Individuals. Working closely with the organisation's founder Matt Boyd, I learned to re-frame common misconceptions in society that I had previously internalised about my own neurodiversity and advocate it to others. This experience was so powerful and transformative that colleagues in the Cabinet Office Press Office proactively pushed a piece out to the Financial Times about my experiences.

One year ago I decided to put effort into building the network and formalising it, as I felt there was so much neurodiverse potential in our organisation which we hadn’t yet tapped into. What I saw all around me at GDS motivated me to change things. I once had a colleague open up to me and tell me they were dyslexic only on their last day. I realised from instances like this that more work needed to be done to ensure people at GDS were able to feel open about their neurodiversity, and supported from day one. This will ensure we harness the neurodiverse superpowers we have at our disposal to solve the challenging user problems we face.

Neurodiverse superpowers

Spiky profiles: while too many focus on the troughs of a neurodiverse person’s abilities, the spikes in abilities are often much higher than a neurotypical person and in areas that can really help organisations solve challenging user problems. (Source: Exceptional Individuals)

The Neurodiversity Network was a very small Slack channel a year ago with only 20 or so members (estimated 2.5% of staff). Today it has 90 members (estimated 12% of staff). But there is still more work to do as studies by the National Career Institute suggest the overall neurodiverse population is close to 20%. The network runs a rich programme of events and leads on key work streams to leverage neurodiversity right across the organisation and those close to it.

Manual of Me

Last year the focus of celebrating neurodiversity week was all about establishing the network and introducing the concept of neurodiverse superpowers. At GDS we celebrate neurodiversity over a whole month to maximise involvement across the staff body. This year we will be covering the strategy we have developed, with a big focus on trialling ‘Manual of Me’ across the business. We have already incorporated ‘Manual of Me’ into the GDS Smarter working toolkit. During the events this month we will seek to promote widespread testing and trialling of them across delivery and product communities. Some Delivery and Product managers are already using ‘Manual of Me’ and we are working closely with them to help promote trialling of it across the organisation.

Our strategy

There are three key pillars to our strategy as a network:

• leveraging and supporting the neurodiverse ‘superpowers’ in the workplace
• making recruitment work for neurodiverse candidates
• supporting colleagues with neurodiverse family and friends

Throughout March our network will host a series of events for staff to flesh out these objectives, with hard actions to empower the organisation and leverage the neurodiverse superpowers of our highly talented staff.

Programme of events

Our programme of events mirrors our strategy with each week focusing on a key pillar:

Week 1: Raising awareness

A series of drop in sessions for different types of neurodiversity.

Week 2: Making recruitment work for neurodiverse candidates

A series of show and tells on what we are doing with Heads of Community to create a more inclusive recruitment process.

Week 3: Leveraging neurodiverse superpowers in the workplace

Showcase the utilisation of Manual of Me to leverage, and level up, neurodiverse talent across GDS.

Week 4: Supporting colleagues with neurodiverse family and friends

A range of open sessions requested from across GDS to understand how we can best help.

How you can get involved

We have events happening all throughout March:

Wednesday 22 March

12:30 to 1:20pm: Manual of Me — find out how to leverage neurodiverse superpowers in your workplace

Tuesday 28 March

12:30 to 1:20pm: ADHD (awareness session)

Thursday 30 March

11:00 to 11:50am: Dyscalculia (awareness session)

12:30 to 1:20pm: The Dyslexic Advantage (awareness session)

Useful resources

Video: Emily - Made By Dyslexia Baby Film

Article: How dyslexic capabilities can help organisations of the future

Autism test for adults

Quotes from neurodiverse leaders

Richard Branson, founder of Virgin

“My dyslexia has shaped Virgin right from the very beginning and imagination has been the key to many of our successes. It helped me think big but keep our messages simple. The business world often gets caught up in facts and figures — and while the details and data are important, the ability to dream, conceptualise and innovate is what sets the successful and the unsuccessful apart.”

Simone Biles, most decorated U.S. women's gymnast 

“Having ADHD, and taking medicine for it is nothing to be ashamed of, nothing that I'm afraid to let people know.” 

Steve Jobs, founder of Apple and fellow dyslexic

“Everything around you that you call life was made up by people that were no smarter than you and you can change it, you can influence it, you can build your own things that other people can use. Once you learn that, you'll never be the same again."

We want GOV.UK Notify to be the public sector’s first choice for sending messages. Common components like Notify make it easier for government to deliver better digital services.

Here’s how we plan to deliver our vision.

Approaching product growth strategically

In early 2020 the world changed dramatically with the onset of the COVID-19 global pandemic. The pandemic also transformed GOV.UK Notify’s product growth. 

Setting up a service on Notify is quick and easy, with no need for procurement or funding. Because of this, many public sector services used Notify to support their rapid response to the pandemic. Nearly 2,000 services started using our product in 2020. Notify sent all the COVID test results and vaccination reminders, and we saw our transaction volumes increasing by up to ten times.

Whilst this growth was great for our product and for public outcomes, it created a lot of reactive work and support. Our team was under a lot of pressure to keep Notify safe, reliable and performing.  

Since February 2021, growth has normalised and our annual rate of growth is around 30%. So that we can continue to offer a great product for the best value for money, we need to be more targeted and considered in our work.

In the last year we’ve updated our product strategy, which sets the direction for Notify’s growth. The strategy is helping us to prioritise and support choices about our focus. 

Our product strategy has three broad goals:

Goal 1: Scaling our operations to match our growth

When we first started this work, our operations weren’t scaled effectively to support the growth from the past few years. For example, we had to support users to complete simple manual tasks. This distracted us from being able to focus on the truly value-adding product work. 

To achieve this goal we will: 

1. Let users set their own branding for emails and letters (read our blog post).
2. Allow users with special organisational permissions to make services for their organisation live.
3. Make it easier for users to find answers that they need by transforming our product documentation. 
4. Replatform our applications to a new hosting provider as GOV.UK PaaS is getting decommissioned by the end of this year.
5. Allow users to complete basic tasks without our support (for example, changing authentication methods or receiving inbound SMS).

We’ve already completed a lot of this work and are starting to see a reduction in support tickets.

Goal 2: Maintaining a great service and improving our features to retain existing users

From engagement with our users and research such as our annual satisfaction survey, we know that overall our users are happy with our product. We are also aware of some improvements that we need to make to our features.

To achieve this goal we will:

1. Give users some more flexibility and features when sending emails and letters with Notify. This includes things like better support for our Welsh users, additional formatting and components, sending out emails from outside of the gov.uk domain.
2. Help our users better understand their use of Notify by improving our reporting and data capabilities.
3. Support our users, especially those who might not have access to content design to write better notifications (read our blog post).
4. Ensure we continue to deliver best value for money by providing the most competitive price to offer SMS notifications that meet our stringent technical, performance and security requirements. We never make any profit from sending messages, and we spend a lot of effort and time in getting the best deal from the mobile telecommunications market. Getting this right is as important as ever in the context of growing wholesale prices for text messages.

Work on this mission is also underway and we hope to be able to share more about some of the exciting improvements to our notifications soon.

Goal 3: Growing our product proposition to attract new users

Whilst we’ll continue to support our steady organic growth, we also want to extend Notify’s reach to services whose needs Notify does not currently meet. In particular - and in line with GDS’s wider strategy - we want to meet the needs of high-volume central government teams and those users with low or no access to Digital, Data and Technology (DDaT) capabilities. 

As part of this mission we might do mini-discoveries and explorations into new functionality like integrations with other channels, more complex two-way communication, as well as integrations with other GDS products, such as GOV.UK Forms.

We may find there is little or limited value in some of these improvements. If so, we will stop and focus on something else.

Get involved

Annex

This table shows the growth in number of services and organisations on Notify since 2016:

The Government Digital Service exists to make digital government simpler, clearer and faster for everyone. To do this, we need strong data foundations to help us make better evidence-based decisions about our digital services.

One key tool we use to understand the users of GOV.UK and other government digital products is Google Analytics (GA). Google is replacing the current version of GA, Universal Analytics or UA, with a new version, GA4. We’ve already talked about our preparations for upgrading to GA4, and the technical side of our implementation. Now, we want to open up access to our data to others across government and to share more information on the specifics of our approach.

Here we’ll explain:

• what we’ve done already
• what we’ve learned so far
• how to sign up for early access to the GOV.UK GA4 data and tracking schema 

What we’ve done already

In September 2022, we began tracking a small number of high priority events (a data point that is collected to track user behaviour on our website e.g. a button click) in GA4, alongside UA.

Since then, we have continued to develop our GA4 tracking schema (a framework detailing what data we will collect and how it will be structured) and have now defined, built and deployed a large number of our medium priority events. These cover user interactions with elements of GOV.UK such as tabs within pages and accordion components.

We’ve also been comparing our UA and GA4 data sources and using GA4 data in reports and dashboards. This has helped us:

• understand how the two data sources are different
• discover issues with the GA4 data
• explore the feature differences

We’ve audited our current GA accounts, properties, and views, and have started thinking about how to rebuild these to best meet our users’ needs given the constraints of the new tool.

Recently, we’ve also been looking into how we would use GA4 for the Single Data Environment. More on this to follow!

What we’ve learned so far

How we deploy tracking updates

We have developed an agreed process involving both developers and performance analysts. Using Google Tag Manager’s versioning and environments capabilities has enabled us to control which events are available in which environment (for example, testing, production). This has meant we can test the function and usability of each individual piece of tracking before it goes live.

How we can work with GA4 data

The numbers of events we are recording in GA4 approximately match those in UA (within a few percent). However, definitions of key concepts such as sessions have changed, making comparison between the two data sets difficult.

Limitations of the GA4 data collection 

Character limits on the data that we send to GA4 affect our ability to collect data. GA4 also groups events in batches when sending them, which causes issues plotting user journeys and comparing UA and GA4 data.

Limitations of the GA4 interface 

Unlike in UA, you can’t have multiple reporting views in GA4 — sub-properties, the most comparable feature, cost more and do not have the same functionality.

Also, issues with the reports and explorations — for example, limited column widths, date ranges not updating — mean that the interface currently does not meet our needs. In particular, the number of unique values we are collecting means large amounts of our data is currently grouped into one line and labelled (other) in the GA4 interface and in Looker Studio.

Limits on API usage

Some tools, such as Looker Studio, rely on the GA4 API to retrieve data. Strict API quota limits hamper our ability to use additional tools for reporting and analysis.

We’re still learning

Due to the limitations of GA4, we’ve often had to rely on querying the raw GA4 data in BigQuery (Google’s data warehouse where they send raw GA data) to get the information we want. This comes with its own potential problems. Changes to the data structure mean that our daily data tables are almost 20% larger in GA4, so we expect storage costs to increase. We hope that query costs may decrease if we take advantage of the more granular row level data to be more specific with our queries.

GA4 is still under development, so we are having to constantly relearn and reevaluate how we can best use GA4 and our GA4 data. New features, such as the recently released expanded datasets, may alleviate some of the issues mentioned above.

Opening up access

We’re keen to open up access to our GA4 data and to share our schema and Data Protection Impact Assessment (DPIA) with other government departments.

We’d like you to test and feedback on the data we’re collecting, so that we can check our implementation meets our users’ needs. Providing visibility of what the new data will look like will also help the migration of any reports and tools departments have which rely on GOV.UK GA data. We’d love to learn from other departments’ experiences, and hear how others are managing the issues we’re facing with GA4.

We also hope departments may be able to learn from our approach and use our schema or our DPIA to support their own GA4 implementations. Increasing standardisation would create efficiencies across government, and hopefully save departments a lot of time and resource.

The Government Digital Service (GDS) exists to make the user experience of the government simple, consistent and welcoming for everyone. To do this, GDS builds, iterates and maintains digital tools and platforms that are part of everyday life in the UK; platforms like GOV.UK - the online home of government information and services.

GOV.UK is an integral part of our national infrastructure. Millions of people visit daily for everything from registering a death to paying their taxes.

But, we know GOV.UK can do more - and must do more - for its users. This is why we have growth at the heart of our plans. We want to grow our product offer, grow our user base, and grow our team.

Why does growth matter? It’s about making sure more people are getting better outcomes from government. We want to make it quicker and easier for users to access information and services, in formats and channels of their choice, helping them to receive the full support that they are entitled to.

So in this blog post I’m going to set out our strategy for GOV.UK until 2025.

Where GOV.UK is now 

Before we look ahead, let’s explain where we are now. 

Since we launched 10 years ago, GOV.UK is now the established digital interface between government and the public. It’s well used - with over 28 billion page views since go-live - and is consistently among the most-visited websites in the country. It's extremely well known according to YouGov, and the signature design and logo is instantly recognisable as the trusted source of government information and services. GOV.UK has become truly essential to living, working and studying in the UK.

Over that time, we’ve experimented with providing a more interactive service on GOV.UK, for example: 

• we built (and subsequently retired) the Brexit Checker, used over 5.6 million times, which returned tailored guidance for individuals and business about what they needed to do to prepare for Brexit, and allowed them to subscribe to updates about future changes that may impact them
• during the pandemic, more than 700,000 questions were submitted through gov.uk/ask for the COVID-19 press conferences; and a postcode checker, which allowed people to find lockdown rules in their area, received 32 million unique page views

These case studies show that by providing a more proactive, interactive and relevant GOV.UK user experience, we can help people to get the right support from government, and help government to communicate and deliver services to people. 

What more GOV.UK can do 

We want to:

• grow GOV.UK to reach people when and where they need government information
• make GOV.UK more proactive in helping people
• evolve to match user expectations for new technologies

How does this translate into delivery for the next year or so? Below are our programme priorities:

1. Develop a GOV.UK app
2. Explore whether emerging technologies can help users when interacting with GOV.UK, for example to find information more easily
3. Develop our presence on social channels, such as YouTube
4. Create new content types and expand use of existing formats such as video
5. Update our homepage and site search
6. Improve the user experience around specific, targeted journeys
7. Evolve our content operating model
8. Reduce the complexity of our publishing tools
9. Expand and update GOV.UK brand guidelines

What's next?

GDS is adopting a new, ambitious and exciting direction for GOV.UK, and we know it will be challenging to deliver, but it's what government needs and what users expect.

It will involve trying new things. Like any good product team, we should never stand still. We're going to experiment, test our ideas and iterate quickly, using data and evaluation to measure the impact on users. 

It will also involve revisiting product decisions we've made in the past. A decade is a lifetime in tech and some of the choices made at the birth of GOV.UK may no longer be the right ones.

We also know this can’t be done within a single team. Within GDS, we’re working closely with colleagues on GOV.UK One Login for Government to make sure we’re delivering an even more useful experience for the growing number of signed-in users. This includes aligning our app work with the GOV.UK One Login programme’s identity checking app. And, looking across government, it’s only through partnerships with departments that we’re able to provide a consistent and high-quality user experience on GOV.UK, so we’ve created this strategy by engaging with cross-government stakeholders and will continue to partner with them every step of the way.

There’s a lot of information in here, and while this is my first blog post, it definitely won’t be my last. We’re going to be shortly updating our public roadmap over on Inside GOV.UK to show our new workstreams, along with a deep dive into how we created the strategy and data behind it; and I’ve committed to quarterly updates on this blog to give more details. 

It's now been over 18 months since I became the SRO of GOV.UK One Login so I thought it was high time I provided an update on our progress lest you think we’ve been twiddling our thumbs. 

Nothing could be further from the truth here in the GOV.UK One Login programme. We've been heads down - pedal to the metal - building, testing, iterating and operating our product set over the past six months. 

We’ve now got 8 services onboard, multiple routes through our product to prove an identity to a medium GPG 45 profile, an Apple and Android app, so it’s been go, go, go…

Shipping product

When I last wrote I talked about how we were shaping our backlogs and working to understand what it was that users and departments needed. Since then our focus has shifted to building and shipping the things on the list, while continuing to run a robust and resilient live service. 

Our numbers have swelled in every area, we have three times the number of teams working in parallel on features and incremental improvements to existing functionality. We’re making over 500 separate releases a month to production through our automated pipeline. We’re running continuous discovery sessions in parallel to development and have conducted over 100 end user research sessions in the last six months alone. 

We now have 8 services using GOV.UK One Login, these are:

• Disclosure and Barring Service: Request a basic check
• Department for Business and Trade: Licensing for International Trade and Enterprise 
• Driver and Vehicle Standards Agency: Apply for your vehicle operator licence
• Social Work England: Register to be a social worker
• HM Revenue and Customs: Government Gateway services (via the identity checking app)
• HM Land Registry: Sign your mortgage deed
• Ofqual: Apply to be an advisor
• Modern Slavery Unit: Modern slavery statement registry

We have driving licence and passport based routes for users, with both forms of ID able to be used with our identity checking app or via a web journey. Our identity check apps have now been downloaded over 2 million times and we’ve issued over 1.5 million verified identities since our first users last summer. More than 815,000 individuals now have GOV.UK One Login accounts that can now be accessed using authenticator apps as one of their factors. We’re accepting overseas telephone numbers for One Time Passwords and we’ve got Welsh Language versions. 

Tackling inclusion 

That’s pretty impressive but we haven’t stopped there. We’re really focussed on broadening out the reach of our service to users who don’t have a driving licence or passport as well as those with low digital skills. This is why we’re going to open a face-to-face route for identity verification in the summer, which will allow users with additional forms of identification to prove their identities to a medium GPG 45 profile and give us our first asynchronous journey!

We’re also just finalising our plans for our first ‘low’ profiles and our first ‘medium profile’ that won’t need users to have any form of photo ID at all. More on these exciting new features as they mature. 

Goodbye GOV.UK Verify 

The pace at the end of 2022 was determined at least in part by the desire to retire our predecessor service GOV.UK Verify. I’m happy to say that before Christmas GOV.UK Verify issued its last identity and in March this year, the last service using it migrated away and it has now been decommissioned. We often talk about new features and products -  so it's good to be able to celebrate the brilliant efforts, teamwork and collaboration that enabled us to elegantly switch something off. GOV.UK Verify successfully proved the identity of 10m end-users and facilitated 18.6m transactions across 27 services using seven identity providers. I hope everyone who worked on Verify is proud of their contribution.

The lessons learned from GOV.UK Verify formed a core part of the thinking around the way we have set up and been running GOV.UK One Login. We’ve been focussed on increasing user success rates from the GOV.UK Verify baseline since we first went into beta last summer. Our user-level data shows that for our most popular service we’re achieving an identity verification rate of 62% which is a huge achievement for a platform that didn’t exist 12 months ago. This is over 10% higher than the target rates agreed with the service before it onboarded. 

So with one last glance backwards we are now firmly focussed forwards and on the exciting roadmap we have in place for 2023-24 and beyond. 

Over the next 18 months we’ll be onboarding the vast bulk of government services. As part of that many of you reading this will create GOV.UK One Login accounts in that period. Please let us know what you think when this happens by dropping us a line to govuk-one-login@digital.cabinet-office.gov.uk.

Until next time, Natalie.

GOV.UK Pay can now take repeated payments and allow your users to pay for things in instalments without needing to pay manually every time. This feature works with cards and future payment types like open banking.

We’ve also made it a lot easier for services to get real-time information about payments by introducing webhooks.

Taking recurring payments

GOV.UK Pay was initially built to take single payments. As it became more popular, services told us they needed the ability to take repeated payments for things like licences and subscriptions.

We've now built functionality to allow services to create 'agreements', which allow payments to be taken on a schedule without the user being present, if they have agreed to it. To find out more about how this works, you can read our technical documentation about recurring payments.

If you're interested in taking recurring payments, whether you use GOV.UK Pay at the moment or not, get in touch with the Pay team.

Webhooks make real-time data easier

Webhooks automatically send information when something happens, rather than requiring services to make API calls. Webhooks have a wide range of potential applications, such as integrating with existing enterprise resource planning software. This can make accounting and financial management a lot simpler and more effective.

For example, you could set up GOV.UK Pay to send a webhook when a payment is received along with the relevant information (like an invoice number) to allow it to be automatically registered and processed by your financial software. Using webhooks means that your finance systems are always up to date without the need for staff to manually input data. This can help with managing refunds.

If your organisation has lots of services that use GOV.UK Pay, webhooks mean you can use a single financial system to keep track of all your payments. You could also use webhooks to set up a system to send an email using GOV.UK Notify when something happens with a payment.

This is all faster than existing bank transfer methods. You can read more about webhooks in our technical documentation. If you already use GOV.UK Pay you can start using webhooks in your service immediately. Contact the team if you need further support.

The future of recurring payments and webhooks

We want to develop recurring payments and webhooks further, particularly to make it easier for services to set up recurring payments without using our API. Later this year we will also be expanding beyond card payments, looking at future payment types like open banking.

Thanks to our partners

We've developed these new features in partnership with the Environment Agency and Kent County Council. The Environment Agency are introducing recurring payments for rod fishing licences which will make renewing your licence a lot easier for users. Kent County Council are using it for the Trading Standards Checked service to allow vetted businesses to pay in instalments so they can appear on the fair trader directory.

This blog post is an experience report of conducting performance tests, highlighting the value and importance of conducting this kind of test. 

Performance testing has given us confidence that our webhooks mechanism can deal with large transaction volumes. Along the way, it helped us identify problems that had not shown up in other tests. As a result, what we deliver will be more efficient and robust. 

GOV.UK Pay is used by hundreds of services across the public sector to take payments. The GOV.UK Pay team recently released our Recurring Card Payments feature into production. An important part of this feature is the webhooks mechanism. Webhooks allow services that use Pay to receive messages about payment events, for example when a payment succeeds or fails. They can be used in a variety of ways, for example to trigger messages to a service's paying users or connect with enterprise resource planning (ERP) software to help automate accounting.

Preparing for the release, we were confident that the mechanism worked from a functional point of view, as we’ve had webhooks running in our sandbox environment for several months. However, we had never tested them under any load, and we wanted to be sure that the mechanism would cope when partner services start subscribing to webhooks.

How webhooks work

There are several stages in the payment lifecycle, from the point when a payment is created through to authorisation and capture. As a payment progresses through this journey, these events are published to an SQS queue. 

The webhooks mechanism has 2 main responsibilities:

Subscription to the event stream

There are several stages in the payment lifecycle, from the point when a payment is created through authorisation or rejection - and authorised payments proceed to capture. As a payment progresses through this journey, these payment events are published to an SQS queue. The Webhooks app polls the SQS queue for new events, as illustrated in Diagram 1 below.

The payment events are checked against the Webhooks database for a match against existing webhook subscriptions. Services that use Pay can subscribe to receive updates on particular payment events (e.g. authorised, rejected) relating to their transactions with customers. If the new payment event matches an existing webhook subscription, the relevant payment details are retrieved from Pay’s Ledger database, and then the webhook message containing a summary of that event is then written to a delivery queue (a table in the Webhooks database).

Diagram 1 - Subscription to the event stream

Message sending process

The Webhooks app also polls the delivery queue table in the Webhooks database and sends the corresponding webhook message to the relevant service (by sending a POST request to the callback URL specified by the service in the webhooks subscription). If delivery fails, a retry is scheduled.

Diagram 2 - Message sending process

If you're interested to find out more about how webhooks work, you can take a look at our source code which is coded in the open.

Our way of working

We organised ourselves into sub-teams to complete the Recurring Card Payments implementation. The Webhooks sub-team that carried out the performance testing included engineers with varying degrees of experience. We decided to employ a ‘mobbing’ or ‘ensemble programming’ approach to the work. This approach is particularly well suited to work where the team needs to quickly learn and explore new technologies, techniques or problem areas together. This would allow us to benefit from a range of perspectives when planning the tests and responding to any problems, and to introduce less experienced team members to performance testing techniques.

We carried out these tests over a couple of days. We were working remotely, so we collaborated by screen-sharing over a number of calls, with breaks as needed to clear heads and catch up with other tasks. We rotated responsibilities, with one person running the tests, another monitoring the logs, others observing. We used a Slack channel to keep in touch outside of the calls.

Our test setup

We used our performance testing environment, which is a replica of our production environment where we can safely test high levels of traffic. We used an AWS lambda to simulate the service’s URL that the webhook messages could be posted to.

The performance tests are controlled by the following parameters:

• the rate of payment transactions
• how long the test runs for
• the number of test accounts used in parallel in the simulation
• the number of webhooks subscribed to our simulated payment events
• the response delay of the simulated service URL

We monitored the system under test by collecting logs and metrics in our log aggregation tool, Splunk. The Splunk dashboard showed us various performance measures but the one we were most interested in was the "time to send". “Time to send" is the time between the creation of a webhooks message ("Add message to delivery queue" in diagram 1) and the message sending process picking it up to send ("POST send message" in diagram 2).

We examined this metric because it could be calculated from existing logs, would be heavily influenced by our application’s performance, and examining how this metric was affected in different scenarios could reveal any existing limitations of our design and provide opportunities for optimisation. The time that elapsed subsequently between sending the webhook message and receiving a response was something we could configure by modifying the AWS lambda function that controlled our simulated service URL. 

A simple first test

For our first test, we sent 10 payments per second, for a single test account with one webhook. This meant that for each payment, there should be one webhook message sent. We also configured the simulated service URL to take 100ms to respond.

Chart 1 - Time to send webhook messages throughout the first test

All the webhook messages were sent successfully, and in good time. So far so good!

We then started to apply a bit more pressure, increasing the transaction frequency, having multiple accounts with different webhook subscriptions. We saw the time-to-send increasing, but no alarm bells were ringing yet.

Problem 1 - Database lock

One big unknown in all of this is the speed of response from the service confirming that the webhook message had been received. We can give some guidance to services about how quickly we would like them to respond, but it is largely outside of our control, so we had been wondering what would happen if there was a significant delay in the response.

We increased response time from 100ms to 2 seconds and reran the test.

Straight away our dashboard showed us that there was a problem.

Chart 2 - Time to send webhook messages when there is a 2-second response delay

The chart illustrates that webhook messages were still being sent for more than 30 minutes after the payment events. Clearly our system was not sending out webhook messages fast enough to keep up with new payment events.

At this point we really benefited from the ‘mobbing’ approach to this task. With 4 engineers looking at the same data, we discussed possible causes for this behaviour. We then took a break from the call, following up some different avenues individually, keeping in touch via Slack.

One of the team looked at the logs to investigate concurrency in the sending of messages. They found that messages were only being retrieved from the database at a rate of one every 2 seconds, with threads operating sequentially rather than concurrently, as if the threads were in a queue to send messages and only one could get out at a time. Were threads being blocked from querying the database? This led us to look at how each thread retrieved the webhook messages from the database.

Our database queries use the Hibernate framework (behind the scenes, Hibernate turns our commands into SQL queries to interrogate the database). The query to get the next webhook message included the following statement:

.setLockMode(LockModeType.PESSIMISTIC_WRITE)

Applying a PESSIMISTIC_WRITE lock means that while the lock is in place other transactions cannot modify that row of data. This was employed here to allow the thread carrying out the transaction to read the data, send the webhook message, and on receiving the response from the subscribing service, to update the delivery_status field to confirm that it has been dealt with, without the possibility of other threads attempting to act on the same row of data. The code further specified:

LockOptions.SKIP_LOCKED

This ensures that when a row is locked, the thread trying to read it will skip to the next row, rather than to wait for the lock to be released. Again, this was appropriate, and necessary for concurrency.

So what was going wrong?

We were telling Hibernate the right thing to do, but the next step was to make sure that Hibernate was actually doing it.

We debugged the procedure, logging the raw SQL query generated by Hibernate. It turned out that Hibernate was not including the ‘SKIP_LOCKED’ command in the SQL query that it generated. If a thread trying to access a locked row was not being instructed to skip past it, it would have to wait until the row was unlocked before it could continue - which would explain what we’d observed.

But why was Hibernate ignoring the SKIP_LOCKED command? With a bit of research we found out that the default dialect of PostgreSQL that Hibernate was configured to use did not include SKIP_LOCKED, and so it just ignored the command. We specified a compatible dialect (version 10), and verified that the resulting SQL query did now include SKIP_LOCKED. Now multiple threads could pick up webhook messages concurrently.

Problem 2 - http request bottleneck

With the first problem apparently diagnosed and fixed, we reran the tests, hoping to see our system sending webhook messages in step with the flow of events. This would be demonstrated by a nice flat line on our chart.

Chart 3 - Time to send webhook messages when there is a 2-second response delay (after fixing the SKIP_LOCKED issue)

This shows an improvement, but the backlog is still building up, so something still was not right.

Digging deeper, we found that the number of webhook messages sent per minute had dramatically increased from 30/min to ~445 per minute, but there was a bottleneck in the process that was sending the messages as HTTP requests to our lambda endpoint.

With a bit of arithmetic, we worked out that of the ten threads running, half were always waiting for 2 seconds after picking up a message before they could make the http request to send it.

To remedy this, we introduced a connection pool manager to our webhook message sending service, configuring it to allow sufficient available connections to support the number of threads. 

Problem 3 - Queue ordering

Chart 4 - Time to send webhook messages when there is a varying response delay (after fixing the available connections issue)

Now the rate of sending webhook messages had increased to ~1670 per minute, and everything is going well until towards the end of the test, when we see the final webhook messages taking much longer to send.

We puzzled over this for a while. When looking at the time to send (peaking at over 5 minutes), we realised that this must mean that there were webhook messages that had been created at the beginning of the test run that were not actually sent for several minutes. Why would the messages that had been created first not be retrieved from the database for sending until much later? Logically this meant that the query was not picking the oldest messages first….and then the penny dropped. Our query was just picking the first webhook message that it found, but without any ordering. So with a heavy load of webhook messages, it was quite feasible that a webhook message could be left unattended for a long time.

We introduced an ORDER BY statement to the query to ensure that messages were selected for sending in ascending order of creation date/time.

A finely-tuned machine

Subsequent testing showed that the factor now limiting the rate of sending messages was the maximum number of threads available to process send attempts on.

We increased the number of threads to 20, kept our response time varying between 100ms and 2000ms, and simulated 20 payments per second, which is considerably higher than anything we’ve ever experienced before, or that we would anticipate any time soon.

The final tests confirmed that our webhooks mechanism was able to cope with this load, with a time-to-send generally under 100ms and not exceeding 450ms. 

Chart 5 - Time to send webhook messages after all fixes in place, under significant load

What we will be looking out for

We also discussed what monitoring we should put in place. The results of the early tests had shown us that in certain scenarios, a backlog of webhook messages can develop which means that the overall time-to-send will increase until the backlog is resolved. It’s hard to foresee the circumstances in which this might happen, but we need to be on the lookout for it.

We were already logging the ‘time to send’ for each webhook message, so we decided to build some alerts based on saved searches in Splunk which run at regular intervals. When the ‘time to send’ for an individual webhook message, or the average time-to-send over a period of time, exceeds a certain threshold, an alert will be triggered. We’re adding some guidance to our team manual so that everyone on the team knows what to do if the alert fires.

What we learned

Performance testing brought several problems to the surface that had not been revealed by unit tests, integration tests or the exposure of the mechanism to services using our sandbox environment.

Once each problem had been identified, the fixes were relatively straightforward. Without these fixes, the webhooks mechanism might have struggled with increasing demand, potentially causing knock-on problems for services and damaging confidence in our product. Diagnosing the problems and implementing the fixes would be much harder when dealing with live data, under time pressure, than in the performance test environment where we have more freedom to experiment.

We can never be 100% certain that a solution is able to handle every possible scenario that might occur in the live environment, but by repeating the same tests with increasing load and simulating a range of response times from services, we’re confident that the webhooks mechanism is robust and copes effectively with a high volume of transactions.

When we launched Version 13 of the Prototype Kit we said that the updates made creating and updating prototypes easier for all users, particularly those getting started with the Kit. 

Now that Version 13 is live we wanted to understand more about the people who are not currently using the Kit or are very new to the Kit and what impact the changes in Version 13 have had on new and existing users. 

As a team we are focusing on improving the Kit in line with user needs. In this blog, we will explain how user research has helped us understand what impact the changes in Version 13 had on both new and existing users.   

User research is an essential part of creating and improving digital services. It helps our teams in government to understand the needs, behaviours and preferences of our users, so that we can inform design decisions and effectively improve our services.  

First-time user experience

Our first step was to map out the user journey, looking at what we knew already and where we needed more information.

We mapped the journey phases as awareness, consideration, preparation, creation, completion and support.

To expand our knowledge of the ‘first-time user’ user group we set up research to capture their first interactions with the GOV.UK Prototype Kit, with no preconceived ideas. 

This round of research focused on people who had never used the Prototype Kit before and had very limited coding experience or knowledge. This also included users who were self-declared novices, and had not used Version 13 before but may have used previous versions of the Prototype Kit. 

In the research session, we asked the users to install the Kit and follow the tutorials on their own and to create a prototype.  

Our aims were to:

• Capture the initial first impressions 
• Base findings not on previous experience 
• Identify any reasons why first-time users stop using the Kit or don’t get through the installation process 

Version 13 is much more accessible 

Our user research participants  really liked Version 13 of the Kit because: 

• “It’s much more accessible to use than Figma or Adobe”
• It’s easy and quick to use
• It’s interactive and “much more like the real thing”

This positive feedback came from the ability for the users to quickly learn how to use the Kit, and to get to the point of a working prototype. The users felt like they had accomplished something and demonstrated high feelings of satisfaction. 

The results also showed that Version 13 of the Prototype Kit increased engagement with the product through gamification, with one user stating that Version 13 “feels more like a game, whereas before it felt like a school lesson.” 

Users also noted that Version 13 was more visual which makes it easier to understand cause and effect, as when you make changes on the new user interface, you can see the changes happen live on your prototype. 

You don’t need to know how to code

Our research demonstrated that users do not need to know how to code to use the Kit. Once the users had moved past the installation phase, the research showed that users could competently create a simple prototype, and have it up and running fairly quickly. New users built up confidence quickly during the creation stage of the prototype, due to the step by step instructions they received in the tutorials.   

Although users initially found the installation process “confusing”, “overwhelming” and “difficult”, once using the Kit the positivity increased dramatically. The users soon followed by saying they were “making some progress” and that their “feelings of engagement and comfortableness are improving”.  One user talked about enjoying using the Kit in the browser and that it was “doing the whole magical thing in the background.” Later in the journey, users highlighted pain points when identifying and fixing errors, mainly due to not being as comfortable with the coding language used.

A visual representation showing users’ self reported happiness at each stage of the user journey and highlighting key pain points for users at each stage.

Having some coding experience is still an aid, particularly when working on more complex prototypes, but this should not be a barrier to use. Actually, users felt that the Kit could be used to help designers to learn more about coding. 

Changing the Kit to meet the user needs

Our research has informed several recommendations for the continuous improvements to the Prototype Kit. Our list of improvements so far include:

1. Streamline the installation and set up process, removing steps and automating some of the process. 
2. Simplify the navigation of the Prototype Kit, making it easier for new and existing users to find the information they need.  
3. Improve the content of the error information provided, making it easier for users to fix any errors that occur.  
4. Add visuals and content to the tutorials and guides, to help users understand more about coding and the software they are installing.  

What’s next 

This research showed us that the improvements we have made so far with Version 13 have been beneficial, and have produced a much better user experience. By implementing our recommendations we believe that the GOV.UK Prototype Kit will continue to improve, particularly for new users and for those with no or little coding experience.