One of the particular challenges for Digital Performance Analysts in DWP is that most services’ outcomes aren’t actually online. For example, currently, a user applying for a Budgeting Loan can only get as far as sending their application online; their application then has to be processed to check they’re receiving a qualifying benefit, do not owe more than a certain amount to the Social Fund etc. They might be eligible, and receive a Budgeting Loan, but they might not.
Users who get as far as submitting an application but don’t receive the loan have spent time needlessly. Similarly, DWP staff have spent time processing an application that won’t result in a loan. All of this takes place after the submit button has been clicked, and is invisible to the analytics on the service.
Online retailers don’t tend to have this problem; because users pay for their purchases online, analytics can identify those who bought something and those who didn’t. The Enhanced Ecommerce function in Google Analytics captures vital data like which products were purchased, the value of orders etc, and can be easily set up on most online retailing platforms (though don’t try to add it to opencart unless you really like being frustrated!).
Unfortunately, there’s no ‘enhanced Government service’ plugin for Google Analytics!
All’s not lost though; it’s possible to import data from other sources into Google Analytics. This means we can import data on whether applications were successful or not into Google Analytics, as long as we have some way of matching the data on outcomes with online sessions.
We’ve had to be a bit creative about how we match these data sources, because it’s not always straightforward matching admin data with analytics. In the case of Budgeting Loans, we record a unique random character string in Google Analytics for each online session, which we call the session ID. When someone hits the final ‘submit’ button, the session ID is added to the claim details that are recorded in the ‘back end’ system that handles claim and payment details. When the application is processed, the processing staff add either a Y (for successful applications) or an N (for unsuccessful ones) to the end of the session ID field. We run regular scans of the back end system, which pull out all the session IDs. We can then upload to Google Analytics the outcome for each online session where the user reached the end-point of the digital journey.
This means we can identify and create segments of users whose applications were successful, and those whose weren’t. We can then look at the behaviour of users who submit an application but aren’t awarded the benefit, to see if there’s a way we can let them know they’re not eligible sooner in their journey, for example. For services where we’re undertaking marketing, we can also measure which channels are most effective at delivering high-quality traffic (ie users who submit an application and end up being awarded the benefit/grant) to the service.
If we don’t link online behaviour and outcomes, we’re not looking at the whole journey, and there’s a risk of optimising the service for clicks on the submit button rather than the service’s actual objective. That’s taking a very partial view of what success is, and could lead to more time (and therefore taxpayers’ money) being spent processing claims from people who aren’t eligible for the benefit or grant.