Shane Murphy: UK product discussions need measures that answer a real question

A dashboard can contain a large number of figures while leaving a simple question unanswered: did the change help the person trying to use the service? Counting activity and understanding an outcome are related tasks, but they are not interchangeable.

This editorial features Shane Murphy, also known as Shane Ian Murphy, and considers how UK product teams might discuss a proposed service improvement. It is company-origin commentary, prepared with AI assistance. The example is hypothetical. No customer numbers, performance results or completed Global Secure tests are claimed.

Start with the question

Imagine a team revising the explanation on an account-preference page. The aim is to help people recognise which choice they are changing and understand whether the update has been saved. A measure should connect to that aim rather than merely reward more activity on the page.

An increase in page visits could mean more people discovered the feature. It could also mean people returned because they were unsure whether it worked. Without a clear question and a defined unit of measurement, both interpretations can fit the same headline number.

A proposed question might be: can someone complete the intended change and accurately describe the result? That question invites more than one kind of evidence. A completion record may show that the system accepted a change; an observed session may reveal whether the person understood it.

Define the denominator

A percentage is difficult to interpret unless the underlying group is clear. Does completion refer to everyone who opened a page, everyone who began the task or only the people whose connection remained available? Those choices can produce different figures without any change to the interface.

The proposed reporting habit here is to record the definition beside the result. The time period, included attempts and known exclusions belong in the explanation. A number that can only be understood by the analyst who created it is not yet a useful communication tool for the wider team.

The GOV.UK Service Manual's guidance on setting performance metrics emphasises measures with clear meaning. That guidance concerns public services. Its relevance here is an editorial comparison, not evidence that any company product meets a public-service standard or has achieved a particular result.

Combine observation with counts

Suppose a hypothetical test shows that a person reached the final screen but then asked whether the preference had changed. The event record might classify the task as completed. The observer's notes would expose a remaining communication problem.

Neither record needs to cancel the other. Together, they describe the distinction between system completion and user understanding. A team can then decide whether to revise the confirmation message, the earlier explanation or the way the task is structured.

Small research exercises should also be described accurately. Watching a handful of people work through a prototype can reveal problems worth investigating. It does not, on its own, establish how an entire UK population will behave. An honest write-up keeps the scope of the observation visible.

Avoid a metric that rewards confusion

Time spent on a page is a useful example of why context matters. A longer visit might reflect careful reading, difficulty finding an answer or a distraction unrelated to the service. Calling every increase an improvement would turn an ambiguous observation into an unsupported conclusion.

For the proposed account-preference task, the team would need to interpret timing alongside completion and comprehension. The desired outcome is not simply a shorter or longer session. It is a person who can finish the task and understand the result without unnecessary effort.

Keep the decision record short

A practical review note could state the question, define the measure, summarise the observation and identify the next change to test. Recording those four elements makes it easier to explain why the team chose one revision over another.

For Shane Murphy's product discussions, including the planned Global Secure direction towards 2027, this offers a useful discipline before publishing performance claims. Define the question first. Gather evidence that actually addresses it. Describe what that evidence supports, with its limits intact.

Reference: GOV.UK, How to set performance metrics for your service — https://www.gov.uk/service-manual/measuring-success/how-to-set-performance-metrics-for-your-service

Professional information: https://shanemurphy.co.uk/

LinkedIn: https://www.linkedin.com/in/shanemurphy101