Measure what the product changes for people

It is easy to count activity in a digital product. Page views, generated responses, completed forms, and feature usage all produce numbers. The harder question is whether those numbers explain a useful change for the people the product serves.

I want measurement to begin with that change. What should become easier, more reliable, or more understandable? What would we expect to observe if the product were helping? What else could explain the same observation?

Those questions are practical safeguards against collecting a dashboard of numbers that look positive but do not guide a decision. They also help a team describe its results accurately. A product can have a meaningful outcome without a dramatic percentage attached to it, and a promising number still needs context before it becomes evidence of improvement.

Write the intended change in ordinary language

Imagine an operations team that receives requests through several channels. A product improvement brings those requests into a shared queue. Counting requests in the new queue would show that the system is being used. It would not yet show whether requests receive better follow-through.

A more useful starting point is the intended change: people responsible for a request can see what needs action and who owns the next step. That description suggests what to investigate. Are requests assigned? Are unresolved items visible? Can staff identify the next action? What happens when ownership changes?

At Hoopes Vision, my documented work included researching patient and staff workflows, designing an internal CRM and operations platform, coordinating engineers, and conducting user acceptance testing with staff. The work addressed inquiries, referrals, and operational follow-through. I describe that contribution through the work and team outcomes the evidence supports.

That experience is part of why I care about choosing measurements before making claims. A number becomes useful when the team knows what it represents and how it relates to the intended change.

The Spera case study gives another example of the distinction. It describes my research, design, prioritization, and release responsibilities, along with the team's delivery of working software. Those are supported outcomes. A claim about adoption or revenue would need additional evidence, with a defined period and a reliable way to attribute the result.

Understand the starting point

A baseline is an account of how the relevant task works before the change. It does not have to be a perfect historical dataset, but its limitations should be clear.

For the hypothetical request queue, the team might review the current process, observe how staff identify unresolved work, and examine available records. It should distinguish what is measured consistently from what depends on memory or incomplete reporting.

If reliable historical information is unavailable, I would say so and establish a starting measurement the team can maintain going forward. I would avoid reconstructing an impressive comparison from incompatible records. A smaller, credible comparison is more useful for deciding what to improve.

The GOV.UK performance metrics guidance connects measures with service purpose and recommends combining data with user research. I use that as a useful principle: the measurement plan should reflect the question the team needs to answer.

Separate adoption, task success, and organizational results

These measures describe related parts of a product, but they should not be treated as interchangeable. Adoption indicates that people use something. Task success concerns whether they can complete the intended work. Organizational results concern the wider change the business hopes to achieve.

For the request queue, active use could be encouraging while staff still struggle to find the right item. Better task completion could be real while a wider outcome remains affected by staffing, demand, or another process. Those relationships deserve investigation rather than an automatic success claim.

I would choose a small group of measures that help explain the path. Usage could show whether the new workflow is entering daily work. Task observations could reveal whether people understand it. Operational records could show what happens after the interaction ends.

The definitions matter. A completed form may be different from a complete request. An assigned owner may be different from an owner who has accepted responsibility. The team should agree on those meanings before turning the events into a report.

Keep context attached to the numbers

Suppose a hypothetical team sees more requests completed after a release. That is worth investigating. It is not enough on its own to establish that the release caused the change.

The team might have changed staffing, introduced a new process, or received a different mix of requests. Reporting might also have improved, making previously invisible work countable. A product manager should ask about those possibilities before choosing an explanation.

I would look at the relevant period, user groups, task types, and process changes. I would also ask the people doing the work what changed from their perspective. Their explanation is evidence with its own limits, but it can help identify questions the quantitative view misses.

This is where collaboration with analysts or other specialists matters. Product leadership needs enough understanding to frame a useful question and interpret a result responsibly. It should draw on the expertise needed for stronger analysis when the decision calls for it.

Measure AI assistance across the whole task

AI introduces especially attractive activity measures. The product can report how many drafts it generated or how quickly it returned an answer. I would want to know what happened next.

Did the user keep the draft? How much did they change? Could they identify errors? Did the feature help them complete the task? Did review effort move to another person? These questions connect the AI interaction with the workflow it is meant to improve.

For a hypothetical summary feature, generating a response quickly would be one observation. A more complete assessment would include source checking, correction, and the quality of the decision the summary supports. A feature that saves drafting time but creates difficult verification work may need a different design.

I would also review failures individually when their consequences matter. An average can hide a small group of cases that the product handles poorly. The team needs examples alongside summary measures so it can understand which changes would be useful.

Turn the review into a decision

Measurement earns its place when it changes what the team does. A review should lead to a practical judgment: continue, adjust, investigate, narrow, or stop. The evidence may justify a modest improvement rather than a broad conclusion.

I would record what the team observed, the interpretation it considers most plausible, and the uncertainty that remains. Then I would connect the finding with a specific next step. If users cannot identify ownership in a queue, the next step may be a design change and another observation rather than a new feature.

The same discipline matters when describing work publicly. I want to distinguish what I contributed, what the team delivered, and what was measured. Where a reliable metric is unavailable, the product story can still explain the problem, the decisions, and the observable outcome.

Good measurement helps a team see its product more clearly. It gives people a reason to change direction, support an investment, or question a comfortable assumption. That is the standard I would use for a product dashboard: does it help us make a better decision about the work?