Measurement
Part of Business licences measurement: how to judge licensing services in 2027
Checking licensing metrics by sampling records: state, evidence and handover tests
Test England licensing metrics before each review by sampling case records for state accuracy, evidence quality, condition ownership and handover recovery.
A metric is only worth reporting if a sampled record can confirm it. Before each review, pull a small sample behind every headline measure and ask whether the case file supports the number. Where it does not, the definition or the data is wrong, and the team's reporting dashboard needs repair.
What to take away
- Sample the records behind each headline measure before the review, not after the report is published.
- Test state accuracy by looking for the supporting document, not for the status label.
- Test handover recovery by asking a deputy to find the authority, version, decision, conditions and next date.
- Pull one positive, one negative and one unknown record for every headline measure.
- Keep the sample and its corrections with the report so a later editor can see what the rate meant.
What a sample can and cannot show
A sample tests whether a metric measures what it claims, not how large the number is. It cannot replace a clear denominator or a written definition of each event. If the sample contradicts the count, restate any earlier report that the error materially affected before building on it.
Sampling also separates two kinds of failure. Some records fail because the evidence is poor, and some fail because the event was defined loosely. Each needs a different fix, so note which one you found before anyone is asked to change their work.
Four measures, four sample tests
Each measure needs its own sample rule. The table gives a sample, a failure condition and the evidence to keep for the four measures that rest on record quality.
| Measure | Sample | A record fails when | Keep as evidence |
|---|---|---|---|
| Evidence quality | One case stopped before submission and one stopped by an authority request afterwards | Names, sites, activities, dates, declarations or attachments disagree | The gate that found the defect |
| State accuracy | Records marked submitted, acknowledged, decided or conditions implemented | A green status has no supporting document | The document, and the time taken to repair the status |
| Condition ownership | A condition from a real decision, not a generic list | There is no named operational owner or no evidence of the control | The owner's name and the control evidence |
| Handover recovery | A case the deputy has not handled | The deputy cannot find the authority, version, decision, conditions or next date | Success, time and missing elements |
Separate defects found before submission from authority requests afterwards. The first reflects an internal gate, while the second may have several causes and needs case review. The quality checklist keeps those two groups apart.
Check the route field first
Every sampled record should show a confirmed authority route. The Department for Business and Trade's Find a licence service can support discovery, but warns that its UK catalogue may not be complete, so a search completion is not a confirmed route.
Confirmed routing rate
- Activity and English location
- Responsible authority identified
- Authority confirmed
- Count as confirmed route
- Unresolved cases stay in denominator
A record with no confirmed route cannot pass any later test. Keep such cases in an unknown column rather than removing them, because dropping difficult journeys would only raise the rate without improving anything.
Test state accuracy
A status label is a claim, and the case file is the evidence. Open each sampled record marked submitted, acknowledged, decided or conditions implemented, and look for the document behind the label. A green status with no supporting document fails.
Measure the time to repair status errors as well as their number. The sample then shows how quickly a wrong status is corrected as well as how often one appears.
Test handover recovery
Ask a deputy to locate the authority, submitted version, decision, conditions and next date for one case they have not handled. Record success, time and the missing elements. This tests operational resilience rather than staff memory.
In the animal activity licensing process, for example, a local authority must advise each licence holder that they need to renew their licence 3 months before it is due to expire. A deputy who cannot find the renewal date has found a real gap.
Sample before every review
Run the same routine before each monthly review. It takes one record of each kind per headline measure and asks the same questions each time.
- Choose one positive, one negative and one unknown record for every headline measure.
- Ask whether the underlying evidence fits the event definition.
- Ask whether the resulting action still makes sense.
- Record the named reviewer and every correction.
- Restate earlier reports if the error was material.
Ask another analyst to reproduce the count from the underlying cases, and keep that reconciliation with the report. Show the source system, data owner, refresh date and last definition change beside each figure. A threshold should trigger case review, not automatic blame.
Turn a failed sample into action
The GOV.UK Service Manual describes metrics as measurements that tell you how well something is performing, which is why a failed sample matters. Give each affected metric a threshold and an action owner, and retire any measure that no longer influences work.
Do not use approval rate as a simple team score. Authorities decide individual cases, and case mix changes. Report any such outcome with definitions, exclusions and attribution limits, and read the wider measurement and reporting guide for how those limits fit a full reporting cycle.
Before you act
- Choose one positive, one negative and one unknown record for each headline measure.
- Check each sampled record for a confirmed authority route.
- Look for the supporting document behind every status label.
- Ask a deputy to run a handover test on a case they have not handled.
- Keep the sample, corrections and reconciliation with the report.
Common questions
How many records should the sample include?
The routine uses one positive, one negative and one unknown record for every headline measure. If any sampled record fails, widen the sample for that measure and review the definition before the number is used again.
Who should run the sampling?
Someone other than the person who built the report. Ask another analyst to reproduce the count from the underlying cases, then check whether the evidence fits the event definition and whether the resulting action still makes sense.
What happens when a sampled record fails?
Note whether the evidence or the definition failed, correct the record and record the reviewer's name. If the error was material, restate any earlier report it affected. Do not treat one failed record as a verdict on the team.
