Business licence support: attribution methods covering before and after, cohort and process tracing

Attribution asks what contribution a licensing service made to an observed change. It is difficult because applicants, case mix, authority processes and operating decisions also change. The honest result is often a bounded contribution claim rather than a statement that one tool caused approval.

In England, a council licensing authority deciding a premises licence under the Licensing Act 2003 faces the same problem.

What to take away

Before-and-after analysis

Compare a defined measure over equivalent periods before and after a change. This is easy to explain but vulnerable to seasonality, altered case mix and new authority routes. Government Digital Service guidance on using data to improve a service supports baselines and decision-led measurement.

Keep event definitions stable and annotate any break. A shorter internal handling time should not include or exclude authority waiting inconsistently.

Cohort comparison

Compare similar cases exposed and not exposed to the service. Match activity, English location, journey stage and channel. The Department for Business and Trade's Find a licence service shows why activity and place matter to discovery and warns that its catalogue may be incomplete. One cohort may face harder routes than another.

Selection bias remains: people choosing assistance may differ from those completing unaided. Report that limitation rather than presenting a causal estimate automatically.

Process tracing

Follow evidence from intervention to immediate effect. A rewritten activity label may reduce wrong searches; a case register may improve handover recovery. Interviews and observed tasks can test the proposed mechanism. Government Digital Service guidance on planning user research helps connect participant groups and methods to a service decision.

This approach suits small samples but does not supply population prevalence by itself.

Commercial claim check

The Advertising Standards Authority's CAP Code section on misleading advertising covers substantiation, material information and comparisons. A public claim about savings, speed or success needs evidence that fits the wording, population and exclusions.

Choose proportionately

Use before-and-after work for an initial signal, cohorts where comparable groups exist and process tracing to explain plausible contribution. Combine methods when the consequence justifies it. Preserve negative cases and missing data.

When resources are limited, choose the method that answers the decision. If you can examine only 20 cases, use process tracing and report that prevalence is unknown. If you have 200 comparable cases, a cohort comparison may justify the matching work.

Worked contribution example

Imagine that an England-focused guidance page for a premises licence under the Licensing Act 2003 changes its activity wording and council handover. Before-and-after data show an illustrative fall from 20 to 17 returns to the same search per 100 sessions, while observed sessions suggest that participants now recognise the location question.

That combination supports a cautious claim that the change contributed to clearer routing.

It does not show that every applicant reached the correct authority, that the catalogue was complete or that approval became more likely. Check a sample of case records for later confirmation and report the unobserved journeys. A cohort using a different channel may also reveal whether the movement was specific to the page.

Write the public conclusion at the narrowest level supported. Use this template: "The change contributed to [immediate outcome] for [population], based on [method] between [dates]; it does not show [wider outcome]." Preserve the alternative explanations, method dates and analyst. If later evidence contradicts the mechanism, update the conclusion rather than defending the original project narrative.

Have a second analyst challenge the preferred explanation before publication. They should inspect negative cases, changes in case mix and events outside the service's control.

Record why each case was retained or rejected in a versioned methods log (a Google Sheets or Excel tab) with columns for date, method, analyst and negative cases. Date this challenge and keep the reviewed calculation with the publication record.

Do not credit the service with every grant or all revenue after launch. State the immediate outcome measured, the authority boundary and the decision the evidence supports.

Before you act

Common questions

Why is attribution difficult for a licensing service?

Applicants, case mix, authority processes and operating decisions also change, so the honest result is often a bounded contribution claim rather than a statement that one tool caused approval. Before-and-after work gives an initial signal, cohorts suit comparable groups and process tracing explains plausible contribution.

What limits a cohort comparison of licensing cases?

One cohort may face harder routes than another. Match activity, English location, journey stage and channel, then report that limitation rather than presenting a causal estimate automatically.

What should a public conclusion about licensing support claim?

State the immediate outcome measured, the authority boundary and the decision the evidence supports. Do not credit the service with every grant or all revenue after launch, and update the conclusion if later evidence contradicts the mechanism.