With low call volume you cannot detect small lifts, so stop trying. Instead, test only large changes, alternate them over time in a planned pattern, measure a cleaner outcome than "any call", and set your decision threshold by what a wrong decision would cost. Every number below is illustrative.
The formula in our sample size article shows why: at a 4% baseline, detecting a 25% lift needs more than 13,000 visitors. A small-town furnace company may see that in a year. This piece is about what to do instead, and it complements our CRO experiments service rather than replacing it.
Change the outcome you count
Raw calls are noisy because many are not leads at all: existing customers, suppliers, sales pitches, wrong numbers. Counting them adds variance that has nothing to do with your page.
A cleaner metric is the qualified call, defined before the test, for example:
- from inside the service area
- longer than a set duration
- about a service you sell (installation, replacement, repair or maintenance)
Google Ads already supports a version of the duration filter. Its call conversion documentation explains that a call counts only if it lasts at least the minimum length you set, and its website call tracking help describes Google forwarding numbers that show which campaigns and keywords produced each call. Those numbers are only available in certain countries, so confirm eligibility in your account.
Removing non-lead calls usually lowers your count but raises the signal per call. Fewer, cleaner data points can beat more, dirtier ones.
Use a switchback schedule
When you cannot split traffic, split time. Run version A for one week, version B the next, and keep alternating for at least eight weeks. Then compare the A weeks with the B weeks.
A planned schedule matters. An illustrative eight-week pattern might be A, B, B, A, B, A, A, B. The scrambled order means a slow upward trend in demand does not automatically favour whichever version ran last.
Rules that keep a switchback honest:
- Alternate at fixed boundaries (for example, Monday at 00:00 local time).
- Do not change ads, budgets or hours during the test.
- Note any cold snap or heat wave, and decide in advance whether those weeks are excluded.
- Compare the same weekdays across versions.
This is weaker than a true split, because week-to-week changes leak in. Read our piece on seasonality controls before relying on it in shoulder seasons.
Set the threshold by the cost of being wrong
The 5% significance convention came from research settings where a false discovery is expensive. A business decision is different. Ask two questions:
- What does a false positive cost? You ship a change that does nothing. If it is a headline rewrite, the cost is close to zero and you can revert it.
- What does a false negative cost? You reject a change that worked. Over a heating season, that could be several lost jobs.
If the change is cheap and reversible, a 20% false-positive risk (80% confidence) can be the rational choice. If it involves a discount, a new written promise to customers or a price display, hold to a stricter standard. Write the chosen threshold into the test plan before launch.
Review calls with a scoring rubric
When numbers are thin, structured listening fills the gap. If you record calls with consent, score a sample from each version on the same rubric:
| Question | Score |
|---|---|
| Did the caller already know what they needed? | 0–2 |
| Did they mention something from the page (an offer, a photo, a phrase)? | 0–2 |
| Were they inside the service area? | 0–1 |
| Did the call end with a booked visit? | 0–1 |
Have someone who does not know which version the call came from do the scoring. Twenty scored calls per version will not prove a lift, but they often reveal why a page works, which is what the next test needs.
A sample low-volume test plan (illustrative)
- Business: a two-truck heating company with about 40 inbound calls a month.
- Change: replace a generic home page with a page built around one offer (same-week furnace replacement estimates).
- Metric: qualified calls per week, defined as above.
- Design: eight-week switchback, scrambled order, Monday boundaries.
- Threshold: 80% confidence, because the change is easy to reverse.
- Guard rail: booked visits per qualified call.
The output is not certainty. It is a decision you can defend, with a clear record of how it was made. For the measurement side, see measurement setup. For more method pieces, browse the newsroom, or call, text or email to talk a plan through.
Frequently asked questions
Is a switchback test as good as a proper A/B split?
It is weaker, because anything that changes from one week to the next can leak into the result. It is still far better than a single before-and-after comparison, as long as you alternate several times and plan the schedule in advance.
Why would I accept a 20% false-positive risk?
Because the cost of a wrong call is not always symmetrical. If a change is cheap, easy to reverse and has no lead-quality downside, being wrong one time in five may cost less than waiting a year for certainty.
What counts as a qualified call?
You define it. A common version is a call from inside your service area, longer than a set duration, about a service you sell. Write the definition down before the test so nobody adjusts it afterwards.