Forecasts and experiments

Two tools for looking forward and checking a change

Forecast projects where clicks are heading from the property’s own history. Experimentation compares groups of pages to see whether a change moved them. A forecast gives you a baseline to hold actual results against; an experiment gives you a structured before-and-after comparison. Neither proves a cause on its own.

Forecast: how the baseline is built

1. Load history. Use “Load 16 months” to fetch Search Console clicks, or upload a CSV with the columns date,sessions. The page notes that the numbers are Search Console clicks, not sessions; export GA4 to CSV for real sessions. 12 to 36 months of history gives the best result.
2. Choose granularity. Weekly gives more points; monthly needs fewer. At least 13 weekly or 6 monthly periods are required, and the first and last partial periods are dropped.
3. Remove seasonality. Seasonal indices compare each period with a centred moving average; an index above 1.0 marks a period above average.
4. Fit a trend. Linear, polynomial and logarithmic models are fitted, and exponential and power models too when all values are positive. “Auto (Best Fit)” picks the highest R².
5. Project and re-apply seasonality. The trend continues forward and the seasonal indices are put back. The default horizon is 52 weeks or 12 months.

Growth assumptions and revenue

The chart shows actual values, the baseline, the growth forecast and the trend, and the details panel lists the selected model, all models ranked by R² and the seasonal indices. Export as CSV, HTML or PNG.

What a forecast cannot do

• It extends the past. A Google update, a migration or a change of business breaks the pattern.
• A high R² is not accuracy. It measures fit to history, not future error.
• Revenue is arithmetic. It multiplies the forecast by the conversion and order values you enter.

Save the forecast as a baseline and compare it with actual results regularly; a gap means conditions changed.

Experimentation: compare groups of pages

1. Define groups. Upload a CSV with url and group (for example A, B, C, D), or let the tool suggest groups by URL path, or create a random A/B split among eligible pages.
2. Set thresholds for the split: minimum clicks, minimum impressions and maximum position. Only pages that meet them are included.
3. Preview, name and save. Optionally set a start date. Groups and the start date are stored separately for the project.
4. Read the result. The chart shows each group’s growth, current period against the previous equal one, and a table lists clicks, previous clicks, growth, impressions, CTR and position.

Reading experiment results

• Growth score. The winner is the group with the largest relative click growth against the equal previous period, not the one with the most traffic.
• Significance. The page shows a z-score, a p-value and a 95 percent interval. A p-value below 0.05 is a statistical signal, not proof of cause.
• Limit. This is an observational comparison of Search Console data, not a randomized controlled experiment. Seasonality, Google updates and other site changes can explain a result.

A careful way to test a change

1. Split pages of a similar kind and change only one group.
2. Annotate the date of the change.
3. Wait for full equal periods after the change.
4. Check that the groups behaved alike before the change.
5. Treat a small p-value as a lead and repeat on another set before rolling out.

Forecasts and experiments, answered

Does the forecast use sessions?

From Search Console it uses clicks. For sessions, upload a GA4 export.

Which trend model is chosen?

“Auto” takes the model with the highest R² after seasonality is removed.

Is a significant p-value proof that my change worked?

No. It is a statistical signal in an observational comparison.

Continue

Next, see advanced analysis and dates and comparisons.