8. Hypothesis: Did LLM Rise Cause the Decline? (Devil's Advocate)
We rigorously test whether the observed traffic decline correlates with the rise of LLM tools. This is a devil's advocate analysis - we treat each potential cause separately and let the data speak. Correlation ≠ causation, so we use stationary transforms, first-party behavior, and causal-style time-series tests.
8.1 The Question (Devil's Advocate)
Three competing hypotheses:
- H1: LLM Chatbots (ChatGPT, Claude, etc.) caused the decline - users bypassing search
- H2: Google AI Overviews caused the decline - zero-click SERP answers
- H3: General AI adoption caused the decline - broader market shift + algorithm changes
8.2 The Visual Evidence: Traffic vs LLM Interest (Google Trends)
We compare our traffic index against Google Trends data for LLM tools (ChatGPT, Claude, Perplexity, Gemini, Copilot, AI Overviews) in Austria.
Figure 7a: Side-by-side comparison - Traffic Index (top) vs LLM Interest (bottom). Note the inverse relationship.
Figure 7b: Dual-axis overlay showing traffic (blue, declining) vs LLM interest (red, rising). Level correlation r = 0.73.
Figure 7c: Scatter plot of traffic index vs LLM interest. R-squared = 0.53.
Figure 7d: Individual LLM search interest trends (Google Trends Austria). ChatGPT dominates.
8.3 Methodological Note: Why Level Correlations Are Misleading
Level correlations between two trending series are spurious. To avoid false causality, we analyze YoY % changes (stationary series). This is the peer-reviewable correlation.
Figure 7i: Level correlation (spurious) vs YoY correlation (stationary). Level r = 0.73. YoY r = 0.13 (p = 0.538).
8.3 First-Party Evidence: LLM Referral Behavior
We use GA4 referral sessions from ChatGPT, Perplexity, Gemini, Claude, and Copilot. This is behavioral evidence, not a proxy. Total LLM referral sessions in the study window: 478,142. Post-peak referral trend: RISING.
Figure 7j: Organic traffic vs first-party LLM referrals. Substitution evidence: NO.
Substitution evidence: NO. We only flag substitution if organic falls while LLM referrals rise post-peak.
8.4 H1 Results: LLM Chatbots
On YoY % changes, the correlation shifts from pre-peak to post-peak. Pre-peak YoY r = 0.54 (p = 0.059). Post-peak YoY r = 0.41 (p = 0.184).
Figure 7f: Level-space visualization of the phase shift. Inference uses YoY correlations reported above.
Figure 7e: Correlation heatmap across LLMs and phases. Green = positive, Red = negative.
Figure 7h: Lag analysis - correlation at different time lags. Optimal lag: 6 months.
8.5 H2 Results: AI Overviews (Quantified)
We run Interrupted Time Series (ITS) regression around March 2025 (AI Overviews Austria). This tests immediate level change and slope change.
Figure 7g: GA4 traffic index with ITS fit. Level change: -6.7 (p = 0.219). Slope change: -1.03/mo (p = 0.188).
We validate the same event using GSC Search clicks (subset with Search Console access, n = 6, 2024-10-02 to 2026-01-23).
Figure 7k: GSC Search clicks with ITS fit. Level change: -4.6% (p = 0.730). Slope change: -5.49%/mo (p = 0.159).
AI Overviews impact (quantified): GA4 ITS is not statistically significant. GSC ITS is not statistically significant. The timing still shows the peak in January 2025, before the Austria rollout in March 2025.
8.6 Verdict: What the Data Actually Says
| Hypothesis |
Evidence Strength |
Verdict |
| H1: LLM Chatbots |
YoY correlation shifts; first-party referrals rising |
WEAKLY SUPPORTED |
| H2: AI Overviews |
GA4 ITS + GSC ITS around March 2025 |
NOT PRIMARY CAUSE |
| H3: General AI Shift |
Trend break + multi-factor evidence |
MOST PLAUSIBLE |
Limitations
- GSC provides ~16 months of history; the AI Overviews test is limited to late-2024 onward.
- ITS detects timing alignment and slope changes, not definitive causality.
- LLM referrals are still a small share of total traffic; substitution may be partial.
