Research notebook / Synthetic experiment
Does a better
first screen help?
A report should show the numbers, the assumptions, and the limits—not just the winning headline.
Demonstration data only. No experiment was run. These numbers were invented to show how a shareable research report can be structured. They are not evidence that any product change works.
The example comparison
Illustrative activation event: a visitor creates and previews a first document within 24 hours of assignment.
What the arithmetic says
198 ÷ 600 = 33%.
144 ÷ 600 = 24%.
33% − 24% = 9 percentage points.
9 ÷ 24 = 37.5% relative lift.
Those are descriptive calculations on fabricated counts. They do not establish causality, statistical significance, revenue impact, or real-world retention.
Keep the underlying counts visible
| Group | Visitors | Activated | Rate |
|---|---|---|---|
| A | 600 | 144 | 24% |
| B | 600 | 198 | 33% |
Before this becomes real research
Write the decision rule before collecting results
Define eligibility, assignment, the activation window, the minimum practical improvement, and guardrails. Decide how uncertainty will be evaluated before repeatedly checking results.
Check whether the result survives beyond the first click
A helpful first screen could improve initial activity without improving the second visit. Measure later behavior separately. Do not treat this sample as a forecast.
Reproduce or adapt the brief
The collection includes data.json, analyze.py, and a methodology note. The Python script recomputes descriptive rates locally without external packages or network access.