Confidence Rating System¶
How we assess and communicate certainty.
Last Updated: 2026-01-27
The Scale¶
| Rating | Name | Criteria | Can Change? |
|---|---|---|---|
| ★★★★★ | Established | Multiple replications, peer-reviewed, mechanism understood | Rarely |
| ★★★★☆ | Strong | Replicated, peer-reviewed, minor gaps | Sometimes |
| ★★★☆☆ | Promising | Peer-reviewed, awaiting replication | Often |
| ★★☆☆☆ | Preliminary | Single study, speculative mechanism | Frequently |
| ★☆☆☆☆ | Speculative | Theoretical only, no direct evidence | Very often |
What Each Level Means¶
★★★★★ Established¶
Requirements: - Multiple independent replications - Published in peer-reviewed journals - Mechanism understood (or multiple converging mechanisms) - Cross-cultural or cross-context validity - No serious unaddressed critiques
Examples in our theory: - Social connection reduces mortality (Holt-Lunstad meta-analysis) - HPA axis stress buffering (Cohen & Wills)
What would change it: - Major methodological flaw discovered - Large contradicting meta-analysis - Mechanism shown to be artifact
★★★★☆ Strong¶
Requirements: - At least one replication - Peer-reviewed publication - Plausible mechanism proposed - Minor gaps or caveats acknowledged
Examples in our theory: - Love universality (Jankowiak & Fischer) - CTRA inflammatory response (Cole) - Oxytocin cardiovascular protection (Light)
What would change it: - Replication failure - Alternative mechanism shown more likely - Major confound identified
★★★☆☆ Promising¶
Requirements: - Peer-reviewed publication - Awaiting independent replication - Mechanism plausible but not confirmed
Examples in our theory: - Xenon isotope anesthesia anomaly (Li et al. 2018)
What would change it: - Successful replication → upgrade to ★★★★☆ - Failed replication → downgrade to ★★☆☆☆
★★☆☆☆ Preliminary¶
Requirements: - Single study or preprint - Mechanism speculative - Needs substantial further work
Examples: - Most new findings start here
What would change it: - Peer review + replication → upgrade - Contradicting evidence → downgrade or remove
★☆☆☆☆ Speculative¶
Requirements: - Theoretical only - No direct empirical evidence - Interesting but unproven
Examples: - L0.5 quantum-classical interface (theoretical spec)
What would change it: - Any empirical support → upgrade - Shown to be impossible → remove
Rating Changes Over Time¶
Ratings are NOT permanent. They change as evidence accumulates.
Upgrade triggers: - Successful replication - Mechanism confirmed - Cross-cultural validation - Meta-analysis support
Downgrade triggers: - Failed replication - Methodological critique - Contradicting evidence - Alternative explanation confirmed
Current Ratings Summary¶
| Claim | Rating | Last Changed |
|---|---|---|
| Integration creates complexity | ★★★★★ | Initial |
| Social connection → mortality | ★★★★★ | Initial |
| Love is universal | ★★★★☆ | Initial |
| Alignment → flourishing | ★★★★☆ | Initial |
Honest Uncertainty¶
We distinguish: - What we KNOW (★★★★★) - What we're CONFIDENT about (★★★★☆) - What's PROMISING (★★★☆☆) - What's PRELIMINARY (★★☆☆☆) - What's SPECULATIVE (★☆☆☆☆)
This is not weakness - it's intellectual honesty.
Confidence ratings are living assessments, not permanent labels. ✨