Methodology¶
How we evaluate claims and rate confidence.
Evidence Standards¶
Every claim on this site follows these standards:
1. Peer-Reviewed Sources¶
We prioritize peer-reviewed research from established journals:
| Tier | Source Type | Examples |
|---|---|---|
| 1 | Peer-reviewed journals | Nature, Science, PNAS, PLoS Medicine |
| 2 | Academic preprints | arXiv, bioRxiv, medRxiv |
| 3 | Secondary sources | Wikipedia, news articles |
| 4 | Commentary | Blog posts, social media |
Rule: Claims need ≥2 Tier 1-2 sources for ★★★★+ rating.
2. Confidence Ratings¶
| Rating | Meaning | Criteria |
|---|---|---|
| ★★★★★ | Established | Multiple replications, peer-reviewed, mechanistically understood |
| ★★★★☆ | Strong | Replicated, peer-reviewed, minor gaps |
| ★★★☆☆ | Promising | Initial findings, awaiting replication |
| ★★☆☆☆ | Preliminary | Single study, speculative mechanism |
| ★☆☆☆☆ | Speculative | No direct evidence, theoretical only |
3. Falsification Criteria¶
Every claim includes what would disprove it:
Example: Social Connection Reduces Mortality
Falsification criteria:
- Large-scale RCT showing no mortality benefit from social interventions
- Discovery that correlation is entirely explained by confounders
- Evidence that effect is limited to specific populations
4. Counter-Evidence Tracking¶
We actively track evidence that contradicts our claims:
- Counter-evidence is displayed alongside supporting evidence
- Claims are downgraded when counter-evidence accumulates
- We update immediately when new research emerges
The Knowledge Graph¶
Our claims are stored in a knowledge graph that tracks:
- Nodes: Claims, Evidence, Mechanisms, Gaps, Researchers
- Edges: SUPPORTS, CONTRADICTS, EXPLAINS, BLOCKS
- Temporal: When beliefs changed and why
Correction Process¶
When errors are found:
- Immediate acknowledgment - We don't hide mistakes
- Confidence downgrade - Rating adjusted based on severity
- Changelog entry - All changes documented
- Source update - Primary sources re-verified
What We Don't Do¶
- Cherry-pick - We include counter-evidence
- Overstate - "Suggests" not "proves" for preliminary findings
- Hide uncertainty - Gaps are prominently displayed
- Ignore criticism - All feedback is considered
Philosophical Commitments¶
We are transparent about our assumptions:
- Empiricism - Evidence over intuition
- Falsifiability - Claims must be testable
- Humility - We could be wrong
- Integration - Multiple disciplines inform our view
"The first principle is that you must not fool yourself — and you are the easiest person to fool." — Feynman