Methodology

NutriVouch displays two distinct signals on every substance card. They answer different questions and use different methodologies. This page explains both, plus the legacy Evidence Score that still exists in our API for transparency.

1. Confidence Tier — how well-researched a substance is

The colored Wi-Fi-style bars on every card. Formula-derived from peer-reviewed study counts. Answers: "How confident is science about this substance as a whole?"

How it's computed

For each substance we count meta-analyses, systematic reviews, and randomized controlled trials (RCTs) that passed our quality gate and were human studies. Then we apply multi-factor OR logic — a substance qualifies for a tier if it meets ANY of the listed conditions:

Strong evidence

(≥50 meta-analyses AND ≥200 RCTs) OR ≥100 meta-analyses OR ≥400 RCTs

Moderate evidence

(≥10 meta-analyses AND ≥50 RCTs) OR ≥30 meta-analyses OR ≥150 RCTs

Emerging evidence

≥10 total studies OR ≥1 meta-analysis

Insufficient evidence

below all of the above

We use OR logic (not AND) because a substance with 100 meta-analyses but "only" 150 RCTs (e.g. a well-reviewed but older compound) is still clearly Strong. Same goes for substances with massive RCT counts but fewer aggregations.

2. Effectiveness Score — how effective a substance is for a specific goal

The 0–10 number next to each goal on a substance card. Editorial, AI-curated against scientific consensus. NOT derived from study counts. Answers: "How effective is this substance FOR THIS SPECIFIC GOAL, according to scientific consensus?"

Honest disclosure: these scores are AI judgment calls against published literature, then reviewed and validated. We chose this approach deliberately — a substance with 500 studies can still score 1/10 for a goal where none of those studies showed benefit. Raw study count tells you depth of research; effectiveness tells you whether the research found the substance actually works for that purpose.

Effectiveness rubric (0–10)

9–10

Gold standard

This substance IS the treatment of choice — strongest evidence, recommended by evidence-based guidelines. (e.g. melatonin → sleep, creatine → muscle growth, iron → iron deficiency)

7–8

Strong

Multiple meta-analyses support clinical use. Recommended by evidence-based guidelines. (e.g. ashwagandha → stress, saffron → mild depression)

5–6

Moderate

Some clinical support. Works for some populations or in specific contexts. Not first-line but reasonable.

3–4

Weak

Theoretical mechanism or limited clinical evidence. Small trials, mixed results, or indirect effects.

1–2

Minimal

Marketed for the goal but evidence is very thin or non-existent.

Not applicable

No plausible mechanism or zero evidence. Don't recommend this pairing.

•Validated against meta-analyses, not blog posts or marketing claims.
•Bias toward under-scoring. If unsure between 5 and 6, we pick 5.
•Reserved 9–10 for substances where there's strong consensus they ARE the treatment.
•Popular ≠ proven. Heavy marketing doesn't inflate scores.
•Generated in bulk by parallel AI agents during initial population, then spot-validated. We update when scientific consensus materially changes.

3. Study quality gate

Only studies that meet these criteria contribute to any score (Confidence Tier, Effectiveness, or legacy Evidence Score):

•Human studies only — animal and in vitro studies are excluded.
•Sourced from PubMed — the world's largest peer-reviewed biomedical database.
•Substance keyword must appear in title or abstract (not just MeSH tags, which can be inconsistent).
•Pipeline-level passed_quality_gate flag — set during deterministic extraction; checks for valid structured data.
•Goal relevance check — for per-goal counts, the study must be tagged with that goal based on its content.

4. Study type hierarchy

Different study types carry different weight in the scientific community. This hierarchy is used internally (and visible in the legacy Evidence Score formula below).

Meta-Analysis

Combines results from multiple studies — strongest evidence type.

Systematic Review

Comprehensive review of all available studies on a question.

Randomized Controlled Trial (RCT)

Participants randomly assigned to treatment or control — reduces bias.

Cohort Study

Observational study following groups over time — useful but not causal.

Case-Control Study

Compares groups with and without a condition retrospectively.

5. Legacy Evidence Score (0–10)

Exists in the API for transparency and historical compatibility. NOT shown as the primary visual signal anywhere on the site — Confidence Tier replaced it as the main user-facing badge.

Exact formula

1.For each qualifying study, base points = hierarchy weight (10 / 9 / 7 / 4 / 3).
2.Sample size bonus: +1.0 if sample_size ≥ 1000, +0.5 if ≥ 100, otherwise 0.
3.quality_score = average of (base + bonus) across all qualifying studies.
4.volume_factor = min(1.0, log10(study_count) / 2) — penalizes substances with very few studies.
5.Final score = quality_score × volume_factor, capped at 10.0.

Example: a substance with 100 RCTs (avg sample 1500, so base=7, bonus=1) and 20 meta-analyses (avg sample 5000, so base=10, bonus=1) gives quality_score ≈ 8.5, volume_factor = min(1.0, log10(120)/2) ≈ 1.0, so final = 8.5.

When viewing a goal page, the legacy Evidence Score is recomputed using only studies tagged to that goal — same formula, narrower input.

6. Verify any of this yourself

Every single study we count is linked directly to its PubMed entry. Open any substance page → "All studies" → click any study row → you land on the original abstract on PubMed. If you think we've miscounted or misclassified a study, the methodology is transparent enough that you can audit it. Issues and corrections welcome.