MedBenefit — Methods & Evidence

Every adjustment in the model, the math it uses, and the literature that justifies it. Written to be argued with.

The problem

Why the trial's number is not your patient's number

A randomized trial reports an average effect in a selected population. Applying that summary number to an individual fails in predictable ways (Kent & Hayward, JAMA 2007 [1]; the PATH Statement, Ann Intern Med 2020 [2]; Rothwell's external-validity critique, Lancet 2005 [3]):

MedModel makes those four corrections explicitly, in a fixed order, and shows its work as a step-by-step waterfall so each adjustment can be inspected — and challenged.

The data

Two datasets, one model

MedModel runs on a merged evidence base with explicit provenance:

A lift layer (js/services/lift.js) converts any database entry into engine form. An outcome published as {RRR, NNT over T years} implies, in its own trial population, ARRT = 1/NNT and a control-arm risk R₀T = ARRT/RRR, hence an annual control hazard −ln(1−R₀T)/T and HR = 1−RRR. The engine then individualizes exactly as it does for deep entries. Where a deep entry exists for the same drug-indication pair, it takes precedence; everything else is labeled "lifted from meds.kevinkeet.com" and carries class-level defaults for what the upstream rows don't record (time-to-benefit ramp, in-trial adherence ~85%, approximate trial mean age) — the defaults table with sources is at the top of lift.js. DOAC stroke effects are published upstream versus warfarin; the lift composes them against no-treatment via warfarin's RRR of 64% (Hart 2007): 1−RRRnet = 0.36 × HRvs warfarin.

What deliberately does not lift: symptomatic and replacement medicines (responder rates like "50% pain reduction" are not preventable events), and outcomes with no baseline anchor. Those render as an honest non-model card — felt benefit versus harms and burden — rather than being forced through prevention math.

The model

Structure: a discrete-time competing-hazards model

For each scenario the engine simulates monthly cycles over the chosen horizon. While alive and event-free, a patient faces two hazards: the index outcome (e.g., major vascular event) and death from other causes. Cumulative incidence is accumulated cause-specifically:

CIFevent(T) = Σt≤T S(t−1) · pevent(t) · (1 − pother(t)/2)
S(t) = S(t−1) · (1 − pevent(t)) · (1 − pother(t))

ARR(T) = CIFuntreated(T) − CIFtreated(T); NNT = 1/ARR. This is the standard cause-specific-hazard construction (a discrete Fine-Gray-style cumulative incidence), which automatically produces the geriatric result: as competing mortality rises, achievable absolute benefit falls even at identical event risk.

1 · Baseline risk: the patient's, not the trial's

The first substitution replaces the trial control-arm risk with the individual's predicted risk, applying the trial's relative effect to it. This is the central recommendation of the PATH Statement [2] and is justified empirically: relative effects are approximately constant across baseline-risk strata for the drug classes modeled here —

Baseline risk sources, in order of preference: a validated multivariable equation where one exists (Pooled Cohort Equations for primary-prevention ASCVD [11], implemented exactly and verified against the guideline's published worked examples; CHA₂DS₂-VASc with the Friberg 2012 Swedish-cohort untreated event rates for AF [12]); otherwise the trial control-arm rate anchored to a user-selected risk stratum with transparent multipliers. Renal function uses the race-free CKD-EPI 2021 creatinine equation (Inker, NEJM 2021 [59]), verified against hand-computed values; an eGFR below 30 auto-sets the CKD flag, harm scaling, and contraindication checks.

Bleeding on antithrombotics: HAS-BLED, dynamically

For anticoagulants, static bleeding NNHs are replaced by a patient-specific estimate: the HAS-BLED score (Pisters, Chest 2010 [60]) maps to an annual major-bleed rate on warfarin (1.13 → 12.5%/y across scores 0–5), multiplied by an agent factor from the pivotal vs-warfarin trials (apixaban 0.69, dabigatran 0.94, rivaroxaban ~1.04, edoxaban 0.80; intracranial factors ~0.4–0.7, DOAC class RR 0.48 [61]). The excess versus no antithrombotic is taken as ~55% of the on-drug rate (anticoagulation roughly doubles major bleeding; est., stated on the card), with ~12% of warfarin major bleeds intracranial. The uncontrolled-hypertension criterion is approximated as absent, and labile INR is proxied by poor expected adherence for warfarin only — both noted limitations.

2 · Competing mortality and prognosis

Other-cause mortality follows a Gompertz hazard h(a) = A·eB·a — the classical law of adult mortality, with rates doubling roughly every 8 years — anchored by sex to the NCHS US life tables (q65, q85) and validated in the test suite against published remaining life expectancies (2022 tables: e65 17.5 y men / 20.2 y women) [13].

The five overall-health levels scale this hazard (×0.55 to ×2.9), calibrated so that "excellent / average / poor" reproduce approximately the top-quartile / median / bottom-quartile life expectancies of the Walter & Covinsky individualized-prevention framework (JAMA 2001) [14]. The frailty multiplier magnitude matches meta-analytic frailty mortality hazards (HR ≈ 2.4 [15]) and the survival spread demonstrated for gait speed (Studenski, JAMA 2011 [16]). Selected high-lethality conditions multiply further, using published all-cause mortality hazards: heart failure ≈1.8 [17], moderate-severe dementia ≈2.5 [18], home-oxygen COPD ≈3 (NOTT/MRC [19]), eGFR <30 ≈3 (Go, NEJM 2004 [20]), metastatic cancer ≈4 (SEER-based estimate). The product is capped (×8); this is deliberately a prognosis sketch, not a validated index — for real prognostication use ePrognosis (Lee [21]/Schonberg [22] indices).

3 · Time-to-benefit

Treatment effect ramps in linearly from zero to the full hazard ratio over a class-specific period fitted to the survival-meta-analysis time-to-benefit literature (the displayed TTB quotes the source directly):

TherapyTime to benefitSource
Statins (primary prevention)0.8 y to 1/500; 2.5 y to 1/100 MACEYourman, JAMA IM 2021 [23]
BP lowering (age ≥65)0.9 y to 1/500; 1.7 y to 1/200; 3.0 y to 1/100 (stroke)Ho, JAGS 2022 [24]
Bisphosphonates12.4 mo to 1/100 nonvertebral fx; 20.3 mo to 1/200 hip fxDeardorff, JAMA IM 2022 [25]
SGLT2i in HFrEFsignificant by day 28 (DAPA-HF); day 12 (EMPEROR-Reduced)Berg, JAMA Cardiol 2021 [26]; Packer, Circulation 2021 [27]
Anticoagulation in AFeffectively immediateno formal TTB analysis; KM curves separate from the outset [28]
Intensive glycemic control~6–9 y to microvascular benefitUKPDS 33 KM curves [29]; Huang decision model, Ann IM 2008 [30]

When median survival (from step 2) is shorter than the therapy's TTB, the app raises the Holmes flag [5]: the patient is likely to carry burden and harm-risk without living to collect the benefit. The same logic underpins deprescribing tools (STOPPFrail [31]) and is supported by trial evidence that stopping statins in advanced illness does not worsen survival and improves quality of life (Kutner, JAMA IM 2015 [32]). Lee's cancer-screening lagtimes (BMJ 2013 [33]) established the framework.

4 · Adherence: diluting the trial effect honestly

Under proportional hazards with an on/off drug effect, a patient covered for a fraction f of days experiences an average hazard of h·(1 − f·(1−HRbio)). The trial's ITT hazard ratio already embeds the trial's own adherence ftrial, so:

RRRpatient = (fpatient / ftrial) · RRRtrial,  capped at the implied fully-adherent effect

Anchors: trial adherence ~85–90% (HPS in-trial compliance ≈85% [34]); real-world adherence to chronic preventive therapy 43–78%, typically ~50–65% (Osterberg, NEJM 2005 [35]); half of elderly statin starters stop within a year and only ~1 in 4 maintain PDC ≥80% at 5 years (Jackevicius [6]; Benner, JAMA 2002 [36]). The presets (90% / 65% / 40%) span that literature.

What we deliberately do not do: observational adherence–mortality gradients (e.g., good-vs-poor adherence all-cause mortality RR 0.55 [37]) are heavily confounded — adherence to placebo carries nearly the same association (OR 0.56; Simpson, BMJ 2006 [38]). MedModel therefore uses only the proportional-exposure dilution above, not the observational gradient. This is the conservative, mechanism-based choice for the efficacy-effectiveness gap (Eichler, Nat Rev Drug Discov 2011 [39]).

5 · Representativeness, stated — not faked

Where a patient falls outside the trial population, no defensible multiplier exists — so the model does not invent one. Instead it grades the mismatch (in the trials / extrapolated / outside the evidence) from each medication's actual enrollment (age ranges, exclusions) plus global rules (older than the oldest participants; frailty [4]), states the reason in plain language, and — where subgroup evidence exists — quotes it (e.g., CTT 2019: relative effect similar across ages overall but inconclusive for primary prevention over 75 [40]; SPRINT's exclusion of diabetes, stroke, dementia, and nursing-home residents; ELDERCARE-AF supporting anticoagulation benefit in frail ≥80 [41]; ASPREE showing primary-prevention aspirin ≥70 causes net harm [42] — included in the library deliberately as the cautionary case).

The other side of the ledger

Harms

Harms are modeled as absolute excess rates (vs no treatment) from trial safety tables and pharmacoepidemiology, with three deliberate differences from the benefit side:

Harm rates also pass through the competing-mortality machinery (a bleed cannot happen after death), so harm counts are honest per-1000 figures over the same horizon as benefits. Where an excess-vs-nothing rate had to be derived (e.g., anticoagulant bleeding vs no antithrombotic, from AVERROES arms [54]), the derivation is stated on the card and marked "est."

Treatment burden

Burden — the workload of being a patient — is scored 0–10 from a transparent rubric, not presented as probability. It operationalizes minimally disruptive medicine (May, Montori & Mair, BMJ 2009 [55]) and the Treatment Burden Questionnaire tradition (Tran, BMC Med 2012/2014 [56]); the classic demonstration is Boyd's guideline-perfect 79-year-old taking 12 drugs on a 19-dose day (JAMA 2005 [57]).

FacetPoints
Dosing frequency1×/d +0.5 · 2×/d +1.5 · ≥3×/d +2.5
Monitoring (labs + extra visits/year)1 +0.5 · 2–3 +1.0 · ≥4 +1.5
Administration constraints+0.8 each (fasting rituals, sick-day rules, INR checks…), cap 2.5
Cost tier$ +0 · $$ +0.75 · $$$ +1.5
Interaction potentiallow +0 · moderate +0.5 · high +1.0
The consultation layer

Goals of care, preferences, and the recommendation ladder

The decision layer is ported intact from the original meds.kevinkeet.com application and re-powered by the engine's numbers:

Where arithmetic meets clinical judgment

Six refinements from the gut-check review

Comparing the model's outputs against clinical judgment on the example patients surfaced six places where the arithmetic was missing a mechanism a clinician uses implicitly. All six are now in the model:

Comparing across a regimen

The regimen-review ranking

The regimen view scores every therapy applicable to the patient on one scale so they can be compared. Per 1000 patients over the chosen horizon:

net = Σoutcomes ARR·woutcome·1000 − Σharms excess·wharm·1000 − burdentier·H·1000·0.2

The event counts (ARR, harm excess) come from the competing-hazards engine — fully individualized. The severity weights w are the QALY-utility estimates from the meds.kevinkeet.com engine (death 1.0, disabling stroke 0.7, ICH 0.6, hip fracture 0.5, MACE 0.35, major bleed 0.15, severe hypoglycemia 0.05…), and the burden term uses its annual decrements (low 0.01 / moderate 0.03 / high 0.06 QALY), tempered ×0.2 so burden breaks ties rather than overruling hard outcomes. Three honesty rules: therapies matching a contraindication are scored but held out of the ranking; symptomatic/replacement medicines are listed separately, unranked; and every row exposes its arithmetic (benefit, harm, burden components). The weights are point-estimate utilities, not measured preferences — the ranking is a screen that orders the conversation, and each row links to the full workup.

Trust, but verify

Calibration and validation

The repository ships a test suite (medmodel/test/engine.test.js, run with node) that asserts:

Limitations — read these

Bibliography

References

  1. Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients. JAMA 2007;298:1209–12.
  2. Kent DM, et al. The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement. Ann Intern Med 2020;172:35–45.
  3. Rothwell PM. External validity of randomised controlled trials. Lancet 2005;365:82–93.
  4. Van Spall HGC, et al. Eligibility criteria of randomized controlled trials published in high-impact general medical journals. JAMA 2007;297:1233–40.
  5. Holmes HM, et al. Reconsidering medication appropriateness for patients late in life. Arch Intern Med 2006;166:605–9.
  6. Jackevicius CA, et al. Adherence with statin therapy in elderly patients with and without acute coronary syndromes. JAMA 2002;288:462–7.
  7. Cholesterol Treatment Trialists' (CTT) Collaboration. Efficacy and safety of more intensive lowering of LDL cholesterol: meta-analysis of 170,000 participants in 26 randomised trials. Lancet 2010;376:1670–81.
  8. CTT Collaboration. The effects of lowering LDL cholesterol with statin therapy in people at low risk of vascular disease. Lancet 2012;380:581–90.
  9. Ettehad D, et al. Blood pressure lowering for prevention of cardiovascular disease and death: systematic review and meta-analysis. Lancet 2016;387:957–67.
  10. Blood Pressure Lowering Treatment Trialists' Collaboration. Pharmacological blood pressure lowering for primary and secondary prevention of cardiovascular disease across different levels of blood pressure. Lancet 2021;397:1625–36.
  11. Goff DC Jr, et al. 2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk. Circulation 2014;129(Suppl 2):S49–73 (Pooled Cohort Equations, Appendix 7).
  12. Friberg L, Rosenqvist M, Lip GYH. Evaluation of risk stratification schemes for ischaemic stroke and bleeding in 182,678 patients with atrial fibrillation. Eur Heart J 2012;33:1500–10.
  13. Arias E, et al. United States Life Tables, 2022. Natl Vital Stat Rep (NCHS).
  14. Walter LC, Covinsky KE. Cancer screening in elderly patients: a framework for individualized decision making. JAMA 2001;285:2750–6.
  15. O'Donovan M, et al. Frailty and all-cause mortality: meta-analysis (frail vs robust HR ≈ 2.4). 2022.
  16. Studenski S, et al. Gait speed and survival in older adults. JAMA 2011;305:50–8.
  17. Kaiser Permanente Southern California HF cohort: 1-year all-cause mortality HR 1.70 (HFrEF 2.43). 2023.
  18. Meta-analyses of dementia and mortality (adjusted HR ~2.4–2.7); median survival ~5–6 y from diagnosis. Lancet Healthy Longev 2021.
  19. Nocturnal Oxygen Therapy Trial Group. Ann Intern Med 1980;93:391–8; MRC Working Party. Lancet 1981;1:681–6.
  20. Go AS, et al. Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Engl J Med 2004;351:1296–305.
  21. Lee SJ, et al. Development and validation of a prognostic index for 4-year mortality in older adults. JAMA 2006;295:801–8.
  22. Schonberg MA, et al. Index to predict 5-year mortality of community-dwelling adults aged 65 and older. J Gen Intern Med 2009;24:1115–22.
  23. Yourman LC, et al. Evaluation of time to benefit of statins for the primary prevention of cardiovascular events in adults aged 50 to 75 years: a meta-analysis. JAMA Intern Med 2021;181:179–85.
  24. Ho VS, Cenzer IS, Nguyen BT, Lee SJ. Time to benefit for stroke reduction after blood pressure treatment in older adults: a meta-analysis. J Am Geriatr Soc 2022;70:1558–68.
  25. Deardorff WJ, et al. Time to benefit of bisphosphonate therapy for the prevention of fractures among postmenopausal women with osteoporosis: a meta-analysis. JAMA Intern Med 2022;182:33–41.
  26. Berg DD, et al. Time to clinical benefit of dapagliflozin in patients with heart failure with reduced ejection fraction (DAPA-HF). JAMA Cardiol 2021;6:499–507.
  27. Packer M, et al. Effect of empagliflozin on the clinical stability of patients with heart failure and a reduced ejection fraction (EMPEROR-Reduced). Circulation 2021;143:326–36.
  28. Hart RG, Pearce LA, Aguilar MI. Meta-analysis: antithrombotic therapy to prevent stroke in patients who have nonvalvular atrial fibrillation. Ann Intern Med 2007;146:857–67.
  29. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin (UKPDS 33). Lancet 1998;352:837–53.
  30. Huang ES, et al. The effect of comorbid illness and functional status on the expected benefits of intensive glucose control in older patients with type 2 diabetes: a decision analysis. Ann Intern Med 2008;149:11–19.
  31. Lavan AH, et al. STOPPFrail: consensus validation. Age Ageing 2017;46:600–7 (v2: Curtin D, et al. Age Ageing 2021;50:465–71).
  32. Kutner JS, et al. Safety and benefit of discontinuing statin therapy in the setting of advanced, life-limiting illness. JAMA Intern Med 2015;175:691–700.
  33. Lee SJ, et al. Time lag to benefit after screening for breast and colorectal cancer. BMJ 2013;346:e8441.
  34. Heart Protection Study Collaborative Group. Lancet 2002;360:7–22.
  35. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med 2005;353:487–97.
  36. Benner JS, et al. Long-term persistence in use of statin therapy in elderly patients. JAMA 2002;288:455–61.
  37. Chowdhury R, et al. Adherence to cardiovascular therapy: a meta-analysis of prevalence and clinical consequences. Eur Heart J 2013;34:2940–8.
  38. Simpson SH, et al. A meta-analysis of the association between adherence to drug therapy and mortality. BMJ 2006;333:15 (the "healthy adherer" effect).
  39. Eichler HG, et al. Bridging the efficacy–effectiveness gap. Nat Rev Drug Discov 2011;10:495–506.
  40. CTT Collaboration. Efficacy and safety of statin therapy in older people. Lancet 2019;393:407–15.
  41. Okumura K, et al. Low-dose edoxaban in very elderly patients with atrial fibrillation (ELDERCARE-AF). N Engl J Med 2020;383:1735–45.
  42. McNeil JJ, et al. ASPREE: effect of aspirin on cardiovascular events, bleeding, and mortality in healthy elderly. N Engl J Med 2018;379:1499–1519 (three papers).
  43. Halvorsen S, et al. Efficacy and safety of apixaban compared with warfarin according to age (ARISTOTLE). Eur Heart J 2014;35:1864–72.
  44. Li L, et al. Age-specific risks, severity, time course, and outcome of bleeding on long-term antiplatelet treatment (Oxford Vascular Study). Lancet 2017;390:490–9.
  45. Sattar N, et al. Statins and risk of incident diabetes: meta-analysis. Lancet 2010;375:735–42.
  46. Preiss D, et al. Risk of incident diabetes with intensive-dose compared with moderate-dose statin therapy. JAMA 2011;305:2556–64.
  47. Wood FA, et al. N-of-1 trial of a statin, placebo, or no treatment to assess side effects (SAMSON). N Engl J Med 2020;383:2182–4.
  48. Herrett E, et al. StatinWISE: series of randomised n-of-1 trials. BMJ 2021;372:n135.
  49. Graham DJ, et al. Incidence of hospitalized rhabdomyolysis in patients treated with lipid-lowering drugs. JAMA 2004;292:2585–90.
  50. Black DM, et al. Atypical femur fracture risk versus fragility fracture prevention with bisphosphonates. N Engl J Med 2020;383:743–53.
  51. Tinetti ME, et al. Antihypertensive medications and serious fall injuries in a nationally representative sample of older adults. JAMA Intern Med 2014;174:588–95.
  52. ACCORD Study Group. Effects of intensive glucose lowering in type 2 diabetes. N Engl J Med 2008;358:2545–59.
  53. ADVANCE Collaborative Group. Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes. N Engl J Med 2008;358:2560–72.
  54. Connolly SJ, et al. Apixaban in patients with atrial fibrillation (AVERROES). N Engl J Med 2011;364:806–17.
  55. May C, Montori VM, Mair FS. We need minimally disruptive medicine. BMJ 2009;339:b2803.
  56. Tran VT, et al. Development and description of measurement properties of an instrument to assess treatment burden among patients with multiple chronic conditions. BMC Med 2012;10:68; English validation BMC Med 2014;12:109.
  57. Boyd CM, et al. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases. JAMA 2005;294:716–24.
  58. SPRINT Research Group. A randomized trial of intensive versus standard blood-pressure control. N Engl J Med 2015;373:2103–16.
  59. McMurray JJV, et al. Dapagliflozin in patients with heart failure and reduced ejection fraction (DAPA-HF). N Engl J Med 2019;381:1995–2008.
  60. Inker LA, et al. New creatinine- and cystatin C–based equations to estimate GFR without race (CKD-EPI 2021). N Engl J Med 2021;385:1737–49.
  61. Pisters R, et al. A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation. Chest 2010;138:1093–100.
  62. Ruff CT, et al. Comparison of the efficacy and safety of new oral anticoagulants with warfarin (meta-analysis of RE-LY, ROCKET-AF, ARISTOTLE, ENGAGE AF). Lancet 2014;383:955–62.
  63. The meds.kevinkeet.com medication database (medeval) — upstream source of the 60-medication catalog, purpose taxonomy, severity weights, burden tiers, costs, and contraindication lists.

← Back to the model