Replicating health economic models

04 Aug 2017 -

Due to time or budget constraints, it’s often easier to try to replicate a cost-effectiveness model than it is to produce an entirely new one. This can save time by alerting the modeler in advance to gaps in the data, even when the model ends up being modified from the original. This is the subject of a new paper from Inigo Bermejo, Paul Tappenden, and Ji-Hee Youn in Pharmacoeconomics.

In order to assess the feasibility of comprehensive model replication, we undertook a small pilot study in which we attempted to fully reproduce five recent economic analyses published in Pharmacoeconomics between August and November 2016. For each study, the feasibility of replicating the analyses was explored through consideration of published information relating to model structure, assumptions and model parameter values within the full study publication and any supplementary material available online, but excluding information reported in other papers or reports.

Two of these can be fully replicated, two lack enough information to allow replication, and one is declared “potentially replicable.”

It was surprising to me that they didn’t mention this paper by Hoogendoorn, et al., where seven modeling groups agreed to try to replicate a model of COPD and came to significantly different conclusions.

They spend much of the discussion talking about how the goal of cost-effectiveness models – like other forms of scientific publication – is the ability to be faithfully replicated. And, though, they don’t acknowledge that it logically follows from this that we might as well make our models open source, they include that recommendation in their conclusion.

A recent survey of health economic stakeholders suggested that across all stakeholders and countries, making health economic models available in an open format was considered to be beneficial. The provision of open-source models would circumvent problems around model replicability.