Markov modeling

Markov modeling is a clinical research technique that allows competing medical strategies to be mathematically assessed in order to identify the optimal allocation of health care resources.

Markov models are often employed to represent stochastic processes, that is, random processes that evolve over time. In a healthcare context, Markov models are particularly suited to modelling chronic disease.

Briggs and Sculpher describe the use of Markov models for economic evaluation of healthcare interventions. The intuitive way in which Markov models can handle both costs and outcomes make them a powerful tool for economic evaluation modelling. The time component of Markov models can offer advantages of standard decision tree models, particularly with respect to discounting. Tey give in a paper a comprehensive description of Markov modelling for economic evaluation, including a discussion of the assumptions on which the type of model is based, most notably the memoryless quality of Markov models often termed the 'Markovian assumption'. A hypothetical example of a drug intervention to slow the progression of a chronic disease is employed to demonstrate the modelling technique and the possible methods of analysing Markov models are explored. Analysts should be aware of the limitations of Markov models, particularly the Markovian assumption, although the adept modeller will often find ways around this problem ¹⁾.

Markov models are useful when a decision problem involves risk that is continuous over time, when the timing of events is important, and when important events may happen more than once. Representing such clinical settings with conventional decision trees is difficult and may require unrealistic simplifying assumptions. Markov models assume that a patient is always in one of a finite number of discrete health states, called Markov states. All events are represented as transitions from one state to another. A Markov model may be evaluated by matrix algebra, as a cohort simulation, or as a Monte Carlo simulation. A newer representation of Markov models, the Markov-cycle tree, uses a tree representation of clinical events and may be evaluated either as a cohort simulation or as a Monte Carlo simulation. The ability of the Markov model to represent repetitive events and the time dependence of both probabilities and utilities allows for more accurate representation of clinical settings that involve these issues ²⁾.

Wali et al., present a review of the recently published neurosurgical literature that employs Markov modeling and provide a conceptual framework with which to evaluate, critique, and apply the findings generated from health economics research.

The PubMed online database was searched to identify neurosurgical literature published from January 2010 to December 2017 that had utilized Markov modeling for neurosurgical cost effectiveness studies. Included articles were then assessed with regard to year of publication, subspecialty of neurosurgery, decision analytical techniques utilized, and source information for model inputs.

A total of 55 articles utilizing Markov models were identified across a broad range of neurosurgical subspecialties. Sixty-five percent of the papers were published within the past 3 years alone. The majority of models derived health transition probabilities, health utilities, and cost information from previously published studies or publicly available information. Only 62% of the studies incorporated indirect costs. Ninety-three percent of the studies performed a 1-way or 2-way sensitivity analysis, and 67% performed a probabilistic sensitivity analysis. A review of the conceptual framework of Markov modeling and an explanation of the different terminology and methodology are provided. CONCLUSIONS As neurosurgeons continue to innovate and identify novel treatment strategies for patients, Markov modeling will allow for better characterization of the impact of these interventions on a patient and societal level. The aim of this work is to equip the neurosurgical readership with the tools to better understand, critique, and apply findings produced from cost-effectiveness research ³⁾.